JP7673709B2 - Data processing methods - Google Patents

Description

The present invention relates to a data processing method.

Techniques for removing noise contained in measurement data are known. Non-Patent Document 1 discloses a technique for mapping spectral data to a partially circulant matrix, performing singular value decomposition on this matrix, extracting a specified number of singular values in descending order of size, and restoring a signal from which the noise components have been separated, based on a vector related to the extracted singular values.

X. C. Chen et al. "Denoising scheme based on singular-value decomposition for one-dimensional spectra and its application in precision storage-ring mass spectrometry" PHYSICAL REVIEW E 99, 063320 (2019)

The inventors recognized that with the technology of Non-Patent Document 1, depending on the magnitude of the noise contained in the spectral data, the accuracy of noise separation may decrease, and the spectrum of the signal after noise separation may deteriorate.

The objective of the present invention is to provide a technology that can suppress degradation of signal data after noise separation.

In order to solve the above problems, a data processing method according to one aspect of the present invention includes a first step of acquiring measurement data such as a spectrum, a second step of separating the measurement data into signal data and noise data, a third step of multiplying the separated noise data by a predetermined coefficient smaller than 1, and a fourth step of adding the noise data after multiplication by the coefficient to the signal data separated in the second step to create synthetic data. The second to fourth steps are repeated until the signal data separated in the current second step and the signal data separated in the previous second step satisfy a predetermined condition regarding similarity, and in the second step and thereafter, the synthetic data is separated into signal data and noise data.

The present invention can suppress degradation of signal data after noise separation.

FIG. 1 illustrates a configuration of a data processing device according to an embodiment. 2 is a diagram showing an example of measurement data acquired by an acquisition unit in FIG. 1 ; 3 is a diagram showing an example of noise data separated from the measurement data of FIG. 2 by the separation unit of FIG. 1; 3 is a diagram showing an example of signal data separated from the measurement data in FIG. 2 by the separation unit in FIG. 1 . 2 is a diagram showing an example of signal data separated by the separation unit in FIG. 1 when a predetermined condition is satisfied; FIG. FIG. 6 is a diagram showing an example of background data separated from the signal data of FIG. 5 . FIG. 6 is a diagram showing an example of final signal data separated from the signal data of FIG. 5 . 2 is a flowchart showing a process of the data processing device of FIG. 1 .

The data processing device of the embodiment separates the measurement data into signal data and noise data. Examples of the measurement data include measurement data of various materials. Examples of the material measurement data include spectra obtained by irradiating a material with X-rays, neutron beams, or electron beams, diffraction and scattering patterns, and power spectra derived from images of materials captured with a microscope. Since noise is superimposed on such measurement data, the noise is separated as a preprocessing step for material data analysis.

The inventors recognized that when the measurement data contains a relatively large, sharp intensity (hereinafter referred to as a specific peak), if the technology of Non-Patent Document 1 is used to separate the spectral data into signal data and noise data, the separated signal data may be influenced by the specific peak, resulting in a decrease in smoothness at the position of the specific peak and its neighboring positions. In this case, due to the influence of the specific peak, some components of the signal data at the position of the specific peak and its neighboring positions are included in the noise data. Therefore, the separated signal data has a smaller intensity than the original value at the position of the specific peak and its neighboring positions, and the spectral shape at these positions becomes sawtooth or uneven. Such signal data is different from the original signal data and is degraded, so it is desirable to suppress this degradation.

The data processing device of the embodiment multiplies the separated noise data by a predetermined coefficient to reduce its intensity, adds the noise data after multiplication by the coefficient back to the separated signal data to create synthetic data, and then separates the synthetic data again into signal data and noise data. By repeating this process, smoother signal data can be obtained.

Figure 1 shows the configuration of a data processing device 1 according to an embodiment. The data processing device 1 includes an acquisition unit 10, a separation unit 12, a calculation unit 14, a synthesis unit 16, a determination unit 18, a processing unit 20, and an output unit 22. The configuration of the data processing device 1 can be realized in hardware terms by the CPU, memory, and other LSIs of any computer, and in software terms by programs loaded into memory, but the functional blocks realized by the cooperation of these are depicted here. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware alone, software alone, or a combination of these.

The acquisition unit 10 acquires the measurement data to be processed, for example, via a network.

Figure 2 shows an example of measurement data acquired by the acquisition unit 10 in Figure 1. This is measurement data obtained by X-ray diffraction. The vertical axis is intensity, and the horizontal axis is 2θ. There is a sharp, relatively large specific peak at 2θ = 30 [deg.]. If the specific peak is due to noise, it can also be called an outlier.

The separation unit 12 separates the measurement data acquired by the acquisition unit 10 into signal data and noise data. The separation unit 12 performs the separation process using, for example, the noise removal method described in Non-Patent Document 1, without using a trained model. The separation unit 12 separates peaks with relatively small half-widths, i.e., sharp peaks, as noise, and peaks with relatively large half-widths, i.e., broad peaks, as signal data. The original measurement data is obtained by adding together the numerical data that correspond one-to-one in the separated signal data and noise data. This noise removal method is a publicly known technique, and therefore further explanation will be omitted. The separation unit 12 may perform the separation process using other publicly known techniques.

As described above, if a specific peak exists and is determined to be noise, the separation unit 12 includes some components of the signal data at the position of the specific peak and its neighboring positions in the noise data. The greater the intensity of the peak determined to be noise, the greater the intensity of the some components of the signal data that will be included in the noise data. If a specific peak does not exist, i.e., if the maximum value of the peak intensity is relatively small, the separation unit 12 separates the signal data from the noise data without including some components of the signal data in the noise data.

Figure 3 shows an example of noise data separated from the measurement data of Figure 2 by the separation unit 12 of Figure 1. At and near 2θ = 30 [deg.], which is the position of the specific peak, the noise data also contains some components of the signal data.

Figure 4 shows an example of signal data separated from the measurement data of Figure 2 by the separation unit 12 of Figure 1. At and near 2θ = 30 [deg.], which is the position of a specific peak, the signal data shows a decrease in signal strength and a decrease in smoothness.

The calculation unit 14 multiplies the noise data separated by the separation unit 12 by a predetermined coefficient, and reduces the intensity of the noise data by a factor of the coefficient. Specifically, the calculation unit 14 multiplies each of the multiple numerical data constituting the separated noise data by a coefficient. The coefficient is greater than 0 and less than 1. The coefficient can be determined appropriately through experiments or simulations, and may be determined, for example, in the range from 0.7 to 0.8. This process also reduces the intensity of specific peaks.

The synthesis unit 16 adds the noise data multiplied by the coefficient in the calculation unit 14 to the signal data separated by the separation unit 12 to create synthesized data. The synthesis unit 16 adds together the numerical data that have a one-to-one correspondence between the noise data multiplied by the coefficient and the separated signal data.

The separation unit 12 separates the composite data created by the synthesis unit 16 again into signal data and noise data. In the composite data, the intensity of certain peaks is smaller than in the measurement data acquired by the acquisition unit 10, so the intensity of some components of the signal data that are included in the noise data by the separation process is reduced. This makes it possible to improve the smoothness of the separated signal data. In other words, degradation of the signal data can be suppressed.

When the separation unit 12 executes the separation process two or more times, the determination unit 18 determines whether the signal data separated in the current separation process by the separation unit 12 and the signal data separated in the previous separation process satisfy a predetermined condition regarding similarity. The predetermined condition is, for example, that the similarity between the vector representing the signal data separated this time and the vector representing the signal data separated the previous time is equal to or less than a predetermined threshold. The threshold can be appropriately determined by experiment or simulation. The inner product of these two vectors can be used as the similarity between these two vectors.

The determination unit 18 repeats the separation process by the separation unit 12, the coefficient multiplication process by the calculation unit 14, and the process of creating synthetic data by the synthesis unit 16 until a predetermined condition is met. By repeating these processes, the intensity of the specific peak decreases, and thus the effect of the specific peak on the signal data in the separation process decreases.

When certain conditions are met, the signal data separated in this separation process is similar to the signal data separated in the previous separation process. In this case, the noise data separated in this separation process is also similar to the noise data separated in the previous separation process and is sufficiently small.

Figure 5 shows an example of signal data separated by the separation unit 12 in Figure 1 when certain conditions are met. The smoothness of the signal data is improved compared to Figure 4 at and around 2θ = 30 [deg.], which is the position of a specific peak.

When a predetermined condition is met, the processing unit 20 subtracts the signal data separated in this separation process from the measurement data acquired by the acquisition unit 10, and regards the subtraction result as the final noise data.

When a predetermined condition is satisfied, the processing unit 20 further separates the signal data separated in this separation process into background data and final signal data using known techniques. The final signal data is the signal data obtained after removing noise data and background data from the measurement data acquired by the acquisition unit 10.

Figure 6 shows an example of background data separated from the signal data of Figure 5. Background data can also be called a baseline.

Figure 7 shows an example of the final signal data separated from the signal data in Figure 5. In the final signal data, background and noise have been removed.

When a predetermined condition is met, the processing unit 20 obtains the peak position from the signal data separated in the current separation process or the final signal data.

The output unit 22 outputs the obtained background data, final noise data, final signal data, and peak position data to, for example, an analysis device (not shown). The analysis device may be included in the data processing device 1.

The analysis device uses known techniques to analyze background data, noise data, signal data, and peak position data for multiple samples of various materials, and presents the analysis results to the user. From the background data, information can be obtained, for example, regarding the amorphous components of the material, higher-order structure, fluorescence, and instrumental differences in the measurement device. From the noise data, information can be obtained, for example, regarding instrumental differences in the measurement device and fluctuations in the optical system of the measurement device. From the signal data, information can be obtained, for example, regarding the crystalline phase, crystalline particle size, and interfacial strain. From the peak position data, information can be obtained, for example, regarding the crystalline phase.

The processing of the embodiment can improve the accuracy of noise data, signal data, and background data when the measurement data contains a specific peak, thereby improving the accuracy of the analysis results of this data.

Figure 8 is a flowchart showing the processing of the data processing device 1 of Figure 1. The data processing device 1 acquires measurement data (first step: S10) and separates the measurement data into signal data and noise data (second step: S12). If the separation process has not been performed two or more times (N in S14), the process proceeds to S18. If the separation process has been performed two or more times (Y in S14), and the signal data separated in this S12 and the signal data separated in the previous S12 do not satisfy a predetermined condition (N in S16), the data processing device 1 multiplies the noise data by a predetermined coefficient (third step: S18), adds the noise data after multiplication by the coefficient to the signal data separated in this S12 to create composite data (fourth step: S20), and returns to S12. In S12 after S20, the data processing device 1 separates the composite data created in S20 into signal data and noise data. In S16, if the signal data separated in this S12 and the signal data separated in the previous S12 satisfy a predetermined condition (Y in S16), the process ends.

In this way, the data processing device 1 repeats the processes of S12, S14, S16, S18, and S20 until the signal data separated in this S12 and the signal data separated in the previous S12 satisfy a predetermined condition.

According to the embodiment, even if the measurement data contains a large specific peak, the noise separation accuracy can be improved and degradation of the signal data after noise separation can be suppressed.

The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing process, and that such modifications are also within the scope of the present invention.

For example, the measurement data is not limited to measurement data of a material, but may be measurement data of any object, or may be the frequency spectrum of an electrical signal, or may be any data.

1: data processing device, 10: acquisition unit, 12: separation unit, 14: calculation unit, 16: synthesis unit, 18: judgment unit, 20: processing unit, 22: output unit.

Claims (1)

A first step of acquiring measurement data;
a second step of separating the measurement data into signal data and noise data;
a third step of multiplying the separated noise data by a predetermined coefficient less than 1;
a fourth step of adding the noise data after multiplication by the coefficient to the signal data separated in the second step to generate synthetic data;
Equipped with
repeating the second step to the fourth step until the signal data separated in the second step this time and the signal data separated in the second step previously satisfy a predetermined condition regarding similarity;
In the second step from the second time onwards, the composite data is separated into signal data and noise data.
23. A data processing method comprising:

Images