JP7834528B2 - Microparticle measurement system and microparticle measurement method - Google Patents

Description

Embodiments of the present invention relate to a microparticle measurement system and a microparticle measurement method.

Traditionally, in organic wastewater treatment, for example, various beneficial microorganisms have been used to decompose organic matter and remove nitrogen and phosphorus from wastewater. While wastewater treatment is currently based on indicators such as sludge concentration and treated water quality, it would be beneficial to be able to measure the concentration of beneficial microorganisms that contribute to organic matter decomposition and nitrogen removal.

Conventional techniques for measuring the concentration of beneficial microorganisms (such as Bacillus microparticles) include colony counting and PCR (Polymerase Chain Reaction). However, these methods require highly specialized equipment, involve the hassle of transporting samples to specialized facilities, and have lengthy measurement times.

Therefore, a technique has been proposed that utilizes the characteristics of light refraction by the target microparticles and detects microparticles (e.g., Bacillus spores) from captured images using deep learning-based image processing. This eliminates the need to transport samples to specialized facilities and allows for rapid measurement of microparticle concentrations.

Japanese Patent Publication No. 2016-147044

However, the deep learning methods described above have a problem in that the measurement accuracy is not consistent. For example, the concentration of Bacillus in sludge is not uniform but varies, and there is variability in the number (concentration) of Bacillus particles visible in the captured images. Therefore, if, for example, the number of Bacillus particles detected per image is small, the accuracy of the measurement results will decrease if the number of captured images is small. Furthermore, it is not always easy for operators to determine whether or not there are too few captured images.

Therefore, the present invention has been made in view of the above circumstances, and aims to provide a microparticle measurement system and a microparticle measurement method that can improve the accuracy of measurements related to microparticles using captured images.

The microparticle measurement system of this embodiment comprises: a light source that emits illumination light onto a liquid containing microparticles to be measured; an objective lens that focuses the illumination light; an imaging lens that forms an image of the focused illumination light; an image sensor that captures the image of the illumination light and outputs an image; a detection unit that detects the microparticles visible in the image; a calculation unit that calculates the reliability of microparticle detection based on the detection result from the detection unit and a predetermined index; and a processing unit that determines the next processing step based on the reliability.

Figure 1 is a schematic diagram of the microparticle measurement system according to an embodiment. Figure 2 is an explanatory diagram of the parameters in the ray tracing matrix. Figure 3 is an explanatory diagram showing the relative transmitted light intensity and other parameters for Bacillus spores. Figure 4 is an explanatory diagram showing the relative transmitted light intensity and other properties of acrylic particles. Figure 5 shows an example of an image of sludge. Figure 6 shows examples of captured images and training images used in deep learning. Figure 7 shows an example of Bacillus detection results using deep learning. Figure 8 shows an example of a reference table used for calculating confidence levels. Figure 9 is a flowchart showing the processing performed by the microparticle measurement system of the embodiment.

The embodiments of the microparticle measurement system and microparticle measurement method of the present invention will be described below with reference to the drawings. In the following, Bacillus spores will also be simply referred to as Bacillus.

Figure 1 is a schematic diagram of the microparticle measurement system 10 according to an embodiment. The microparticle measurement system 10 comprises a light source 11, a stage 13, a stage drive unit 14, a laser displacement meter 15, an objective lens 16, an imaging lens 17, an image sensor 18, a measurement control unit 19, and an information processing device 20. The measurement control unit 19 and the information processing device 20 may be configured as a single unit. Alternatively, the information processing device 20 may be configured as two or more separate units.

The light source 11 emits illumination light L towards the measurement sample SP (liquid; specimen) containing the minute particles to be measured.

Stage 13 supports the slide glass (preparation slide) 12 that holds the sample SP for measurement.

The stage drive unit 14 moves the stage 13 along the optical axis in the vertical direction shown in Figure 1.
The laser displacement meter 15 detects the position of the slide glass 12 using a laser.

The objective lens 16 focuses the illumination light L to form parallel light.
The imaging lens 17 focuses the illumination light L, which has become parallel light, to form an image.

The image sensor 18 captures the illumination light imaged by the imaging lens 17 and outputs the captured image.
The measurement control unit 19 controls the stage drive unit 14 and the image sensor 18.

The information processing device 20 comprises an acquisition unit 21, a detection unit 22, a calculation unit 23, a processing unit 24, a storage unit 25, and a display unit 26.

The acquisition unit 21 acquires the captured image from the image sensor 18.

The detection unit 22 detects minute particles, impurities, and other contaminants visible in the captured image.

The calculation unit 23 calculates the reliability of the detection of minute particles based on the detection results from the detection unit 22 and predetermined indicators (details will be described later).

The processing unit 24 performs various information processing tasks. For example, the processing unit 24 determines the next action based on the reliability level. For example, if the reliability level is below a predetermined threshold, the processing unit 24 displays a screen on the display unit 26 requesting the user (operator) to perform a predetermined action to improve the reliability level. For example, if the reliability level is below a predetermined threshold, the processing unit 24 requests the user to acquire additional captured images.

Furthermore, for example, the processing unit 24 may automatically acquire additional images if the confidence level falls below a predetermined threshold.

The memory unit 25 stores the operation programs for each unit 21-24, various parameters, captured images acquired by the acquisition unit 21, detection results from the detection unit 22, calculation results such as reliability scores from the calculation unit 23, and processing results from the processing unit 24.

The display unit 26 displays various information based on instructions from the processing unit 24.

Furthermore, all or part of the processing performed in each of the above-mentioned sections 21-24 may be executed by a single processor (control unit) based on the operation program and various parameters stored in the storage unit 25.

Next, we will explain the measurement principle for minute particles. When illumination light is shone from the back side of minute particles in a liquid, the lens effect of the minute particles causes the illumination light to be focused to a position corresponding to the particle size and refractive index of the minute particles.

Furthermore, the transmitted light intensity increases as you approach the focusing position, reaching its maximum at the focusing position. As you move away from the focusing position, the transmitted light intensity decreases again. In other words, the position where the transmitted light intensity is maximum is the focusing position. At this time, the focusing position can be determined by measuring the distance between the objective lens 16 and the position where the transmitted light intensity is maximum.

In this case, the optical path of the illumination light can be expressed by the following equation. Therefore, if the distance between the objective lens 16 and the position where the transmitted light intensity is maximum, as well as the particle size of the minute particles, are known, the refractive index of the minute particles can be determined by solving the equation expressed by the ray tracing matrix below.

Figure 2 is an explanatory diagram of the parameters in the ray tracing matrix. In the ray tracing matrix described above, the radius of the microparticle PC is r, the refractive index of the microparticle is n, and the distance between the microparticle and the objective lens 16 when the transmitted light intensity of the illumination light L is maximum is z. Also, the distance from the optical axis when the illumination light L is incident on the microparticle is _x0 , and the angle of incidence when the illumination light L is incident on the microparticle is _u0 . Also, the distance from the optical axis of the illumination light L incident on the image sensor 18 is _x1 , and the angle of incidence of the illumination light L incident on the image sensor 18 is _u1 .

Furthermore, let _l1 be the distance between the objective lens 16 and the imaging lens 17, and _l2 be the distance between the imaging lens 17 and the image sensor 18. Also, let _f1 be the focal length of the objective lens, and _f2 be the focal length of the imaging lens 17.

Furthermore, as described above, if the refractive index of a minute particle is known, the particle size of the minute particle can be calculated by solving the equation expressed by the ray tracing matrix mentioned above.

Furthermore, beneficial microorganisms used in organic wastewater treatment can be considered as microparticles under certain conditions. These conditions include, for example, the formation of spores. When spores are formed, their shape and other characteristics remain unchanged, and their shape is almost constant due to the beneficial microorganisms themselves.

Because spores of beneficial microorganisms possess a unique size (e.g., particle diameter) and refractive index, treating them similarly to fine particles makes it possible to detect such beneficial microorganisms and measure their number (and consequently, their concentration) per observation field.

To measure concentration, the number of beneficial microorganisms in a volume corresponding to the observation field of view × scanning distance is measured by scanning the observation position (image acquisition position) along the optical axis, thereby enabling concentration measurement.

Incidentally, it has been found that in spores of Bacillus strains in sludge with a known refractive index and particle size of 1 μm or less (bacillus spores), the position corresponding to the distance z where the transmitted light intensity is maximum is located within the depth of field (effective focal position) corresponding to the focal length _f1 in image acquisition. Therefore, based on a pre-set transmitted light intensity threshold, the portion with a light intensity above the threshold can be considered a bacillus spore.

In this case, the transmitted light intensity of the liquid containing Bacillus spores will be greater than that of the liquid without Bacillus spores.
Therefore, by setting the threshold for transmitted light intensity used to determine whether or not a sample contains Bacillus spores to a value slightly greater than the transmitted light intensity in a solution without spores, Bacillus spores can be reliably detected.

Furthermore, by using a set threshold to continuously move the sample in the optical axis direction while sequentially acquiring images, and combining this with deep learning (machine learning) that uses the transmitted light intensity at each location (per pixel) obtained from the acquired images as criteria, it becomes possible to detect and measure the number of Bacillus spores, and ultimately, the concentration of Bacillus spores, with higher accuracy.

When performing deep learning, for example, multiple samples with different concentrations of microparticles can be prepared in advance. Supervised learning can then be performed for each sample so that the detection results obtained by the user (operator) are equal to the detection results obtained by deep learning. This allows for obtaining detection results for microparticles according to the particle size and refractive index of the microparticles being studied.

Next, Figure 3 is an explanatory diagram illustrating the relative transmitted light intensity for Bacillus spores. More specifically, Figure 3 illustrates the relationship between the relative transmitted light intensity and the difference between the actual position of the objective lens 16 and the distance z between the Bacillus spore and the objective lens 16, when the transmitted light intensity for the Bacillus spore is maximized.

Furthermore, Figure 4 is an explanatory diagram regarding the relative transmitted light intensity for acrylic particles. More specifically, Figure 4 explains the relationship between the relative transmitted light intensity and the difference in the actual position of the objective lens 16 relative to the distance z between the acrylic particle (microparticle) and the objective lens 16, when the transmitted light intensity is maximized for acrylic particles with a particle diameter of 30 μm.

First, images of Bacillus spores and acrylic particles were acquired at the focal point using the image sensor 18.
Subsequently, the stage 13 was moved vertically along the optical axis direction by the stage drive unit 14, and the positional difference Δz between the position of the objective lens 16 when the relative transmitted light intensity of each minute particle on the image sensor 18 was maximum and the actual position of the objective lens 16 was measured by the laser displacement meter 15.

Figure 3(A) shows the image acquired when the relative transmitted light intensity is maximum in a liquid containing Bacillus spores. As shown in Figure 3(A), it can be seen that the relative transmitted light intensity is maximum at the center of the imaging region. Furthermore, as shown in Figure 3(B), it was calculated that in a liquid containing Bacillus spores, the relative transmitted light intensity is maximum at a positional difference Δz = 0 μm.

In contrast, as shown in Figure 4(B), in the case of a liquid containing acrylic particles with a particle size of 30 μm, the relative transmitted light intensity is negative at a position difference of Δ0 μm where the relative transmitted light intensity is maximum in the Bacillus spore. That is, the transmitted light intensity is lower than the background light intensity. Furthermore, as shown in Figure 4(A), the relative transmitted light intensity is minimum around the acrylic particles. Finally, as shown in Figure 4(B), it was calculated that in a liquid containing acrylic particles with a particle size of 30 μm, the relative transmitted light intensity is maximum outside a position difference of Δz = ±15 μm.

Furthermore, Figure 4(C) shows the image captured when the relative transmitted light intensity was maximum in a liquid containing acrylic particles with a particle size of 30 μm. As shown in Figure 4(C), it can be seen that the relative transmitted light intensity is maximum at the center of the imaging region. And, as shown in Figure 4(D), it was calculated that the relative transmitted light intensity is maximum in a liquid containing acrylic particles with a particle size of 30 μm at a position difference Δz = 26 μm.

Based on these measurement results, the position difference Δz, which corresponds to the difference between the distance z from the Bacillus spores and 30 μm acrylic particles (which are the finest particles when the transmitted light intensity is maximum) to the objective lens 16 and the focal length of the objective lens, was calculated using the ray tracing matrix described above. The position difference Δz was found to be 0.9 μm for the liquid containing Bacillus spores and 22.5 μm for the liquid containing 30 μm acrylic particles, which was found to be in close agreement with the measurement results using the laser displacement meter 15. At this time, the distance between the objective lens 16 and the imaging lens 17 was set to _l1 = 130 mm, the distance between the imaging lens 17 and the image sensor 18 was set to _l2 = 164.5 mm, the focal length of the objective lens was set to _f1 = 4.1125 mm, and the focal length of the imaging lens 17 was set to _f2 = 164.5 mm. Furthermore, the r = 1 μm and n = 1.4 were set for Bacillus spores, and n = 1.5 for acrylic particles.

In particular, it was found that the positional difference Δz = 0.9 μm in the liquid containing Bacillus spores is effectively equal to the focal length of the objective lens 16 (within the depth of field), and the relative transmitted light intensity is maximum at the focal point.
This indicates that Bacillus spores can be detected by measuring the transmitted light intensity at the focal length.

In this way, by utilizing the characteristic that the center of a Bacillus spore glows brightly when the transmitted light intensity is maximum, it is possible to detect only Bacillus spores from the acquired image.

Next, Figure 5 shows an example of an image of sludge. As shown in Figures 5(a) and 5(b), the image may also show contaminants C in addition to Bacillus spores B.

Next, Figure 6 shows examples of captured images and training images used in deep learning. (a) is a captured image showing a Bacillus spore B and impurities C. The user provides the central position P of the Bacillus spore B as ground truth data for this captured image, resulting in the training image shown in (b). By training the network to detect the central position P of the Bacillus spore B in the captured image using these images, deep learning can be performed.

Next, Figure 7 shows an example of Bacillus detection results using deep learning. The detection unit 22 calculates the likelihood (the probability (likelihood) that each pixel is the central position of a Bacillus spore) of the Bacillus spore detection result using deep learning-based image processing.

Figure 7(a) is the input image (captured image). The detection unit 22 calculates a likelihood map, for example, shown in Figure 7(b). This likelihood map indicates that brighter areas have a higher likelihood, and darker areas have a lower likelihood. The symbol Q corresponds to the Bacillus spore B (Figure 7(a)), where the likelihood is high.

The detection unit 22 then performs thresholding on this likelihood and selects pixels with a likelihood above a certain level as the central position of the Bacillus spore, thereby obtaining the detection result shown in Figure 7(c). In Figure 7(c), the symbol S indicates the central position of the detected Bacillus spore.

Next, we will explain the confidence threshold in detail. The confidence threshold is set based on factors such as the number of images required to measure the dominance concentration of Bacillus, the minimum concentration that the microparticle measurement system 10 can measure, and the measurement error of the microparticle measurement system 10.

Furthermore, the confidence threshold may be a value pre-set in the fine particle measurement system 10, or a different value may be set for each site where the fine particle measurement system 10 is installed.

Specifically, the confidence threshold is set to 1.0 for each of the following confidence indicators. In this case, the measurement process is performed so that the confidence level is 1.0 or higher. Furthermore, if a higher confidence level for the concentration measurement results is desired, the threshold may be set to a value greater than 1.0. Conversely, if a high level of confidence is not required, the threshold may be set to a value less than 1.0.

The following are examples of confidence indicators.

(The confidence level is measured by the number of microparticles.)
If the confidence index is the number of fine particles, the calculation unit 23 calculates the confidence level based on the number of fine particles detected.

For example, if we measure L Bacillus spores, we can assume that the measured concentration will statistically match the actual concentration. In this case, the confidence level is calculated using the following formula.

One method for biological measurement (standard counting method) is based on the idea that measuring approximately 30 target organisms will statistically correlate the measured concentration with the actual concentration. Therefore, for example, the variable L can be set to 30.

In this case, the confidence level may be calculated using the reference table shown in Figure 8, rather than the formula described above. However, variable L = 30 is just an example; the range of values for the number of Bacillus cells and the confidence level can be set arbitrarily.

(The reliability indicator is the number of captured images)
If the reliability index is the number of captured images, the calculation unit 23 calculates the reliability based on the number of captured images.

In organic wastewater treatment, Bacillus becomes dominant at concentrations of ^10⁵ [cells/ml] or higher. The confidence level is calculated from the number of images required to measure this concentration.

For example, if the Bacillus concentration is ^10⁵ [cells/ml], and "m Bacillus cells are captured in each image," then L/m images are needed to "measure L target organisms." Therefore, the confidence level is calculated using the following formula.

The reliability is calculated based on the number of images, making it easy for the operator to understand. In other words, for example, if the reliability does not meet the standard, all that is needed is to increase the number of images taken.

Furthermore, if the minimum concentration that the fine particle measurement system 10 can measure (or that you wish to measure) is predetermined, the reliability can be similarly determined.

For example, if the minimum concentration that the microparticle measurement system 10 can measure is ^10³ [particles/ml], and "n Bacillus organisms are captured in each image," then L/n images are needed to "measure L target organisms." Therefore, the confidence level is calculated using the following formula.

(The confidence index is the likelihood of detecting Bacillus spores.)
If the confidence index is the likelihood of the Bacillus spore detection result, the calculation unit 23 calculates the confidence level based on the likelihood of the Bacillus spore detection result obtained by image processing using deep learning.

For example, if the confidence criterion is "the mean likelihood of the Bacillus detection result is r or greater," the confidence level is calculated using the following formula.

(The confidence level is measured by the detection results of impurities.)
If the confidence index is the result of detecting impurities other than Bacillus spores, the calculation unit 23 calculates the confidence level based on the detection result of the impurities.

For example, if the confidence criterion is "the number of pixels occupied by non-Bacillus particles is s or less," the confidence level is calculated using the following formula.

Furthermore, in addition to the number of pixels occupied by the interfering elements, the confidence level may also be calculated using the size and number of the interfering elements.

Next, Figure 9 is a flowchart showing the processing performed by the microparticle measurement system 10 of the embodiment. First, we will explain the steps taken prior to this processing.

First, the measurement operator collects water from a water treatment device containing the microorganism (Bacillus) to be measured. The collected sample undergoes a predetermined pretreatment (filtering, heat treatment, etc.). The pretreated sample is then placed in the microparticle measurement system 10.

Next, in step S1 of Figure 9, the acquisition unit 21 acquires the captured image from the image sensor 18.

Next, in step S2, the detection unit 22 detects Bacillus spores visible in the captured image.

Next, in step S3, the calculation unit 23 calculates the reliability of the fine particle detection based on the detection results from step S2 and a predetermined index.

Next, in step S4, the processing unit 24 determines whether the confidence level calculated in step S3 is equal to or greater than the threshold. If the answer is Yes, the process terminates; otherwise, the process proceeds to step S5.

In step S5, the processing unit 24 displays a screen on the display unit 26 requesting the user to perform a predetermined action to improve reliability. For example, the processing unit 24 requests the user to acquire additional images. The user, upon seeing this display, may shift the slide glass 12 to perform imaging with the image sensor 18, or replace the slide glass 12 to perform imaging with the image sensor 18, in order to improve reliability.

Other tasks performed by the user include, for example, filtering to remove impurities, heat treatment to induce spore formation of Bacillus bacteria, and injection of activators.

Furthermore, if the reliability does not reach a threshold level even after performing reliability-enhancing operations a certain number of times, the measurement result may be marked as "concentration below the detection limit" or "unmeasurable," and the measurement may be terminated.

Furthermore, for example, if the configuration of the microparticle measurement system 10 is different and the sample is imaged directly without using the slide glass 12, the sample may be shaken or otherwise altered before imaging with the image sensor 18.

Furthermore, if, for example, the configuration of the microparticle measurement system 10 is different and it is possible to automatically acquire additional new images, the processing unit 24 may be configured to automatically acquire these additional new images. In that case, for example, a device for automatically shifting the slide glass 12 may be provided.

After the predetermined operation requested in step S5 is completed, the processes from step S1 onward are executed again.

In this way, the microparticle measurement system 10 of this embodiment can improve the accuracy of microparticle measurement by calculating the reliability of microparticle detection and determining the next processing step based on that reliability. In other words, by requesting specific actions from the operator when the reliability is low, even operators without specialized knowledge can easily perform the appropriate tasks.

The above description focuses on a standalone configuration of the microparticle measurement system 10, but it is not limited to this configuration. For example, the microparticle measurement system 10 may acquire images using an image sensor 18 on the local terminal side, transfer these images to a cloud server via a communication interface and communication network, process them using an information processing device 20 on the cloud server side, and display the processing results on the local terminal.

Furthermore, the microparticle measurement system 10 of this embodiment includes a control device such as a CPU (Central Processing Unit), a storage device such as a ROM (Read Only Memory) or RAM (Random Access Memory), an external storage device such as an HDD (Hard Disk Drive), a display device such as a display unit, and an input device such as a keyboard or mouse, thus having a hardware configuration that utilizes a standard computer.

Furthermore, the program executed by the microparticle measurement system 10 of this embodiment is provided as an installable or executable file, recorded on a computer-readable recording medium such as a DVD (Digital Versatile Disk), USB (Universal Serial Bus) memory, or SSD (Solid State Drive) or other semiconductor storage device.

Furthermore, the program may be configured to be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program may be configured to be provided or distributed via a network such as the Internet.

Furthermore, the program may be pre-installed and provided in ROM or similar media.

10...Microparticle measurement system, 11...Light source, 12...Slide glass, 13...Stage, 14...Stage drive unit, 15...Laser displacement meter, 16...Objective lens, 17...Imaging lens, 18...Image sensor, 19...Measurement control unit, 20...Information processing device, 21...Acquisition unit, 22...Detection unit, 23...Calculation unit, 24...Processing unit, 25...Storage unit, 26...Display unit

Claims (5)

A light source that emits illumination light onto a liquid containing the fine particles to be measured,
An objective lens that focuses the aforementioned illumination light,
An imaging lens that forms an image of the focused illumination light,
An image sensor captures the imaged illumination light and outputs the captured image,
A detection unit for detecting the minute particles shown in the captured image,
A calculation unit calculates the reliability of minute particle detection based on the detection results from the detection unit and the number of captured images,
A processing unit that determines the next process based on the aforementioned reliability,
A microparticle measurement system equipped with the following features. The fine particle measurement system according to claim 1, wherein the processing unit displays a screen on the display unit requesting the user to perform a predetermined action to improve the reliability when the reliability is below a predetermined threshold. The microparticle measurement system according to claim 1, wherein the processing unit requests the user to acquire additional images when the reliability is below a predetermined threshold. The microparticle measurement system according to claim 1, wherein the processing unit automatically acquires additional captured images when the reliability is below a predetermined threshold. A method for measuring minute particles using a minute particle measurement system comprising: a light source that emits illumination light onto a liquid containing minute particles to be measured; an objective lens that focuses the illumination light; an imaging lens that forms an image of the focused illumination light; an image sensor that captures the imaged illumination light and outputs an image; a detection unit; a calculation unit; and a processing unit, wherein the system comprises:
The detection unit performs a detection step of detecting the minute particles that are visible in the captured image,
The calculation unit performs a calculation step in which it calculates the reliability of the detection of minute particles based on the detection result from the detection step and the number of captured images,
A method for measuring fine particles, comprising a processing step in which the processing unit determines the next processing based on the reliability.