JP7707664B2 - Cell image analysis method - Google Patents

Description

This invention relates to a cell image analysis method.

Conventionally, cell image analysis methods are known (see, for example, Patent Document 1).

Patent Document 1 discloses a cell image analysis method for analyzing an image of a cell captured by an imaging device. Specifically, Patent Document 1 acquires a first cell image and a second cell image, and obtains a correlation value or difference between the cell feature amounts in the first cell image and the second cell image. The configuration disclosed in Patent Document 1 analyzes the cell image based on the time series change in the correlation value or difference between the cell feature amounts in the first cell image and the second cell image.

JP 2010-22318 A

Although not disclosed in Patent Document 1, depending on the shooting conditions when capturing a cell image, for example, the cells may be captured in a blurred state, resulting in a cell image that is not suitable for analysis. Furthermore, when analyzing a cell image, multiple cell images may be captured, for example, by capturing images of multiple locations in a container in which cells are cultured, and the captured multiple cell images may be analyzed together through batch processing. In this way, when multiple cell images are captured, there is the inconvenience that the user may not easily notice that an image that is not suitable for analysis is included. In this case, the analysis of the cell images is performed in a state in which the cell images are not suitable for analysis are included, resulting in a problem of reduced accuracy of the analysis results.

This invention has been made to solve the above problems, and one object of the invention is to provide a cell image analysis method that can prevent a decrease in the accuracy of cell image analysis by preventing analysis of cell images that are not suitable for analysis from being performed in a state where the cell images are included.

In order to achieve the above object, a cell image analysis method according to one aspect of the present invention includes the steps of acquiring a cell image, determining whether the cell image includes at least one of blurred cells in the cell image, reflection of foreign matter in the cell image, reflection of stray light in the cell image, degradation in image quality of the cell image due to the culture medium when culturing the cells, an area in the cell image where pixel values are saturated, and cells where the boundary between the cell area and the non-cell area is unclear, and classifying the cell image into a proper cell image if the cell image does not include blurred cells, reflection of foreign matter, reflection of stray light, degradation in image quality of the cell image due to the culture medium, an area where pixel values are saturated, and cells where the boundary is unclear. The method includes the steps of: classifying the cell image into an inappropriate cell image if the cell image contains at least one of reflections of objects, reflections of stray light, degradation of the image quality of the cell image due to culture medium, areas with saturated pixel values, and cells with unclear boundaries; generating an inappropriate cell image by performing image processing that degrades the image quality of the appropriate cell image; and creating a first trained model that has been trained to determine whether a cell image is an appropriate cell image or an inappropriate cell image by learning from appropriate cell images and inappropriate cell images generated from the appropriate cell images, and in the determination step, a determination is made as to whether the cell image is an appropriate cell image or an inappropriate cell image using the first trained model .

In the cell image analysis method according to the above aspect, as described above, the steps include a step of determining whether the cell image includes at least one of blurred cells in the cell image, reflection of foreign matter in the cell image, reflection of stray light in the cell image, deterioration in image quality of the cell image due to the culture medium when culturing the cells, an area in the cell image where pixel values are saturated, and a cell where the boundary between the cell area and the non-cell area is unclear; and classifying the cell image as a proper cell image if the cell image does not include blurred cells, reflection of foreign matter, reflection of stray light, deterioration in image quality of the cell image due to the culture medium, an area where pixel values are saturated, and a cell where the boundary is unclear. The method includes a step of classifying a cell image into an inappropriate cell image when the cell image includes at least one of the following: a step of performing image processing to degrade the image quality of the appropriate cell image to generate an inappropriate cell image; and a step of creating a first trained model that has learned to determine whether a cell image is an appropriate cell image or an inappropriate cell image by learning the appropriate cell image and the inappropriate cell image generated from the appropriate cell image. In the step of determining, the first trained model is used to determine whether the cell image is an appropriate cell image or an inappropriate cell image. This allows only appropriate cell images that do not include blurred cells, reflections of foreign matter, reflections of stray light, deterioration of the image quality of the cell image due to the culture medium, regions with saturated pixel values, and cells with unclear boundaries to be used for analysis. As a result, it is possible to prevent analysis from being performed using cell images that include inappropriate cell images, thereby preventing a decrease in the accuracy of the analysis of the cell image.

1 is a schematic diagram showing an overall configuration of a cell-image analyzer according to an embodiment. 1A to 1G are schematic diagrams for explaining appropriate cell images and inappropriate cell images. A schematic diagram for explaining a learning method of a first learning model according to one embodiment and a method of classifying cell images using the first trained learning model. 1 is a flowchart for explaining a process for creating a first trained model according to one embodiment. 1 is a flowchart for explaining a process in which an image analysis unit according to one embodiment analyzes a cell image. 1 is a schematic diagram for explaining a configuration in which a cell-image analyzing device according to an embodiment analyzes a cell image when capturing the cell image. FIG. 11 is a schematic diagram for explaining a configuration in which a cell-image analyzing device according to one embodiment notifies a user that an inappropriate cell image has been captured. FIG. 1 is a flowchart for explaining an analysis process performed by a cell-image analyzing device in one embodiment when capturing a cell image. A schematic diagram for explaining a configuration in which a cell image analysis apparatus according to one embodiment generates a second trained model. 13 is a schematic diagram for explaining a configuration in which a cell image analysis device according to one embodiment notifies that generation of a second trained model has been completed. 13 is a flowchart illustrating a process for generating a second trained model by a cell image analysis apparatus according to an embodiment. 1 is a schematic diagram for explaining a configuration for analyzing a cell image by a cell-image analyzing device according to an embodiment. FIG. 1 is a schematic diagram for explaining a configuration in which a cell-image analyzing device according to an embodiment notifies the user that analysis of a cell image has been completed. 1 is a flowchart for explaining a process of analyzing a cell image by a cell-image analyzing device according to an embodiment.

The following describes an embodiment of the present invention with reference to the drawings.

The configuration of a cell image analysis device 100 according to one embodiment will be described with reference to FIG. 1.

(Configuration of cell image analyzer)
As shown in FIG. 1, the cell image analyzer 100 includes an image acquisition unit 1, a processor 2, a storage unit 3, a display unit 4, and an input receiving unit 5.

The image acquisition unit 1 is configured to acquire a cell image 10. The cell image 10 is an image of a cultured cell 90 (see FIG. 2(A)) cultured in a container. In this embodiment, the image acquisition unit 1 is configured to acquire the cell image 10 from a device that captures the cell image 10, such as a microscope 20 equipped with an imaging device. The image acquisition unit 1 includes, for example, an input/output interface.

The processor 2 is configured to analyze the acquired cell image 10. The processor 2 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array) configured for image processing. The processor 2, which is made up of hardware such as a CPU, includes a control unit 2a, an image judgment unit 2b, an image analysis unit 2c, a trained model generation unit 2d, and an inappropriate cell image generation unit 2e as functional blocks of software (programs). The processor 2 functions as a control unit 2a, an image determination unit 2b, an image analysis unit 2c, a trained model generation unit 2d, and an inappropriate cell image generation unit 2e by executing a program stored in the storage unit 3. The control unit 2a, the image determination unit 2b, the image analysis unit 2c, the trained model generation unit 2d, and the inappropriate cell image generation unit 2e may be individually configured by hardware with a dedicated processor (processing circuit).

The control unit 2a is configured to control the cell image analysis device 100. The control unit 2a is also configured to control the display unit 4 to display the analysis results 50.

The image judgment unit 2b is configured to judge whether the image quality of the cell image 10 satisfies the judgment criterion. In this embodiment, the image judgment unit 2b is configured to judge whether the image quality of the cell image 10 satisfies the judgment criterion by combining judgment by the first trained model 6 and judgment by image processing (rule-based processing). The judgment criterion is a criterion for at least one of judging whether the image quality of the cell image 10 is suitable for analysis of the cell image 10 when classifying the cell image 10 for analysis of the cell image 10, and judging whether the image quality of the cell image 10 is suitable for learning analysis for the second learning model 7a when classifying the cell image 10 used for learning the second learning model 7a (see FIG. 9) for analyzing the cell image 10. Specifically, the criteria include the presence or absence of blurring of the cell 90 (see FIG. 2A), the presence or absence of reflection of a foreign body 80 (see FIG. 2C), the presence or absence of reflection of stray light, the presence or absence of degradation in image quality of the cell image 10 due to the culture medium, the presence or absence of a region 83 in the cell image 10 where pixel values are saturated (see FIG. 2F), and the presence or absence of a cell 90a (see FIG. 2G) where the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is unclear. In this embodiment, the image determination unit 2b determines whether the cell image 10 includes at least one of the following: blurring of the cell 90 reflected in the cell image 10, reflection of a foreign body 80 in the cell image 10, reflection of stray light in the cell image 10, degradation in image quality of the cell image 10 due to the culture medium when culturing the cell 90, the region 83 where pixel values are saturated in the cell image 10, and the cell 90a where the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is unclear. In this embodiment, the cell image 10 is an image of a cell 90 having differentiation potential. For example, the cell 90 includes an iPS cell (induced pluripotent stem cell), an ES cell (embryonic stem cell), etc.

The image analysis unit 2c is configured to analyze the cell image 10. In this embodiment, the image analysis unit 2c is configured to analyze, for example, whether a cell 90 (see FIG. 2) shown in the cell image 10 is an undifferentiated cell or an undifferentiated deviation cell. An undifferentiated cell is a cell that has the ability to differentiate. An undifferentiated deviation cell is a cell that does not have the ability to differentiate.

The trained model generation unit 2d is configured to generate the first trained model 6 by training the first trained model 6a (see FIG. 5). The trained model generation unit 2d is also configured to generate the second trained model 7 by training the second trained model 7a (see FIG. 9). Details of the configuration in which the trained model generation unit 2d generates the first trained model 6 and the second trained model 7 will be described later.

The inappropriate cell image generating unit 2e is configured to generate an inappropriate cell image 12 from an appropriate cell image 11. Details of the configuration by which the inappropriate cell image generating unit 2e generates the inappropriate cell image 12 will be described later.

The storage unit 3 is configured to store the cell image 10, the first trained model 6, and the second trained model 7. The storage unit 3 is also configured to store various programs executed by the processor 2. The storage unit 3 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

The display unit 4 is configured to display the analysis results 50 analyzed by the image analysis unit 2c. The display unit 4 includes, for example, a display device such as a liquid crystal monitor.

The input reception unit 5 is configured to be able to receive operational input from an operator. The input reception unit 5 includes input devices such as a mouse and a keyboard.

(Appropriate cell images and inappropriate cell images)
The appropriate cell image 11 and the inappropriate cell image 12 will be described with reference to FIGS. 2(A) to 2(G).

Figure 2 (A) is a schematic diagram of an appropriate cell image 11. The appropriate cell image 11 is an image that satisfies the judgment criteria. Specifically, the appropriate cell image 11 is an image that does not include blurred cells 90, does not include foreign matter 80 (see Figure 2 (C)), does not include stray light, does not include degradation of image quality due to the culture medium, does not include an area 83 with saturated pixel values (see Figure 2 (F)), and does not include cells 90a where the boundary 84 (see Figure 2 (G)) between the cell 90 area and the area other than the cell 90 91 (see Figure 2 (G)) is unclear.

Figure 2 (B) is a schematic diagram of an inappropriate cell image 12a. In the inappropriate cell image 12a shown in Figure 2 (B), the cell 90 is in a blurred state. In other words, the inappropriate cell image 12a is an image in which the cell image 10 was not focused when it was photographed, and the inside of the cell 90 is unclear. That is, the inappropriate cell image 12a shown in Figure 2 (B) is an image that does not satisfy the criteria for determining whether the cell 90 in the cell image 10 is blurred. Note that in the example shown in Figure 2 (B), for convenience, the boundary between the cell 90 and the background is illustrated with a dashed line to represent the blurred state of the cell 90. Also, in the example shown in Figure 2 (B), for convenience, the cell 90 is hatched differently from the cell 90 shown in Figure 2 (A) to represent that the inside of the cell 90 is unclear.

Figure 2 (C) is a schematic diagram of an inappropriate cell image 12b. The inappropriate cell image 12b shown in Figure 2 (C) is an image in which a foreign object 80 is reflected. In other words, the inappropriate cell image 12 shown in Figure 2 (C) is an image that does not satisfy the criteria for determining whether or not a foreign object 80 is reflected in the cell image 10. Note that the foreign object 80 is anything other than a cell 90, and includes objects that have been mixed in from the outside and insoluble substances contained in the culture medium.

Figure 2 (D) is a schematic diagram of an inappropriate cell image 12c. The inappropriate cell image 12c shown in Figure 2 (D) is an image in which stray light is reflected. In other words, the inappropriate cell image 12c shown in Figure 2 (D) is an image that does not satisfy the criteria for determining whether or not stray light is reflected in the cell image 10. Note that the inappropriate cell image 12c shown in Figure 2 (D) is an image in which stray light reflected by a container or the like is reflected as a bright line 81.

2(E) is a schematic diagram of an inappropriate cell image 12d. The inappropriate cell image 12d shown in FIG. 2(E) is an image in which the image quality of the cell image 10 is degraded due to the culture medium when culturing the cell 90. That is, the inappropriate cell image 12d shown in FIG. 2(E) is an image that does not satisfy the criteria for determining whether the image quality of the cell image 10 is degraded due to the culture medium. Here, at the position where the inner circumferential surface of the container in which the cell 90 is cultured contacts the culture medium, the liquid surface of the culture medium is bent due to surface tension. This is called a meniscus. The thickness of the culture medium in the optical axis direction of the optical member of the microscope 20 is different between the portion where the meniscus is generated and the portion where the meniscus is not generated. In the meniscus when the liquid surface of the culture medium is concave, the thickness of the culture medium in the meniscus portion is large. Therefore, when a concave meniscus is generated, the meniscus portion becomes dark. In addition, in the meniscus when the liquid surface of the culture medium is convex, the thickness of the culture medium in the meniscus portion is small. Therefore, when a convex meniscus occurs, the meniscus portion becomes bright. The example shown in FIG. 2(E) is a schematic diagram of a case where a concave meniscus occurs. In other words, the degradation of image quality caused by the medium when culturing cells 90 means that the meniscus causes areas of low brightness or areas of high brightness in cell image 10.

Figure 2 (F) is a schematic diagram of an inappropriate cell image 12e. The inappropriate cell image 12e shown in Figure 2 (F) is an image that includes an area 83 in the cell image 10 where the pixel values are saturated. In other words, the inappropriate cell image 12e shown in Figure 2 (F) is an image that does not satisfy the criteria for determining whether or not an area 83 in the cell image 10 where the pixel values are saturated exists. Note that the inappropriate cell image 12e shown in Figure 2 (F) is an image in which an area 83 in the cell image 10 where the pixel values are saturated has occurred due to stray light or the like.

2(G) is a schematic diagram of an inappropriate cell image 12f. The inappropriate cell image 12f shown in FIG. 2(G) is an image including a cell 90a in which the boundary 84 between the cell 90 region and the region 91 other than the cell 90 is unclear. In other words, the inappropriate cell image 12f is an image in which the cell image 10 is in focus when photographed, and the inside of the cell 90a is clearly shown, but the boundary 84 is unclear due to stray light or the like. The region 91 other than the cell 90 includes, for example, the background part of the cell image 10. In addition, in the example shown in FIG. 2(G), for convenience, the boundary 84 is not clear, which is represented by illustrating the boundary 84 with a dashed line. In addition, in the example shown in FIG. 2(G), for convenience, the inside of the cell 90a is clearly shown, which is represented by applying hatching to the cell 90a in the same manner as the cell 90 shown in FIG. 2(A).

(Image analysis method)
Next, a method of analyzing a cell image 10 using the cell image analysis method according to the present embodiment will be described with reference to FIG. 3. In this embodiment, a configuration will be described in which the cell image analysis device 100 analyzes the cell image 10 to classify the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12. In this embodiment, the cell image analysis device 100 analyzes the cell image 10 using the first trained model 6 to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. The first trained model 6 outputs a cell image classification label 13 when the cell image 10 is input. The cell image classification label 13 includes 0 and 1. When the cell image classification label 13 is 1, it means that the cell image 10 has been classified into an appropriate cell image 11. Also, when the cell image classification label 13 is 0, it means that the cell image 10 has been classified into an inappropriate cell image 12.

The cell image analysis device 100 also uses the second trained model 7 to analyze the appropriate cell image 11 to determine whether the cell 90 depicted in the appropriate cell image 11 is an undifferentiated cell or an undifferentiated deviant cell.

Figure 3 is a block diagram showing the flow of image processing according to this embodiment. As shown in Figure 3, in this embodiment, the cell image analysis method broadly includes an image analysis method 101 and a method 102 for generating a first trained model 6.

(Generation of First Learning Model)
The method 102 for generating the first trained model 6 according to the present embodiment generates the first trained model 6 by having the first trained model 6a learn to classify the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12 using the teacher cell image 30 and the teacher cell image classification label 31. That is, the appropriate cell image 11 as the teacher cell image 30 is used as input data, and 1 as the teacher cell image classification label 31 is used as output data, thereby making the model learn that the input image is an appropriate cell image 11. Also, the inappropriate cell image 12 as the teacher cell image 30 is used as input data, and 0 as the teacher cell image classification label 31 is used as output data, thereby making the model learn that the input image is an inappropriate cell image 12. The first trained model 6 is, for example, a convolutional neural network (CNN) shown in FIG. 3, or includes a convolutional neural network as a part thereof. The first trained model 6 generated by training the first learning model 6a is stored in the memory unit 3 (Figure 1) of the cell image analysis device 100.

(Image analysis method)
The image analysis method 101 according to this embodiment is an image analysis method that classifies a cell image 10 acquired by the image acquisition unit 1 from a microscope 20 or the like into an appropriate cell image 11 and an inappropriate cell image 12. The image analysis method 101 according to this embodiment includes a step of acquiring the cell image 10, a step of determining whether or not the image quality of the cell image 10 satisfies a judgment criterion, and a step of classifying the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12. Detailed processing of each step of the image analysis method 101 will be described later.

In this embodiment, as shown in FIG. 3, the step of acquiring a cell image 10 is performed by the image acquisition unit 1. The image acquisition unit 1 acquires the cell image 10 from an image capture device such as a microscope 20. The image acquisition unit 1 also outputs the acquired cell image 10 to the image evaluation unit 2b.

In addition, in this embodiment, as shown in FIG. 3, the step of classifying the cell image 10 is performed by the image judgment unit 2b. The image judgment unit 2b inputs the cell image 10 to the first trained model 6, and outputs a cell image classification label 13 for the input cell image 10. In other words, the image judgment unit 2b classifies the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12.

(First trained model generation process)
Next, with reference to Figure 4, the process by which the trained model generation unit 2d generates the first trained model 6 will be described.

In step 200, the image acquisition unit 1 acquires a cell image 10.

In step 201, the inappropriate cell image generating unit 2e generates an inappropriate cell image 12 based on the cell image 10. Specifically, the inappropriate cell image generating unit 2e generates the inappropriate cell image 12 by performing image processing that deteriorates the image quality of the appropriate cell image 11. The appropriate cell image 11 used in step 201 is a cell image 10 that has been previously classified as an appropriate cell image 11 by the operator.

The inappropriate cell image generating unit 2e changes the image processing parameters that deteriorate the image quality of the appropriate cell image 11, and creates images that fall within the acceptable range for the appropriate cell image 11 and images that do not fall within the acceptable range. In this embodiment, in the processing of step 201, the inappropriate cell image generating unit 2e performs a filter process (for example, processing using a Gaussian filter, etc.) on the appropriate cell image 11 to reduce the contrast of the appropriate cell image 11, and generates an inappropriate cell image 12a (see FIG. 2(B)) in which the cell 90 is blurred, or an inappropriate cell image 12f (see FIG. 2(G)) including the cell 90a in which the boundary 84 between the cell 90 area and the area 91 other than the cell 90 is unclear. The inappropriate cell image generating unit 2e also generates an inappropriate cell image 12b (see FIG. 2C) in which a foreign object 80 is reflected, or an inappropriate cell image 12d (see FIG. 2E) in which image quality has been deteriorated due to the culture medium, by setting the pixel values of some pixels of the appropriate cell image 11 below a threshold value. The inappropriate cell image generating unit 2e also generates an inappropriate cell image 12c (see FIG. 2D) in which stray light is reflected, or an inappropriate cell image 12 in which image quality has been deteriorated due to the culture medium, by setting the pixel values of some pixels (several pixels to several tens of pixels) of the appropriate cell image 11 above a threshold value. The inappropriate cell image generating unit 2e also generates an inappropriate cell image 12e (see FIG. 2F) including a region 83 in which pixel values are saturated, by saturating the pixel values of some pixels (several pixels to several tens of pixels) of the appropriate cell image 11. The acceptable range is the range of image quality set by the operator.

In step 202, the trained model generating unit 2d creates a first trained model 6 that has learned to judge whether or not a cell image 10 satisfies a judgment criterion by training the appropriate cell image 11 and the inappropriate cell image 12. Specifically, in the step of creating the first trained model 6, the trained model generating unit 2d creates the first trained model 6 based on the appropriate cell image 11 and the inappropriate cell image 12 generated from the appropriate cell image 11.

(Image analysis processing)
Next, a process in which the processor 2 analyzes the cell image 10 will be described with reference to Fig. 5. The cell image analysis method according to this embodiment includes step 300 of acquiring the cell image 10, step 301 of determining whether the image quality of the cell image 10 satisfies a judgment criterion, step 302 of classifying the cell image 10 as an appropriate cell image 11 if the image quality of the cell image 10 satisfies the judgment criterion, and step 303 of classifying the cell image 10 as an inappropriate cell image 12 if the image quality does not satisfy the judgment criterion.

In step 300, the image acquisition unit 1 acquires a cell image 10.

In step 301, the image determination unit 2b determines whether the cell image 10 includes at least one of the following: a foreign object 80 reflected, stray light reflected, a deterioration in the image quality of the cell image 10 due to the culture medium, a region 83 where the pixel value is saturated, and a cell 90a with an unclear boundary 84. In other words, in step 301, the image determination unit 2b determines whether the cell image 10 satisfies at least one of the following criteria: whether the image quality of the cell image 10 is suitable for analyzing the cell image 10 when classifying the cell image 10 for analyzing the cell image 10, and whether the image quality of the cell image 10 is suitable for learning the analysis of the second learning model 7a when classifying the cell image 10 used for learning the second learning model 7a for analyzing the cell image 10. If the cell image 10 does not include reflections of foreign matter 80, reflections of stray light, deterioration in the image quality of the cell image 10 due to the culture medium, a region 83 where pixel values are saturated, or a cell 90a with an unclear boundary 84, the process proceeds to step 302. If the cell image 10 includes at least one of reflections of foreign matter 80, reflections of stray light, deterioration in the image quality of the cell image 10 due to the culture medium, a region 83 where pixel values are saturated, or a cell 90a with an unclear boundary 84, the process proceeds to step 303.

In step 302, the image assessment unit 2b classifies the cell image 10 as an appropriate cell image 11. Specifically, the image assessment unit 2b classifies the cell image 10 as an appropriate cell image 11 if the cell image 10 does not include any foreign matter 80, any stray light, any degradation in image quality of the cell image 10 due to the culture medium, any region 83 with saturated pixel values, or any cell 90a with an unclear boundary 84. The image assessment unit 2b also stores the appropriate cell image 11 in the memory unit 3. The process then ends.

Furthermore, if the process proceeds from step 301 to step 303, in step 303, the image assessment unit 2b classifies the cell image 10 as an inappropriate cell image 12. Specifically, the image assessment unit 2b classifies the cell image 10 as an inappropriate cell image 12 if the cell image 10 contains at least one of the following: a foreign object 80, stray light, deterioration in image quality of the cell image 10 caused by the culture medium, an area 83 where pixel values are saturated, and a cell 90a whose boundary 84 is unclear. The process then ends.

In this embodiment, the image determination unit 2b determines whether the cell image 10 includes at least one of the following: blurring of the cell 90 in the cell image 10, reflection of a foreign object 80 in the cell image 10, reflection of stray light in the cell image 10, deterioration of the image quality of the cell image 10 caused by the culture medium when culturing the cell 90, a region 83 in which the pixel values in the cell image 10 are saturated, and a cell 90a in which the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is unclear, as the process of step 301. Specifically, the image determination unit 2b uses the first trained model 6 (see FIG. 1) to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 in the process of step 301. In addition, in the processing of step 301, the image determination unit 2b performs image processing on the cell image 10 to determine whether the cell image 10 includes at least one of the following: a foreign object 80 reflected, stray light reflected, deterioration in image quality of the cell image 10 due to the culture medium, a region 83 with saturated pixel values, and a cell 90a with an unclear boundary 84. That is, the image determination unit 2b uses both the judgment based on the trained model 6 and the judgment based on image processing. Note that, in the processing of step 301, when the image determination unit 2b judges the cell image 10 by image processing, the image determination unit 2b judges whether the cell 90 is blurred and whether the cell 90a with an unclear boundary 84 is present based on the contrast of the cell image 10, and judges whether the foreign object 80 is reflected, whether stray light is reflected, whether the image quality of the cell image 10 is deteriorated due to the culture medium, and whether the region 83 with saturated pixel values is present based on the distribution of the luminance values of the cell image 10.

When making a judgment based on the contrast of the cell image 10, the image judgment unit 2b calculates the difference between the pixel value of a specified pixel in the cell image 10 and the pixel value of the specified pixel and the pixel adjacent to the specified pixel. The image judgment unit 2b also calculates the difference between the pixel value of each pixel and the pixel value of the adjacent pixel in the horizontal direction of the cell image 10, and accumulates the calculated difference in the horizontal direction of the cell image 10. The image judgment unit 2b also digitizes the contrast of the entire cell image 10 by accumulating the accumulated pixel value differences in the vertical direction. Then, the image judgment unit 2b judges whether or not the cell 90 in the cell image 10 is blurred depending on whether or not the digitized contrast is equal to or greater than a threshold value. In other words, when the digitized contrast is equal to or greater than a threshold value, the image judgment unit 2b determines that the cell 90 depicted in the cell image 10 is not blurred, and judges that the cell image 10 satisfies the judgment criterion. Furthermore, if the quantified contrast is smaller than the threshold, the image evaluation unit 2b determines that the cell 90 depicted in the cell image 10 is blurred or contains a cell 90a whose boundary 84 is unclear, and determines that the cell image 10 does not satisfy the evaluation criteria.

When judging based on the distribution of the brightness values of the cell image 10, the image judgment unit 2b quantifies the proportion of pixels in the cell image 10 that have a brightness value below a certain level, and judges whether or not the quantified proportion is equal to or greater than a threshold value to determine whether or not the foreign object 80 is reflected or the image quality of the cell image 10 is degraded due to the culture medium. That is, when the proportion of pixels with a minimum brightness value is equal to or greater than a threshold value, the image judgment unit 2b judges that the foreign object 80 is reflected and that the cell image 10 does not satisfy the judgment criterion. When the proportion of pixels with a brightness value below a certain level and not the minimum brightness value is equal to or greater than a threshold value, the image judgment unit 2b judges that the image quality of the cell image 10 is degraded due to the culture medium and that the cell image 10 does not satisfy the judgment criterion. When the proportion of pixels with a brightness value below a certain level is smaller than the threshold value, the image judgment unit 2b judges that the foreign object 80 is not reflected or that the image quality of the cell image 10 is degraded due to the culture medium and that the cell image 10 satisfies the judgment criterion.

The image determination unit 2b also quantifies the proportion of pixels in the cell image 10 that have a certain brightness value or more, and determines whether or not there is reflection of stray light, whether or not there is deterioration in the image quality of the cell image 10 due to the culture medium, or whether or not there is an area 83 where the pixel value is saturated, depending on whether or not the quantified proportion is equal to or greater than a threshold value. That is, if the proportion of pixels whose brightness value is the maximum value (brightness value is saturated) is equal to or greater than a threshold value, the image determination unit 2b determines that there is reflection of stray light or that there is an area 83 where the pixel value is saturated, and that the cell image 10 does not meet the determination criterion. If the proportion of pixels whose brightness value is equal to or greater than a certain value and whose brightness value is not saturated is equal to or greater than a threshold value, the image determination unit 2b determines that there is deterioration in the image quality of the cell image 10 due to the culture medium, and that the cell image 10 does not meet the determination criterion. Furthermore, if the proportion of pixels having a certain or higher brightness value is smaller than the threshold value, the image evaluation unit 2b determines that there is no stray light, no degradation in image quality of the cell image 10 due to the culture medium, or no area 83 with saturated pixel values, and determines that the cell image 10 meets the evaluation criteria.

In this embodiment, if the analysis result 50 based on the first trained model 6 differs from the analysis result 50 based on the image processing, the image assessment unit 2b displays a confirmation dialog 44 (see FIG. 7) described below on the display unit 4, or classifies the image as an inappropriate cell image 12.

The analysis of the cell image 10 using the first trained model 6 is mainly performed when the cell image 10 is captured, when the second trained model 7 is generated, and when the cell image 10 is analyzed using the second trained model 7.

(Analysis process during cell image capture)
Next, a configuration in which the cell-image analyzing device 100 analyzes the cell image 10 when capturing the cell image 10 will be described with reference to FIGS. 6 and 7. FIG.

Figure 6 shows a screen displayed on the display unit 4 when the cell image 10 is captured. As shown in Figure 6, the screen displayed on the display unit 4 when the cell image 10 is captured displays an image of the cell 90 being observed with the microscope 20 (see Figure 1) in the first display area 40. In this state, when the operator presses the capture button 41, the capture of the cell image 10 is started. That is, when the capture button 41 is pressed, multiple cell images 10 are captured at a preset capture position. Note that, as shown in Figure 6, when the automatic check box 42 for image quality is checked, the image evaluation unit 2b analyzes the cell image 10. When the automatic check box 42 for image quality is not checked, the image evaluation unit 2b does not analyze the cell image 10. Note that, as shown in Figure 6, the display unit 4 displays an image of the captured cell image 10 in the second display area 43.

When the captured cell images 10 include images that do not meet the judgment criteria, as shown in FIG. 7, a confirmation dialog 44 for asking whether or not to register the cell images 10 is displayed on the display unit 4 together with the images that do not meet the judgment criteria. A discard button 45 and a register button 46 are displayed on the confirmation dialog 44. When the discard button 45 is pressed, the control unit 2a discards the displayed cell image 10. When the register button 46 is pressed, the control unit 2a registers the cell image 10. In other words, even if the cell image 10 is judged to be an inappropriate cell image 12, if the operator presses the register button 46, the cell image 10 judged to be an inappropriate cell image 12 is stored in the memory unit 3 as an appropriate cell image 11.

Next, the image analysis process performed when capturing the cell image 10 will be described with reference to FIG.

In step 400, the image acquisition unit 1 acquires a cell image 10.

In step 401, the image assessment unit 2b assesses whether the image quality of the cell image 10 meets the assessment criteria. If the image quality of the cell image 10 meets the assessment criteria, the process proceeds to step 402. If the image quality of the cell image 10 does not meet the assessment criteria, the process proceeds to step 404. Note that the process of step 401 is similar to the process of step 301, and therefore a detailed description thereof will be omitted.

In step 402, the image assessment unit 2b classifies the cell image 10 into an appropriate cell image 11.

In step 403, the image assessment unit 2b stores the appropriate cell image 11 in the storage unit 3. The process then ends.

When the process proceeds from step 401 to step 404, in step 404, the control unit 2a notifies the user that the cell image 10 does not satisfy the judgment criteria. Specifically, when the cell image 10 is classified as an inappropriate cell image 12, the control unit 2a notifies the user that the cell image 10 is not suitable for analysis when analyzing the cell image 10 by displaying a confirmation dialog 44 on the display unit 4 as shown in FIG. 7, or that the cell image 10 is not suitable for learning the analysis of the second learning model 7a when training the second learning model 7a for analyzing the cell image 10.

In step 405, the image determination unit 2b determines whether or not an operation input to store a cell image 10 classified as an inappropriate cell image 12 has been received. If an operation input to store a cell image 10 classified as an inappropriate cell image 12 has been received, the process proceeds to step 402. If an operation input to store a cell image 10 classified as an inappropriate cell image 12 has not been received, the process proceeds to step 406.

In step 406, the image determination unit 2b determines whether or not there has been an operation input to discard the cell image 10. If there has been an operation input to discard the cell image 10, the process proceeds to step 407. If there has not been an operation input to discard the cell image 10, the process returns to step 405.

In step 407, the image determination unit 2b classifies the cell image 10 as an inappropriate cell image 12. Then, the process ends.

(Analysis process when generating second trained model)
9 and 10, a configuration for analyzing the cell image 10 when the cell-image analyzing device 100 generates the second trained model 7 by training the second learning model 7a will be described. Note that the second learning model 7a is, for example, a convolutional neural network or includes a convolutional neural network as a part thereof.

As shown in FIG. 9, the trained model generating unit 2d according to this embodiment is configured to generate the second trained model 7 by having the second trained model 7a analyze the cell image 10 and learn to output the analysis result 50. Specifically, the image acquiring unit 1 acquires the cell image 10 and outputs the acquired cell image 10 to the image determining unit 2b. The analysis result 50 includes a label indicating what type of cell the cell 90 depicted in the cell image 10 is. That is, the second trained model 7 is trained using the cell image 10 as input data and a label indicating whether the cell 90 depicted in the input cell image 10 is an undifferentiated cell or an undifferentiated deviant cell as output data.

The image determination unit 2b, to which the cell image 10 is input, determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. In this embodiment, the image determination unit 2b classifies the cell image 10 using the first trained model 6 or image processing.

The trained model generating unit 2d trains the second trained model 7a to output an analysis result 50 using the appropriate cell image 11 classified by the image determining unit 2b. In this embodiment, the trained model generating unit 2d trains the second trained model 7a to output, as the analysis result 50, whether the cell 90 depicted in the appropriate cell image 11 is an undifferentiated cell or an undifferentiated deviant cell. In addition, the generated second trained model 7 is stored in the storage unit 3.

Figure 10 is a schematic diagram of the screen displayed on the display unit 4 when learning of the second learning model 7a is completed and generation of the second trained model 7 is completed.

The control unit 2a causes the display unit 4 to display a message 51 indicating that the generation of the second trained model 7 has been completed. The control unit 2a also causes the display unit 4 to display a list of the learning results 52.

In this embodiment, the control unit 2a causes the display unit 4 to display, as the learning result list 52, the performance of the learning result 52a, a list of learning target images 52b, and a list of images excluded from learning 52c. In the example shown in FIG. 10, the control unit 2a causes the accuracy, precision, recall, and IoU (Intersection over Union) to be displayed as the performance of the learning result 52a.

Next, the generation process of the second trained model 7 will be described with reference to FIG. 11.

In step 500, the image assessment unit 2b acquires a cell image 10 that is a candidate for learning.

In step 501, the image assessment unit 2b judges whether the image quality of the cell image 10 meets the judgment criteria. If the image quality of the cell image 10 meets the judgment criteria, the process proceeds to step 502. If the image quality of the cell image 10 does not meet the judgment criteria, the process proceeds to step 503. Note that when performing the process of step 501, if a cell image classification label 13 is assigned to the cell image 10, the cell image 10 is judged by giving priority to the cell image classification label 13. In other words, if a cell image classification label 13 of 0 has been assigned in advance, the cell image 10 is classified as an appropriate cell image 11 even if it does not meet the judgment criteria.

In step 502, the image assessment unit 2b classifies the cell image 10 into an appropriate cell image 11.

If the process proceeds from step 501 to step 503, in step 503, the image assessment unit 2b classifies the cell image 10 as an inappropriate cell image 12.

In step 504, the image assessment unit 2b excludes the inappropriate cell image 12 from learning. That is, when the cell image 10 is classified as an inappropriate cell image 12, the image assessment unit 2b excludes the cell image 10 from learning of the analysis of the cell image 10 for the second learning model 7a.

In step 505, the image judgment unit 2b judges whether it has been confirmed that all images that are candidates for learning objects satisfy the judgment criteria. If the image judgment unit 2b has confirmed that all images that are candidates for learning objects satisfy the judgment criteria, the process proceeds to step 506. If the image judgment unit 2b has not confirmed that all images that are candidates for learning objects satisfy the judgment criteria, the process proceeds to step 500.

In step 506, the trained model generation unit 2d trains the second trained model 7a. In this embodiment, the trained model generation unit 2d generates the second trained model 7 by training the second trained model 7a to perform analysis using the cell images 10 classified as appropriate cell images 11.

In step 507, the control unit 2a causes the display unit 4 to display a list 52b of images to be learned and a list 52c of images excluded from learning. In this embodiment, the control unit 2a also causes the display unit 4 to display a message 51 indicating that the generation of the second trained model 7 has been completed, and the performance of the learning result 52a. Then, the process ends.

(Analysis and processing of cell images)
Next, a configuration for the image analysis unit 2c to analyze the cell image 10 will be described with reference to FIGS.

As shown in FIG. 12, the image analysis unit 2c according to this embodiment is configured to output an analysis result 50 by analyzing a cell image 10. Specifically, the image acquisition unit 1 acquires a cell image 10 and outputs the acquired cell image 10 to the image evaluation unit 2b.

The image determination unit 2b determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. In this embodiment, the image determination unit 2b classifies the cell image 10 using the first trained model 6 and image processing.

The image analysis unit 2c performs image analysis using the appropriate cell image 11. The image analysis unit 2c outputs an analysis result 50 obtained by analyzing the appropriate cell image 11. In this embodiment, the control unit 2a causes the display unit 4 to display the analysis result 50.

Figure 13 is a schematic diagram showing the analysis result 50 displayed on the display unit 4 after the analysis of the cell image 10 is completed.

The control unit 2a causes the display unit 4 to display a message 60 indicating that the analysis of the cell image 10 has been completed. The control unit 2a also causes the display unit 4 to display a list of the analysis results 50.

In this embodiment, the control unit 2a causes the display unit 4 to display, as a list of the analysis results 50, analysis items 50a, a list of images to be analyzed 50b, and a list of images excluded from the analysis 50c. In the example shown in FIG. 13, the control unit 2a causes the display unit 4 to display, as the analysis items 50a, labels indicating the type of cell depicted in the cell image 10. That is, the control unit 2a causes the analysis items 50a to display the probability that the cell 90 depicted in the cell image 10 is A, the probability that it is B, and the probability that it is C.

Next, the analysis process of the cell image 10 will be described with reference to FIG. 14.

In step 600, the image evaluation unit 2b acquires a cell image 10 that is a candidate for analysis.

In step 601, the image assessment unit 2b judges whether the image quality of the cell image 10 meets the assessment criteria. If the image quality of the cell image 10 meets the assessment criteria, the process proceeds to step 602. If the image quality of the cell image 10 does not meet the assessment criteria, the process proceeds to step 603. Note that when performing the process of step 601, if a cell image classification label 13 has been assigned to the cell image 10, the cell image 10 is assessed with priority given to the cell image classification label 13. In other words, if a cell image classification label 13 of 0 has been assigned in advance, the cell image 10 is classified as an appropriate cell image 11 even if it does not meet the assessment criteria.

In step 602, the image assessment unit 2b classifies the cell image 10 into an appropriate cell image 11.

If the process proceeds from step 601 to step 603, in step 603, the image assessment unit 2b classifies the cell image 10 as an inappropriate cell image 12.

In step 604, the image determination unit 2b excludes the inappropriate cell image 12 from the analysis. That is, if the cell image 10 is classified as an inappropriate cell image 12, the image determination unit 2b excludes the cell image 10 from the analysis.

In step 605, the image assessment unit 2b determines whether or not all images that are candidates for analysis satisfy the assessment criteria. If the image assessment unit 2b has confirmed whether or not all cell images 10 that are candidates for analysis satisfy the assessment criteria, the process proceeds to step 606. If the image assessment unit 2b has not confirmed whether or not all cell images 10 that are candidates for analysis satisfy the assessment criteria, the process proceeds to step 600.

In step 606, the image analysis unit 2c analyzes the cell image 10. In this embodiment, the image analysis unit 2c performs the analysis using the cell image 10 classified as an appropriate cell image 11.

In step 607, the control unit 2a causes the display unit 4 to display a list 61b of images to be analyzed and a list 50c of images excluded from the analysis. In this embodiment, the control unit 2a also causes the display unit 4 to display a message 60 indicating that the analysis of the cell image 10 has been completed, and the analysis items 50a. The process then ends.

(Effects of this embodiment)
In this embodiment, the following effects can be obtained.

In this embodiment, as described above, the cell image analysis method includes the steps of acquiring a cell image 10, determining whether the cell image 10 includes at least one of the following: blurring of the cell 90 in the cell image 10, reflection of a foreign body 80 in the cell image 10, reflection of stray light in the cell image 10, deterioration of the image quality of the cell image 10 due to the medium used in culturing the cell 90, a region 83 in the cell image 10 where pixel values are saturated, and a cell 90a where the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is unclear; and and classifying the cell image 10 into an appropriate cell image 11 if the cell image 10 does not include blurred cells 90, reflection of foreign matter 80, reflection of stray light, deterioration of image quality of the cell image 10 due to the culture medium, region 83 where pixel values are saturated, and cells 90a with unclear boundaries 84, and classifying the cell image 10 into an inappropriate cell image 12 if the cell image 10 includes at least one of blurred cells 90, reflection of foreign matter 80, reflection of stray light, deterioration of image quality of the cell image 10 due to the culture medium, region 83 where pixel values are saturated, and cells 90a with unclear boundaries 84. As a result, only the appropriate cell image 11 that does not include blurred cells 90, reflection of foreign matter 80, reflection of stray light, deterioration of image quality of the cell image 10 due to the culture medium, region 83 where pixel values are saturated, and cells 90a with unclear boundaries 84 can be used for analysis. As a result, it is possible to prevent analysis from being performed using cell images 10 that contain inappropriate cell images 12, thereby preventing a decrease in the accuracy of the analysis of the cell images 10.

In addition, the above embodiment has the following additional advantages:

That is, in this embodiment, as described above, the cell image 10 is an image of the cultured cell 90 cultured in a container, and the determining step determines whether or not the cell image 10 satisfies at least one of the following criteria: determining whether the image quality of the cell image 10 is suitable for analyzing the cell image 10 when classifying the cell image 10 for analyzing the cell image 10; and determining whether the image quality of the cell image 10 is suitable for learning the analysis of the second learning model 7a when classifying the cell image 10 used for learning the second learning model 7a for analyzing the cell image 10. As a result, by determining the cell image 10 according to the above criteria, it is possible to easily determine whether the cell image 10 is suitable for analysis of the cell image 10 and whether it is suitable for learning the analysis of the second learning model 7a.

In addition, in this embodiment, as described above, the judgment criteria include the presence or absence of blurring of the cell 90, the presence or absence of reflection of the foreign matter 80, the presence or absence of reflection of stray light, the presence or absence of degradation in image quality of the cell image 10 due to the culture medium, the presence or absence of an area 83 where the pixel value is saturated, and the presence or absence of a cell 90a where the boundary 84 is unclear. As a result, the cell image 10 in which the cell 90 is blurred, the cell image 10 in which the foreign matter 80 is reflected, the cell image 10 in which stray light is reflected, the cell image 10 in which the image quality of the cell 90 is degraded due to the culture medium, the cell image 10 including the area 83 where the pixel value is saturated, and the cell image 10 including the cell 90a where the boundary 84 is unclear can be classified as an inappropriate cell image 12. As a result, it is possible to prevent a decrease in the accuracy of the analysis of the cell image 10 or the learning of the second learning model 7a due to a cell image 10 in which the cell 90 is blurred, a cell image 10 in which a foreign body 80 is reflected, a cell image 10 in which stray light is reflected, a cell image 10 whose image quality has been reduced due to the culture medium, a cell image 10 that includes an area 83 in which pixel values are saturated, and a cell image 10 that includes a cell 90a whose boundary 84 is unclear.

In addition, this embodiment further includes a step of creating a first trained model 6 that has been trained to judge whether or not a cell image 10 satisfies a judgment criterion by having the model learn appropriate cell images 11 and inappropriate cell images 12 as described above, and in the judgment step, the first trained model 6 is used to judge whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. This makes it possible to judge whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 even when it is difficult to make a judgment by image processing.

In addition, in this embodiment, as described above, the method further includes a step of generating inappropriate cell images 12 by performing image processing that deteriorates the image quality of the appropriate cell image 11, and in the step of creating the first trained model 6, the first trained model 6 is created based on the appropriate cell image 11 and the inappropriate cell image 12 generated from the appropriate cell image 11. As a result, even if it is difficult to obtain a sufficient number of inappropriate cell images 12 by photographing or the like when creating the first trained model 6, a sufficient number of inappropriate cell images 12 can be obtained by generating the inappropriate cell images 12 from the appropriate cell images 11. As a result, it is possible to use a sufficient number of appropriate cell images 11 and inappropriate cell images 12 when generating the first trained model 6, and therefore the learning accuracy of the first trained model 6 can be improved.

In addition, in this embodiment, as described above, in the step of generating the inappropriate cell image 12, the image processing parameters that degrade the image quality of the appropriate cell image 11 are changed, and images that fall within the acceptable range for the appropriate cell image 11 and images that do not fall within the acceptable range are created. In this way, when generating the inappropriate cell image 12 from the appropriate cell image 11, an image that falls within the acceptable range for the appropriate cell image 11 is generated, thereby increasing not only the number of inappropriate cell images 12 but also the number of appropriate cell images 11. As a result, it is possible to further increase the number of images used to generate the first trained model 6, thereby further improving the learning accuracy of the first trained model 6.

In addition, in this embodiment, as described above, in the determination step, image processing is performed on the cell image 10 to determine whether the cell image 10 includes at least one of the following: blurred cells 90, reflections of foreign matter 80, reflections of stray light, deterioration of the image quality of the cell image 10 due to the culture medium, an area 83 with saturated pixel values, and a cell 90a with an unclear boundary 84. As a result, in cases where it is possible to easily determine by image processing, such as the reflection of foreign matter 80, it is possible to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 without using a trained model.

In addition, in this embodiment, as described above, in the determination step, the presence or absence of blurring of the cell 90 is determined based on the contrast of the cell image 10, and the presence or absence of reflection of foreign matter 80, the presence or absence of reflection of stray light, the presence or absence of degradation in image quality of the cell image 10 due to the culture medium, the presence or absence of an area 83 where pixel values are saturated, and the presence or absence of a cell 90a with an unclear boundary 84 are determined based on the distribution of brightness values of the cell image 10. In this way, by acquiring the contrast and the distribution of brightness values of the cell image 10, it is possible to easily determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12.

In addition, as described above, this embodiment further includes a step of excluding the cell image 10 from the analysis when the cell image 10 is classified as an inappropriate cell image 12, and a step of performing analysis using the cell image 10 classified as an appropriate cell image 11. This makes it possible to prevent the inclusion of inappropriate cell images 12 in the analysis of the cell image 10. As a result, the accuracy of the analysis of the cell image 10 can be improved.

In addition, in this embodiment, as described above, when the cell image 10 is classified as an inappropriate cell image 12, a step is further provided in which, when analyzing the cell image 10, the cell image 10 is not suitable for analysis, or, when training the second learning model 7a for analyzing the cell image 10, the cell image 10 is not suitable for learning analysis for the second learning model 7a. This allows, for example, when photographing the cell image 10, the user to know whether the photographed image is an appropriate cell image 11 or an inappropriate cell image 12. As a result, the user's convenience can be improved. In addition, since it is possible to allow the user to know that the cell image 10 contains an inappropriate cell image 12 when photographing the cell image 10, re-photographing the cell image 10 can be suppressed. As a result, the burden on the operator caused by re-photographing can be reduced.

In addition, as described above, this embodiment further includes a step of excluding the cell image 10 from the learning of the analysis of the cell image 10 for the second learning model 7a when the cell image 10 is classified as an inappropriate cell image 12, and a step of training the second learning model 7a to analyze using the cell image 10 classified as an appropriate cell image 11. This makes it possible to suppress the second learning model 7a from learning using the inappropriate cell image 12 when training the second learning model 7a to analyze the cell image 10. As a result, it is possible to improve the learning accuracy of the second learning model 7a, and therefore the analysis accuracy of the second trained model 7 can be improved.

[Modification]
The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims rather than the description of the above embodiments, and further includes all modifications (variations) within the meaning and scope of the claims.

For example, in the above embodiment, an example of a configuration in which the image determination unit 2b determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 by using the first trained model 6 in combination with image processing has been shown, but the present invention is not limited to this. For example, the image determination unit 2b may be configured to classify the cell image 10 by either the first trained model 6 or image processing.

In addition, in the above embodiment, an example of a configuration in which the image judgment unit 2b classifies an inappropriate cell image 12 into an appropriate cell image 11 based on the operation input of the operator has been shown, but the present invention is not limited to this. For example, when the image judgment unit 2b classifies an image into an inappropriate cell image 12, it may be configured not to classify the image into an appropriate cell image 11 regardless of the operation input of the operator. In other words, once the image judgment unit 2b classifies an image into an inappropriate cell image 12, it may be configured not to classify the image into an appropriate cell image 11 thereafter.

In addition, in the above embodiment, an example of a configuration in which the trained model generating unit 2d generates the first trained model 6 and the second trained model 7 is shown, but the present invention is not limited to this. For example, the first trained model 6 and the second trained model 7a may be generated by a device other than the cell image analyzing device 100.

In addition, in the above embodiment, an example of a configuration in which the inappropriate cell image generating unit 2e generates the inappropriate cell image 12 by performing image processing on the appropriate cell image 11 has been shown, but the present invention is not limited to this. For example, the cell image analyzing device 100 may be configured to acquire the inappropriate cell image 12 generated by a device other than the cell image analyzing device 100.

In addition, in the above embodiment, an example of a configuration in which the inappropriate cell image generating unit 2e generates an image that falls within the acceptable range of the appropriate cell image 11 is shown, but the present invention is not limited to this. For example, the inappropriate cell image generating unit 2e may be configured to generate only the inappropriate cell image 12.

In addition, in the above embodiment, an example of a configuration in which the cell image analysis device 100 is used when photographing the cell image 10, when generating the first trained model 6, and when analyzing the cell image 10 is shown, but the present invention is not limited to this. For example, the cell image analysis device 100 may be used in any one of photographing the cell image 10, when generating the first trained model 6, and when analyzing the cell image 10. Furthermore, the cell image analysis device 100 may be used in any two of photographing the cell image 10, when generating the first trained model 6, and when analyzing the cell image 10.

[Aspects]
It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

(Item 1)
acquiring a cell image;
determining whether the cell image includes at least one of the following: blurred cells in the cell image; foreign matter in the cell image; stray light in the cell image; degradation of image quality of the cell image caused by a culture medium when culturing the cells; an area in the cell image where pixel values are saturated; and a cell where the boundary between the cell and an area other than the cell is unclear;
a step of classifying the cell image into an appropriate cell image if the cell image does not include blurring of the cell, reflection of the foreign matter, reflection of the stray light, deterioration of image quality of the cell image due to the culture medium, an area where the pixel values are saturated, and the cell whose boundary is unclear, and a step of classifying the cell image into an inappropriate cell image if the cell image includes at least one of blurring of the cell, reflection of the foreign matter, reflection of the stray light, deterioration of image quality of the cell image due to the culture medium, an area where the pixel values are saturated, and the cell whose boundary is unclear.

(Item 2)
The cell image is an image of a cultured cell cultured in a container,
2. The cell image analysis method according to item 1, wherein in the determining step, when classifying the cell image for analysis of the cell image, a determination is made as to whether or not the image quality of the cell image is suitable for the analysis of the cell image, and when classifying the cell image for use in training a learning model for analyzing the cell image, a determination is made as to whether or not the image quality of the cell image is suitable for training the analysis of the learning model.

(Item 3)
3. The cell image analyzing method according to item 2, wherein the determination criteria include whether or not the cell is blurred, whether or not the foreign matter is reflected, whether or not the stray light is reflected, whether or not the image quality of the cell image is reduced due to the culture medium, whether or not the pixel value is saturated, and whether or not the cell has an unclear boundary.

(Item 4)
The method further includes a step of creating a first trained model that is trained to determine whether or not the cell image satisfies the judgment criterion by training the appropriate cell image and the inappropriate cell image,
4. The cell image analysis method according to item 2 or 3, wherein in the determining step, the first trained model is used to determine whether the cell image is the appropriate cell image or the inappropriate cell image.

(Item 5)
The method further comprises the step of generating the inappropriate cell image by performing image processing that deteriorates the image quality of the appropriate cell image,
5. The cell image analysis method according to item 4, wherein in the step of creating the first trained model, the first trained model is created based on the appropriate cell image and the inappropriate cell image generated from the appropriate cell image.

(Item 6)
A cell image analysis method as described in item 5, in which, in the step of generating the inappropriate cell image, image processing parameters that degrade the image quality of the appropriate cell image are changed, and an image that falls within an acceptable range for the appropriate cell image and an image that does not fall within the acceptable range are created.

(Item 7)
7. The cell image analysis method according to any one of items 1 to 6, wherein in the determining step, image processing is performed on the cell image to determine whether or not the cell image includes at least one of blurring of the cell, reflection of the foreign matter, reflection of the stray light, deterioration of image quality of the cell image caused by the culture medium, a region where the pixel values are saturated, and the cell whose boundary is unclear.

(Item 8)
8. The cell image analyzing method according to item 7, wherein in the determining step, the presence or absence of blurring of the cell is determined based on the contrast of the cell image, and the presence or absence of the reflection of the foreign matter, the presence or absence of the reflection of the stray light, the presence or absence of degradation in image quality of the cell image caused by the culture medium, the presence or absence of the region where pixel values are saturated, and the presence or absence of the cell whose boundary is unclear are determined based on a distribution of luminance values of the cell image.

(Item 9)
excluding the cell image from analysis if the cell image is classified as the inappropriate cell image;
9. The cell image analysis method according to any one of items 1 to 8, further comprising a step of performing the analysis using the cell image classified into the appropriate cell image.

(Item 10)
The cell image analysis method according to any one of items 1 to 8, further comprising a step of notifying, when the cell image is classified as the inappropriate cell image, that the cell image is not suitable for analysis when performing the analysis of the cell image, or that the cell image is not suitable for training the learning model for the analysis when training the learning model for analyzing the cell image.

(Item 11)
If the cell image is classified as the inappropriate cell image, excluding the cell image from learning of an analysis of the cell image for a learning model;
9. The cell image analysis method according to any one of items 1 to 8, further comprising a step of training the learning model to perform the analysis using the cell image classified into the appropriate cell image.

6 First trained model 7 Second trained model 7a Second trained model (training model)
10 Cell image 11 Appropriate cell image 12, 12a, 12b, 12c, 12d Inappropriate cell image 80 Foreign object 83 Region where pixel value is saturated 84 Boundary (Boundary between cell region and non-cell region)
90 cells (cultured cells)
90a cells (cells with unclear boundaries)

Claims (10)

acquiring a cell image;
determining whether the cell image includes at least one of the following: blurred cells in the cell image; foreign matter in the cell image; stray light in the cell image; degradation of image quality of the cell image caused by a culture medium when culturing the cells; an area in the cell image where pixel values are saturated; and a cell where the boundary between the cell and an area other than the cell is unclear;
classifying the cell image as an appropriate cell image when the cell image does not include blurring of the cell, reflection of the foreign matter, reflection of the stray light, deterioration of image quality of the cell image due to the culture medium, an area where the pixel value is saturated, and the cell whose boundary is unclear, and classifying the cell image as an inappropriate cell image when the cell image includes at least one of blurring of the cell, reflection of the foreign matter, reflection of the stray light, deterioration of image quality of the cell image due to the culture medium, an area where the pixel value is saturated, and the cell whose boundary is unclear;
generating the inappropriate cell image by performing image processing that deteriorates the image quality of the appropriate cell image;
and creating a first trained model that is trained to determine whether the cell image is the appropriate cell image or the inappropriate cell image by training the appropriate cell image and the inappropriate cell image generated from the appropriate cell image ;
A cell image analysis method , in which in the determining step, a determination is made as to whether the cell image is an appropriate cell image or an inappropriate cell image using the first trained model . The cell image is an image of a cultured cell cultured in a container,
The cell image analysis method according to claim 1, wherein in the determining step, when classifying the cell image for analysis of the cell image, it is determined whether the image quality of the cell image is suitable for the analysis of the cell image, and when classifying the cell image to be used for training a learning model for analyzing the cell image, it is determined whether the image quality of the cell image satisfies a determination criterion for at least one of the following: determining whether the image quality of the cell image is suitable for training the analysis of the learning model. The cell image analysis method according to claim 2, wherein the criteria include whether the cell is blurred, whether the foreign matter is reflected, whether the stray light is reflected, whether the image quality of the cell image is reduced due to the culture medium, whether there is an area where the pixel values are saturated, and whether there is a cell whose boundary is unclear. The cell image analysis method of claim 2 or 3, wherein in the step of creating the first trained model , the first trained model is created by learning the appropriate cell image and the inappropriate cell image, to determine whether or not the cell image satisfies the judgment criterion. The cell image analysis method of claim 1, wherein in the step of generating the inappropriate cell image, image processing parameters that degrade the image quality of the appropriate cell image are changed to create an image that falls within an acceptable range for the appropriate cell image and an image that does not fall within the acceptable range . The cell image analysis method according to any one of claims 1 to 5, wherein in the determining step, image processing is performed on the cell image to determine whether or not the cell image includes at least one of blurring of the cell, reflection of the foreign matter, reflection of the stray light, degradation of image quality of the cell image caused by the culture medium, an area where the pixel values are saturated, and the cell whose boundary is unclear. 7. The cell image analysis method according to claim 6, wherein in the determining step, the presence or absence of blurring of the cell is determined based on the contrast of the cell image, and the presence or absence of reflection of the foreign matter, the presence or absence of reflection of the stray light, the presence or absence of degradation in image quality of the cell image due to the culture medium, the presence or absence of an area where pixel values are saturated, and the presence or absence of the cell whose boundary is unclear are determined based on the distribution of brightness values of the cell image. excluding the cell image from analysis if the cell image is classified as the inappropriate cell image;
The cell image analysis method according to claim 1 , further comprising: performing the analysis using the cell image classified as the appropriate cell image. The cell image analysis method according to any one of claims 1 to 7, further comprising a step of notifying, when the cell image is classified as an inappropriate cell image, that the cell image is not suitable for analysis when performing the analysis of the cell image, or that, when training a learning model for analyzing the cell image, the cell image is not suitable for training the analysis on the learning model . If the cell image is classified as the inappropriate cell image, excluding the cell image from learning of an analysis of the cell image for a learning model;
The cell image analysis method according to any one of claims 1 to 7 , further comprising a step of training the learning model to perform the analysis using the cell images classified as the appropriate cell images.