JP7037438B2 - Colony detector, colony detection method, and colony detection program - Google Patents

Description

Publication on the website to which Article 30, Paragraph 2 of the Patent Law is applied Publication date: May 22, 2018, publication address, etc .: https: // www. nikkei. com / article / DGXMZ030817750S8A520C1TJ2000 /

The present invention relates to a technique for detecting a colony derived from a microorganism.

Companies involved in food manufacturing, processing, cooking, etc. are always required to carry out thorough quality control in order to provide consumers with safe food. Therefore, we pay attention to all processes from raw materials, manufacturing and processing, cooking, storage, distribution, sales, etc. to final products, and inspect them by an objective scientific method (see Non-Patent Document 1). ..

As an inspection in quality control, for example, a culture method, which is the most basic technique in a microbiological inspection, is performed. The culture method separates nutrients necessary for the growth of the target microorganism, salts for adjusting pH and osmotic pressure, selection agents such as bile salt, sugars, amino acids, pH indicators and color-developing enzyme substrates. This is a method for obtaining the number of bacteria by testing a sample in a medium composed of a differentiating agent, agar for solidification, and the like, and measuring the number of colonies formed in this medium. The number of bacteria obtained by the culture method is one of the most powerful indicators of the degree of microbial contamination of foods, as it is the subject of inspection for all foods and environmental materials (containers, cutting boards, etc.) that handle them. In addition, according to the number of bacteria, the presence or absence of spoilage or spoilage of food, the risk of developing food poisoning, and the like can be comprehensively estimated (see Non-Patent Document 2).

As a technique for measuring the number of colonies, not only a conventional method for visually measuring the number of colonies but also a technique for mechanically measuring the number of colonies by image processing or the like is known. As a technique for mechanically measuring the number of colonies by image processing or the like, a technique using a device using a monochromatic CCD camera (see, for example, Patent Document 1) and a technique using a still imaging device such as an image scanner (for example, see). (See Patent Document 2) is known. Further, as another technique, a technique is known in which pixels satisfying a predetermined condition in the medium image data are specified as colony pixels, a statistical standard is calculated, and colonies are detected based on the specified colony pixels. (For example, see Patent Document 3).

Japanese Unexamined Patent Publication No. 2006-345750 Japanese Unexamined Patent Publication No. 2017-035042 Japanese Unexamined Patent Publication No. 2016-26500

Journal of Japanese Food Microbiology 31 (2) 64-69 (2014) Food Sanitation Inspection Guidelines Microorganisms 2015, Japan Food Sanitation Association March 31, 2015

However, in the techniques disclosed in Patent Document 1 and Patent Document 2, it is necessary for the service provider or the user to set an optimum threshold value in advance as a reference value for colony discrimination. Therefore, it is necessary to set a fixed threshold value for colony discrimination in advance, and when the properties of the colony change due to the influence of gene mutation or impurities contained in the sample, or the colony has an atypical shape. It was necessary to manually reset the threshold value considering such colonies for each image when measuring. In addition, statistical criteria such as those in Patent Document 3 do not always provide reference values that can detect and measure appropriate colonies, so there is no guarantee that colonies can be measured accurately for images with various imaging conditions and colony shapes. do not have.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technique capable of easily and appropriately detecting colonies from image data obtained by imaging a culture medium after culturing. be.

In order to achieve the above object, the colony detection device according to one aspect is a colony detection device that detects a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism, and is a target for detecting the colony. A medium image acquisition unit for acquiring a medium image, a region extraction unit for extracting a colony candidate region for detecting the presence or absence of a colony from the acquired medium image, and a region extraction unit for extracting the presence or absence of a colony in the colony candidate region. A colony discrimination unit that determines whether or not there are colonies in the colony candidate region using a neural network model, and a calculation unit that calculates the total number of colonies in the culture medium image based on the discrimination result by the colony discrimination unit. Be prepared.

According to the present invention, colonies can be easily and appropriately detected from image data.

It is an overall block diagram of the colony counting system which concerns on one Embodiment. It is a block diagram of the trained model correspondence table which concerns on one Embodiment. It is a block diagram of the threshold value correspondence table which concerns on one Embodiment. It is a schematic diagram of the dry type simple culture medium which concerns on one Embodiment. It is a flowchart of the colony count processing which concerns on one Embodiment. It is a flowchart of the candidate area extraction process which concerns on one Embodiment. It is a figure explaining the state of the image in the candidate area extraction process which concerns on one Embodiment. It is a flowchart of the trained model generation process which concerns on one Embodiment. It is a figure explaining the learning of the model which concerns on one Embodiment, and the discrimination of a candidate area. It is a figure explaining the trained model generation processing and the counting of the number of colonies which concerns on one Embodiment.

The embodiments will be described with reference to the drawings. It should be noted that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are indispensable for the means for solving the invention. Is not always.

FIG. 1 is an overall configuration diagram of a colony counting system according to an embodiment.

The colony counting system 1 is a system for taking an image of a medium for measuring microorganisms in which microorganisms are cultured and counting colonies in an image (medium image) of the medium. The colony counting system 1 acquires one or more mobile terminals 10 for capturing / managing a culture medium image, one or more fixed terminals 20 for inputting / managing a culture medium image, a network 30, and a culture medium image. It has an image analysis server 40 as an example of a colony detection device that performs image analysis processing. In the present embodiment, the colony counting system 1 shows an example of having a portable mobile terminal 10 and a fixed terminal 20, but it seems that the colony counting system 1 has only one of the mobile terminal 10 and the fixed terminal 20. You may do it.

The network 30 is, for example, a network such as a wired LAN (Local Area Network), a wireless LAN, or a WAN (Wide Area Network).

The mobile terminal 10 is a terminal having an image pickup function (hereinafter referred to as a camera function) such as a still image, and is, for example, a smartphone, a digital camera, a tablet, a mobile phone, or the like. The mobile terminal 10 has an image data generation unit 11, an application control unit 12, and a data transmission / reception unit 13.

The image data generation unit 11 has a camera function for capturing a range including a predetermined object, and generates an image (image data) in a range including the predetermined object by the camera function. In the present embodiment, the image data generation unit 11 generates an image (medium image) of the dry simple medium 801 described later according to the operation of the user (user) of the mobile terminal 10. The application control unit 12 registers / deletes a user account, previews image data, sets image data attributes, specifies a medium image to be counted as a colony, displays a colony count result for the medium image, and the like. The data transmission / reception unit 13 communicates with another device (for example, an image analysis server 40 or the like) via the network 30. In the present embodiment, the data transmission / reception unit 13 transmits the medium image to be counted to the image analysis server 40, receives the count result from the image analysis server 40, and the like.

In the present embodiment, the mobile terminal 10 is a terminal having a communication function, but the present invention is not limited to this, and a terminal having no communication function may be used. In this case, the captured medium image is stored in a recording medium (memory card, etc.) mounted on the mobile terminal 10, and the medium image stored in the recording medium is read by a device having another communication function to obtain an image. It may be sent to the analysis server 40.

The fixed terminal 20 is, for example, a PC (Personal Computer) or the like, and has an image data input unit 21, an application control unit 22, and a data transmission / reception unit 23.

The image data input unit 21 inputs a medium image stored in the recording medium from a recording medium such as a memory card, a USB memory, or a CD-ROM. The image data input unit 21 has a camera function, and the medium image may be captured and input by the camera function, or the medium image read by an image reading device (not shown) may be input. You may. The application control unit 22 registers / deletes a user account, previews image data, sets image data attributes, specifies a medium image to be counted as a colony, displays a colony count result for the medium image, and the like. The data transmission / reception unit 23 communicates with another device (for example, an image analysis server 40 or the like) via the network 30. In the present embodiment, the data transmission / reception unit 23 transmits the medium image to be counted to the image analysis server 40, receives the count result from the image analysis server 40, and the like.

The image analysis server 40 is a server device for interlocking with the mobile terminal 10 or the fixed terminal 20 via the network 30 and executing a process of counting colonies of a culture medium image. The image analysis server 40 includes a processor (CPU), a memory, and the like. The image analysis server 40 includes a data transmission / reception unit 41 as an example of a medium image acquisition unit, an image data analysis unit 42 as an example of a region extraction unit, a colony discrimination unit, a calculation unit, an output unit, and a medium discrimination unit. A completed model generation unit 43 and a storage unit 44 as an example of the model storage unit are provided.

The data transmission / reception unit 41 communicates with another device (for example, a mobile terminal 10, a fixed terminal 20, etc.) via the network 30. In the present embodiment, the data transmission / reception unit 41 receives the medium image to be counted from the mobile terminal 10 or the fixed terminal 20, and transmits the count result to the mobile terminal 10 or the fixed terminal 20.

The image data analysis unit 42 analyzes the received medium image and executes a process of counting colonies. The trained model generation unit 43 creates a trained model based on the image data (learning data) for the teacher.

The storage unit 44 is, for example, an HDD (Hard Disk Drive), SSD (Solid State Drive), or the like, and includes image data 45, a trained model correspondence table 46, a threshold correspondence table 47, and learning for one or more colony discrimination. A trained model 48 and a trained model 49 for discriminating product information are provided. The storage unit 44 is executed by a processor (not shown) to form a data transmission / reception unit 41, an image data analysis unit 42, and a trained model generation unit 43 (colony detection program). Remember. The image data 45 includes a culture medium image received for colony discrimination and learning data.

The learned model 48 for colony discrimination is a neural network model for discriminating the presence or absence of colonies in a partial image (colony candidate image) extracted from a medium image. The trained model 48 for colony discrimination may be a CNN (convolutional neural network model). The learned model 48 for colony discrimination may be one, but in the present embodiment, since the target for colony discrimination is a plurality of types of media, the learning for colony discrimination is performed from the viewpoint of improving the accuracy of colony discrimination. The completed model 48 is prepared for each medium type (for each product type of medium). A learned model 48 for colony discrimination may be provided for each type of fungus targeted by the medium and for each color tone of the colony.

The trained model 49 for discriminating product information is a neural network model for discriminating product information from the image of the product label 805 described later in the culture medium image.

FIG. 2 is a configuration diagram of a trained model correspondence table according to an embodiment.

The trained model correspondence table 46 is a table for managing the trained model for colony discrimination used at the time of colony discrimination for each product type of the medium, and each entry includes a field of the product type 46a and the trained model 46b. The product type 46a stores the product type of the medium. The trained model 46b stores the identification information of the trained model 48 for colony discrimination, which should be used when performing colony discrimination on the medium image of the culture medium of the product type corresponding to the entry. According to the trained model correspondence table 46, it is possible to specify a trained model for colony discrimination suitable for the product type of the medium.

FIG. 3 is a configuration diagram of a threshold value correspondence table according to an embodiment.

The threshold value correspondence table 47 is a table that manages the threshold value used in the foreground / background separation of the candidate area extraction process described later and the process of setting an area larger than a predetermined area as a colony candidate area, and each entry is a product type 47a. , Includes fields of threshold X47b and threshold Y47c. The product type 47a stores the product type of the medium. In the threshold value X47b, a threshold value for separating the foreground and the background with respect to the medium image of the medium of the product type corresponding to the entry is stored. The threshold value Y47c stores a threshold value when a region having a predetermined area or more is used as a colony candidate region. According to the threshold value correspondence table 47, it is possible to specify a threshold value suitable for the product type of the medium.

Next, the medium to be imaged will be described.

FIG. 4 is a schematic diagram of a dry simple medium according to an embodiment.

The dry simple medium 801 includes a base material 802, a circular portion 803 arranged on the base material 802, and a product label 805. A medium layer 806 for culturing microorganisms is housed in the circular portion 803. The medium of the culture medium layer 806 may have a composition containing at least components necessary for the growth of microorganisms, agar, a gelling agent, and the like, and may be capable of culturing microorganisms and isolating their colonies. Just do it. The dry simple medium 801 can be stored at room temperature in an unused state and does not require refrigerating equipment, and is therefore preferable in terms of storage. Examples of the dry simple medium 801 include Compact Dry (registered trademark). The medium that is the basis of the medium image to be the target of the colony counting process is not limited to the above-mentioned dry simple medium 801 but may be a medium that can isolate colonies, and is a simple medium such as a film-like medium or a sheet-like medium. An agar medium such as a medium or a plate medium may be used.

When culturing microorganisms, the sample as it is or a diluted solution of the sample is tested on the medium layer 806, the cover sheet 804 is put on the upper part of the circular portion 803, and then the sample is cultured under predetermined conditions. .. Examples of the sample include, but are not limited to, food and drink, environmental sample, clinical sample, and the like. In the present embodiment, a medium image obtained by imaging a dry simple medium 801 after culturing is used for determining a colony.

The product label 805 is arranged on the lower side of the drawing of the circular portion 803 of the base material 802. The product label 805 may be placed anywhere as long as it does not affect the colony counting process and can be recognized from the captured medium image. On the product label 805, product information 807 that can identify the type of the dry simple medium 801 is displayed. Product information 807 includes various information about the medium (product) such as the type of medium, the type of colonies generated in the medium, the manufacturer of the medium, and information about the production, or information that can identify these information (product identification information). There is. In the present embodiment, it is assumed that at least the information regarding the type (product type) of the medium (product) is displayed as the product information 807. The product information 807 may be in a readable form such as an alphanumerical character, a two-dimensional code, or a three-dimensional code.

Next, the processing operation in the colony counting system 1 will be described.

FIG. 5 is a flowchart of the colony counting process according to the embodiment.

First, the data transmission / reception unit 41 of the image analysis server 40 receives (inputs) the image data (medium image data) of the dry simple medium 801 after culturing from the mobile terminal 10 (or the fixed terminal 20) and stores it in the storage unit 44. (S201). Next, the image data analysis unit 42 loads the trained model 49 for discriminating product information for discriminating the product information 807 of the product label 805 of the dry simple medium 801 (S202), and the trained model 49 for discriminating product information 49. Is used to determine the product information 807 (S203). Here, the product information 807 is discriminated using the trained model 49 for discriminating product information, but instead of this, the product information 807 is discriminated using OCR (optical character recognition) or barcode reading technology. You may try to do it.

Next, the image data analysis unit 42 refers to the trained model correspondence table 46, identifies the trained model 48 for colony discrimination corresponding to the product information 807 (product type) discriminated in step S203, and discriminates the identified colonies. Load the trained model 48 for use (S204).

Next, the image data analysis unit 42 extracts the colony candidate region by executing the candidate region extraction process (see FIG. 6) on the medium image acquired in step S201 (S205).

Next, the image data analysis unit 42 executes the loop A processing (S206, S207) with each of the colony candidate regions extracted in step S205 as the processing target. Here, in the description of FIG. 5, the colony candidate region to be processed is referred to as a target colony candidate region.

In the loop A, the image data analysis unit 42 discriminates the label of the discrimination class given to the target colony candidate region by using the colony discrimination learning model 48 (S206). Here, the discrimination class is whether or not there is a colony in the target colony candidate area (for example, background), and if there is a colony in the target colony candidate area, how many colonies are included in the target colony candidate area? It is a class for identifying whether it is. For example, there are class 00 indicating that it is not a colony, class 01 indicating that there is one colony, class 02 indicating that there are two colonies, class 03 indicating that there are three colonies, and the like. If you want to measure the number of colonies in more subdivided units, you can set the identification class according to the category for which you want to measure colonies. For example, when identifying how many bacterial species (for example, specified by the color of the colony) are in the target colony candidate region, if there is one colony of Bacterial A and 0 colony of Bacterial B, class 10 and Bacterial A When there is one colony and one colony of Bacteria B, it may be set as in class 11.

Next, the image data analysis unit 42 counts the number of colonies contained in all the colony candidate regions to be processed in the medium image based on the label of the identification class (S207). When the identification information counted for each category such as a bacterial species is used, the colony may be measured for each category.

When the loop A processing is executed for the target colony candidate area, the loop A processing is executed with the next colony candidate area as the processing target, and the loop A processing is executed for all the colony candidate regions. If it is executed, the process of loop A is exited and the colony count process is terminated.

When this colony counting process is completed, the image data analysis unit 42, via the data transmission / reception unit 41, tells the terminal of the transmission source of the culture medium image the number of colonies counted by the colony counting process, that is, The total number of colonies contained in the medium image is transmitted (output). In the terminal that has received the number of colonies, for example, a display device (not shown) may display the number of colonies. When the number of colonies is measured for each bacterial species, the number of colonies for each bacterial species may be displayed.

Next, the candidate region extraction process (S205) will be described in detail.

FIG. 6 is a flowchart of the candidate region extraction process according to the embodiment. FIG. 7 is a diagram illustrating a state of an image in the candidate region extraction process according to the embodiment.

In the candidate region extraction process, in order to discriminate colonies using the learned model 48 for colony identification in the subsequent step (S206), it is necessary to extract characteristic regions in the medium image without leaking as much as possible. I am aiming.

First, the image data analysis unit 42 performs grayscale conversion on the medium image (medium image data) of the dry simple medium 801 (S301). According to this step, for example, the original medium image shown in FIG. 7 (a) (original image: a black-and-white image due to the limitation of illustration, but actually a color image) is as shown in FIG. 7 (b). Converted to a grayscale image.

Next, the image data analysis unit 42 performs binarization processing (for example, adaptive threshold processing) on the grayscale image (S302). Here, the adaptive thresholding process is a binarization process that is effective for an image that has a different light source environment in the image region depending on the imaging conditions, and is a medium image that may be imaged in various light source environments. However, it is an effective means. According to this process, colonies can be accurately discriminated even if the medium image is not captured by an image pickup device capable of capturing images under uniform conditions such as a stationary device. An example in which adaptive thresholding is used as the binarization process has been shown, but the present invention is not limited to this, and any other binarization process that can alleviate unevenness due to the light source included in the image data is used. It may be a process. Here, according to this step, for example, the grayscale image shown in FIG. 7 (b) is converted into an image (referred to as a binarized image) that has been binarized as shown in FIG. 7 (c). To.

Next, the image data analysis unit 42 applies noise reduction processing to the binarized image using a smoothing filter using a Gaussian filter as an example (S303).

Next, the image data analysis unit 42 smoothes the line segment appearing in the binarized image by performing an expansion process on the binarized image, and further expands the binarized image by the foreground / background separation process. A specific region is extracted from the region formed by the line segment (S304). This process deletes, for example, a region in which the distance from the center of gravity in the region is smaller than the threshold value (threshold value of the threshold value X47b) corresponding to the product type of the threshold value correspondence table 47 for the region surrounded by the expanded line segment. The area where the distance from the center of gravity is equal to or greater than the threshold value is extracted. By this processing, it is possible to efficiently remove the background, that is, the region that is unlikely to be a colony, such as the pattern of the medium appearing in the binarized image. According to this step, the binarized image shown in FIG. 7 (c) becomes an image (referred to as a foreground image) as shown in FIG. 7 (d). Comparing the foreground image with the binarized image, it can be seen that the composition that becomes the background (the pattern of the medium and the base material of the medium) is removed from the image.

Next, the image data analysis unit 42 applies the Watershed method to the foreground image to separate the areas of adjacent colonies (S305). Here, the Watershed method is an algorithm for determining the contour of an object by using the brightness of an image. In the subsequent step (S206), in order to perform classification assuming that a plurality of colonies are included in one colony candidate region in the process of discriminating colonies using the learned model 48 for colony identification. , In step S305, it is not necessary to separate the regions of all adjacent colonies in detail. The Watershed method has been mentioned as a method for dividing the colony area, but any other method may be used as long as it is a treatment method capable of separating a plurality of colony areas. According to this step, the foreground image shown in FIG. 7 (d) is converted into an image as shown in FIG. 7 (e).

Next, the image data analysis unit 42 sets only a region having an area equal to or larger than a predetermined threshold as a colony candidate region based on the empirical value that the region having a relatively small area is not a colony (S306). Here, the threshold value used may be the threshold value of the threshold value Y47c corresponding to the product type of the threshold value correspondence table 47. As described above, by using the threshold value corresponding to the product type, processing suitable for the image of the medium can be performed, and a more appropriate colony candidate region can be extracted.

The candidate area extraction process focuses on removing areas that are unlikely to be colonies, rather than extracting areas that look like colonies. It can be left and handed over to the colony discrimination process by the trained model to improve the colony detection accuracy.

Next, the trained model generation process for generating the trained model 48 for colony discrimination will be described.

Since the colony discrimination process S206 is performed by a trained model trained by a predetermined number or more of training data in advance, it is necessary to generate a trained model 48 for colony discrimination.

FIG. 8 is a flowchart of the trained model generation process according to the embodiment. FIG. 8 is a process for generating one trained model, and when a plurality of types of trained models are generated, the process of FIG. 8 is executed using different training data.

In the trained model generation process, the loop B process (S701 to S705) is executed for each of the training data, and the result is the learning being generated each time the learning by each learning data is completed. It will be fed back to the finished model.

First, the trained model generation unit 43 of the image analysis server 40 accepts the input of the training data and the label of the identification class corresponding to the training data by the worker (S701). Here, the training data is an image prepared assuming a form that can be taken as a colony candidate region in the culture medium image targeted by the trained model, and is, for example, an image of a colony cut out from an actual culture medium image. It is an image of a region or a non-colony region. By learning training data of various forms (typical image, atypical image, each number of images with a predetermined number of colonies or less, images in which colonies of multiple bacterial species are combined, etc.) , The number of colonies, the combination of different bacterial types, etc., to determine how many colonies are contained in the atypical shape even if the shape of the colony has an atypical shape. Is possible. In order to design the discrimination class in consideration of various lighting environments and colony morphologies, it is important to prepare a sufficient number of learning data corresponding to various lighting environments and colony morphologies.

Here, as a class identified by the trained model (that is, a class prepared as training data), for example, the following class can be considered.
-Although there are differences in the size and shape of colonies, the number of colonies is a class consisting of image data counted as one.-A class consisting of image data with multiple colonies in one area.-1 A class consisting of image data with colonies of different colors (different bacterial species) in one region-A class consisting of image data that is not a culture medium background or other colony region

The trained model generation unit 43 applies an arbitrary number of filters to the input training data in the convolution process, and extracts feature amount data for each filtered image data (S702). Next, the trained model generation unit 43 performs a pooling process in order to reduce the number of dimensions of the extracted feature data (S703). Next, the trained model generation unit 43 inputs the feature amount data obtained in S703 into the neural network model, and outputs the predicted value of the label of the identification class corresponding to the training data (S704). Next, the trained model generation unit 43 compares the predicted value of the identification class label output in S704 with the label of the correct identification class input in S701, and in the neural network model in the trained model. Update to minimize the parameter deviation (S705).

The processing of loop B (S701 to S705) is repeatedly executed for the number of training data, and the trained model generation processing is completed. By executing this trained model generation process, the trained model 48 for colony discrimination is generated. When generating trained models for culture medium images of different product types, it is sufficient to prepare training data for the culture medium images of the product type and perform the same processing as described above.

FIG. 9 is a diagram illustrating learning of the model and determination of the candidate region according to the embodiment.

The trained model 48 for colony discrimination shown in FIG. 9 includes a class (background class: class 00) that is a background image (no colonies), a class that has one colony (number of colonies 1 class: class 01), and a class (class 01). It is a trained model that can be discriminated into either a class having two colonies (number of colonies 2 class: class 02) and a class having three colonies (number of colonies 3 classes: class 03). The learned model 48 for colony discrimination uses an image as shown in the image sample of the classification in FIG. 9A as training data, and is input and trained as a label of the identification class corresponding to the training data. Has been generated. The learned model 48 for colony discrimination is trained and generated by learning data of a predetermined number or more of various imaging conditions and modes corresponding to each class.

When the image of the colony candidate region (without colony) shown in FIG. 9A is the processing target, it is determined to be a background image according to the learned model 48 for colony discrimination, and the colony candidate region is used. On the other hand, the identification class (class 00) of the background class is given. Further, when the image of the colony candidate region shown in FIG. 9B (the number of colonies is one) is the processing target, it is determined that one colony exists according to the learned model 48 for colony discrimination. Therefore, an identification class (class 01) of one colony number class in which one colony exists is assigned. Further, when the image of the colony candidate region shown in FIG. 9C (the number of colonies is two) is the processing target, it is determined that there are two colonies according to the learned model 48 for colony discrimination. , The identification class (class 02) of the number of colonies 2 in which two colonies exist is given.

Although an example of identification for a relatively simple image is shown in FIG. 9, the identification class corresponding to the combination of the number of colonies for a plurality of bacteria is larger than the identification class shown in FIG. In some cases.

FIG. 10 is a diagram illustrating the generation process of the trained model and the counting of the number of colonies according to the embodiment.

FIG. 10A shows a case where the medium image contains a bacterium (bacterial species A) that forms a red colony (red colony) and a bacterium (bacterial species B) that forms a blue colony (blue colony). An example is shown. As described above, when there are bacterial species A and bacterial species B, the discrimination class in the trained model 48 for colony discrimination includes a class having one red colony as shown in (a1) and (a2). ), A class having one blue colony and a class having one red colony and one blue colony as shown in (a3) are included. That is, a large amount of learning data corresponding to these classes is prepared for learning.

According to this trained model for colony discrimination, the colony candidate regions include a class having one red colony, a class having one blue colony, and a class having one red colony and one blue colony. Can be determined. When the determination is made in this way, for example, the image data analysis unit 42 counts the colonies for each colony of each color in S207 of the colony counting process of FIG. In this way, colonies for each bacterial species can be distinguished and appropriately counted.

FIG. 10B shows an example in which a large-sized colony and a small-sized colony coexist in the culture medium image. As described above, when the size-sized colonies and the small-sized colonies coexist, the learned model 48 for colony discrimination can be used when there is only one large-sized colony as shown in (b1). A trained model is used in which the same colony is discriminated into one class when there is one small colony as shown in (b2). That is, as learning data corresponding to one class with one colony, a large number of learning data having one colony and different sizes are prepared and trained.

According to this trained model for colony discrimination, the colony candidate region is discriminated as a class in which the same colony is one regardless of whether there is one large-sized colony or one small-sized colony. can. When the determination is made in this way, for example, the image data analysis unit 42 counts the colonies regardless of the size of the colonies in S207 of the colony counting process of FIG. By doing so, the number of colonies can be appropriately counted regardless of the size of the colonies.

FIG. 10 (c) shows an example in which uneven coloring occurs in the medium due to various factors. On the other hand, as the trained model 48 for colony discrimination, when there is one colony even in the colored region as shown in (c1), it is discriminated into the class having one colony, and ( In the case of non-colony such as test solution residue in the colored region as shown in c2), a trained model discriminated into the non-colony class is generated. That is, as learning data corresponding to one class with one colony, learning data having one colony in the colored region is prepared and trained, and as learning data corresponding to a non-colony class. , The learning data which is a non-colony in the colored region is prepared and trained.

According to this learned model for colony discrimination, when the colony candidate region is a colored region and has one colony, it can be discriminated into a class having one colony, and it is a colored region and is a non-colony. In some cases, it can be discriminated into a non-colony class. When determined in this way, for example, the image data analysis unit 42 can appropriately count the colonies in the colored region in S207 of the colony counting process of FIG.

FIG. 10D shows an example in which colonies are present at high density in the medium image. In this way, when colonies are present at high density, as the learned model 48 for colony discrimination, when there are three colonies as shown in (d1), the class has three colonies. In addition, when there are four colonies as shown in (d2), a trained model is generated so that the class has four colonies. That is, as learning data corresponding to three classes of colonies, learning data having three colonies is prepared and trained, and as learning data corresponding to four classes of colonies, colonies are used. Prepare four learning data and let them learn.

According to this learned model 48 for colony discrimination, when the colony candidate region has four colonies, it can be discriminated into a class having four colonies, and when there are three colonies, there are three colonies. It can be identified as one class. When determined in this way, for example, the image data analysis unit 42 appropriately counts the number of colonies by adding the number of colonies in each class in S207 of the colony counting process of FIG. be able to.

The present invention is not limited to the above-described embodiment, and can be appropriately modified and implemented without departing from the spirit of the present invention.

For example, in the above embodiment, the image analysis server 40 has shown an example of generating a learned model 48 for colony discrimination and a learned model 49 for product information discrimination, but the present invention is not limited to this, and the image analysis server is not limited to this. At least one of the learned model 48 for colony discrimination or the trained model 49 for product information discrimination is generated by the same processing by an information processing device different from 40, and the generated trained model is stored in the image analysis server 40. You may do it.

Further, in the above embodiment, a part or all of the processing performed by the processor may be performed by the hardware circuit. Further, the program in the above embodiment may be installed from the program source. The program source may be a program distribution server or storage media (eg, portable storage media).

1 ... Colony counting system, 10 ... Mobile terminal, 20 ... Fixed terminal, 30 ... Network, 40 ... Image analysis server, 41 ... Data transmission / reception unit, 42 ... Image data analysis unit, 43 ... Learned model generation unit, 44 ... Storage Part, 45 ... image data, 46 ... trained model correspondence table, 47 ... threshold correspondence table, 48 ... colony discrimination trained model, 49 ... product information discrimination trained model

Claims (13)

It is a colony detection device that detects a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
A medium image acquisition unit that acquires a medium image to be detected as a colony, and a medium image acquisition unit.
From the acquired culture medium image, a region extraction unit for extracting a colony candidate region, which is a region for detecting the presence or absence of the colony, and a region extraction unit.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, a colony discriminating unit for determining whether or not there is a colony in the colony candidate region, and
A calculation unit that calculates the total number of colonies in the culture medium image based on the discrimination result by the colony discrimination unit, and a calculation unit.
Equipped with
A model storage unit that stores a plurality of learning models for determining the presence or absence of the colony, and
Further, a medium discriminating unit for discriminating the medium type of the medium corresponding to the medium image is provided.
The colony discriminating unit acquires a learning model suitable for the discriminated medium type from the model storage unit, and discriminates the presence or absence of colonies.
Colony detector. The colony detection device according to claim 1, further comprising an output unit that outputs information regarding the calculated total number of colonies. The learning model is a model capable of discriminating the number of colonies in the colony candidate region, and the colony discriminating unit determines the presence or absence of colonies in the colony candidate region and the number of colonies when there are colonies. Item 2. The colony detection device according to Item 2. The culture medium image includes an image of a label showing information that can identify the culture medium type.
The colony detection device according to claim 1 , wherein the culture medium discrimination unit identifies the information of the label from the culture medium image and specifies the culture medium type. The region extraction unit grayscales the acquired medium image, executes binarization processing, performs background / background separation processing, divides the region based on the brightness, and colonies the divided region. The colony detection device according to any one of claims 1 to 4 , which is a candidate region. The colony detection device according to any one of claims 1 to 5 , wherein the culture medium image acquisition unit acquires a culture medium image from a mobile terminal via a network. The colony detection device according to any one of claims 1 to 6 , which uses a neural network model as the learning model. The colony detection device according to claim 7 , wherein the neural network model is a convolutional neural network model. It is a colony detection device that detects a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
A medium image acquisition unit that acquires a medium image to be detected as a colony, and a medium image acquisition unit.
From the acquired culture medium image, a region extraction unit for extracting a colony candidate region, which is a region for detecting the presence or absence of the colony, and a region extraction unit.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, a colony discriminating unit for determining whether or not there is a colony in the colony candidate region, and
A calculation unit that calculates the total number of colonies in the culture medium image based on the discrimination result by the colony discrimination unit, and a calculation unit.
Equipped with
The region extraction unit grayscales the acquired medium image, executes binarization processing, performs background / background separation processing, divides the region based on the brightness, and colonies the divided region. Candidate area
Colony detector. It is a colony detection method by a colony detection device that detects a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
When you get a medium image to detect colonies,
A colony candidate region, which is a region for which the presence or absence of the colony is to be detected, is extracted from the acquired medium image.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, it is determined whether or not there is a colony in the colony candidate region.
Based on the result of the discrimination, the total number of colonies in the medium image was calculated .
When determining whether or not there is a colony in the colony candidate region, the medium type of the medium corresponding to the medium image is determined, and a learning model suitable for the determined medium type is used to determine the presence or absence of the colony. Obtain from the model storage unit that stores multiple learning models to determine the presence or absence of colonies.
Colony detection method. It is a colony detection program executed by a computer constituting a colony detection device for detecting a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
The colony detection program
The computer has a medium image acquisition unit that acquires a medium image to be detected as a colony, and a medium image acquisition unit.
From the acquired culture medium image, a region extraction unit for extracting a colony candidate region, which is a region for detecting the presence or absence of the colony, and a region extraction unit.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, a colony discriminating unit for determining whether or not there is a colony in the colony candidate region, and
Based on the discrimination result by the colony discrimination unit, it functions as a calculation unit for calculating the total number of colonies in the culture medium image .
In addition,
A model storage unit that stores a plurality of learning models for determining the presence or absence of the colony, and
It functions as a medium discriminating unit for discriminating the medium type of the medium corresponding to the medium image.
The colony discriminating unit acquires a learning model suitable for the discriminated medium type from the model storage unit, and discriminates the presence or absence of colonies.
Colony detection program. It is a colony detection method by a colony detection device that detects a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
When you get a medium image to detect colonies,
A colony candidate region, which is a region for which the presence or absence of the colony is to be detected, is extracted from the acquired medium image.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, it is determined whether or not there is a colony in the colony candidate region.
Based on the result of the discrimination, the total number of colonies in the medium image was calculated .
When extracting the colony candidate region, the acquired medium image is grayscaled, binarization processing is performed, foreground / background separation processing is performed, and the region is divided based on the brightness and obtained by dividing. Use the area as a colony candidate area
Colony detection method. It is a colony detection program executed by a computer constituting a colony detection device for detecting a colony derived from a microorganism from a medium image which is an image of a medium after culturing the microorganism.
The colony detection program
The computer has a culture medium image acquisition unit that acquires a culture medium image to be detected as a colony, and a culture medium image acquisition unit.
From the acquired culture medium image, a region extraction unit for extracting a colony candidate region, which is a region for detecting the presence or absence of the colony, and a region extraction unit.
Using a learning model for determining the presence or absence of the colony in the colony candidate region, a colony discriminating unit for determining whether or not there is a colony in the colony candidate region, and
Based on the discrimination result by the colony discrimination unit, it functions as a calculation unit for calculating the total number of colonies in the culture medium image .
The region extraction unit grayscales the acquired medium image, executes binarization processing, performs background / background separation processing, divides the region based on the brightness, and colonies the divided region. Candidate area
Colony detection program.

Images