JP7375161B2 - Learning data creation device, method, program, and recording medium - Google Patents

Description

The present invention relates to a learning data creation device, method, program, learning data, and machine learning device, and particularly relates to a technique for efficiently creating a large amount of learning data.

2. Description of the Related Art Conventionally, an appearance inspection apparatus has been proposed that learns based on a large amount of teaching data stored in a teaching file, performs pattern recognition, and determines defects (Patent Document 1).

This visual inspection device includes a teaching data generation device that transforms specific teaching data with a small number of data out of a large number of teaching data in the teaching file to generate new teaching data. By supplementing the teaching file with teaching data generated by the teaching data generating device for learning, it is possible to inspect defects with a small amount of data.

Furthermore, when the teaching data to be generated is image data, the teaching data generation device performs affine transformation including image enlargement, reduction, and rotation, and attribute conversion including brightness, contrast, and edge strength. New teaching data is being generated.

Japanese Patent Application Publication No. 2006-48370

By the way, in order to accurately recognize the drug area in a photographed image of a drug as a target object using a trained learning model, it is necessary to It is necessary to create many pairs with region information (correct data) indicating the region of the drug, and perform machine learning on the learning model using a learning data set consisting of many pairs.

Conventionally, this type of correct answer data is created by displaying a photographed image on a display and having the user fill in the drug area pixel by pixel while looking at the photographed image displayed on the display, which takes time and effort to create correct answer data. The problem is that it takes time.

On the other hand, the visual inspection device described in Patent Document 1 uses a camera to image a printed matter or a plain surface (paper, film, metal, etc.), recognizes a printing defect from the captured image, and recognizes the type of defect ( It is used to separate ``holes'', ``stains'', ``protrusions'', ``streaks'', etc.).

Therefore, when the teaching data generation device transforms one piece of data (image data) with a small number of data to generate a plurality of new teaching data, the data corresponding to the plurality of teaching data generated by transforming the same image data. The correct data is data indicating the same defect type. That is, Patent Document 1 does not describe the problem that it takes time and effort to create correct answer data for teaching data (teaching images), and also does not disclose a technique to solve the problem.

The present invention has been made in view of the above circumstances, and it is possible to efficiently create learning data for machine learning a learning model that recognizes area information corresponding to the area of a drug from photographed images of drugs. The purpose is to provide a learning data creation device, method, program, learning data, and machine learning device that can perform the following tasks.

In order to achieve the above object, the invention according to a first aspect is a learning data creation device comprising a processor and a memory, the processor creates learning data for machine learning, the processor creates learning data for machine learning, and the processor creates learning data for machine learning. An acquisition process that acquires a photographed image, a learning image generation process that generates a learning image in which drugs are arbitrarily arranged from the acquired photographic image, and second area information corresponding to the area of the drug in the generated learning image. Correct data generation processing is performed in which the generated second area information is used as correct data for the learning image, and storage control is performed in which a pair of the generated learning image and correct data is stored in a memory as learning data. .

According to the first aspect of the present invention, a learning image in which medicines are arbitrarily arranged is generated from a photographed image in which medicines are photographed. Further, second area information corresponding to the drug area in the generated learning image is generated, and the generated second area information is used as correct data for the learning image. This generation of correct answer data can be performed by correct answer data generation processing by a processor, so that no effort or time is required to create the correct answer data.

By using the pair of the learning image and correct answer data generated in this way as learning data, it is possible to generate (inflate) a large amount of learning data.

In the learning data generation device according to the second aspect of the present invention, the acquisition process of the processor acquires first area information corresponding to the drug area in the acquired photographic image, and the correct data generation process acquires first area information corresponding to the acquired first area. Preferably, the second region information is generated based on the region information.

In the learning data creation device according to the third aspect of the present invention, the acquisition process of the processor includes acquiring captured images in which a plurality of drugs are captured, or a plurality of captured images in which drugs are captured, and It is preferable to acquire a plurality of pieces of corresponding first area information.

In the learning data creation device according to the fourth aspect of the present invention, the first region information includes region information that manually sets the drug region in the captured image, and region information that automatically extracts the drug region in the captured image by image processing. Preferably, the area information is area information obtained by automatically extracting the drug area in the photographed image through image processing and manually adjusting the area information.

In the learning data creation device according to the fifth aspect of the present invention, the correct data includes a correct image corresponding to the drug region, bounding box information surrounding the drug region with a rectangle, and edge information indicating edges of the drug region. It is preferable to include at least one of the following. Note that the correct image includes a mask image.

In the learning data generation device according to the sixth aspect of the present invention, the learning image generation process generates the learning image by translating, inverting, rotating, or scaling the captured image, and the correct data generation process includes the first It is preferable to generate correct data by translating, inverting, rotating, or scaling the region information in accordance with the photographed image. The learning image and the correct data may be generated simultaneously through parallel processing, or one of the learning image and the correct data may be generated before the other is generated.

In the learning data creation device according to the seventh aspect of the present invention, the learning image generation process generates a learning image by synthesizing two or more images obtained by translating, inverting, rotating, or scaling a captured image; Preferably, the correct data generation process generates correct data by translating, inverting, rotating, or scaling the first area information corresponding to each of the two or more images in accordance with the captured image. Thereby, it is possible to generate a learning image consisting of images of a plurality of drugs and its correct answer data.

In the learning data creation device according to the eighth aspect of the present invention, the processor includes a drug image acquisition process for acquiring a drug image in which a drug region is cut out from a captured image based on the acquired first region information, and The generation process involves translating, inverting, rotating, or scaling the acquired drug image to generate a learning image, and the correct data generation process involves translating, inverting, or rotating the first region information in accordance with the drug image. It is preferable to generate the correct data by enlarging or reducing the data.

In the learning data creation device according to the ninth aspect of the present invention, the processor includes a drug image acquisition process for acquiring a drug image in which a drug region is cut out from a captured image based on the acquired first region information; In the generation process, a learning image is generated by combining two or more images obtained by translating, inverting, rotating, or scaling the acquired drug image, and in the correct data generation process, the first area information is made to correspond to the drug image. It is preferable to generate correct data by synthesizing two or more pieces of second area information that have been translated, reversed, rotated, or scaled. Thereby, it is possible to generate a learning image consisting of images of a plurality of drugs and its correct answer data.

In the learning data creation device according to the tenth aspect of the present invention, in the learning image generation process, when generating a learning image including a plurality of drugs, all or part of the plurality of drugs are in contact with each other at points or lines. It is preferable to generate training images.

In the learning data creation device according to the eleventh aspect of the present invention, it is preferable that the correct data includes an edge image that shows only the points or lines of the plurality of drug images that are in contact with each other. As correct answer data for a learning image in which all or part of a plurality of medicines are in contact with a point or a line, an edge image showing only a portion of a plurality of medicine images that are in contact with a point or a line can be included. This learning data is useful for separating points or lines that touch each other in a plurality of drug images.

In the learning data creation device according to the twelfth aspect of the present invention, it is preferable that the drug is at least partially transparent. Since drugs that are at least partially transparent are more difficult to extract than drugs that are entirely opaque and require less learning data, it is particularly effective to generate learning data for drugs that are at least partially transparent.

In the learning data creation device according to the thirteenth aspect of the present invention, the learning image generation process by the processor may include arbitrarily arranging drugs other than transparent drugs when generating the learning image including a plurality of drugs. preferable. In the case of a drug that is at least partially transparent, an image of the drug arranged arbitrarily and an image taken with the transparent drug placed in the same position will result in different images depending on the positional relationship with the illumination light. be.

In the learning data creation device according to the fourteenth aspect of the present invention, it is preferable that the photographed image is an image of a drug handled by the own pharmacy. This is because it is only necessary to create learning data for drugs handled by the own pharmacy.

The invention according to the fifteenth aspect provides learning that includes a pair of a learning image generated by arbitrarily arranging photographed drugs, and correct answer data having second area information indicating the area of the drug in the learning image. It is data.

A machine learning device according to a sixteenth aspect of the present invention includes a learning model and a learning control unit that performs machine learning on the learning model using the above learning data.

In the machine learning device according to the seventeenth aspect of the present invention, the learning model is preferably configured with a convolutional neural network.

The invention according to the 18th aspect is a learning data creation method in which a processor creates learning data for machine learning by performing processing in the following steps, the step of acquiring a photographed image in which a drug is photographed; , a step of generating a learning image in which drugs are arbitrarily arranged from the acquired photographed image, generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as a learning image. The method includes a step of setting the image as correct data, and a step of storing the generated pair of the learning image and the correct data in a memory as the learning data.

In the learning data creation method according to the nineteenth aspect of the present invention, the correct data includes a correct image corresponding to a drug region, bounding box information surrounding the drug region with a rectangle, and edge information indicating an edge of the drug region. It is preferable to include at least one of the following.

In the learning data creation method according to the 20th aspect of the present invention, it is preferable that in the step of generating a learning image, when arranging a plurality of drugs, all or part of the plurality of drugs are brought into contact with a point or a line. .

In the learning data creation method according to the 21st aspect of the present invention, it is preferable that the correct data includes an edge image showing only the points or lines of contact between the plurality of drugs.

In the learning data creation method according to the twenty-second aspect of the present invention, it is preferable that the drug is at least partially transparent.

In the learning data creation method according to the twenty-third aspect of the present invention, the step of generating a learning image may include arbitrarily arranging drugs other than transparent drugs when generating a learning image including a plurality of drugs. preferable.

The invention according to the twenty-fourth aspect provides a function of acquiring a photographed image in which a drug is photographed, a function of generating a learning image in which a drug is arbitrarily arranged from the acquired photographic image, and a region of the drug in the generated learning image. A function that generates second area information corresponding to and uses the generated second area information as correct answer data for the learning image, and a function that stores the pair of the generated learning image and correct answer data in memory as learning data. This is a learning data creation program that uses a computer to realize the following.

According to the present invention, it is possible to efficiently create learning data for machine learning a learning model that recognizes area information corresponding to a drug area from a captured image of a drug.

FIG. 1 is a diagram showing captured images input to a trained learning model and output results to be obtained from the learning model. FIG. 2 is a diagram showing an example of learning data. FIG. 3 is a conceptual diagram showing image processing when correct data is automatically created. FIG. 4 is a diagram showing a first embodiment in which learning data is created by simulation. FIG. 5 is a diagram showing a second embodiment in which learning data is created by simulation. FIG. 6 is a block diagram showing an example of the hardware configuration of the learning data creation device according to the present invention. FIG. 7 is a plan view showing a medicine package containing a plurality of medicines. FIG. 8 is a block diagram showing a schematic configuration of the imaging device shown in FIG. 6. FIG. 9 is a plan view showing a schematic configuration of the photographing device. FIG. 10 is a side view showing a schematic configuration of the photographing device. FIG. 11 is a block diagram showing an embodiment of a learning data creation device according to the present invention. FIG. 12 is a diagram illustrating an example of a photographed image acquired by the image acquisition unit and first area information indicating a drug area in the photographed image acquired by the first area information acquisition unit. FIG. 13 is a diagram showing an example of learning data generated from the captured image and mask image shown in FIG. 12. FIG. 14 is a diagram showing an example of an edge image showing only points or lines of contact between a plurality of drugs. FIG. 15 is a block diagram showing an embodiment of a machine learning device according to the present invention. FIG. 16 is a flowchart showing an embodiment of the learning data creation method according to the present invention.

Hereinafter, preferred embodiments of a learning data creation device, method, program, learning data, and machine learning device according to the present invention will be described with reference to the accompanying drawings.

[Summary of the present invention]
FIG. 1 is a diagram showing captured images input to a trained learning model and output results to be obtained from the learning model.

The output result of the learning model shown in FIG. 1(B) is the inference result of inferring the drug region (drug region) in the photographed image shown in FIG. 1(A), and in this example, the drug region and the background region are This is a mask image that is classified into regions. Note that the inference result is not limited to the mask image, but may be, for example, a bounding box surrounding the drug region with a rectangular frame, coordinates of two diagonal points of the bounding box, or a combination thereof.

In order to obtain a desired output result (inference result) from an arbitrary input image using a trained learning model, it is necessary to prepare a large amount of learning data for performing machine learning on an untrained learning model.

FIG. 2 is a diagram showing an example of learning data.

In Figures 2 (A) to (C), the left side is the photographed image of the drug (learning image), the right side is the correct data for the photographed image, and the pair of the left and right photographed images and the correct data is the learning data. It is. The correct data in this example is a mask image that distinguishes each drug region from the background.

Basically, the training data requires the captured images on the left side shown in Figures 2 (A) to (C), but for example, there are new drugs and other drugs that are rare in number, so many images cannot be collected. The problem is that there is no.

To create correct answer data for a photographed image (learning image), it is common for the user to display the photographed image on a display and fill in the drug area pixel by pixel while looking at the photographed image displayed on the display. be.

Furthermore, when correct data is automatically created, it can be obtained, for example, by calculating the position and rotation angle of the drug using template matching from the photographed image.

FIG. 3 is a conceptual diagram showing image processing when correct data is automatically created.

A template image I tpl, which is an image showing the drug, is prepared for the photographed image _ITP of the drug. If the shape of the drug is not circular, it is preferable to prepare a plurality of template images I _tpl for each rotation angle to be searched.

Then, by searching for a template image at a position and rotation angle that has the highest correlation with the template image I _tpl from among the photographed images ITP (template matching), the position of the template image I _tpl when the correlation is highest, and Based on the rotation angle of the template image I _tpl , correct data indicating the area of the drug in the photographed image ITP can be created.

Alternatively, the photographed image ITP and the correct data (for example, a mask image) may be superimposed and displayed on the display, and if there is an error in the mask image, the user may correct the mask image pixel by pixel.

<First embodiment of creating learning data through simulation>
FIG. 4 is a diagram showing a first embodiment in which learning data is created by simulation.

FIG. 4A is a diagram showing a pair of a photographed image of a drug and a mask image manually or automatically generated based on the photographed image.

In the present invention, learning data is created by simulation from a pair of a captured image and a mask image (the learning data is padded).

FIG. 4(B) is a diagram showing a pair of a photographed image and a mask image, which are obtained by inverting the photographed image and mask image shown in FIG. 4(A), respectively.

The reversed (horizontally reversed) mask image on the right side shown in FIG. 4(B) becomes a mask image showing the area of the drug in the reversed captured image on the left side shown in FIG. 4(B). Therefore, the inverted captured image can be used as a new learning image, and the inverted mask image can be used as correct data for the newly generated learning image. Note that image reversal is not limited to horizontal reversal, but also includes vertical reversal. Alternatively, the right image may be created by first creating the left image and detecting the area of the drug image therefrom.

FIG. 4(C) is a diagram showing an image obtained by adding the images shown in FIG. 4(A) and FIG. 4(B).

The photographed image on the left side shown in FIG. 4(C) can be created by combining the photographed image shown in FIG. 4(A) and the inverted photographed image shown in FIG. 4(B). That is, the photographed image shown in FIG. 4(C) is obtained by adding an image (drug image) obtained by cutting out the drug area in the inverted photographed image shown in FIG. 4(B) to the photographed image shown in FIG. 4(A). It can be created by pasting. Note that cutting out the drug image from the inverted captured image is performed by cutting out the drug area from the captured image shown in FIG. 4(A) based on the inverted mask image in FIG. 4(B). I can do it. Furthermore, the method of combining two or more drug images is not limited to the method of using mask images as described above. For example, by using a background image of only the background in which no drug is photographed, only the drug images are extracted from each of FIGS. 4(A) and 4(B), and each extracted drug image is combined with the background image. Thus, the photographed image shown in FIG. 4(C) can be generated.

Further, the right mask image shown in FIG. 4(C) can be created by adding the mask image shown in FIG. 4(A) and the inverted mask image shown in FIG. 4(B). Note that when adding two mask images, in order to separate instances, in this example, the pixel value of the drug area in the inverted mask image of FIG. By adding the value as "0", the pixel values of the two drug regions in the generated mask image are made different.

In this way, the two learning data shown in FIGS. 4(B) and (C) can be created from one learning data consisting of the pair of photographed image and mask image shown in FIG. 4(A). .

In addition, in the first embodiment described above, the captured image and the mask image shown in FIG. 4(A) are each inverted, and the learning data consisting of the new captured image and mask image pair shown in FIG. 4(B) is created. However, the present invention is not limited to this, and the captured image and mask image shown in FIG. You may also create learning data. Note that if a blank space is created in the background by synchronously moving, rotating, or reducing the captured image and the mask image, it is preferable to fill the blank space with pixel values similar to those of the background.

Furthermore, in the first embodiment described above, a new photographic image and a mask image are created from a photographic image and a mask image in which one drug is photographed, but a new photographic image and a mask image are created from a photographic image and a mask image in which a plurality of different drugs are photographed separately. A new photographed image and a mask image may be created from a mask image or a photographed image and a mask image in which a plurality of different drugs are photographed simultaneously.

<Second embodiment of creating learning data through simulation>
FIG. 5 is a diagram showing a second embodiment in which learning data is created by simulation.

FIG. 5(A) is a diagram showing a pair of a photographed image of a drug and a mask image generated manually or automatically based on the photographed image, and is the same as the pair shown in FIG. 4(A). be.

FIG. 5(B) is a diagram showing drug regions cut out from the captured image and mask image shown in FIG. 5(A), respectively.

In this example, the area within the rectangular frame surrounding the drug area is set as the area from which the image is cut out (cutout area). Note that since the drug area is known from the mask image, an image within a rectangular frame surrounding the drug area can be cut out based on the mask image.

FIG. 5(C) shows an image of a cutout area cut out from the photographed image and mask image shown in FIG. 5(A), respectively. Cutting out the drug image from the captured image is performed by the process of cutting out the drug area from the captured image shown in FIG. 5(A) (drug image acquisition process) based on the mask image shown in FIG. 5(A). I can do it. Note that since the mask image shown in FIG. 5(A) has information indicating the drug area (first area information), it is possible to cut out the drug area (hereinafter referred to as "drug mask image") from the mask image. can. Further, the drug image acquisition process may include a process of reading out the image after it has been cut out from a memory or the like.

FIG. 5(D) is a diagram showing a new captured image and a mask image created by pasting the cut out drug image and drug mask image at an arbitrary position and an arbitrary rotation angle.

The photographed image and mask image shown in FIG. 5(D) are learning data consisting of a pair of a new learning image and correct answer data created by the above image processing from the photographed image and mask image shown in FIG. 5(A). becomes.

FIG. 5E is a diagram showing a new captured image and mask image created by pasting the cut out drug image and drug mask image at an arbitrary position and arbitrary rotation angle, and in particular, a diagram showing a new captured image and a mask image created by pasting the cut out drug image and drug mask image at an arbitrary position and at an arbitrary rotation angle. are created so that they touch at a point or line.

In order to improve the inference results of the learning model, it is necessary to create a large amount of learning data in which drugs are in contact with each other at points or lines. This is because it is more difficult to accurately infer the area of each drug from a photographed image in which the drugs are in contact with each other at points or lines than when the drugs are isolated without contact.

The mask image on the left side shown in FIG. 5(E) is preferably image-processed so that the respective drug regions do not touch each other. Since the locations where each drug area contacts are known, by replacing the contact locations with the background color, it is possible to prevent each drug area from contacting each other.

Furthermore, if the drugs that are in contact with each other at points or lines are the same drug, it is preferable to make the pixel values of the drug regions different for instance separation. In this case, since each drug region in the mask image can be recognized by the difference in pixel values, there is no need to replace the portions where the drug regions come into contact with the background color.

As described above, a large amount of learning data can be created based on a photographed image in which a drug is photographed and the first area information (mask image) indicating the region of the drug in the photographed image.

[Configuration of learning data creation device]
FIG. 6 is a block diagram showing an example of the hardware configuration of the learning data creation device according to the present invention.

The learning data creation device 1 shown in FIG. 6 can be configured by, for example, a computer, and mainly includes an image acquisition section 22, a CPU (Central Processing Unit) 24, an operation section 25, a RAM (Random Access Memory) 26, and a ROM (Read 27, a memory 28, and a display section 29.

The image acquisition unit 22 acquires, from the photographing device 10, a photographed image of the drug taken by the photographing device 10.

The medicine photographed by the photographing device 10 is, for example, a single dose of medicine or any medicine, and may be contained in a medicine package or not contained in a medicine package.

FIG. 7 is a plan view showing a medicine package containing a plurality of medicines.

In the medicine package TP shown in FIG. 7, a plurality of medicines to be taken at one time are stored in a transparent package and packed one by one. As shown in FIGS. 9 and 10, the medicine packages TP are connected in a band shape, and each medicine package TP has a cutoff line that allows the medicine packages TP to be separated. Note that six medicines T are packaged in the medicine package TP shown in FIG. 7 .

FIG. 8 is a block diagram showing a schematic configuration of the imaging device shown in FIG. 6.

The photographing device 10 shown in FIG. 8 includes two cameras 12A and 12B that photograph the medicine, two illumination devices 16A and 16B that illuminate the medicine, and a photographing control section 13.

9 and 10 are a plan view and a side view, respectively, showing the schematic configuration of the photographing device.

The medicine package TP is placed on a transparent stage 14 installed horizontally (xy plane).

The cameras 12A and 12B are arranged facing each other with the stage 14 in between, in a direction (z direction) orthogonal to the stage 14. The camera 12A directly faces the surface of the medicine package TP and photographs the medicine package TP from above. The camera 12B directly faces the back side of the medicine package TP and photographs the medicine package TP from below.

A lighting device 16A is provided on the side of the camera 12A with the stage 14 in between, and a lighting device 16B is provided on the side of the camera 12B.

The illumination device 16A is arranged above the stage 14 and illuminates the medicine package TP placed on the stage 14 from above. The illumination device 16A has four light emitting sections 16A1 to 16A4 arranged radially, and emits illumination light from four orthogonal directions. Light emission from each of the light emitting sections 16A1 to 16A4 is individually controlled.

The illumination device 16B is arranged below the stage 14 and illuminates the medicine package TP placed on the stage 14 from below. The lighting device 16B has four light emitting parts 16B1 to 16B4 arranged radially like the lighting device 16A, and emits illumination light from four orthogonal directions. The light emission of each light emitting section 16B1 to 16B4 is individually controlled.

Photographing is performed as follows. First, the medicine package TP is photographed from above using the camera 12A. When photographing, the light emitting parts 16A1 to 16A4 of the illumination device 16A are made to emit light in sequence to take four images, and then each of the light emitting parts 16A1 to 16A4 are made to emit light at the same time to take one image. I do. Next, each of the light emitting parts 16B1 to 16B4 of the lower illumination device 16B is made to emit light at the same time, a reflector (not shown) is inserted, the medicine package TP is illuminated from below through the reflector, and the medicine package TP is illuminated from above using the camera 12A. Take pictures of TP.

The four images taken by sequentially emitting light from each of the light emitting parts 16A1 to 16A4 have different illumination directions, and if there is an imprint (unevenness) on the surface of the drug, the appearance of shadows due to the imprint will be different. Become. These four captured images are used to generate a stamp image that emphasizes the stamp on the front side of the medicine T.

One image taken by simultaneously emitting light from each of the light emitting units 16A1 to 16A4 is an image without uneven brightness, and is used, for example, when cutting out an image of the surface side of the drug T (drug image), and This is a photographed image on which a stamped image is superimposed.

In addition, an image in which the medicine package TP is illuminated from below through a reflector and the medicine package TP is photographed from above using the camera 12A is a photographed image used when recognizing areas of a plurality of medicines T. .

Next, the medicine package TP is photographed from below using the camera 12B. When photographing, each of the light emitting sections 16B1 to 16B4 of the illumination device 16B is made to emit light in sequence to take four images, and then each of the light emitting sections 16B1 to 16B4 is made to emit light at the same time to take one image. I do.

The four captured images are used to generate a stamp image that emphasizes the stamp on the back side of the drug T, and one image captured by simultaneously emitting light from each of the light emitting sections 16B1 to 16B4 is used to eliminate uneven brightness. For example, this is an image that is used when cutting out a drug image on the back side of the drug T, and is also a photographed image on which a stamp image is superimposed.

The photographing control unit 13 shown in FIG. 8 controls the cameras 12A, 12B and the illumination devices 16A, 16B, and photographs one medicine package TP 11 times (6 times with the camera 12A, 5 times with the camera 12B). (photographing)).

Furthermore, the photographing is performed in a dark room, and the only light that is irradiated onto the medicine package TP during photographing is the illumination light from the illumination device 16A or the illumination device 16B. Therefore, among the 11 photographed images taken as described above, the image in which the medicine packet TP is illuminated from below through the reflector and the medicine packet TP is photographed from above using the camera 12A has the background as the light source. (white), and the area of each drug T is shielded from light and becomes black. On the other hand, in the other 10 photographed images, the background is black and the area of each drug is the color of the drug.

Note that even if the image is obtained by illuminating the drug package TP from below through a reflector and photographing the drug package TP from above using the camera 12A, the drug may be entirely transparent (semi-transparent), or only a portion of the drug may be transparent. In the case of capsules that are completely transparent and filled with powder or granular medicine (partially transparent medicines), light passes through the medicine area, so the capsules do not turn pitch black like opaque medicines.

Returning to FIG. 6, the learning data creation device 1 is configured to perform machine learning on a learning model that infers a drug from a captured image in which the drug is captured (in particular, infers the region of each drug T present in the captured image). This is to create learning data.

Therefore, the image acquisition unit 22 of the learning data creation device 1 uses a photographic image (i.e., a reflector It is preferable to illuminate the drug package TP from below through the camera 12A and obtain a photographed image of the drug package TP from above using the camera 12A.

The memory 28 is a storage part that stores learning data, and is, for example, a nonvolatile memory such as a hard disk device or a flash memory.

The CPU 24 uses the RAM 26 as a work area, uses various programs including a learning data creation program stored in the ROM 27 or the memory 28, and executes various processes of the apparatus by executing the programs.

The operation unit 25 includes a keyboard and a pointing device (mouse, etc.), and is a part through which various information and instructions are input by user operations.

The display unit 29 displays a screen necessary for operations on the operation unit 25, functions as a part that implements a GUI (Graphical User Interface), and can also display photographed images and the like.

[Embodiment of learning data creation device]
FIG. 11 is a block diagram showing an embodiment of a learning data creation device according to the present invention.

The learning data creation device 1 shown in FIG. 11 is a functional block diagram showing functions executed by the hardware configuration of the learning data creation device 1 shown in FIG. 6, and includes a processor 2 and a memory 28.

The processor 2 includes the image acquisition unit 22, CPU 24, RAM 26, ROM 27, memory 28, etc. shown in FIG. 6, and performs various processes shown below.

The processor 2 functions as an acquisition unit 20 , a learning image generation unit 30 , a correct data generation unit 32 , and a storage control unit 34 .

The acquisition unit 20 includes an image acquisition unit 22 and a first area information acquisition unit 23.

The image acquisition unit 22 acquires the photographed image ITP of the drug T from the photographing device 10 as described above (performs the photographed image acquisition process).

The first area information acquisition unit 23 acquires information (first area information) indicating the area of the drug in the captured image ITP acquired by the image acquisition unit 22. This first region information is correct data for the inference result inferred by the learning model when the captured image is used as an input image for machine learning of the learning model. Note that the first area information, which is the correct data, includes a correct image (for example, a mask image) that indicates the area of the drug in the captured image, bounding box information that surrounds the area of the drug with a rectangle, and information that indicates the edges of the area of the drug. It is preferable that at least one piece of edge information is included.

FIG. 12 is a diagram illustrating an example of a photographed image acquired by the image acquisition unit and first area information indicating a drug area in the photographed image acquired by the first area information acquisition unit.

The captured image ITP shown in FIG. 12(A) is an image obtained by illuminating the medicine package TP from below through a reflector and photographing the medicine package TP (see FIG. 7) from above using the camera 12A. This medicine package TP contains six medicines T1 to T6.

The drugs T1 to T3 shown in FIG. 12(A) are photographed in black because they are opaque drugs that block illumination light from below. Since the drug T4 is a transparent drug, the illumination light from below is transmitted through it and the drug is photographed as white. Drugs T5 and T6 are the same type of capsules, and some of the illumination light from below leaks, so some parts are photographed slightly white.

FIG. 12(B) is first area information indicating the area of each drug T1 to T6 in the photographed image ITP, which is a mask image IM in this example.

The mask image IM is created by, for example, displaying the captured image ITP on the display unit 29, and while viewing the captured image ITP displayed on the display unit 29, the user uses a pointing device such as a mouse to select the area of each drug T1 to T6. It can be created by filling in pixel by pixel. For example, a binarized mask image IM can be created by setting the pixel value of each of the filled-in areas of the drugs T1 to T6 to "1" and the pixel value of the background area to "0".

Note that although the capsule-shaped medicines T5 and T6 are of the same type, it is preferable that the pixel values of the regions of the two medicines T5 and T6 be made different for instance separation. For example, the pixel value of the drug T5 region can be set to "1", and the pixel value of the drug T6 region can be set to "0.5".

In the above example, the mask image IM, which is the first area information, is area information generated by the user manually setting the area of each drug T1 to T6 in the captured image ITP using a pointing device. However, the invention is not limited to this, and it may be generated by automatically extracting the drug area in the captured image by image processing, or it may be generated by automatically extracting the drug area in the captured image by image processing and manually adjusted. It may be generated by

Returning to FIG. 11, the learning image generation unit 30 inputs the photographed image ITP in which the drug is photographed from the image acquisition unit 22, and generates a learning image (I _A , I _B , I _C , ...) are generated. That is, the learning image generation unit 30 performs learning image generation processing to generate a plurality of learning images (I _A , I _B , I _C , . . . ) based on the photographed image ITP.

Arbitrary placement of the drug photographed in the photographed image ITP may be performed by the user instructing the position and rotation of the drug image using a pointing device, or as explained using FIG. This may also be done by inverting or adding. Alternatively, the position and rotation of the drug image may be randomly determined using random numbers, and the drug may be arbitrarily placed. In this case, it is necessary to prevent drug images from overlapping.

The correct data generation unit 32 inputs the mask image IM, which is the first area information, from the first area information acquisition unit 23, and generates a plurality of learning images (I _A , I _B , I _C , . . . ) from the input mask image IM. ) is generated. A plurality of correct answer data (I _a , I _b , I _c , ...) corresponding to That is, the correct data generation unit 32 generates second area information indicating the drug area in the plurality of learning images (I _A , I _B , I _C , ...) based on the mask image IM, and Correct data generation processing is performed in which region information is set as a plurality of correct data (I _a , I _b , I _c , . . . ) for each of a plurality of learning images (I _A , I _B , I _C , . . . ).

Note that the generation of multiple learning images (I _A , I _B , I _C ,...) and multiple correct answer data (I _a , I _b , I _c ,...) is the first step in creating learning data through simulation. As described in the embodiment and the second embodiment, the captured image and the Generated by synchronously inverting, translating, rotating, scaling, etc. each mask image, or by translating, rotating, scaling, or pasting the drug image and drug mask image cut out from the captured image and mask image. can do.

The storage control unit 34 stores learning images (I _A , I _B , I _C , ...) generated by the learning image generation unit 30 and correct data (I _a , I _b ) generated by the correct data generation unit 32 . , I _c , ...), respectively corresponding pairs (learning image I _A and correct data I _a , learning image I _B and correct data I _b , learning image I _C and correct data I _c , ... , ) is stored in the memory 28.

As a result, a large amount of learning data is stored and accumulated in the memory 28. Although not shown in FIG. 6, it is preferable that the pair of photographed image ITP and mask image IM, which are input to the learning image generation unit 30 and the correct data generation unit 32, respectively, is also stored in the memory 28 as learning data. .

FIG. 13 is a diagram showing an example of learning data generated from the captured image and mask image shown in FIG. 12.

FIG. 13(A) shows learning data consisting of a pair of learning image _IA and correct data (mask image) _Ia , and FIG. 13(B) shows a pair of learning image _IB and mask image _Ib . The training data consisting of pairs is shown.

In the learning image _IA shown in FIG. 13(A), capsule-shaped medicines T5 and T6 are in contact with each other in a line, and medicines T2, T3, and T4 are in contact with each other in points. The mask image _{Ia corresponding to this learning image IA makes} it possible to separate instances of the regions of drugs T5 and T6 by differentiating the pixel values of the regions of drugs T5 and T6, which are the same drug, and The boundary between drugs T5 and T6, which are in contact with each other, can also be distinguished.

In addition, in the mask image _Ia , the points where the drugs T2, T3, and T4 are in contact with each other are made the same as the pixel values of the background, so that the drugs T2, T3, and T4 do not come into contact with each other. The regions of each drug T2, T3, and T4 are made clear.

In addition, in the learning image _IB shown in FIG. 13(B), capsule-shaped medicines T5 and T6 are in contact with each other at a line, and medicine T6 and medicine T3 are in contact at a point. The mask image _Ib corresponding to this learning image _IB has different pixel values in the regions of drugs T5 and T6, which are the same drug (for example, the pixel value in the region of drug T6 is set to "0.5"). ), it is possible to separate the instances of the regions of drugs T5 and T6, and it is also possible to distinguish the boundary between drugs T5 and T6, which are in contact with each other with a line, and the border between drugs T6 and drug T3, which are in contact with each other at a point.

The learning data shown in FIG. 13 is an example, and the drug images showing each drug T1 to T6 are arranged by arbitrarily combining translation and rotation, etc., and the drug mask image showing the area of each drug T1 to T6 is similarly arranged. You can create a lot of learning data by placing it in

In this case, it is preferable to generate learning data by arranging some or all of the plurality of drug images so that they are in contact with each other at points or lines. This is because the trained learning model, which has been machine learned using such learning data, can correctly infer the area of each drug when the input image is a photographed image of a drug in contact with a point or a line.

Furthermore, when a transparent drug T4 is photographed as in the photographed image ITP shown in FIG. The light transmission situation changes. That is, the drug image of the transparent drug T4 is an image in which the brightness distribution and the like vary depending on the position and angle of the transparent drug T4 in the imaging region.

Therefore, when generating a learning image by arbitrarily arranging drugs from captured images of multiple drugs including transparent drugs, drugs other than the transparent drugs are arbitrarily placed without moving the transparent drugs. It is preferable to generate a learning image by

Further, in this example, a mask image is generated as the correct data, but edge information (edge image) for each drug image indicating the edge of the region of the drug image may be used. Furthermore, when the drugs are in contact with each other at points or lines, it is preferable to replace the points or lines where they contact with the background color and separate the edge images for each drug.

Furthermore, if the drugs contact each other at a point or line, an edge image showing only the point or line contact may be generated as correct data.

FIG. 14 is a diagram showing an example of an edge image showing only points or lines of contact between a plurality of drugs.

The edge image IE shown in FIG. 14 is an image showing only the locations E1 and E2 where two or more of the plurality of drugs T1 to T6 are in contact with each other in points or lines, and is an image shown by a solid line in FIG. It is. Note that in FIG. 14, the area indicated by a dotted line indicates an area where a plurality of drugs T1 to T6 are present.

The edge image of the line contact point E1 is an image of the line contact point between capsule-shaped drugs T5 and T6, and the edge image of the point contact point E2 is an image of the point point contact point E2 where the three drugs T2 to T4 are in contact with each other. This is an image of locations that are in contact with each other at points.

Since the arrangement of each drug image in the learning image is known, the locations where two or more of the plurality of drugs are in contact with each other by points or lines are also known. Therefore, the correct data generation unit 32 shown in FIG. 11 automatically generates an edge image (correct data) showing only points or lines that touch the learning image generated by the learning image generation unit 30. can be created.

The edge image IE shown in FIG. 14 can be correct data corresponding to the learning image _IA shown in FIG. 13(A). That is, the learning data can be made up of a pair of the learning image _IA shown in FIG. 13(A) and the edge image IE shown in FIG. 14.

Such learning data is used when machine learning a learning model that uses a drug image captured by a point or line contact as an input image and outputs an edge image of only the point or line contact point as an inference result. can be used.

In addition, an edge image (inference result) of only a point or line contact can be, for example, a drug image taken of multiple drugs that are in contact with a point or line, and an edge image of only a point or line contact. can be used as an input image (multichannel input image) for a learning model that infers regions of multiple drugs. According to this learning model, in addition to the input image, information on points of contact with points or lines is input, so that the area of each drug can be more accurately inferred.

[Machine learning device]
FIG. 15 is a block diagram showing an embodiment of a machine learning device according to the present invention.

The machine learning device 50 shown in FIG. 15 includes a learning model (convolution neural network (CNN), which is one of the learning models) 52, a loss value calculation unit 54, and a parameter control unit 56. .

This machine learning device 50 uses learning data created by the learning data creation device 1 shown in FIG. 11 and stored in the memory 28 to cause the CNN 52 to perform machine learning.

The CNN 52 is a part that infers the region of the drug in the input image when an image of a drug is taken as an input image, and has a multiple layer structure and holds multiple weight parameters. . The weight parameters include filter coefficients of a filter called a kernel used for convolution operations in the convolution layer.

The CNN 52 can change from an untrained learning model to a trained learning model by updating the weight parameters from initial values to optimal values.

This CNN 52 includes an input layer 52A, an intermediate layer 52B having multiple sets composed of convolutional layers and pooling layers, and an output layer 52C, and each layer has a structure in which a plurality of "nodes" are connected by "edges". It has become.

A learning image, which is a learning target, is input as an input image to the input layer 52A. The learning image is a learning image in the learning data (learning data consisting of a pair of a learning image and correct data) stored in the memory 28.

The intermediate layer 52B has a plurality of sets including a convolution layer and a pooling layer, and is a part that extracts features from the image input from the input layer 52A. The convolution layer filters (performs a convolution operation using a filter) nearby nodes in the previous layer to obtain a "feature map." The pooling layer reduces the feature map output from the convolution layer to create a new feature map. The "convolution layer" is responsible for extracting features such as edge extraction from an image, and the "pooling layer" is responsible for providing robustness to the extracted features so that they are not affected by parallel movement.

Note that the intermediate layer 52B is not limited to the case where a convolutional layer and a pooling layer are set as one set, but may also include a case where convolutional layers are continuous, an activation process using an activation function, and a normalization layer.

The output layer 52C is a part that outputs a feature map showing the features extracted by the intermediate layer 52B. Further, the output layer 52C outputs an inference result in which the trained CNN 52 performs area classification (segmentation) of the drug area etc. in the input image in pixel units or in units of several pixels grouped together.

Any initial values are set for the filter coefficients and offset values applied to each convolutional layer of the CNN 52 before learning.

Of the loss value calculation unit 54 and parameter control unit 56 that function as a learning control unit, the loss value calculation unit 54 uses the feature map output from the output layer 52C of the CNN 52 and the correct answer data for the input image (learning image). A certain mask image (a mask image read out corresponding to the learning Yuo image from the memory 28) is compared, and an error between the two (a loss value that is a value of a loss function) is calculated. Possible methods for calculating the loss value include, for example, softmax cross entropy and sigmoid.

The parameter control unit 56 adjusts the weight parameters of the CNN 52 using the error backpropagation method based on the loss value calculated by the loss value calculation unit 54. In the error backpropagation method, errors are backpropagated in order from the final layer, stochastic gradient descent is performed in each layer, and parameter updates are repeated until the error converges.

This weight parameter adjustment process is repeated, and learning is performed repeatedly until the difference between the output of the CNN 52 and the mask image that is the correct data becomes small.

The machine learning device 50 repeats machine learning using the learning data stored in the memory 28, so that the CNN 52 becomes a trained model. When the trained CNN 52 receives an unknown input image (a captured image of a drug), it outputs an inference result such as a mask image indicating the region of the drug in the captured image.

Note that as the CNN 52, R-CNN (Regions with Convolutional Neural Networks) can be applied. In R-CNN, bounding boxes of different sizes are slid within the photographed image ITP to detect the region of the bounding box into which the drug can enter. Then, by evaluating only the image portion within the bounding box (extracting CNN features), edges of the drug are detected. Furthermore, instead of R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, etc. can be used.

The inference results of the trained model configured in this manner can be used, for example, when cutting out images of each drug from a photographed image of a medicine package containing a plurality of drugs. Note that the cut out images of each drug are used when inspecting and differentiating each drug contained in a drug package.

By the way, as described above, the memory 28 stores a lot of learning data created by simulation based on the photographed image of the drug and the correct data indicating the area of the drug in the photographed image. The image is preferably an image of a drug that is handled by the pharmacy. By creating learning data using captured images of drugs handled by the own pharmacy and configuring a learning model using the learning data, auditing and differentiation of drugs handled by the own pharmacy is performed. This is because the learning model can be used effectively in certain cases.

[Learning data creation method]
FIG. 16 is a flowchart showing an embodiment of the learning data creation method according to the present invention.

The processing of each step shown in FIG. 16 is performed, for example, by the processor 2 of the learning data creation device 1 shown in FIG.

In FIG. 16, the image acquisition unit 22 acquires a captured image ITP of a drug (for example, the captured image ITP shown in FIG. 12(A)) from the imaging device 10 (step S10). Note that the captured image ITP shown in FIG. 12(A) is an image obtained by illuminating the drug package from below through a reflector and photographing the drug package from above. It is not limited to photographs taken in this manner. Further, the medicine to be photographed may not be in a medicine package, and the number of medicines may be one.

The first area information acquisition unit 23 also receives a mask image IM (for example, the mask image IM shown in FIG. 12(B)) as first area information indicating the area of the drug in the captured image acquired by the image acquisition unit 22. Acquire (step S12). Note that the mask image IM is generated manually or automatically based on the photographed image ITP, and is stored in the memory 28 or the like.

Subsequently, the learning image generation unit 30 generates a learning image in which drugs T1 to T6 are arbitrarily arranged from the photographed image ITP acquired in step S10 (step S14). The learning images can be generated by image processing that translates, inverts, rotates, or scales the drug images representing each drug.

Further, the correct data generation unit 32 generates correct data (mask image) corresponding to the learning image generated in step S14, based on the mask image IM acquired in step S12 (step S16). That is, in step S16, the region of each drug in the mask image IM is arranged in the same way as each drug in the learning image, second region information indicating the region of each drug arranged is generated, and the generated second region information Image processing is performed using this as correct data (mask image) for the learning image.

The storage control unit 34 stores the pair of the learning image generated in step S14 and the mask image generated in step S16 in the memory 28 as learning data (step S18). 13(A) and 13(B) show an example of learning data that is generated as described above and is stored in the memory 28 and is composed of a pair of a learning image and a mask image.

Subsequently, the processor 2 determines whether to end the generation of learning data (step S20). For example, when the user inputs an instruction to end the generation of learning data, or when a preset number of learning data has been created from a pair of one captured image ITP and a mask image, the learning data is It can be determined that generation has ended.

If it is determined that the generation of learning data has not been completed (in the case of "No"), the process returns to step S14 and step S16, and the next learning data is created in steps S14 to S20.

If it is determined that the generation of the learning data is to be completed (in the case of "Yes"), the creation of the learning data based on the photographed image ITP and the mask image IM acquired in steps S10 and S12 is terminated.

It goes without saying that when another captured image ITP and mask image IM are acquired in steps S10 and S12, a plurality of learning data are created based on the captured image ITP and mask image IM.

[others]
The hardware structure of a processing unit such as the CPU 24 that executes various processes in the learning data creation device according to the present invention is the following various processors. Various types of processors include CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays), which are general-purpose processors that execute software (programs) and function as various processing units.The circuit configuration can be changed after manufacturing. This includes programmable logic devices (PLDs), which are processors, and dedicated electrical circuits, which are processors with circuit configurations specifically designed to execute specific processes, such as ASICs (Application Specific Integrated Circuits). It will be done.

One processing unit may be composed of one of these various types of processors, or may be composed of two or more processors of the same type or different types (for example, multiple FPGAs, or a combination of a CPU and FPGA). You can. Further, the plurality of processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, one processor is configured with a combination of one or more CPUs and software, as typified by computers such as clients and servers. There is a form in which a processor functions as multiple processing units. Second, there are processors that use a single IC (Integrated Circuit) chip to implement the functions of the entire system, including multiple processing units, as typified by System On Chip (SoC). be. In this way, various processing units are configured using one or more of the various processors described above as a hardware structure.

More specifically, the hardware structure of these various processors is an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements.

Further, the present invention includes a learning data creation program that is installed on a computer to realize various functions as the learning data creation device according to the present invention, and a recording medium on which this learning data creation program is recorded.

Furthermore, it goes without saying that the present invention is not limited to the embodiments described above, and that various modifications can be made without departing from the spirit of the present invention.

1 Learning data creation device 2 Processor 10 Photographing device 12A, 12B Camera 13 Photographing control section 14 Stage 16A, 16B Illumination device 16A1 to 16A4, 16B1 to 16B4 Light emitting section 20 Acquisition section 22 Image acquisition section 23 First area information acquisition section 24 CPU
25 Operation unit 26 RAM
27 ROM
28 Memory 29 Display unit 30 Learning image generation unit 32 Correct data generation unit 34 Storage control unit 50 Machine learning device 52 Learning model (CNN)
52A Input layer 52B Intermediate layer 52C Output layer 54 Loss value calculation section 56 Parameter control section _IA , _IB , _IC learning image IE Edge image IM, _Ia , _Ib , _Ic Mask image (correct data)
ITP Photographed image I _tpl template image S10-S20 Step T, T1-T6 Drug TP Medicine package

Claims (18)

A learning data creation device comprising a processor and a memory, the processor creating learning data for machine learning,
The processor includes:
an acquisition process for acquiring a photographed image of the drug;
a learning image generation process that generates a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
Correct data generation processing that generates second area information corresponding to the area of the drug in the generated learning image, and sets the generated second area information as correct data for the learning image;
performing storage control to store a pair of the generated learning image and the correct answer data in the memory as learning data;
When generating the learning image including a plurality of drugs, the learning image generation process generates the learning image in which all or part of the plurality of drugs are in contact with a point or a line,
The correct answer data includes an edge image showing only the points or lines of the plurality of drugs that are in contact with each other.
Learning data creation device. A learning data creation device comprising a processor and a memory, the processor creating learning data for machine learning,
The processor includes:
an acquisition process for acquiring a photographed image of the drug;
a learning image generation process that generates a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
Correct data generation processing that generates second area information corresponding to the area of the drug in the generated learning image, and sets the generated second area information as correct data for the learning image;
performing storage control to store a pair of the generated learning image and the correct answer data in the memory as learning data;
When generating the learning image including a plurality of drugs, the learning image generation process generates the learning image in which all or part of the plurality of drugs are in contact with a point or a line,
The correct answer data is a mask image corresponding to the drug area or an edge image showing an edge of the drug area, and the points or lines that touch are replaced with a background color.
Learning data creation device. A learning data creation device comprising a processor and a memory, the processor creating learning data for machine learning,
The processor includes:
an acquisition process for acquiring a photographed image of the drug;
a learning image generation process that generates a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
Correct data generation processing that generates second area information corresponding to the area of the drug in the generated learning image, and sets the generated second area information as correct data for the learning image;
performing storage control to store a pair of the generated learning image and the correct answer data in the memory as learning data;
When generating the learning image including a plurality of drugs, the learning image generation process generates the learning image in which all or part of the plurality of drugs are in contact with a point or a line,
The correct answer data is a mask image corresponding to the region of the drug, and the mask images corresponding to the plurality of drugs that are in contact with the points or lines have different pixel values.
Learning data creation device. A learning data creation device comprising a processor and a memory, the processor creating learning data for machine learning,
The processor includes:
an acquisition process for acquiring a photographed image of the drug;
a learning image generation process that generates a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
Correct data generation processing that generates second area information corresponding to the area of the drug in the generated learning image, and sets the generated second area information as correct data for the learning image;
performing storage control to store a pair of the generated learning image and the correct answer data in the memory as learning data;
The drug is at least partially transparent,
The learning image generation process by the processor includes arbitrarily arranging drugs other than the transparent drug when generating the learning image including a plurality of drugs.
Learning data creation device. The acquisition process of the processor acquires first area information corresponding to a drug area in the acquired captured image;
The correct data generation process generates the second area information based on the acquired first area information.
The learning data creation device according to any one of claims 1 to 4 . The acquisition processing of the processor acquires the photographed images in which a plurality of drugs are photographed, or the plurality of photographed images in which drugs are photographed, and acquires the plurality of first region information corresponding to the regions of the plurality of drugs. get,
The learning data creation device according to claim 5 . The first region information may be region information that manually sets the drug region in the captured image, region information that automatically extracts the drug region in the captured image by image processing, or region information that indicates the drug region in the captured image. The area is automatically extracted by image processing and the area information is manually adjusted.
The learning data creation device according to claim 5 or 6 . The learning image generation process generates the learning image by translating, inverting, rotating, or scaling the photographed image,
The correct data generation process generates the correct data by translating, inverting, rotating, or scaling the first region information in accordance with the photographed image.
The learning data creation device according to claim 5 or 6 . The learning image generation process generates the learning image by combining two or more images obtained by translating, inverting, rotating, or scaling the photographed image,
The correct data generation process generates the correct data by translating, inverting, rotating, or scaling the first region information corresponding to each of the two or more images in accordance with the photographed image.
The learning data creation device according to claim 5 or 6 . The processor includes a drug image acquisition process of acquiring a drug image in which the drug region is cut out from the photographed image based on the acquired first region information,
The learning image generation process involves translating, inverting, rotating, or scaling the acquired drug image to generate the learning image;
The correct data generation process generates the correct data by translating, inverting, rotating, or scaling the first region information in accordance with the drug image.
The learning data creation device according to claim 5 or 6 . The processor includes a drug image acquisition process of acquiring a drug image in which the drug region is cut out from the photographed image based on the acquired first region information,
The learning image generation process generates the learning image by combining two or more images obtained by translating, inverting, rotating, or scaling the obtained drug image,
The correct data generation process generates the correct data by combining two or more pieces of second area information obtained by translating, inverting, rotating, or scaling the first area information corresponding to the drug image.
The learning data creation device according to any one of claims 5, 6, and 10 . The photographed image is an image of a drug handled by the own pharmacy;
The learning data creation device according to any one of claims 1 to 11 . A learning data creation method in which a processor creates learning data for machine learning by processing each of the following steps,
a step of acquiring a photographed image in which the drug is photographed;
generating a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as correct data for the learning image;
a step of storing the pair of the generated learning image and the correct data in a memory as learning data;
In the step of generating the learning image, when arranging the plurality of drugs, all or part of the plurality of drugs are brought into contact with a point or a line,
The correct answer data includes an edge image showing only the points or lines of the plurality of drugs that are in contact with each other.
How to create learning data. A learning data creation method in which a processor creates learning data for machine learning by processing each of the following steps,
a step of acquiring a photographed image in which the drug is photographed;
generating a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as correct data for the learning image;
a step of storing the pair of the generated learning image and the correct data in a memory as learning data;
In the step of generating the learning image, when arranging the plurality of drugs, all or part of the plurality of drugs are brought into contact with a point or a line,
The correct answer data is a mask image corresponding to the drug area or an edge image showing an edge of the drug area, and the points or lines that touch are replaced with a background color.
How to create learning data. A learning data creation method in which a processor creates learning data for machine learning by processing each of the following steps,
a step of acquiring a photographed image in which the drug is photographed;
generating a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as correct data for the learning image;
a step of storing the pair of the generated learning image and the correct data in a memory as learning data;
In the step of generating the learning image, when arranging the plurality of drugs, all or part of the plurality of drugs are brought into contact with a point or a line,
The correct answer data is a mask image corresponding to the region of the drug, and the mask images corresponding to the plurality of drugs that are in contact with the points or lines have different pixel values.
How to create learning data. A learning data creation method in which a processor creates learning data for machine learning by processing each of the following steps,
a step of acquiring a photographed image in which the drug is photographed;
generating a learning image in which the drug is arbitrarily arranged from the acquired photographed image;
generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as correct data for the learning image;
a step of storing the pair of the generated learning image and the correct data in a memory as learning data;
In the step of generating the learning image, when arranging the plurality of drugs, all or part of the plurality of drugs are brought into contact with a point or a line,
The drug is at least partially transparent,
The step of generating the learning image includes arbitrarily arranging drugs other than the transparent drug when generating the learning image including a plurality of drugs.
How to create learning data. A function to obtain images of drugs,
The function is to generate a learning image in which the drugs are arbitrarily arranged from the acquired photographic image, and when generating the learning image including a plurality of drugs, all or a part of the plurality of drugs are A function of generating the learning image that contacts with a point or a line ;
a function of generating second area information corresponding to the area of the drug in the generated learning image, and using the generated second area information as correct data for the learning image;
A computer realizes a function of storing a pair of the generated learning image and the correct answer data in a memory as learning data ,
The correct answer data includes an edge image showing only the points or lines of the plurality of drugs that are in contact with each other.
Learning data creation program. A non-transitory computer-readable recording medium on which the program according to claim 17 is recorded.

Images