JP7329463B2 - drug identification system - Google Patents

Description

TECHNICAL FIELD The present invention relates to a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug.

Mechanical identification of drugs is often required in drug handling facilities such as pharmacies. For example, mechanical identification of drugs is used to determine whether or not drugs arranged based on a prescription actually match the contents of the prescription by an entity other than the operator (pharmacist, etc.). A device (medicine inspection device) may be used. By using such a device, it is possible to prevent operational errors in delivering drugs to patients.

Patent Literature 1 discloses a weighing device applied to a drug inspection device. In this weighing device, in addition to a weighing unit for measuring the weight of the drug to be measured, an imaging means (camera) for capturing an image of the target to be measured is provided, and based on the captured image data, It is determined whether or not the drug to be measured matches the type of drug specified in the prescription.

Japanese Patent No. 5744776

However, in the weighing device disclosed in Patent Literature 1, proper determination cannot be made unless the drug to be measured is imaged under proper imaging conditions. Specifically, it is necessary to position the measurement target in an imaging target region in which the imaging means can properly capture an image of the measurement target from above. For this purpose, a positioning means is provided for positioning the container holding the object to be measured at an appropriate position for measurement. In addition, in order to prevent the object to be measured from being shaded by ambient light, it is necessary to house the container holding the object to be measured and the imaging means together in a housing to shield them from ambient light.

Thus, the weighing device disclosed in Patent Document 1 is provided with a mechanism for placing the drug to be measured under appropriate imaging conditions (position, lighting conditions, etc.). A proper judgment cannot be made. This is because the weighing device of Patent Document 1 discriminates the drug type based on the image data of the drug taken and the image data of the drug as master data, and the master data and imaging conditions are different. This is because image data obtained by imaging cannot be used for proper discrimination.

Therefore, the weighing device disclosed in Patent Document 1 requires work (positioning, housing in a housing, etc.) for placing the drug under appropriate imaging conditions when capturing an image of the drug. In order to use this weighing device, it was necessary to learn a dedicated operation method, and the operation itself was also large and troublesome. For this reason, it has not been possible to casually perform mechanical identification of drugs. In addition, this weighing device is a large-scale device having a dedicated mechanism for placing the drug under appropriate imaging conditions, and cannot be made portable.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to make it possible to identify a drug from images captured under various conditions without placing the drug under specific appropriate imaging conditions.

In order to solve the above problems, a drug identification system as an example of an embodiment according to the present invention is a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug . A learning data storage unit that stores a combination of information and information of a captured image as learning data; and a drug identification unit that identifies a drug included in the captured image . A drug contained in an input photographed image is identified, and the AI model is an AI model created based on the learning data stored in the learning data storage unit, wherein the photographed image is used as input data, and If the AI model returns information on the corresponding drug as output data, and an identification index indicating identification information of the drug is written on the drug or on the drug package that packages the drug, the drug identification unit is configured such that the identification index is A drug is identified based on the indicated drug identification information, drug imaging example data is created by combining information on the identified drug and information on the photographed image at that time, and the drug imaging example data is stored as the learning data. When the learning data storage unit receives the drug imaging example data from the drug identification unit, the learning data storage unit stores a combination of drug information and photographed image information included in the drug imaging example data, It is characterized by updating learning data .

Further preferably , the drug identification unit detects the identification index from the captured image by image analysis separately from the AI model, and the inclusion of the identification index in the captured image is detected by the image analysis . In this case, the drug identification unit identifies the drug based on the identification information of the drug corresponding to the identification index detected by the image analysis , and combines the information of the identified drug with the information of the captured image at that time. It is preferable to create drug imaging example data.

Also preferably, a main device having the learning data storage unit, and a plurality of local devices each having the drug identification unit and capable of accessing the main device via a network, wherein the local device When the drug photography example data is created in the identification of the drug performed by Preferably, the learning data stored in the learning data storage unit is updated based on.

Further preferably, the main device has a main AI model, and the main AI model is an AI model created based on learning data stored in the learning data storage unit of the main device. is an AI model that uses a photographed image as input data and returns corresponding drug information as output data, and the main device recreates the main AI model when the learning data is updated, and the local device incorporates, as the AI model used by the drug identification unit, a local AI model that uses a captured image as input data and returns corresponding drug information as output data. data received from the main device, data constituting the local AI model is data of the main AI model at the time when the local device receives data from the main device, and the local device The drug identification unit of uses the local AI model to identify the drug contained in the captured image, and if the drug contained in the captured image cannot be identified even using the local AI model, via a network The main device may be accessed and the drug may be identified using the main AI model of the main device.

Preferably, when the photographed drug is a medical drug, an optical mark written on the drug package of the medical drug and indicating information about the medical drug may be used as the identification index.

Further, preferably, the drug identification system is used in a system including a database that stores prescription information input at the discretion of a doctor, and information on one or more drugs identified by the drug identification unit is It is good to judge whether it agrees with the prescription information.

Also preferably, the drug identification system is used in a system including a database that stores inventory information that varies according to information on drugs received and shipped from a pharmacy, and the drug identification unit identifies It may be determined whether the information for one or more drugs matches the inventory information.

Also preferably, the drug identification system is used in a system including a database for managing a medication schedule instructed by a medical institution for drugs dispensed to a patient, and the one identified by the drug identification unit It is preferable to determine whether or not the above drug information matches the drug administration schedule.

Preferably, the learning data storage unit and the drug identification unit are configured as separate devices, and the drug identification unit can access the learning data storage unit via a network.

According to the medicine identification system as an example of the embodiment of the present invention, information about how various medicines appear in photographed images, that is, various photographed images and the The correspondence between drug information (type and quantity) is stored (accumulated) in the learning data storage unit as learning data. Then, the AI model created based on the learning data is used to identify the drug contained in the input photographed image. Here, if the AI model is created from learning data that has been sufficiently accumulated (capturing a large amount of captured images), it can be used from images captured under various shooting conditions (lighting conditions, positions, distances, angles, etc.). , the information (type and quantity) of the drug shown in the image can be properly identified. Therefore, the drug does not have to be placed under specific proper imaging conditions. Therefore, an operator who intends to identify the medicine does not need to perform work for placing the medicine under appropriate imaging conditions, and can easily identify the medicine.

When an identification index is written on the medicine or medicine package, the medicine identification unit identifies the medicine based on the identification information indicated by the identification index, and furthermore, information on the identified medicine and information on the photographed image at that time. If the learning data is updated based on drug photography example data combined with The learning data can be appropriately updated (accumulated) in association with correct information as to whether or not the drug is contained. For this reason, even a person without specialized knowledge (for example, a patient who received a drug dispensed at a pharmacy and who is neither an AI engineer nor a medical professional) can obtain information about what is the correct answer. It can contribute to accumulation of learning data including so-called teacher data. Therefore, in this drug identification system, it is possible to capture images taken by people who do not have expert knowledge of AI and drugs, and accumulate learning data on drugs. The learning data accumulation speed increases as the number of people who perform training increases. In addition, the accuracy of the AI model created based on the learning data will improve as the number of system users increases.

Further, when the AI model determines that the captured image includes the identification index, the identification of the drug and the drug imaging example data are performed based on the identification information of the drug corresponding to the identification index determined by the AI model. If created, drug photography example data can be obtained simply by photographing drugs, and learning data can be updated more efficiently.

In addition, when the drug identification system is a system having a plurality of local devices that can access the main device via a network, the local devices function as devices that can be carried by individuals, such as drug identification units. By using a smartphone on which a predetermined application is installed, it becomes possible to collect drug photography example data from a large number of local devices, further improving the accumulation speed of learning data.

In addition, when identifying a drug contained in a captured image using a local AI model incorporated in the local device, the local device does not need to access the main device if the drug can be identified by the local AI model. Therefore, the frequency of communication via the network can be minimized, the average speed of drug identification can be improved, and the communication volume of the entire system can be suppressed.

In addition, when the drug photographed is a prescription drug, by using the optical mark written on the drug packaging of the prescription drug, which indicates information about the drug, as an identification index, the packaging of the prescription drug It is possible to identify a drug using an identification index such as a bar code that is required to be displayed, for which the encoding method of drug information is predefined, and the accuracy of drug identification based on the identification index improves.

Also, if the photographed drug is a drug dispensed based on the prescription information input by the doctor's judgment, whether the information of one or more drugs identified by the drug identification unit matches the prescription information By determining whether the prescription is correct or not, it is possible to prevent delivery of a drug different from the content of the prescription when delivering the drug to a patient at a dispensing pharmacy or the like.

Further, when the photographed drug is a drug managed based on inventory information that varies according to information on received and delivered drugs at the pharmacy, one or more drugs identified by the drug identification unit. By judging whether or not the information matches the inventory information, it is possible to collate the inventory information with the information on drugs actually present in the pharmacy when taking an inventory of the pharmacy.

Also, if the photographed drug is a drug that has been dispensed for a patient and should be taken according to a drug administration schedule instructed by a medical institution, one or more of the drugs identified by the drug identification unit By judging whether or not the drug information matches the medication schedule, it is possible to confirm whether the patient is taking the medication correctly according to the medication schedule.

In addition, if the learning data storage unit and the drug identification unit are configured by separate devices, and the drug identification unit can access the learning data storage unit via a network, the system can be freely configured. degree increases. For example, it becomes possible to provide the learning data storage unit and the drug identification unit at distant locations, or to downsize each device.

1 is a block diagram schematically showing the configuration of a drug identification system as an example of an embodiment according to the present invention; FIG. The figure which shows the tablet simple substance in which the identification index was written. FIG. 2B is a diagram showing a PTP sheet with an identification index for packaging the tablet of FIG. 2A. The figure which shows the whole PTP sheet of FIG. 2B. The figure which shows the back side of the PTP sheet of FIG. 2C. FIG. 1 shows an individual package of a powder, marked with an identification index. The figure which shows the PTP sheet with which the identification index was written which packages a capsule. FIG. 4 is a diagram showing an example of an identification index table; The figure which shows an example of drug imaging|photography example data. FIG. 2 is a flow chart showing procedures for drug identification and learning data update in the drug identification system of FIG. 1; FIG. 1 is a block diagram schematically showing the configuration of a drug identification system in which a plurality of local devices access a main device via a network; FIG. The figure which shows roughly the structure of a medicine identification sensor.

The block diagram of FIG. 1 schematically shows the configuration of a drug identification system 10 as an example of an embodiment according to the present invention. As examples of drugs 12 to be identified in this drug identification system 10, FIG. 1 shows a single tablet 12a and a tablet PTP sheet 12b (10 tablets per sheet).

A drug identification system 10 shown in FIG. 1 includes a monitor 14 for displaying identification results, a camera 20 for capturing an image of a drug 12, and a drug 12 included in a captured image 22 captured by the camera 20. A drug identification unit 30, an identification index table storage unit 40 that stores an identification index table 42 (FIG. 3), which will be described later, and an AI model 50 that uses the photographed image 22 as input data and returns information on the corresponding drug 12 as output data. As an entity, it includes a storage unit that stores data of the AI model 50) and a learning data storage unit 60 that stores learning data 62 that is the basis for creating the AI model 50. FIG. These components may be integrated into one device, or may be divided and arranged in a plurality of devices. For example, the drug identification system 10 may be configured in one smart phone with the screen of the smart phone as the monitor 14, the camera incorporated in the smart phone as the imaging device 20, and the memory and processor as other components. Alternatively, the drug identification system 10 is built on a network (such as the Internet), and each of a plurality of devices connected via the network functions as a component, The configuration may be such that the drug identification system 10 can be used from devices at various locations.

(basic operation)
A user (operator) who intends to perform identification work using this drug identification system 10 first captures the drug 12 (the area containing the drug 12 ) with the imaging device 20 to obtain a captured image 22 . Then, this photographed image 22 is input to the medicine identifying section 30 . The drug identification unit 30 performs predetermined processing (such as conversion into a data format accepted by the AI model 50) for inputting the captured image 22 to the AI model 50, and then transfers the captured image 22 to the AI model 50 as input data. hand over as

The AI model 50 used in the drug identification system 10 is a model that describes the input/output relationship such that when some input data is received, output data corresponding to the input data is returned. have a job Specifically, when the AI model 50 receives the captured image 22 as input data, the AI model 50 returns information about the drug 12 shown in (included in) the captured image 22 as output data. This AI model 50 can output various information related to the drug 12, such as product name, dosage form, active ingredient, efficacy, etc., as output data. Here, for the sake of explanation, it is assumed that the product name of the drug 12 and the quantity appearing in the photographed image 22 are output as output data.

This AI model 50 is created based on the learning data 62 stored in the learning data storage unit 60 . The learning data 62 is obtained by accumulating a plurality of example data, each of which is a set of example data including the photographed image 22 photographed in the past and information about the drug 12 shown in the photographed image 22. is. Here, in the photographed images 22 photographed in the past, the medicine 12 is in various states (the position, distance, angle, packaging state, lighting conditions at the time of photographing, resolution, etc. of the medicine 12 are different for each photographed image 22). in) is in the picture. In other words, the learning data storage unit 60 stores, as the learning data 62, how the various medicines 12 appear in the photographed images 22, respectively.

The AI model 50 created based on such learning data 62 uses as input data the photographed images 22 photographed under various photographing conditions, and accurately outputs the information of the drug 12 contained in the photographed images 22. It is possible to

Then, the drug identification unit 30 receives the output data from the AI model 50, and the information (here, product name and quantity) of the drug 12 identified in the captured image 22 identified by the AI model 50 is used as the identification result on the monitor 14. to display. At this time, if multiple types of drugs 12 are shown in the photographed image 22 , the drug identification unit 30 can identify all of the multiple types of drugs 12 . In addition, it is also possible to identify the quantity of the medicines 12 of the same kind shown in the photographed image 22 . The monitor 14 can display information on the type (for example, product name) and quantity for each of the multiple types of medicines 12 shown in the captured image 22 . For example, when the drug 12 to be identified is the single tablet 12a and the tablet PTP sheet 12b shown in FIG. 1, the product name of the tablet 12a is "ExampleX 5mg" and the product name of the tablet PTP sheet 12b is "ExampleA 1mg". Then, the monitor 14 displays information such as "<Drug 1: Product name/ExampleX 5 mg, quantity/1 tablet>-<Drug 2: Product name/ExampleA 1 mg, quantity/1 sheet>". The display of such identification results may be displayed together with the captured image 22 . In particular, the medicine in the captured image 22 and the identification result display of the medicine may be superimposed and displayed as a kind of AR image.

(Regarding identification index)
The above is the basic operation of the drug identification system 10. In the drug identification system 10 of the present embodiment, in addition to identification by the AI model 50, it is possible to identify the drug 12 using the "identification index". It's becoming The identification index is written in the form of engraving, printing, or the like on the drug 12 or the drug packaging that packages the drug 12, and indicates what the drug 12 is, that is, identification information of the drug 12. As an identification index, an optical mark such as a bar code or two-dimensional code (which displays encoded information by combining multiple parts with different optical properties such as light refractive index and reflection spectrum). In addition, a display in which the product name itself or an identification code (for example, a symbol string consisting of a company code of a pharmaceutical company and a product code determined by the company) is written as a character string/symbol string can be used. Although the information to be indicated is recorded as it is (the information is not coded), such display can also be used as an identification index. Specifically, an identification code stamped on the drug 12 itself, a character string printed with the product name of the drug 12 directly on the drug package (for example, "ExampleA 1mg"), or a barcode printed on the drug package ( For example, the GS1 data bar specified in Japanese Industrial Standard X0509, which is required to be displayed on prescription drugs, can be used as an identification index.

2A-2F show specific examples of identification indices. In addition to these identification indicators, drugs and drug packaging may display the corporate logo of the pharmaceutical company, the recycling type mark on the packaging, efficacy, and directions for use, but these are omitted here for identification. Shows indicators only. FIG. 2A shows a single tablet 70a, and this tablet 70a has an identification code 71a "XY123" stamped on its surface. FIG. 2B shows a PTP sheet piece 73a from which a portion (for two tablets) of the PTP sheet packaging the tablets 70a in FIG. 2A has been cut off. An identification code 71a is printed nearby, and a product name 72a of "ExampleA 1mg" is printed as it is as a character string.

FIG. 2C shows the entire PTP sheet 75a (10 tablets) before the PTP sheet piece 73a of FIG. 2B is cut off. As in FIG. 2B, an identification code 71a and a product name 72a are printed near the tablet 70a, and the product name 72a is also printed on the edge of the PTP sheet 75a (upper edge in the drawing).

FIG. 2D shows the back side of the PTP sheet 75a shown in FIG. 2C. On the back side, a product name 72a is printed for each area of two tablets separated by a separation line, and a bar code 76a is printed on the edge of the PTP sheet 75a (lower edge in the drawing). there is The bar code 76a represents a number with a predetermined number of digits by combining lines and blanks. The number represented by the GS1 data bar, which is required to be written on prescription drugs, is usually 10 digits or more, but for the sake of explanation, the bar code 76a represents a 6-digit number. .

FIG. 2E shows an individual package 73b in which a powder 70b (powder medicine) is packaged by a nylon SP package (Strip Package). The identification code 71b (“ZZ789”) of the powder 70b, the product name 72b (“ExampleB 10%”), and the bar code 76b are printed on this individual package 73b.

FIG. 2F shows a PTP sheet piece 73c from which a portion (for three tablets) of the PTP sheet packaging the capsule 70c has been cut off. ”), a product name 72c (“ExampleC 200”), and a bar code 76c are printed.

Each piece of identification information indicated by the identification index as shown in FIGS. 2A to 2F is associated with drug information (information on drugs). Note that the product names 72a, 72b, and 72c are identification indexes and can be said to represent drug information themselves. The correspondence between identification information and drug information is managed by the drug identification system 10, and in this embodiment, the correspondence is stored in the identification index table storage unit 40 as an identification index table 42 as shown in FIG. there is

FIG. 3 shows an example of the identification index table 42. As shown in FIG. 2A-2D, the powder 70b of FIG. 2E, and the capsule 70c of FIG. 2F, the corresponding relationship between identification information and drug information is shown. For example, for the tablet 70a, the number "123456" represented by the bar code 76a and the identification code 71a "XY123" are associated as the identification information with the product name 72a "ExampleA 1 mg", which is the drug information (in the table, data belonging to the same row, i.e. the same record). Similarly, for the powder 70b, the product name 72b "ExampleB 10%" is associated with the number "456789" represented by the bar code 76b and the identification code 71b "ZZ789" and stored. For the capsule 70c, the product name 72c "ExampleC 200" is associated with the number "024680" indicated by the bar code 76c and the identification code 71c "VW456" and stored. In addition to this information, the identification index table 42 can include various information related to drugs such as "dosage form" (the shape of the drug) and "packaging" (how it is packaged).

The identification index table 42 naturally includes information on a large number of drugs other than tablets 70a, powders 70b, and capsules 70c. Especially for pharmaceuticals, it is desirable to include information on all products that can be identified. Such an identification index table 42 is prepared independently of the AI model 50 of FIG. Created on the basis of

As an example of drugs other than tablets 70a, powders 70b, and capsules 70c, the identification index table 42 shown in FIG. This drug "ExampleX 5 mg" is a bottled tablet, as indicated by the "packaging" of "100 tablets per bottle". As an identification index for the drug "ExampleX 5 mg", a bar code indicating the number "975310" is attached to the bottle, but the bar code is not attached to the drug in units of one tablet. On the other hand, a character string with the identification code "NM321" is engraved on each tablet. The identification index table 42 can also include information on various forms of drugs, such as ointments in tubes and liquid drugs in ampoules.

(Identification using identification index)
The drug identification system 10 (FIG. 1) of the present embodiment uses an identification index as shown in FIGS. 2A to 2F and an identification index table 42 as shown in FIG. It is also possible to identify drugs 12 contained in .

That is, when an identification index is written on the medicine 12 to be imaged or on the medicine package, the medicine identification system 10 does not use the output data of the AI model 50 as the identification result as it is, but rather the identification information indicated by the identification index. Identification results can be obtained by specifying drug information from the For example, in addition to the AI model 50, if the drug identification unit 30 has a detection function for detecting an identification index made up of characters or barcodes from the photographed image 22 by image analysis, the identification index obtained by the detection function By comparing the identification information indicated by , with the identification index table 42, information on the drug 12 can be obtained without using the AI model 50. FIG.

In other words, the detection and reading of the identification index may be performed by an image analysis program separate from the AI model 50. An image analysis program that specializes in detecting a part where an identification index (barcode, character string, etc.) exists and reading it by focusing on only that part is compared to the AI model 50 that identifies from the entire appearance of the identification target. Therefore, depending on the photographing conditions (for example, when the overall lighting is dark but only the barcode portion is bright), identification with higher accuracy than the AI model 50 may be possible.

However, the AI model 50 can also detect the identification index included in the captured image 22 . That is, the AI model 50 can identify the drug 12 shown in the photographed image 22 based on the appearance (contour shape, color, etc.) other than the identification index, and can also determine the presence or absence of the identification index. When the AI model 50 determines that the identification index is included in the photographed image 22, the AI model 50 notifies the identification information of the medicine 12 corresponding to the identification index to the medicine identifying section 30. FIG. Then, the medicine identification unit 30 compares the notified identification information with the identification index table 42 to identify the medicine 12 based on the identification index while using the AI model 50 .

When the information of the drug 12 is obtained based on the identification information as described above, the information of the drug 12 obtained based on the identification information is displayed on the monitor 14 as the identification result, and the identification result and the photographed image are displayed. 22 are combined and transmitted to the learning data storage unit 60 . Specifically, drug imaging example data 32a and 32b as shown in FIG. In this case, that is, when the identification index is detected by the image analysis program or the AI model 50 , the identification based on the identification information indicated by the identification index is prioritized over the identification result by the AI model 50 .

(Update learning data)
The drug imaging example data 32a and 32b include information on the drug 12 identified by the drug identification unit 30 based on the identification information, and a photographed image 22 of the drug 12 to be identified at that time. For example, if the objects to be identified are one tablet 12a alone and one tablet PTP sheet 12b as shown in FIG. 1, as shown in FIG. A drug identification unit stores drug photographing example data 32a representing information "a photographed image showing a tablet" and drug photographing example data 32b representing information "a photographed image showing one tablet of ExampleX 5 mg drug with product name". 30 creates and transmits these drug imaging example data 32 a and 32 b to the learning data storage unit 60 . Here, the photographed image 22 is divided into drug photographed example data 32a showing "one sheet of Example A 1 mg" and drug photographed example data 32b showing "one tablet of Example X 5 mg". may be sent to the learning data storage unit 60 as drug imaging example data in which "Example A 1 mg is 1 sheet and Example X 5 mg is 1 tablet".

Upon receiving the drug imaging example data 32a and 32b from the drug identification unit 30, the learning data storage unit 60 updates the learning data 62 based on the drug imaging example data 32a and 32b. Specifically, the combination of the information of the medicine 12 (what kind of medicine was photographed) included in the medicine photographing example data 32a and 32b and the information of the photographed image 22 (what kind of image it is) is , ``this drug 12 (here, ``ExampleA 1mg'' and ``ExampleX 5mg'') may appear in such a form in the photographed image 22" is stored as an example.

After the learning data 62 is updated, the AI model 50 is recreated (educated) at a predetermined timing based on the updated learning data 62 . This re-creation is desirably performed at a timing that does not interfere with the operation of the drug identification system 10 . For example, if the learning data storage unit 60 itself can recreate the AI model 50 and the recreating does not take much time, the AI model 50 can be recreated every time the learning data 62 is updated. good. On the other hand, if the time required to re-create the AI model 50 is long and the drug identification system 10 becomes unavailable during the re-creation work, the drug identification system 10 is not operating (the system is stopped). ), the AI model 50 is preferably recreated. For example, for re-creation, it is necessary to connect an external device (device different from the components shown in FIG. 1) that analyzes the learning data 62 to the drug identification system 10, and the AI model 50 cannot be used during that time. In such a case, it is desirable to re-create it while the system is stopped. In addition to this, the timing of re-creation can be set variously, for example, when a predetermined period has passed since the previous re-creation, when learning data 62 has accumulated to some extent (learning data 62 is updated after a predetermined The timing may be, for example, when the update has been performed a number of times, or when the amount of data added by the update exceeds a predetermined amount.

The AI model 50 recreated based on the updated learning data 62 has a higher ability to identify the drug 12 appearing in the captured image 22 than the AI model 50 before recreation. For example, in the drug photographing example data 32a in FIG. 4, the back side of the PTP sheet is shown, and the figure of the drug itself is not shown. It has become. Therefore, based on the identification information indicated by the bar code, the drug identification unit 30 can identify that the photographed image contains "ExampleA 1mg". When the learning data 62 is updated using the medicine photographing example data 32a at this time, and the AI model 50 is recreated (educated) based on it, the AI model 50 will be able to store the medicine alone in the input photographed image from now on. Even if the product name does not appear and it is difficult to read the product name, it is possible to identify that "ExampleA 1mg" is captured in the photographed image similar to the medicine photographed example data 32a.

In addition, by accumulating a large amount of medicine photographing example data 32a and 32b regarding individual medicines 12a and 12b, the AI model 50 in which the information is reflected is able to: It becomes possible to identify that the individual medicines 12a and 12b are shown therein. In other words, it is possible to identify a plurality of drugs 12 at the same time by collecting the drug imaging example data 32a and 32b regarding individual drugs 12a and 12b. On the other hand, even if the photographed image 22 is not segmented as medicine photographing example data 32a and 32b relating to individual medicines 12a and 12b, and is accumulated in the form of "pharmaceutical photographing example data showing a plurality of medicines". , it is possible to accurately and quickly identify a plurality of medicines in photographed images similar to the medicine photographing example data.

(Procedure for identification by drug identification system)
FIG. 5 shows a flow chart of procedures for identifying the drug 12 and updating the learning data 62 performed by the drug identification system 10 of FIG. First, the photographed image 22 photographed by the user is input to the medicine identifying section 30 (step S01). The drug identification unit 30 starts image recognition of the input photographed image 22 (step S02), and determines whether or not the photographed image 22 includes an identification index by the AI model 50 or a different detection function. (Step S03).

If the photographed image 22 contains an identification index (step S03-YES), the drug identification unit 30 extracts the photographed image from the identification index table 42 of the identification index table storage unit 40 based on the identification information indicated by the identification index. Information (drug information) on the drug 12 in the image 22 is obtained (step S04), and the drug information is used as the identification result. Based on the captured image 22 and the obtained drug information, the drug identification unit 30 creates drug imaging example data 32a and 32b (step S05), and transmits it to the learning data storage unit 60 (step S06). The learning data storage unit 60 that has received the drug imaging example data 32a and 32b updates the learning data 62 based on the drug imaging example data 32a and 32b (step S07).

On the other hand, if no identification index is included in the photographed image 22 (step S03-NO), the medicine information returned as output data by the AI model 50 that received the photographed image 22 as input data is used as the identification result (step S08). ).

Then, drug identification unit 30 transmits the identification result obtained in step S04 or step S08 to monitor . The monitor 14 that has received the identification result displays the identification result to the user (step S09).

As described above, the identification result of the drug 12 taken by the user is displayed on the monitor 14, and if the identification index is included in the captured image 22, the learning data 62 is updated. According to this drug identification system 10, an AI model 50 is created based on learning data 62 in which captured images 22 captured under various past conditions are accumulated, and identification is performed using the AI model 50. . Therefore, the medicine identification system 10 can identify medicines 12 in various postures (for example, medicines 12 that are obliquely oriented, such as the tablet PTP sheet 12b shown in FIG. 1, or medicine packages containing the medicines) without changing their postures. It can be performed. Therefore, identification is possible without performing the work of placing the medicine 12 under appropriate imaging conditions, which is performed by the conventional weighing device described in Patent Document 1, so that the user can easily identify the medicine. can. To give an example of "putting in proper imaging conditions", in the prior art, the background color of the drug 12 in the captured image 22 had to be uniformed to a specific color (typically black). This is the detection of ``from where to where is the drug in the image'', that is, to detect the contour (edge) of the drug. was required to be able to be detected as an area in which the drug exists. On the other hand, a well-educated AI model 50, such as that used in this embodiment, will be ) edges can be detected, and the task of unifying the background color (for example, the task of arranging the medicine 12 on a black solid mat prepared in advance) is not required.

Further, if the photographed image 22 contains an identification index, the learning data 62 is updated simply by photographing the drug 12 as the user's work. Information) can contribute to updating (accumulating) the learning data 62 . In addition, it is possible to accumulate the photographed images 22 photographed in the process of drug identification as part of the learning data 62, and the accuracy of identification by the AI model 50 can be improved by repeatedly performing drug identification. is possible.

In addition, even if the identification index is not detected, the AI model 50 can still identify the medicine 12. For example, the AI model 50 can detect the appearance of the medicine 12 for which the product name and bar code are unknown due to being dispensed in one package. , the identification can be done on the basis of contours. In addition, since it is possible to use the product code imprinted on the drug 12 alone as an identification index, it is possible to identify the drug 12 alone taken out of the drug package, and also to identify drugs other than medical drugs. , it can also identify over-the-counter drugs that do not have barcodes. Since the identification code is determined independently by the manufacturer, the same identification code may be assigned to different drugs. By combining the reading of and shape identification by the AI model 50, highly accurate identification can be performed.

(system via network)
FIG. 6 shows a drug identification system 80 in which multiple local devices 86 access a main device 84 via network 82 . In this drug identification system 80, it is the local device 86 that the user directly operates. The local devices 86 may be devices of different forms, and FIG. 6 shows a smartphone 86a, a dispensing inspection device 86b (equipment installed in a dispensing pharmacy or the like), and a desktop PC 86c as examples of the local devices 86. It is In the drawing, each of these local devices 86 is connected to the network 82 by a line, but in reality, access to the main device 84 may be performed by wireless communication.

A main device 84 is also connected to the network 82 . The main device 84 is a target device (such as a server device) to which each local device 86 accesses via the network 82. As shown in FIG. and a main AI model 52 created based on the data 62 . The main AI model 52, like the AI model 50 in the drug identification system 10 shown in FIG. , has a function of using a photographed image as input data and returning corresponding drug information as output data.

Each local device 86 has a function corresponding to the drug identification unit 30 in the drug identification system 10 of FIG. For example, by installing an identification application that provides a drug identification function in the local device 86 , it becomes possible to use the calculator (CPU or processor) of the local device 86 as the drug identification unit 30 . Also, each local device 86 incorporates a local AI model 56 . For example, if the identification application described above contains data that configures the local AI model 56 , the local device 86 will be in a state in which the local AI model 56 is incorporated. The data constituting the local AI model 56 can be obtained by receiving the current data of the main AI model 52 from the main device 84 at the time of installation or update of the identification application, for example.

As with the main AI model 52, the local AI model 56 also has a function of returning a photographed image as input data and corresponding drug information as output data. Although not shown in FIG. 6, each local device 86 also has a function of taking an image of the drug. For example, in the case of the smart phone 86a and the dispensing inspection device 86b, the cameras incorporated therein are used, and in the case of the desktop PC 86c, the camera externally attached via a USB connection or the like serves as means for obtaining the photographed image of the medicine.

By using the local AI model 56, each local device 86 can identify the drug shown (included) in the photographed image taken by itself, and whether the photographed image contains an identification index. If the identification index is included, it is possible to identify the medicine based on the identification information indicated by the identification index. Hereinafter, as an example, the function of the drug identification system 80 when the smartphone 86a functions as the local device 86 will be described. Drug identification is basically performed in the same flow as the flow chart of FIG.

As a result of the user taking a photographed image of the drug with the camera of the smartphone 86a and inputting it to the local AI model 56, if the local AI model 56 returns drug information as output data, the drug information is used as the identification result on the smartphone. It is displayed on the screen of 86a. Further, when it is determined that the captured image contains an identification index, the medicine is identified based on the identification information corresponding to the identification index. In the identification of drugs based on this identification information, if data corresponding to the identification index table 42 of FIG. 3 is installed in the smartphone 86a together with an identification application, the data can be used to obtain drug information corresponding to the identification information. can be done. Alternatively, if a drug DB 88a (DB: database) containing drug information on various drugs as shown in FIG. , the smartphone 86a may access the drug DB 88a via the network 82 and acquire information about the taken drug from the data in the drug DB 88a.

When the drug is identified using the identification index, the smartphone 86a creates drug imaging example data by combining the captured image at that time and the identification result (information on the identified drug), and transmits the data via the network 82. to the main device 84. It should be noted that this transmission does not have to be performed immediately after identifying the drug, and may be performed when there is an opportunity to access the main device 84 . The main device 84 updates the learning data 62 stored in its own learning data storage unit 60 based on the drug imaging example data transmitted from the smartphone 86a (local device 86). After the learning data 62 is updated, the main AI model 52 is recreated (educated) at a predetermined timing. After that, when a request to update the local AI model 56 is received from the smartphone 86a, copy data of the recreated main AI model 52 is transmitted to the smartphone 86a.

By the way, the main AI model 52 improves in identification ability as the learning data 62 is updated (accumulated), but the identification ability of the local AI model 56 remains constant unless the model itself is updated. Therefore, even if the main AI model 52 can identify what appears in the input photographed image, a situation may occur in which the local AI model 56 cannot identify it. Therefore, when the identification by the local AI model 56 cannot identify the drug contained in the captured image (the type of drug contained in the captured image cannot be identified), the smartphone 86a is connected to the main device 84 via the network 82. , and transmits the photographed image at that time to the main device 84 to make an "inquiry" about the drug shown in the photographed image. The main device 84 that has received the "inquiry" inputs the photographed image at that time to the main AI model 52, identifies the drug shown in the photographed image, and transmits the identification result to the smartphone 86a. "Answer". In this way, it is preferable that the smart phone 86a, which is the local device 86, be able to identify drugs using the main AI model 52 via the network 82.

When other devices such as the dispensing inspection device 86b and the desktop PC 86c work as the local device 86, the same operation as the smart phone 86a is performed. In this way, when the drug identification system 80 has a main device 84 and a plurality of local devices 86 connected via a network 82, the main device 84 is a multitude of local It is possible to collect drug imaging example data from the equipment 86, and it is expected that the accumulation speed of the learning data 62 will be improved. When collecting drug photography example data from a large number of devices in this way, in order to ensure data consistency and improve the efficiency of data analysis, the local device 86 stores the drug photography example data as the date and time of photography, the photography location, and the user. may be sent to the main device 84 by adding a terminal identification number or the like.

Further, when identifying a drug contained in a photographed image using the local AI model 56 incorporated in the local device 86, if the drug can be identified only by the local AI model 56, the local device 86 is the main device 84 Since there is no need to access the network 82, the frequency of communication over the network 82 can be minimized, increasing the average speed of drug identification and reducing overall system traffic.

It should be noted that each local device 86 may be configured to have means for obtaining a captured image, a communication function for accessing the network 82, and the drug identification unit 30, but not the local AI model 56. In this case, each local device 86 transmits the captured image to the main device 84 and uses the AI model 52 of the main device 84 to identify the medicine, in the same manner as in the above-described "inquiry". The main device 84 that has received the captured image passes output data resulting from identification by the AI model 52 regarding the captured image to each local device 86 , thereby identifying the drug in each local device 86 .

Various components other than the main device 84 and the local device 86 can be connected to the network 82 . FIG. 6 shows a state in which a prescription DB 88b, an inventory DB 88c, and a medication management DB 88d are connected to the network 82 as an example.

(Dispensing inspection)
The prescription DB 88b is a database that stores prescription information that is input according to the doctor's judgment. Prescription information is information on drugs that a doctor has judged to be prescribed to a patient as a result of a doctor's diagnosis of the patient at a medical institution such as a hospital. In a so-called dispensing pharmacy, when a patient submits a document (prescription) describing prescription information, a drug is dispensed and delivered to the patient based on the prescription information. At this time, if the prescription DB 88b is connected to the network 82 in the work of "prescription inspection" to confirm whether the drug to be delivered to the patient really matches the content described in the prescription, the drug identification system 80 are available. That is, the drug to be delivered to the patient is photographed by the smartphone 86a or the camera of the dispensing inspection device 86b, etc., the drug to be delivered is identified, and the identification result is collated with the prescription DB 88b. It is possible to check whether the drug dispensed by the patient really matches the prescription information. Here, when there are a plurality of drugs to be dispensed, the drug identification system 80 collectively identifies the plurality of drugs to determine whether the information of the plurality of drugs matches the prescription information in terms of type and quantity. is preferably determined. Alternatively, the drug identification system 80 may perform identification in a plurality of times and determine whether or not the synthesis of the identification results matches the prescription information.

(Inventory control)
The inventory DB 88c is a database that stores inventory information that fluctuates according to information on drugs received and information on drugs delivered from the pharmacy. In other words, inventory information is information that indicates what drugs should be left in the pharmacy at present. If the inventory DB 88c is connected to the network 82, the medicine identification system 80 can be used in the work of checking what kind of medicines are currently left in the pharmacy, so-called inventory. For example, every time a medicine left in the pharmacy is taken out, it is photographed by the camera of the smartphone 86a or the like to be identified, and when all the medicines are taken out, it is determined whether or not the synthesis of the identification results up to that point matches the inventory information. By making the determination, it is possible to check whether there is any discrepancy between the inventory information and the drugs actually remaining. At this time, if the number of drugs actually remaining is less than the inventory information, there is a possibility that the drugs have been lost. There is a possibility of forgetting to deliver the medicine that should be taken. By utilizing the drug identification system 80, it is possible to determine the likelihood that such an eventuality has occurred.

(Medication management)
The medication management DB 88d is a database for managing medication schedules instructed by medical institutions for medications dispensed for patients. Specifically, for each type of dispensed drug, the date and time when the drug was dispensed, the total amount at the time of dispensation, the amount to be consumed per dose, and the desired frequency of taking the drug (after each meal, etc.) is recorded, and based on this information, it is possible to estimate how much of the dispensed medicine should remain (how much has been consumed) at the current time. If the medication management DB 88d is connected to the network 82, it is possible to check the patient's medication compliance status and improve medication adherence. That is, at home nursing care sites, etc., the medicines remaining in the patient's hand are photographed and identified, and whether or not the identification result matches the current number of remaining medicines estimated from the medication management DB 88d is determined. By judging, it is possible to confirm whether or not the patient is taking the medicine according to the dosing schedule. In addition, if the patient is not taking the medication according to the medication schedule, the staff of the medical institution should ask the patient why they are not following the medication schedule, and if there is any reason why they cannot follow the medication schedule, By reviewing the medication schedule according to the time of day, improvement of medication adherence is expected.

(Billing process)
An administrator who operates a drug identification system 80 configured via a network 82 as shown in FIG. ) is also possible. For example, when performing identification using the main AI model 52, when updating the local AI model 56, when accessing the prescription DB 88b, the inventory DB 88c, and the medication management DB 88d, the user attempts to use resources on the network 82. At that time, it is possible to permit the use of resources only to users who have agreed to pay a fee set according to the resource usage mode.

(identification sensor)
In the drug identification system 10 of the present invention as shown in FIG. The drug identification accuracy of the model 50 also improves as the drug identification system 10 is used. As a result, by incorporating the AI model 50 with improved drug identification accuracy to some extent into the drug identification sensor, it is possible to construct a drug identification sensor with high drug identification accuracy.

An example of a drug identification sensor 90 is shown schematically in FIG. This drug identification sensor 90 includes an arithmetic element 93 (CPU, processor, PLD, etc.) and a memory element 95 (memory element, etc.) arranged on a substrate 91, and an image capturing device connected to the substrate 91 via a flexible cable 92. It has an element 94 (such as a camera) and a connector 98 for connecting the board 91 and an external device.

The data of the AI model 50 created (sufficiently trained) based on the learning data 62 sufficiently accumulated by repeated drug identification in the drug identification system 10 of FIG. be done. In the drug identification sensor 90, the computing element 93 and the memory element 95 function as a processing section in which the AI model 50 is incorporated. The medicine identification sensor 90 can identify medicines contained in the photographed image by performing image recognition processing on the photographed image photographed by the image photographing device 94 in the processing unit. Specifically, a photographed image may be passed to the AI model 50 as input data, and drug information returned by the AI model 50 as output data may be used as the identification result. The identification result can be displayed on a monitor or the like connected to the connector 98 .

Since the drug identification sensor 90 of FIG. 7 incorporates the AI model 50 created based on the learning data 62 sufficiently accumulated in the drug identification system 10 of FIG. 1, the drug can be identified with extremely high accuracy. It can be carried out. Also, since the minimum components constituting the drug identification sensor 90 are only the image capturing element 94 and the processing section (storage element 95 and arithmetic element 93), the dimensions of the drug identification sensor 90 should be very compact. can be done.

It is also possible to provide the drug identification sensor 90 with a learning data storage unit 60 as shown in FIG. Furthermore, by connecting the drug identification sensor 90 to the network 82 shown in FIG. It is also possible to transmit to the main device 84 and update the learning data 62 of the main device 84 using the drug identification results performed by the drug identification sensor 90 .

(Confirmation and correction of data by the user)
In the embodiment of FIG. 1, when the medicine is identified based on the identification information indicated by the identification index, the identification result is combined with the photographed image 22 to create medicine photographing example data 32a and 32b. Although the drug imaging example data 32 a and 32 b are transmitted to the learning data storage unit 60 , the user may confirm and input the data before the drug imaging example data 32 a and 32 b are transmitted to the learning data storage unit 60 . For example, the monitor 14 displays the identification result and also displays the "Confirm" button and the "Correct" button. If the displayed identification result is correct, the user selects the "Confirm" button (using an input means such as a mouse or a touch panel) to inform the drug identification unit 30 that the identification result is correct. If the displayed identification result is incorrect, the user selects the "Correct" button to notify the drug identification unit 30 that the identification result is incorrect, and to input the drug 12 (using an input means such as a keyboard or touch panel). correct information is input and transmitted to the drug identification unit 30 . When the “Confirm” button is selected, the drug identification unit 30 determines that the generated identification result is confirmed, and transmits the drug imaging example data 32a and 32b to the learning data storage unit 60 as they are. When the "correction" button is selected, the identification result is corrected to the correct information input by the user, and combined with the photographed image 22 to recreate drug photographing example data 32a and 32b, and then the learning data storage unit 60. In this way, correct information can be sent to the learning data storage unit 60 even if the identification index such as the bar code is read incorrectly. In addition, in inputting the "correct data", the user inputs an identification indicator that can be read by humans, such as the identification code 71a and the product name 72a written on the drug or drug package as shown in FIG. 2B. It is also possible to enter the identification information In particular, in the case of medical drugs, such an identification index as a character string is written on the packaging of the drug, and in many cases, the number represented by the bar code 76a (FIG. 2D) is also printed as a character string near the bar code 76a. . Therefore, even if the user does not have specialized knowledge about drugs, it is possible to input "correct data". In this way, even when the user inputs data, the identification information indicated by the identification index is input, and the medicine is identified based on the identification information.

Moreover, even if the AI model 50 identifies the object without using the identification index, it is possible to allow the user to confirm and input the data. For example, similar to the above example, the monitor 14 displays the identification result by the AI model 50 and the "confirm" button and the "correction" button. Then, for each button, the identification result is confirmed or corrected in the same manner as described above, and the confirmed or corrected identification result and the photographed image 22 are combined and sent to the learning data storage unit 60 . In addition, when the identification index is not detected and identification is impossible even by the AI model 50, a display indicating that identification was not possible (such as “identification impossible”) is performed, and the “data input” button is displayed on the monitor 14. , and the correct data may be entered by the user when this button is pressed. In this way, even when identification by the AI model 50 is performed, or when identification is not possible by the AI model 50 or by the identification index, an example of medicine photographing including the photographed image 22 at that time can be performed. Data 32a and 32b can be transmitted to the learning data storage unit 60. FIG. In this case, when inputting the "correct data", the user re-photographs the portion marked with the identification index with the camera 20 (for example, if the front side of the PTP sheet was first photographed, the back side of the PTP sheet is photographed). ), or by reading the barcode with a barcode reader provided separately from the imaging device 20, the identification index written on the drug to be identified or the drug package is reidentified. You may make it detect to the part 30. FIG. In addition, the drug name written on the cosmetic box, which is the sales packaging unit of the drug, is also a kind of identification index written on the drug packaging, and the identification index written on the individual drug packaging such as the PTP sheet is difficult to read due to stains. In such a case, the identification index of such a vanity case may be read. In the case of a bottled drug, an identification index such as a bar code attached to the bottle may be read.

(Evaluation of identification confidence)
Also, when identification by the AI model 50 is performed, an evaluation of how reliable the identification may be performed. For example, according to the degree to which the input photographed image 22 matches the past photographed image, a value of "certainty degree" representing the degree of certainty with which the identification result is determined can be calculated. Just do it. Then, if this "certainty degree" exceeds a predetermined threshold value (for example, 90%), the identification result may be combined with the photographed image 22 and sent to the learning data storage unit 60 (the higher the certainty degree, the For example, drug imaging example data is created even for identification by the AI model 50). Also, regardless of whether the "certainty factor" is high or low, the identification result in that case is displayed on the monitor 14 together with the confidence factor value. At this time, the "data input button" is displayed on the monitor 14 in the same manner as in the case of "unidentifiable", and the user can input "correct data" when the degree of certainty is particularly low. In this way, the user only has to input "correct data" when the degree of certainty is low, thereby reducing the troublesomeness of the operation.

(Assistance in data entry by users)
In addition, the identification index or the AI model 50 may be used as an aid to data input by the user instead of using the identification by the identification index or the AI model 50 as the identification result. For example, the data is basically manually input by the user, and the data may be input by the user entering the data in the input fields displayed on the monitor 14 . On the other hand, when identification by the identification index or the AI model 50 is performed, the data of the identification result is automatically inserted into the input field. The user checks the automatically inserted data, corrects it if necessary, and confirms it, thereby notifying the drug identification unit 30 and the learning data storage unit 60 of the identification result. In this way, all data to be entered is entered after confirmation by the user, which increases the reliability of the data. It's not bothersome.

(Combination of AI model and identification index)
Further, instead of using only the AI model 50 or only the identification index for identification, the identification result may be determined by combining the identification by the AI model 50 and the identification by the identification index. In particular, it is expected that a combination of identification by the AI model 50 and identification by the identification index will increase the accuracy of determining the quantity such as "1 sheet" or "1 tablet".

For example, if a PTP sheet has only one bar code on one sheet, it can be identified by reading the bar code if only the identification index is used to identify the presence of "one PTP sheet" of medicine. be done. However, PTP sheets that are actually handled at pharmacies and patients' homes are sometimes cut off and used while leaving the bar code. Also, even if the cutting is not performed, it is possible that only some of the tablets have already been taken out. In such a case, if it is identification by the AI model 50, based on "the shape of the cut PTP sheet" or "the shape of the PTP sheet in which a part of the tablet area is torn", how many tablets are currently in the PTP sheet? remaining can be identified. Since the shape of the PTP sheet is generally the same even if the type of medicine to be packaged is different, the type of medicine is identified based on an identification index such as a bar code, while the quantity is determined from the shape of the PTP sheet. By identifying with the AI model 50, both types and quantities can be identified with high accuracy. On the other hand, if an identification index such as an identification code written on each tablet can be detected by image analysis, it is possible to identify the quantity of medicine based on the identification index.

It should be noted that the specific correspondence relationship between the shape and quantity of drug packaging (for example, PTP sheet) can be accumulated in the learning data 62 as drug imaging example data in which the shape and quantity of the drug packaging are combined. be. For example, regarding a PTP sheet from which only one tablet has been taken out of a PTP sheet containing 10 tablets (only an area corresponding to one tablet has been torn), the user who photographed the PTP sheet may say, The number of tablets currently in use is 9", and the information is transmitted to the learning data storage unit 60. FIG. When a large amount of information indicating the correspondence relationship between the shape of the drug package and the quantity of the drug contained therein is accumulated in the learning data 62 in this way, the AI model 50 reflecting the information can, from the shape of the drug package, It becomes possible to identify the quantity of medicine contained in the medicine package. For example, if the drug imaging example data related to the PTP sheet from which only one tablet has been taken out of the PTP sheets containing 10 tablets is accumulated, the AI model 50 will It becomes possible to identify that "out of a sheet containing 10 tablets, 9 tablets remain".

(Regarding Identification of Bringing Medicine)
Furthermore, according to the well-educated AI model 50, it is possible to identify even a drug that does not have an identification index, based on the shape and color of the drug itself. Therefore, identification using the AI model 50 is very effective for identification of bring-your-own-medicine. Bring-in medications are medications presented to patients as the medications they usually take at the time of examination, examination, treatment, or hospitalization at a medical institution. For medical institutions, it is very important to understand the content of medicines brought by patients in prescribing medicines after examination, during treatment, and at the time of hospitalization, and in performing surgery and examinations appropriately. That is, it is possible to avoid double administration of prescription drugs and bring-your-own drugs, and to avoid dangerous combinations of drugs. In addition, it is possible to identify drugs that have been or should be stopped prior to treatment, surgery, or testing. In addition, it is possible for the medical institution to properly advise the patient on how to take the medicine, and also to give advice on taking the medicine together with health foods and supplements. Even without drug identification, it is possible to grasp the content of the drug that the patient is taking from the information in the drug manual or so-called medicine notebook. However, these information cannot completely grasp whether the patient is actually taking the medicine. The surest way is to ask the patient to bring the drugs they are actually taking and to check the drugs they bring. When confirming the medicine brought in, if the product name and identification code of the medicine are written on the packaging of the medicine as an identification index, the medicine can be identified relatively easily without using the medicine identification system. However, as a result of dispensation in one package, the PTP sheet being cut off, or the tablet being taken out of the package and stored separately, the identification index is not written in a readable form on the package of the drug, or the package of the drug itself may not exist. In this case, it becomes difficult to identify the brought medicine with the naked eye. The medicine identification system of the present invention can identify the medicine even in such a case, and is very effective for identifying and confirming the medicine brought by the patient.

(Data input/output for external devices)
In the embodiment shown in FIG. 1, the imaging device 20 and the monitor 14 are components of the drug identification system 10, but these may be devices external to the drug identification system 10, and the drug identification system 10 may be external to the drug identification system 10. It is sufficient that the data input of the photographed image 22 can be received from and the data of the identification result can be output to the outside.

(Regarding data placement)
In the embodiment shown in FIG. 1, the identification index table storage unit 40 and the AI model 50 are separate from the drug identification unit 30. 30 may be incorporated. On the other hand, the data of the identification index table 42 and the data of the AI model 50 may be arranged in a device remote from the drug identification unit 30 and accessible via a network.

(About shooting time)
The drug 12 does not need to be photographed at the time of identification, and the data of the image taken in advance at a location different from the drug identification system 10 can be captured as the photographed image 22 (input to the drug identification unit 30). is also possible.

(Regarding the AI model creation method)
Also, the AI model 50 created based on the learning data 62 is not limited to one created by supervised learning using the learning data 62 as teacher data. For example, an AI model obtained as a result of deep learning based on the learning data 62 may be used as the AI model 50 of the drug identification system 10 .

REFERENCE SIGNS LIST 10 drug identification system 12 drug 12a tablet 12b tablet PTP sheet 14 monitor 20 imaging device 22 captured image 30 drug identification unit 32a, 32b drug imaging example data 40 identification index table storage unit 42 identification index table 50 AI model 52 main AI model 56 local AI model 60 Learning data storage unit 62 Learning data 70a Tablet 70b Powder 70c Capsule 71a, 71b, 71c Identification code 72a, 72b, 72c Product name 73a, 73c Piece of PTP sheet 73b Individual package 75a PTP sheet 76a, 76b, 76c Bar code 80 drug identification system 82 network 84 main device 86 local device 88a drug DB
88b Prescription DB
88c Inventory DB
88d Medication Management DB
90 drug identification sensor 91 substrate 92 flexible cable 93 computing element 94 imaging element 95 memory element 98 connector S01-S09 steps

Claims (9)

In a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug,
a learning data storage unit that stores a combination of drug information and captured image information as learning data;
a drug identification unit that identifies a drug included in a captured image;
with
The drug identification unit uses an AI model to identify a drug contained in the input captured image,
The AI model is an AI model created based on the learning data stored in the learning data storage unit, and is an AI model that uses a captured image as input data and returns information on the corresponding drug as output data. can be,
When an identification index indicating identification information of the medicine is written on the medicine or the medicine package for packaging the medicine, the medicine identification unit identifies the medicine based on the identification information of the medicine indicated by the identification index, and identifies the medicine. creating drug imaging example data by combining the information of the drug obtained and the information of the photographed image at that time, and transmitting the drug imaging example data to the learning data storage unit;
Upon receiving the drug imaging example data from the drug identification unit, the learning data storage unit stores a combination of drug information and photographed image information included in the drug imaging example data, and updates the learning data. A drug identification system characterized by: The drug identification unit detects the identification index from the photographed image by image analysis separately from the AI model, and when it is detected by the image analysis that the identification index is included in the photographed image, the The drug identification unit identifies the drug based on the identification information of the drug corresponding to the identification index detected by the image analysis, and the drug imaging example in which the information of the identified drug and the information of the captured image at that time are combined 2. The drug identification system according to claim 1, wherein data is created. a main device having the learning data storage unit;
a plurality of local devices each comprising the drug identification unit and capable of accessing the main device via a network;
when the drug imaging example data is created in the drug identification performed by the local device, the drug imaging example data is transmitted to the main device;
3. The medicine according to claim 1, wherein the main device updates the learning data stored in the learning data storage unit based on the drug imaging example data transmitted from the local device. identification system. The main device has a main AI model,
The main AI model is an AI model that is created based on the learning data stored in the learning data storage unit of the main device, and uses the captured image as input data and the corresponding drug information as output data. is an AI model to return,
The main device recreates the main AI model when the learning data is updated,
The local device incorporates, as the AI model used by the drug identification unit, a local AI model that returns a captured image as input data and corresponding drug information as output data,
The local AI model is composed of data received by the local device from the main device. AI model data,
The drug identification unit of the local device identifies the drug contained in the captured image using the local AI model, and if the drug contained in the captured image cannot be identified using the local AI model, the network 4. The drug identification system according to claim 3, wherein the main AI model of the main device is used to identify the drug by accessing the main device via the . If the photographed drug is a medical drug, the identification index is an optical mark that is written on the drug package of the medical drug and that indicates information about the medical drug. Item 5. The drug identification system according to any one of Item 4. The drug identification system is used in a system that includes a database that stores prescription information entered at the discretion of a doctor,
6. The drug identification according to any one of claims 1 to 5, wherein it is determined whether or not information on one or more drugs identified by the drug identification unit matches the prescription information. system. The drug identification system is used in a system that includes a database that stores inventory information that varies according to information on drugs received and delivered from the pharmacy,
6. The drug identification according to any one of claims 1 to 5, wherein it is determined whether or not information on one or more drugs identified by the drug identification unit matches the inventory information. system. The drug identification system is used in a system that includes a database that manages a medication schedule prescribed by a medical institution for drugs dispensed to patients,
6. The drug identification according to any one of claims 1 to 5, characterized in that it is determined whether or not information on one or more drugs identified by the drug identification unit matches the medication schedule. system. 3. The learning data storage unit and the drug identification unit are configured as separate devices, respectively, and the drug identification unit can access the learning data storage unit via a network. A drug identification system according to any one of claims 1 to 8.

Images