JP7583077B2 - Mobile devices - Google Patents

Description

The present invention relates to a mobile terminal.

Research and development is being conducted on technologies that will assist the operations of facilities that provide at least one of the following services: medical care, nursing care, and rehabilitation, such as hospitals and nursing homes.

In this regard, there is known an information processing device that identifies medical information necessary for instruction information that a doctor gives to medical personnel from medical record information entered by a doctor into an electronic medical record, the information processing device comprising type identification information for identifying medical information from the medical record information for each type of medical information to be identified, and an extraction means for extracting the specific medical information identified by the type identification information (see Patent Document 1).

JP 2018-163406 A

Here, the facility may need to identify an item associated with a user of the facility from among items related to the facility's business. However, the facility may mistakenly identify an item that is not associated with the user as an item associated with the user. For example, if it is difficult to verify the identity of a user due to dementia or the like, an item that is not associated with the user may be mistakenly identified as an item associated with the user. As a result, the facility may use an item that is not associated with the user on the user.

The present invention has been made in consideration of these circumstances, and aims to provide a mobile terminal that can prevent items not associated with a person to whom items related to facility operations are to be provided from being mistakenly identified as items associated with that person.

One aspect of the present invention is a mobile terminal, in which each of a plurality of items related to the services of a facility that provides at least one of medical care, nursing care, and rehabilitation is affixed with an identification image representing item identification information that identifies the individual item, and the mobile terminal includes a display unit, an imaging unit, and a first storage unit that stores, for each of one or more users that use the facility, first correspondence information in which face information indicative of the user's face is associated with user identification information indicative of the user, and second correspondence information in which, for each of the one or more users, the user identification information indicative of the user is associated with the item identification information indicative of the item associated with the user, and a face of a person to whom at least one of the plurality of items is to be provided is imaged by the imaging unit. a second captured image captured by the imaging unit, in which one of the plurality of items having different contents or types is taken as a target item, and the second captured image is an image of an identification image affixed to a target item selected from the plurality of items by a user other than the one or more users; and the mobile terminal determines whether or not the user identification information associated with the user identified by facial authentication based on a first captured image captured by the imaging unit matches the user identification information associated with the item identification information represented by the identification image captured in the second captured image, in which the identification image is affixed to a target item selected from the plurality of items by a user other than the one or more users; and, depending on the result of the determination, displays first information on the display unit indicating whether the target item is associated with the person.

In one aspect of the present invention, a configuration may be used in which, in a mobile terminal, the facility is a hospital, the item is a medicine or a container containing a medicine, the user is a patient, and the identification image is a one-dimensional code or a two-dimensional code.

In one aspect of the present invention, a mobile terminal may be configured such that, when the mobile terminal displays the information on the display unit, the mobile terminal displays on the display unit a UI (User Interface) in which information indicating that the item captured in the second captured image has been provided to the person is input.

In one aspect of the present invention, a configuration may be used in which the mobile terminal outputs the information to another device.

The present invention can prevent items that are not associated with a person to whom items related to facility operations are to be provided from being mistakenly identified as items associated with that person.

1 is a diagram illustrating an example of a configuration of an information processing system 1 according to an embodiment. FIG. 2 is a diagram showing an example of a flow of processing performed by an information processing device 20.

<Embodiment>
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

<Outline of the information processing system>
First, an overview of an information processing system according to an embodiment will be described.

The information processing system outputs information indicating whether or not a target item is associated with a person based on a first captured image capturing the face of a person to whom an item related to the facility's services of at least one of medical care, nursing care, and rehabilitation is to be provided, and a second captured image capturing one of a plurality of items having different contents or types as the target item. This makes it possible for the information processing system to prevent an item that is not associated with a person to whom an item related to the facility's services is to be provided from being erroneously identified as an item associated with the person.

The configuration of such an information processing system and the processing performed by the information processing system are described in detail below.

<Configuration of Information Processing System>
Hereinafter, a configuration of an information processing system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment.

Here, as shown in FIG. 1, the information processing system 1 includes a mobile terminal 10 and an information processing device 20. In the example shown in FIG. 1, the mobile terminal 10 and the information processing device 20 are separate. Note that in the information processing system 1, the mobile terminal 10 and the information processing device 20 may be configured as an integrated unit.

In the information processing system 1, the face of a person to whom an item related to the services of a facility that provides at least one of medical care, nursing care, and rehabilitation is provided is captured by the mobile terminal 10. In the information processing system 1, the information processing device 20 acquires an image of the person's face captured by the mobile terminal 10 as a first captured image. For ease of explanation, the items related to the services of the facility will be referred to simply as items. Also, for ease of explanation, the person will be referred to as a target person. Also, for ease of explanation, the facility will be referred to simply as a facility. Here, the facility is, for example, a hospital, nursing care facility, rehabilitation facility, etc., but is not limited to these.

For example, if the facility is a hospital, items related to the facility's operations include, but are not limited to, medicines, medical charts, nursing records, slips, etc. In this case, the subjects include, but are not limited to, patients admitted to the hospital, patients visiting the hospital, etc. (i.e., users who use the hospital, etc.). In this case, the users who operate the mobile terminal 10 include, but are not limited to, nurses, doctors, various types of therapists, medical office staff, etc.

For example, if the facility is a care facility, items related to the facility's operations include, but are not limited to, various tools used in care, care records, slips, etc. In this case, the target persons are, but are not limited to, users of the care facility. In this case, the users who operate the mobile terminal 10 are, but are not limited to, caregivers, office staff at the care facility, etc.

For example, if the facility is a correctional facility, items related to the facility's operations include, but are not limited to, various tools used in correctional facilities, correctional records, slips, etc. In this case, the target persons are, but are not limited to, users of the correctional facility. In this case, the users who operate the mobile terminal 10 are, but are not limited to, staff of the correctional facility.

In addition, in the information processing system 1, one of a plurality of items having different contents or types is imaged as a target item by the mobile terminal 10. Then, in the information processing system 1, the information processing device 20 acquires an image of the target item imaged by the mobile terminal 10 as a second image.

Here, multiple items with different contents are, for example, medical records, slips, etc. created in a hospital. Also, multiple items with different types are, for example, medicines, etc.

Each of a plurality of items having different contents or types is affixed with an identification image that represents item identification information for identifying the item. The item identification information of a certain item is information that indicates the item, for example, an identification ID that indicates the item. The item identification information of the item may be other information that can indicate the item. The identification image is, for example, a one-dimensional code or a two-dimensional code. The one-dimensional code is, for example, a barcode, a color code, etc. The two-dimensional code is, for example, a QR code (registered trademark), etc. The identification image may be affixed to the item itself, or to a container that contains the item. That is, in the embodiment, the identification image affixed to the item means an identification image affixed to the item itself, or an identification image affixed to a container that contains the item.

In other words, the second captured image in which the target item is captured is, more specifically, an image in which the identification image attached to the target item is captured.

In the information processing system 1, the information processing device 20 outputs information indicating whether or not the target item is associated with the target person based on the first captured image acquired from the mobile terminal 10 and the second captured image acquired from the mobile terminal 10.

More specifically, in the information processing system 1, the information processing device 20 outputs information indicating whether or not the target item is associated with the target person based on the first captured image acquired from the mobile terminal 10, the second captured image acquired from the mobile terminal 10, the first correspondence information, and the second correspondence information.

Here, the first correspondence information is information in which, for each of one or more users who use the facility, facial information indicating the user's face is associated with user identification information indicating the user. The facial information indicating the user's face is, for example, a captured image of the user's face. Note that the facial information indicating the user's face may be other information capable of indicating the user's face, such as CG showing the user's face. The first identification information indicating the user is, for example, an identification ID indicating the user, but may also be other information capable of indicating the user.

The second correspondence information is information in which, for each of one or more users who use the facility, user identification information indicating the user is associated with item identification information indicating the item associated with the user. The item identification information indicating the item is, for example, an identification ID indicating the item, but may be other information capable of indicating the item.

The information processing device 20 identifies user identification information associated with a user who matches the subject through face authentication based on the acquired first captured image from the first correspondence information. The information processing device 20 also identifies user identification information associated with the item identification information represented by the identification image captured in the second captured image from the second correspondence information based on the acquired second captured image. The information processing device 20 outputs information indicating whether the target item captured in the second captured image is associated with the subject captured in the first captured image based on the user identification information identified from the first correspondence information and the user identification information identified from the second correspondence information. This allows the information processing system 1 to prevent an item that is not associated with the subject from being erroneously identified as an item associated with the subject. That is, for example, the information processing system 1 can prevent a medicine that is not scheduled to be administered to the subject from being erroneously identified as a medicine to be administered to the subject.

Here, the mobile terminal 10 is, for example, an information processing device such as a multi-function mobile phone terminal (smartphone), a mobile phone terminal, a tablet PC (Personal Computer), a notebook PC, etc., but is not limited to these. The mobile terminal 10 is communicatively connected to the information processing device 20 by wireless communication using a mobile communication network, a wireless LAN (Local Area Network), etc. Note that the mobile terminal 10 may also be configured to be communicatively connected to the information processing device 20 by wired communication via a cable.

The mobile terminal 10 includes an imaging unit 11 and a display unit 12.

The imaging unit 11 is a camera equipped with an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that converts collected light into an electrical signal. The imaging unit 11 captures still images or moving images. In the following, as an example, a case where the imaging unit 11 captures still images will be described. Note that when the imaging unit 11 captures moving images, the information processing system 1 performs processing based on any of the frames of the moving images instead of the captured images described below.

The display unit 12 is a display device equipped with a liquid crystal display panel, an organic EL (ElectroLuminescence) display panel, or the like. The display unit 12 may be a touch panel integrated with a touch pad as an input device.

The information processing device 20 is, for example, an information processing device such as a workstation, a desktop PC, or a server, but is not limited to these.

The information processing device 20 also includes, for example, a storage unit 22, a communication unit 23, and a control unit 24. The information processing device 20 communicates with the mobile terminal 10 via the communication unit 23. Note that the communication may be relayed by another device such as a server.

The storage unit 22 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), etc. Note that the storage unit 22 may be an external storage device connected via a digital input/output port such as a universal serial bus (USB) instead of being built into the information processing device 20. The storage unit 22 stores various information processed by a processor (not shown) included in the information processing device 20, various commands executable by a computer, various images, etc. Note that the processor is a central processing unit (CPU), but may also be another processor such as a field programmable gate array (FPGA). The processor included in the information processing device 20 executes various commands stored in the storage unit 22 included in the information processing device 20.

The memory unit 22 also has a first memory unit 221 and a second memory unit 222.

The first memory unit 221 is a part of the memory area of the memory unit 22. The first memory unit 221 stores the first correspondence information and second correspondence information described above.

The second storage unit 222 stores history information, which will be described later.

The communication unit 23 includes, for example, a digital input/output port such as a USB, an Ethernet (registered trademark) port, etc.

The information processing device 20 may be configured to include an input device such as a keyboard, a mouse, or a touchpad. The information processing device 20 may also be configured to include a display device such as a liquid crystal display panel, an organic EL (ElectroLuminescence) display panel, or the like.

The control unit 24 controls the entire information processing device 20. The control unit 24 includes an image acquisition unit 241, a first identification unit 242, a second identification unit 243, a determination unit 244, an output unit 245, and a memory control unit 246. These functional units of the control unit 24 are realized, for example, by a processor (not shown) included in the information processing device 20 executing various commands stored in the memory unit 22. In addition, some or all of these functional units may be hardware functional units such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

The image acquisition unit 241 acquires from the mobile terminal 10 an image captured by the imaging unit 11 of the mobile terminal 10. More specifically, the image acquisition unit 241 acquires, for example, an image in which the face of a target person is captured by the imaging unit 11 from the mobile terminal 10 as a first captured image. In addition, the image acquisition unit 241 acquires, for example, an image in which a target item is captured by the imaging unit 11 from the mobile terminal 10 as a second captured image.

The first identification unit 242 identifies, from the first correspondence information, user identification information associated with a user that matches the subject captured in the first captured image by face recognition based on the first captured image acquired by the image acquisition unit 241.

The second identification unit 243 identifies, based on the second captured image acquired by the image acquisition unit 241, from the second correspondence information, the user identification information associated with the item identification information represented by the identification image captured in the second captured image.

The determination unit 244 determines whether the user identification information identified by the first identification unit 242 matches the user identification information identified by the second identification unit 243.

When the determination unit 244 determines that the user identification information identified by the first identification unit 242 and the user identification information identified by the second identification unit 243 match, the output unit 245 outputs information indicating that the target item is associated with the target person, and when the determination unit 244 determines that the user identification information identified by the first identification unit 242 and the user identification information identified by the second identification unit 243 do not match, the output unit 245 outputs information indicating that the target item is not associated with the target person.

The memory control unit 246 stores information indicating the result of the judgment by the judgment unit 244 in the second memory unit 222 as history information associated with time information indicating the time when the judgment by the judgment unit 244 was made.

<Processing performed by information processing device>
The process performed by the information processing device 20 will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of the flow of the process performed by the information processing device 20.

The first identification unit 242 waits until the image acquisition unit 241 acquires the first captured image from the mobile terminal 10 (step S110). Here, for example, the mobile terminal 10 has a dedicated application program installed thereon, and when a user of the mobile terminal 10 captures an image of a subject using the imaging unit 11, the captured image captured by the imaging unit 11 is output to the information processing device 20 together with information indicating that the captured image is the first captured image. In this case, when the image acquisition unit 241 acquires the captured image together with the information from the mobile terminal 10, it acquires the captured image as the first captured image. This allows the first identification unit 242 to determine whether the image acquisition unit 241 has acquired the first captured image.

When the image acquisition unit 241 determines that the first captured image has been acquired from the mobile terminal 10 (step S110-YES), the first identification unit 242 performs a first identification process (step S120). Here, in step S120, the first identification unit 242 performs a process of identifying, from the first correspondence information, user identification information associated with a user who matches the target person through face authentication based on the first captured image.

Next, the second identification unit 243 waits until the image acquisition unit 241 acquires the second captured image from the mobile terminal 10 (step S130). Here, for example, a dedicated application program is installed on the mobile terminal 10, and when a user of the mobile terminal 10 captures an image of a target item using the imaging unit 11, the image captured by the imaging unit 11 is output to the information processing device 20 together with information indicating that the captured image is the second captured image. In this case, when the image acquisition unit 241 acquires the captured image together with the information from the mobile terminal 10, it acquires the captured image as the second captured image. This allows the second identification unit 243 to determine whether the image acquisition unit 241 has acquired the second captured image.

When the image acquisition unit 241 determines that the second captured image has been acquired from the mobile terminal 10 (step S130-YES), the second identification unit 243 performs a second identification process (step S140). Here, in step S140, the second identification unit 243 reads out from the identification image captured in the second captured image, the item identification information represented by the identification image. Then, the second identification unit 243 identifies the user identification information associated with the read item identification information from the second correspondence information. The second identification unit 243 performs such a process as the second identification process.

Next, the determination unit 244 determines whether the user identification information identified by the first identification unit 242 matches the user identification information identified by the second identification unit 243 (step S150).

Next, the output unit 245 outputs information indicating the result of the determination made by the determination unit 244 in step S150 (step S160). Here, if the determination unit 244 determines in step S150 that the user identification information identified by the first identification unit 242 and the user identification information identified by the second identification unit 243 match, the output unit 245 outputs information indicating that the target item is associated with the target person.

More specifically, in this case, the output unit 245 outputs the information to the mobile terminal 10 and displays the information on the display unit 12 of the mobile terminal 10. In this case, the output unit 245 may be configured to display a UI (User Interface) into which information indicating that the target item has been provided to the subject is input on the display unit 12. In this case, the information processing system 1 can record that the target item has been provided to the subject, and can prevent the target item from being forgotten to be provided to the subject, the target item from being excessively provided to the subject, etc. In this case, the storage control unit 246 stores the information in the second storage unit 222 as history information associated with time information indicating the time when the determination unit 244 made the determination. In this case, the information processing system 1 can record the history of the determination by the information processing system 1. In addition, the output unit 245 may be configured to output the information to another device in this case. In addition, in this case, if the information processing device 20 is equipped with a display device, the output unit 245 may be configured to display the information on the display device.

On the other hand, when the determination unit 244 determines in step S150 that the user identification information identified by the first determination unit 242 and the user identification information identified by the second determination unit 243 do not match, the output unit 245 outputs information indicating that the target item is not associated with the subject. In this case, the output unit 245 may be configured to display warning information that warns that the target item is not associated with the subject on the display unit 12. In addition, the output unit 245 may be configured to output a sound that warns that the target item is not associated with the subject to the mobile terminal 10. In this way, the information processing system 1 can prevent an item that is not associated with the subject from being erroneously identified as an item associated with the subject. In addition, in this case, the memory control unit 246 stores the information in the second memory unit 222 as history information associated with time information indicating the time when the determination by the determination unit 244 was made. In this way, the information processing system 1 can record the history of the determination by the information processing system 1. In addition, in this case, the output unit 245 may be configured to output the information to another device. In addition, in this case, and if the information processing device 20 is equipped with a display device, the output unit 245 may be configured to display the information on the display device.

After processing in step S160 is performed, the information processing device 20 ends the processing.

As described above, the information processing device (information processing device 20 in the example described above) of the information processing system according to the embodiment (information processing system 1 in the example described above) outputs information indicating whether or not a target item is associated with a person (subject in the example described above) to whom an item (e.g., medicines, etc.) related to the services of a facility (e.g., a hospital, etc.) that performs at least one of medical care, nursing care, and rehabilitation is to be provided, based on a first captured image capturing the face of the person, and a second captured image capturing one of a plurality of such items having different contents or types as the target item. This allows the information processing system or information processing device to prevent an item that is not associated with a person to whom an item related to the services of the facility is to be provided from being erroneously identified as an item associated with the person.

In addition, the information processing device of the information processing system has an identification image (e.g., a QR code or the like) attached to each of the multiple items, which represents item identification information that identifies the individual item, and the identification image attached to an item selected by a user from the multiple items is captured in the second captured image, and a first memory unit (in the example described above, the first memory unit 221) stores, for each of one or more users using the facility, first correspondence information in which face information indicating the user's face is associated with user identification information indicating the user, and second correspondence information in which user identification information indicating the user is associated with item identification information indicating the item associated with the user, and a second memory unit (in the example described above, the first memory unit 221) stores, for each of one or more users using the facility, first correspondence information in which user identification information indicating the user is associated with item identification information indicating the item associated with the user, and A configuration including a first identification unit (first identification unit 242 in the example described above) that identifies, from the first correspondence information, user identification information associated with a user who is captured in the second captured image; a second identification unit (second identification unit 243 in the example described above) that identifies, from the second correspondence information, user identification information associated with item identification information represented by an identification image captured in the second captured image based on the second captured image; and an output unit (output unit 245 in the example described above) that outputs information indicating whether or not the item captured in the second captured image is associated with a person captured in the first captured image based on the user identification information identified by the first identification unit and the user identification information identified by the second identification unit.

In addition, the information processing device of the information processing system may be configured such that the facility is a hospital, the item is a medicine or a container containing a medicine, the user is a patient, and the identification image is a one-dimensional code or a two-dimensional code.

In addition, the information processing device of the information processing system may be configured to include a determination unit (determination unit 244 in the example described above) that determines whether the user identification information identified by the first identification unit and the user identification information identified by the second identification unit match, and an output unit that outputs information indicating that the item captured in the second captured image corresponds to the person when the determination unit determines that the user identification information identified by the first identification unit and the user identification information identified by the second identification unit match, and outputs information indicating that the item captured in the second captured image does not correspond to the person when the determination unit determines that the user identification information identified by the first identification unit and the user identification information identified by the second identification unit do not match.

The information processing device of the information processing system may also be configured to include a second storage unit (second storage unit 222 in the example described above) and a storage control unit (storage control unit 246 in the example described above) that stores information indicating the result of the determination by the determination unit in the second storage unit as history information associated with time information indicating the time when the determination by the determination unit was made.

The information processing device of the information processing system may also be configured to include a display unit that displays an image (in the example described above, the display unit 12 when the information processing device 20 is integrated with the mobile terminal 10, or a display device included in the information processing device 20), and the output unit may output information indicating whether or not the item captured in the second captured image is associated with the person to the display unit, and cause the display unit to display the information.

In addition, in the information processing device of the information processing system, when the output unit causes the display unit to display information indicating that the item captured in the second captured image is associated with the person, the output unit may be configured to display a UI on the display unit into which information indicating that the item captured in the second captured image has been provided to the person is input.

In addition, in the information processing device of the information processing system, the output unit may be configured to output information indicating whether or not the item captured in the second captured image is associated with the person to another device (in the example described above, to the mobile terminal 10 or another device when the information processing device 20 is separate from the mobile terminal 10).

The information processing device of the information processing system may also be configured to include an imaging unit (in the example described above, imaging unit 11 when information processing device 20 is integrated with mobile terminal 10), and the first captured image and the second captured image may be captured images captured by the imaging unit.

The embodiment of the present invention has been described above in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and may be changed, replaced, deleted, etc., without departing from the spirit of the present invention.

A program for implementing the functions of any of the components of the device described above may be recorded on a computer-readable recording medium, and the program may be read and executed by a computer system. Here, the device in question may be, for example, a mobile terminal 10 or an information processing device 20. Note that the term "computer system" here includes hardware such as an OS (Operating System) and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD (Compact Disk)-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" also refers to storage devices that hold a program for a certain period of time, such as volatile memory inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

The above program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has a function of transmitting information, such as a network such as the Internet or a communication line such as a telephone line.
The above program may be for implementing some of the above functions. Furthermore, the above program may be a so-called differential file or differential program that can implement the above functions in combination with a program already recorded in the computer system.

1...information processing system, 10...mobile terminal, 11...imaging unit, 12...display unit, 20...information processing device, 22...storage unit, 23...communication unit, 24...control unit, 221...first storage unit, 222...second storage unit, 241...image acquisition unit, 242...first identification unit, 243...second identification unit, 244...determination unit, 245...output unit, 246...storage control unit

Claims (4)

A mobile terminal,
An identification image representing item identification information for identifying each of a plurality of items relating to the services of the facility that performs at least one of medical care, nursing care, and rehabilitation is affixed to each of the items;
The mobile terminal includes:
A display unit;
An imaging unit;
a first storage unit that stores, for each of one or more users using the facility, first correspondence information in which face information indicating the user's face is associated with user identification information indicating the user, and second correspondence information in which, for each of the one or more users, the user identification information indicating the user is associated with the item identification information indicating the item associated with the user;
Equipped with
a determination is made as to whether or not the user identification information associated with the user who matches the person, the face of which is identified by face authentication based on a first captured image captured by the imaging unit, matches the user identification information associated with the item identification information represented by the identification image captured in a second captured image captured by the imaging unit in which one of the multiple items having different contents or types is taken as a target item, the identification image being affixed to a target item selected from the multiple items by a user other than the one or more users ; and, depending on the result of the determination, display first information indicating whether or not the target item is associated with the person on the display unit.
Mobile device. the facility is a hospital;
The article is a medicine or a container containing a medicine,
The user is a patient,
The identification image is a one-dimensional code or a two-dimensional code.
The mobile terminal according to claim 1 . When the mobile terminal causes the display unit to display the first information, the mobile terminal causes the display unit to display a user interface (UI) for inputting second information indicating that the article captured by the imaging unit has been provided to the person.
3. The mobile terminal according to claim 1 or 2. The mobile terminal outputs the first information to another device.
A mobile terminal according to any one of claims 1 to 3.

Images