JP7520664B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

Embodiments of the present invention relate to an information processing device, an information processing method, and an information processing program that use image data.

Wearing a mask is recommended to prevent infection in shopping malls, stores, and other commercial facilities, as well as hospitals and event venues.

Various technologies have been proposed to determine whether or not a mask is being worn. For example, Patent Document 1 discloses a technology that detects differences in color or the amount of edges in specific areas of the upper and lower parts of a person's face in an image of the person's face to determine whether or not the person is wearing a mask.

Patent No. 4322537

However, the degree of risk of spreading an infectious disease is not determined solely by whether or not a mask is worn. The degree of risk of spreading an infectious disease varies depending on various factors in addition to whether or not a mask is worn. Therefore, the technology to determine whether or not a mask is being worn, as disclosed in Patent Document 1, is not sufficient as a measure to prevent infectious diseases.

This invention was made in light of the above-mentioned circumstances, and in one aspect aims to provide a technology that realizes notifications according to each person's risk of spreading infectious diseases.

In order to solve the above problem, one aspect of the information processing device of the present invention includes a first determination unit that determines a fever state of a person based on image data from a first camera, a second determination unit that determines whether the person is wearing a mask based on image data from a second camera, and an output unit that outputs an alert based on at least one of the first determination result regarding the fever state by the first determination unit and the second determination result regarding the mask wearing state by the second determination unit.

According to one aspect of the invention, it is possible to provide notifications according to each person's risk of spreading infectious diseases.

FIG. 1 is a diagram showing an example of the configuration of a monitoring system including an information processing apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of a hardware configuration of a Web server device used as an information processing device according to an embodiment of the present invention. FIG. 3 is a block diagram showing an example of the software configuration of a Web server device used as an information processing device according to an embodiment of the present invention. FIG. 4 is a flowchart showing the procedure and contents of the output process based on the detection of a person, which is executed by the Web server device. FIG. 5 is a flowchart showing the procedure and contents of an output process based on tracking of a monitored person executed by a Web server device.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[One embodiment]
(Configuration example)
(1) System
FIG. 1 is a diagram showing the overall configuration of a system including an information processing apparatus according to an embodiment of the present invention.

For example, a thermal camera C1 is placed at the entrance of a facility such as a shopping mall, a store, a hospital, or an event venue. The thermal camera C1 is an example of a sensor for non-contact measurement of the body temperature of each person entering the facility. The thermal camera C1 is an example of a first camera. The thermal camera C1 is attached to, for example, a ceiling or a wall, and outputs image data acquired within the imaging range. Hereinafter, the image data acquired by the thermal camera C1 is also referred to as the first captured image data. The first captured image data is an example of image data from the thermal camera C1. The first captured image data is data showing the distribution of heat of a subject reflected in an image based on the first captured image data. The first captured image data is image data that can determine a fever state. The fever state is "with fever" or "without fever". "With fever" refers to a body temperature state equal to or higher than a predetermined temperature. "Without fever" refers to a body temperature state below a predetermined temperature. The predetermined temperature can be set or changed arbitrarily.

In addition, multiple surveillance cameras C2-1 to C2-n (n is 2 or more) are distributed around the facility. For example, the surveillance cameras C2-1 to C2-n are visible light cameras. The surveillance cameras C2-1 to C2-n are attached to, for example, the ceiling or a wall, and output image data acquired in their respective imaging ranges. The surveillance camera C2-1 is located at the entrance to the facility. The surveillance camera C2-1 is an example of a sensor for detecting each person entering the facility. The imaging range of the surveillance camera C2-1 overlaps with the imaging range of the thermal camera C1. Each of the surveillance cameras C2-1 to C2-n is an example of a sensor for detecting each person entering the area in which the surveillance cameras are installed. Each of the surveillance cameras C2-1 to C2-n is an example of a second camera. Hereinafter, the image data acquired by each of the surveillance cameras C2-1 to C2-n is also referred to as second captured image data. The second captured image data acquired by the surveillance camera C2-1 is an example of image data from the surveillance light camera C2-1. The second captured image data acquired by each of the surveillance cameras C2-1 to C2-n is an example of image data from the surveillance cameras C2-1 to C2-n. Here, an example in which multiple surveillance cameras are distributed in a facility is described, but it is sufficient for one or more surveillance cameras to be installed in a facility.

Each of the surveillance cameras C2-1 to C2-n is provided with a video analysis engine VE1 to VEn. The video analysis engine VE1 detects whether a person is wearing a mask based on the second captured image data, as exemplified below. The video analysis engine VE1 detects whether a person is wearing a mask in an image based on the second captured image data by analyzing the second captured image data acquired by the surveillance camera C2-1. The video analysis engine VE1 generates information indicating the mask wearing state based on the second captured image data. The mask wearing state is "mask worn" or "mask not worn". "Mask worn" refers to a state in which a mask is worn so as to cover a person's mouth. "Mask not worn" may include a state in which a mask is not worn, as well as a state in which a mask is shifted so as not to cover a person's mouth.

The video analysis engine VE1 generates a first video analysis result. For example, the first video analysis result refers to a video analysis result that is generated each time a person entering the facility is detected, regardless of whether or not the person is a surveillance target, as described below.

The first video analysis result includes first person image data based on the second captured image data. The first person image data is image data of a person appearing in an image based on the second captured image data. The first person image data includes at least one of face image data and whole body image data. The first person image data is an example of image data from a surveillance camera C2-1. The first video analysis result includes the second captured image data on which the first person image data is based. The first video analysis result includes information indicating whether the person appearing in the first person image data is wearing a mask. The first video analysis result includes a person ID that identifies the person appearing in the first person image data. The person ID may be set arbitrarily. The first video analysis result includes a camera ID that identifies the surveillance camera C2-1.

The video analysis engines VE1 to VEn further extract second person image data of a person having image features similar to those of the monitored person by analyzing the second captured image data. For example, the image features of the monitored person are obtained from a registered image based on the registered image data of the monitored person. The registered image data includes at least one of face image data and whole-body image data. The registered image data of the monitored person is image data used for comparison with the second person image data to detect the monitored person.

Each of the video analysis engines VE1 to VEn generates a similarity based on the second person image data and the registered image data of the person being monitored, as exemplified below. Each of the video analysis engines VE1 to VEn determines a similarity of the person image based on the second person image data to the registered image based on the registered image data, based on the image feature amounts of the person being monitored.

Each of the video analysis engines VE1 to VEn detects whether a mask is being worn based on the second captured image data, as exemplified below. Each of the video analysis engines VE1 to VEn detects whether a person appearing in an image based on the second captured image data is wearing a mask by analyzing the second captured image data. Each of the video analysis engines VE1 to VEn generates information indicating whether a mask is being worn based on the second captured image data.

Each of the video analysis engines VE1 to VEn generates a second video analysis result. The second video analysis result refers to the video analysis result generated each time a monitored person is detected.

The second video analysis result includes second person image data based on the second captured image data. The second person image data is image data of a person appearing in an image based on the second captured image data. The second person image data includes at least one of face image data and whole body image data, similar to the first person image data. The second person image data is an example of image data from the surveillance cameras C2-1 to C2-n. The second video analysis result includes the second captured image data on which the second person image data is based. The second video analysis result includes information indicating whether the person appearing in the second person image data is wearing a mask. The second video analysis result includes a monitored person ID that identifies a monitored person having image features similar to those of the person appearing in the second person image data. The second video analysis result includes a camera ID that identifies a surveillance camera from among the surveillance cameras C2-1 to C2-n that acquired the second person image data. The second video analysis result includes a similarity of the person image based on the second person image data to the registered image based on the registered image data.

In addition, the video analysis engines VE1 to VEn do not have to be arranged one-to-one with the surveillance cameras C2-1 to C2-n. Instead, a smaller number of video analysis engines may be arranged for multiple surveillance cameras, and the video data from the multiple surveillance cameras may be processed collectively by the smaller number of video analysis engines.

In addition, multiple output devices MT1-1 to MT1-n are distributed throughout the facility. Each of the output devices MT1-1 to MT1-n is installed near (in the same area as) the positions of the surveillance cameras C2-1 to C2-n. The output devices MT1-1 to MT1-n are devices that include a display that displays information. The output devices MT1-1 to MT1-n are also devices that include a speaker that outputs information audibly. The output devices MT1-1 to MT1-n are examples of display devices, audio output devices, or output devices. Visitors to the facility can check the information output by the output devices MT1-1 to MT1-n.

The system of one embodiment includes a Web server device SV used as an information processing device. The Web server device SV is a device that performs information processing based on data acquired by a thermal camera C1 and each of the distributed surveillance cameras C2-1 to C2-n. The video analysis engines VE1 to VEn are capable of data communication with the Web server device SV via a network NW. The video analysis engines VE1 to VEn transmit the first video analysis result or the second video analysis result to the Web server device SV via the network NW. The network NW is, for example, a wired LAN (Local Area Network) or a wireless LAN, but may be any other network.

The Web server device SV may be equipped with video analysis engines VE1 to VEn or one video analysis engine, and the video analysis engines VE1 to VEn or one video analysis engine of the Web server device SV may receive image data from each of the surveillance cameras C2-1 to C2-n via the network NW and analyze the received image data.

The system of one embodiment includes an output device MT2 and an administrator terminal OT. The output device MT2 is connected to the Web server device SV. The output device MT2 is a device including a display that displays information output from the Web server device SV. The output device MT2 is also a device including a speaker that outputs the information output from the Web server device SV as audio. The output device MT2 may be a wearable device or smart glasses that can be carried by each monitor. The output device MT2 is an example of a display device, audio output device, or output device. The monitor can check the information output by the output device MT2. The administrator terminal OT is connected to the Web server device SV. The administrator terminal OT is a terminal into which the administrator inputs various settings for the Web server device SV.

(2) Web server device SV
FIG. 2 is a block diagram showing an example of a hardware configuration of the Web server device SV.
The Web server device SV includes a control unit 1, a program storage unit 2, a data storage unit 3, an input/output I/F (interface) 4, and a communication I/F 5. The elements constituting the Web server device SV are connected to each other via a bus 6.

The control unit 1 corresponds to the central part of the Web server device SV. The control unit 1 has a hardware processor such as a central processing unit (CPU). The control unit 1 has a ROM (Read Only Memory) as a non-volatile memory area. The control unit 1 has a RAM (Random Access Memory) as a volatile memory area.

The program storage unit 2 is configured with a non-volatile memory such as a hard disk drive (HDD) or solid state drive (SSD) as a storage medium that can be written to and read from at any time. The program storage unit 2 stores programs necessary to execute various control processes. For example, the program storage unit 2 stores an information processing program.

The data storage unit 3 is configured with a non-volatile memory such as an HDD or SSD as a storage medium that can be written to and read from at any time. The data storage unit 3 stores management data 31, detection history data 32, a management table 33, a detection history table 34, a camera information table 35, and an area information table 36.

The management data 31 includes registered image data of each monitored person. The registered image data of the monitored person corresponds to the first person image data of a person whose fever state is "fever" or whose mask wearing state is "no mask". For example, the registered image data of the monitored person corresponds to the first person image data of a person whose fever state is "fever". The registered image data of the monitored person corresponds to the first person image data of a person whose mask wearing state is "no mask". The registered image data of the monitored person corresponds to the first person image data of a person whose combination of fever state and mask wearing state is "no fever" and "no mask", "fever" and "mask wearing", or "fever" and "no mask". Hereinafter, the combination of fever state and mask wearing state is also referred to as a combination of states. The registered image data can be used not only by the Web server device SV but also by the video analysis engines VE1 to VEn. The registered image data is associated with the monitored person ID. The monitored person ID is an ID given to each monitored person. The monitored person ID can be set as appropriate. The monitored person ID may correspond to the person ID of the person who is the monitored person. The management data 31 may be updated based on the addition or deletion of registered image data.

The detection history data 32 includes detection image data of each monitored person. The detection image data includes second person image data of the monitored person detected by the control unit 1 as described below. The detection image data may include second captured image data that is the basis of the first person image data registered as registered image data of the monitored person. The detection image data may include second captured image data that is the basis of the second person image data. The detection image data is associated with the monitored person ID. The detection history data 32 may be updated based on the addition or deletion of detection image data.

The management table 33 manages records for each monitored person ID in which the monitored person ID, the monitored person name, information indicating the monitored person's fever state, and information indicating the monitored person's mask wearing state are associated with each other. The monitored person ID may correspond to the person ID of the monitored person. The monitored person name is a name such as the name of the monitored person. The monitored person name may be set as appropriate. The information indicating the monitored person's fever state is information indicating the monitored person's "fever" or "no fever." If the monitored person is not judged to be in a fever state, the information indicating the monitored person's fever state is not registered. The information indicating the monitored person's mask wearing state is information indicating the monitored person's "mask wearing" or "no mask wearing." If the monitored person is not judged to be in a mask wearing state, the information indicating the monitored person's mask wearing state is not registered. The management table 33 may be updated based on updates to the data constituting the record. The management table 33 may be updated based on addition or deletion of a record.

The detection history table 34 manages records for each detection ID in which the detection ID, the monitored person ID, the detection camera ID, and the detection date and time are associated with each other. The detection ID is an ID given to each detection image data. The detection ID can be set as appropriate. The detection camera ID is an ID that identifies the monitoring camera that acquired the detection image data. The detection camera ID corresponds to the camera ID. The camera ID is an ID given to each of the monitoring cameras C2-1 to C2-n. The camera ID can be set as appropriate. The detection date and time is the date and time when the monitoring target person is detected based on the second person image data by the control unit 1 as described later. The date and time when the monitoring target person is detected based on the second person image data by the control unit 1 corresponds to the date and time when the second person image data is captured by any of the monitoring cameras C2-1 to C2-n. The detection date and time is the date and time for each detection. The detection history table 34 can be updated based on updates to the data that constitutes the record. The detection history table 34 can be updated based on addition or deletion of a record.

The camera information table 35 manages records for each camera ID that associates the camera ID with the camera name, etc. The camera name is a name related to the surveillance camera. For example, the camera name is the name of the installation location of the surveillance camera, etc. The camera name can be set as appropriate. The camera information table 35 can be updated based on updates to the data that constitutes the record. The camera information table 35 can be updated based on the addition or deletion of a record.

The area information table 36 manages area information regarding the no-entry areas set based on at least one of the fever state and the mask wearing state. For example, the area information includes information regarding the no-entry areas for people with a fever, the no-entry areas for people without a mask, the no-entry areas for people without a fever and not wearing a mask, the no-entry areas for people with a fever and wearing a mask, and the no-entry areas for people with a fever and not wearing a mask. Hereinafter, the no-entry areas for people without a fever and not wearing a mask are also referred to as the first no-entry areas. The no-entry areas for people with a fever and wearing a mask are also referred to as the second no-entry areas. The no-entry areas for people with a fever and not wearing a mask are also referred to as the third no-entry areas. The area information only needs to indicate which areas in the facility the first no-entry areas, the second no-entry areas, and the third no-entry areas correspond to.

The first no-entry area may be set to one or more areas in the facility. For example, the first no-entry area may be set to an event space where people tend to gather, an area where elderly people tend to stay, a kids' space, and the like. The second no-entry area may be set to one or more areas in the facility. For example, the second no-entry area may be set to an area other than an area where people with a fever and wearing a mask are likely to usually come, such as a hospital examination area or an infected person reception area. The third no-entry area may be set to one or more areas in the facility. For example, the third no-entry area may be set to all areas in the facility. An area in the facility where none of the first no-entry area, the second no-entry area, and the third no-entry area is set is an area where any person can enter, regardless of the combination of conditions. There may be areas in the facility that are set as two or more of the first no-entry area, the second no-entry area, and the third no-entry area. No-entry areas for people with a fever and no-entry areas for people not wearing a mask can also be set in one or more areas within the facility.

The area information table 36 manages records in which a first no-entry area and the camera IDs of one or more surveillance cameras are associated with each other. The camera ID associated with the first no-entry area is the camera ID of one or more surveillance cameras installed in one or more areas in the facility set as the first no-entry area. The area information table 36 manages records in which a second no-entry area and the camera IDs of one or more surveillance cameras are associated with each other. The camera ID associated with the second no-entry area is the camera ID of one or more surveillance cameras installed in one or more areas in the facility set as the second no-entry area. The area information table 36 manages records in which a third no-entry area and the camera IDs of one or more surveillance cameras are associated with each other. The camera ID associated with the third no-entry area is the camera ID of one or more surveillance cameras installed in one or more areas in the facility set as the third no-entry area. Similarly, the area information table 36 manages records in which a no-entry area for a person with a fever and the camera IDs of one or more surveillance cameras are associated with each other. The area information table 36 manages records that associate areas where people who are not wearing masks cannot enter with the camera IDs of one or more surveillance cameras.

The setting and cancellation of the first no-entry area, the second no-entry area, and the third no-entry area for each area within the facility can be updated as appropriate. Similarly, the setting and cancellation of the no-entry areas for people with a fever and people not wearing a mask for each area within the facility can be updated as appropriate. The area information table 36 can be updated based on updates to the data that makes up the records. The area information table 36 can be updated based on the addition or deletion of records.

The input/output I/F 4 is configured with an interface that connects to the output device MT and the administrator terminal OT.
The communication I/F 5 transmits data between the thermal camera C1 and the image analysis engines VE2-1 to VE2-n using a communication protocol defined by the network NW under the control of the control unit 1. The communication I/F 5 transmits data between the output devices MT1-1 to MT1-n under the control of the control unit 1. For example, the communication I/F 5 is configured with an interface compatible with a wired LAN or a wireless LAN.

FIG. 3 is a block diagram showing an example of the software configuration of the Web server device SV.
The control unit 1 includes a first determination unit 11, a second determination unit 12, an output unit 13, a recording unit 14, a detection unit 15, and a judgment unit 16. Each unit is realized by causing a hardware processor of the control unit 1 to execute an information processing program stored in the program storage unit 2. Each unit can also be said to be included in the hardware processor. Each unit can also be said to be each function. Note that each unit described below can be read as the control unit 1 or the hardware processor.

The first determination unit 11 determines the fever state of the person based on the first captured image data acquired by the thermal camera C1.
The second determination unit 12 determines whether the person is wearing a mask based on the second captured image data acquired by the surveillance camera C2-1.
The output unit 13 outputs an alert based on at least one of the first judgment result on the fever state by the first judgment unit 11 and the second judgment result on the mask wearing state by the second judgment unit 12. The first judgment result on the fever state by the first judgment unit 11 indicates "no fever" or "fever". The second judgment result on the mask wearing state by the second judgment unit 12 indicates "mask wearing" or "mask not wearing". Hereinafter, the combination of the first judgment result on the fever state by the first judgment unit 11 and the second judgment result on the mask wearing state by the second judgment unit 12 is also referred to as a combination of judgment results.

The recording unit 14 records data on the monitored person in the data storage unit 3 .
The detection unit 15 detects a monitoring target. The detection unit 15 may detect the same monitoring target only once. The detection unit 15 may detect the same monitoring target multiple times at different times. The detection unit 15 may detect the same monitoring target in different areas at different times. The detection unit 15 acquires the detection date and time based on the detection.
The determination unit 16 determines, based on the detection of the monitored person by the detection unit 15, whether or not the monitored person is in a no-entry area.

(Example of operation)
Next, an example of the operation of the Web server device SV configured as above will be described.
4 is a flowchart showing the procedure and contents of the output process based on the detection of a person executed by the Web server SV. For example, the Web server SV performs the process shown in FIG. 4 for each person entering the facility.

The first determination unit 11 determines the fever state of the person based on the first captured image data acquired by the thermal camera C1 (step S1). In step S1, for example, the first determination unit 11 acquires the first captured image data from the thermal camera C1. Based on the first captured image data, the first determination unit 11 recognizes the position of the person's face shown in the image based on the first captured image data and estimates the person's body temperature. The first determination unit 11 determines the fever state based on the estimated body temperature. The first determination unit 11 determines the fever state as "fever" or "no fever". Note that the above-mentioned process of estimating the person's body temperature may be performed by the engine of the thermal camera C1.

The second determination unit 12 determines whether the person is wearing a mask based on the second captured image data acquired by the surveillance camera C2-1 (step S2). In step S2, for example, the second determination unit 12 acquires a first video analysis result related to the surveillance camera C2-1 from the video analysis engine VE2-1. The second determination unit 12 determines the mask wearing state of the person determined to be in a fever state by the first determination unit 11 based on information indicating the mask wearing state included in the first video analysis result. The person determined to be in a fever state by the first determination unit 11 and the person determined to be in a mask wearing state by the second determination unit 12 may be associated with the same person as follows. The angle of view of the thermal camera C1 and the angle of view of the surveillance camera C2-1 are assumed to be the same. Each coordinate of the image based on the first captured image data and each coordinate of the image based on the second captured image data are associated with each other for each position at which the image is taken. The second determination unit 12 compares the coordinates of the person in the image who has been determined to have a fever with the coordinates in the image of the person who has been determined to have a mask. If the second determination unit 12 determines that the people are in the same position based on the comparison, it determines that the person who has been determined to have a mask and the person who has been determined to have a fever are the same person. Note that the information indicating the mask-wearing state is generated based on the second captured image data. Therefore, determining whether a person has a mask on based on the information indicating the mask-wearing state is an example of determining whether a person has a mask on based on the second captured image data.

The second determination unit 12 may have a video analysis function similar to that of the video analysis engine VE1. In this case, the second determination unit 12 may analyze the second captured image data acquired by the surveillance camera C2-1 and determine whether the person is wearing a mask.

The output unit 13 outputs an alert based on at least one of the first judgment result and the second judgment result (step S3). In step S3, for example, the output unit 13 outputs an alert according to the first judgment result, the second judgment result, or a combination of the judgment results. The output unit 13 outputs the alert to the output device MT2. The output unit 13 may output the alert to at least one of the output devices MT1-1 to MT1-n.

The output unit 13 outputs a first alert based on the combination of judgment results for the person being judged being "no fever" and "not wearing a mask". The output unit 13 outputs a second alert based on the combination of judgment results being "fever" and "wearing a mask". The output unit 13 outputs a third alert based on the combination of judgment results being "fever" and "not wearing a mask". The output unit 13 does not output an alert when the combination of judgment results is "no fever" and "wearing a mask".

An example of an alert output by the output unit 13 will be described.
For example, the output unit 13 outputs a message indicating that a person with "no fever" and "no mask" has been detected to the output device MT2 as a first alert. The displayed message may be a visually identifiable message, and may be a text message or a mark message. The output device MT2 may output the first alert by voice or by display. The output unit 13 may output a map showing the location of a person with "no fever" and "no mask" to the output device MT2 as the first alert. The output device MT2 outputs the map by display. For example, the output unit 13 outputs a message indicating that a person with "fever" and "wearing a mask" has been detected to the output device MT2 as a second alert. The output unit 13 may output a map showing the location of a person with "fever" and "wearing a mask" to the output device MT2 as the second alert. For example, the output unit 13 outputs a message indicating that a person with a fever and not wearing a mask has been detected as the third alert to the output device MT2. The output unit 13 may output a map indicating the locations of people with a fever and not wearing a mask as the third alert to the output device MT2.

The risk of spreading infectious diseases may be higher for a person with no fever and not wearing a mask, a person with a fever and wearing a mask, and a person with a fever and not wearing a mask. In particular, it is preferable to refuse entry to a facility for a person with a fever and not wearing a mask. Therefore, the priority of the supervisor to take action such as calling out to a person may be higher for a person with no fever and not wearing a mask, a person with a fever and wearing a mask, and a person with a fever and not wearing a mask. The output unit 13 may increase the range of output destinations (the number of supervisors carrying the output device MT2) to which the alerts are output in the order of the first alert, the second alert, and the third alert. The output unit 13 may change the output mode of the alerts for the first alert, the second alert, and the third alert. The output unit 13 may output only the display output for the first alert, only the audio output for the second alert, and both the display output and the audio output for the third alert.

Similarly, the output unit 13 may output, as an alert, a message indicating that a person with a fever has been detected. The output unit 13 may output, as an alert, a map indicating the location of the person with a fever. The output unit 13 may output, as an alert, a message indicating that a person not wearing a mask has been detected. The output unit 13 may output, as an alert, a map indicating the location of the person not wearing a mask.
The contents of the alert output by the output unit 13, the range of output destinations, the output format, and the like are not limited to those described above and can be changed as appropriate.

As described above, the Web server device SV outputs an alert based on at least one of the fever state and the mask wearing state. In a typical example, the Web server device SV outputs an alert according to the fever state, the mask wearing state, or a combination of states. This allows the Web server device SV to realize a notification according to the risk of spreading infection of each person by using at least one of the fever state and the mask wearing state. The monitor can take measures according to the degree of risk of spreading infection of the detected person. For example, the monitor can track people with "fever" and "not wearing a mask" so that they do not enter the facility. The monitor can take measures against people with "no fever" and "not wearing a mask" or people with "fever" and "wearing a mask" without multiplying the number of people. The monitor can take measures by giving priority to people with "fever" and "not wearing a mask" over people with "fever", people without "fever" and "not wearing a mask", people with "fever" and "wearing a mask".

The output unit 13 may output an alert to the output devices MT1-1 to MT1-n according to the distance from the position of the surveillance camera C2-1 to each of the output devices MT1-1 to MT1-n. In this example, the output unit 13 outputs, as an alert, a message indicating that at least a person with a fever or not wearing a mask has been detected to the output device MT1-1 installed near the position of the surveillance camera C2-1. On the other hand, the output unit 13 outputs, as an alert, a map indicating the positions of at least a person with a fever or not wearing a mask to at least one of the output devices MT1-2 to MT1-n.

An example will be described in which the combination of judgment results is "fever" and "not wearing a mask." The output unit 13 outputs a message indicating that a person with "fever" and "not wearing a mask" has been detected as an alert to the output device MT1-1 installed near the position of the surveillance camera C2-1. On the other hand, the output unit 13 outputs a map indicating the positions of people with "fever" and "not wearing a mask" as an alert to at least one of the output devices MT1-2 to MT1-n.

That is, the output unit 13 outputs a message indicating that at least a person with a fever or not wearing a mask has been detected as an alert to the output device of the first output device and the second output device that is closest to the position of the surveillance camera C2-1. For example, the output unit 13 outputs a message indicating that a person with a fever and not wearing a mask has been detected as an alert to the output device of the first output device and the second output device that is closest to the position of the surveillance camera C2-1. The first output device and the second output device are output devices among the output devices MT1-1 to MT1-n. For example, the output device closest to the position of the surveillance camera C2-1 is the output device MT1-1 installed in the vicinity of the position of the surveillance camera C2-1. The output unit 13 outputs a map indicating the positions of at least a person with a fever or not wearing a mask as an alert to the output device of the first output device and the second output device that is farthest from the position of the surveillance camera C2-1. For example, the output unit 13 outputs a map showing the location of a person who has a fever and is not wearing a mask as an alert to the output device farthest from the surveillance camera C2-1 out of the first output device and the second output device. For example, the output device farthest from the surveillance camera C2-1 is any one of the output devices MT1-2 to MT1-n. In this example, the Web server device SV may store information showing the correspondence between each of the surveillance cameras C2-1 to C2-n and each of the output devices MT1-1 to MT1-n. The Web server device SV may store information showing the positional relationship (distance) between the surveillance cameras C2-1 to C2-n.

According to this example, the Web server device SV can notify other people at the detection location in real time that a person who poses a risk of spreading an infectious disease has been detected. This allows people who are near a person who poses a risk of spreading an infectious disease to be vigilant against people who pose a risk of spreading an infectious disease who are near them. The Web server device SV can notify people who are away from the detection location of a person who poses a risk of spreading an infectious disease in real time. This allows people in the facility to avoid approaching the detection location of a person who poses a risk of spreading an infectious disease.

Figure 5 is a flowchart showing the processing steps and processing contents of the output processing based on tracking of a monitored person executed by the web server device SV.

The recording unit 14 judges whether or not the fever state of the person to be judged is at least "fever" or the mask wearing state is "no mask" (step S11). If the fever state is not at least "fever" or the mask wearing state is not "no mask" (step S11, NO), the process ends. In other words, a person with "no fever", a person wearing a mask, and a person with "no fever" and "wearing a mask" are not monitored. If at least the fever state is "fever" or the mask wearing state is "no mask" (step S11, YES), the process transitions from step S11 to step S12. If at least the fever state is "fever" or the mask wearing state is "no mask", the fever state is "fever" and the mask wearing state is "no mask", or the combination of the fever state and the mask wearing state is "no fever" and "no mask", "fever" and "mask wearing", or "fever" and "no mask".

The recording unit 14 registers as a person to be monitored at least when the fever state is "fever present" or the mask wearing state is "mask not worn" (step S12). In step S12, for example, the recording unit 14 records the first person image data included in the first video analysis result as registered image data of the person to be monitored in the management data 31 based on at least one of the first judgment result and the second judgment result. The recording unit 14 may record the second captured image data that is the basis of the first person image data included in the first video analysis result as detection image data in the detection history data 32. The recording unit 14 records at least one of information indicating the fever state of the person to be monitored and information indicating the mask wearing state of the person to be monitored in the management table 33 based on at least one of the first judgment result and the second judgment result.

The detection unit 15 detects the person to be monitored (step S13). In step S13, for example, the detection unit 15 acquires a second video analysis result from one of the video analysis engines VE1 to VEn after the recording unit 14 registers the person to be monitored. The second video analysis result is based on the second captured image data acquired by one of the surveillance cameras C2-1 to C2-n. The detection unit 15 compares the similarity included in the second video analysis result with a threshold value. The threshold value is a reference value for determining whether or not the person shown in the second person image data included in the second video analysis result is considered to be a person to be monitored. The threshold value can be set appropriately. If the similarity is equal to or greater than the threshold value, the detection unit 15 detects the person shown in the second person image data as a person to be monitored. Note that the similarity is generated based on the second person image data and the registered image data of the person to be monitored. Therefore, detecting a person to be monitored based on the similarity is an example of detecting a person to be monitored based on the second captured image data and the registered image data of the person to be monitored.

The detection unit 15 may have a video analysis function similar to that of the video analysis engines VE1 to VEn. In this case, the detection unit 15 may analyze the second captured image data acquired by one of the surveillance cameras C2-1 to C2-n to detect the person being monitored.

Based on the detection of the monitored person by the detection unit 15, the recording unit 14 records the data of the monitored person in the data storage unit 3, as exemplified below. The recording unit 14 records the second person image data showing the monitored person as detection image data in the detection history data 32 in association with the monitored person ID. The recording unit 14 may record the second captured image data that is the basis of the second person image data showing the monitored person as detection image data in the detection history data 32 in association with the monitored person ID. Based on the detection of the monitored person by the detection unit 15, the recording unit 14 adds a record in which the detection ID, monitored person ID, detection camera ID, and detection date and time are associated with each other to the detection history table 34. The recording unit 14 uses the monitored person ID included in the second video analysis result. The recording unit 14 uses the camera ID included in the second video analysis result as the detection camera ID. The recording unit 14 uses the detection date and time acquired by the detection unit 15. The recording unit 14 adds a record to the detection history table 34 each time the detection unit 15 detects a monitored person.

The second determination unit 12 determines whether the monitored person is wearing a mask based on the second captured image data related to the detection of the monitored person by the detection unit 15 based on the detection of the monitored person by the detection unit 15 (step S14). In step S14, for example, the second determination unit 12 determines whether the monitored person is wearing a mask based on information indicating the mask wearing state included in the second video analysis result used to detect the monitored person by the detection unit 15. The second captured image data related to the detection of the monitored person by the detection unit 15 is the second captured image data showing the monitored person detected by the detection unit 15. Note that the information indicating the mask wearing state is generated based on the second captured image data related to the detection of the person who will be the monitored person. The second captured image data related to the detection of the person who will be the monitored person is the second captured image data showing the person who will be the monitored person. Therefore, determining whether the monitored person is wearing a mask based on information indicating the mask wearing state is an example of determining whether the monitored person is wearing a mask based on the second captured image data.
The second determination unit 12 may have a video analysis function similar to that of the video analysis engines VE1 to VEn. In this case, the second determination unit 12 may analyze the second captured image data acquired by any one of the surveillance cameras C2-1 to C2-n to determine whether the surveillance subject is wearing a mask.

The recording unit 14 judges whether the mask wearing state of the monitored person has changed (step S15). In step S15, for example, the recording unit 14 compares the second judgment result for the monitored person by the second judgment unit 12 with the information indicating the mask wearing state of the monitored person stored in the management table 33. If the second judgment result is different from the information indicating the mask wearing state of the monitored person stored in the management table 33, the recording unit 14 judges that the mask wearing state of the monitored person has changed. On the other hand, if the second judgment result is the same as the information indicating the mask wearing state of the monitored person stored in the management table 33, the recording unit 14 judges that the mask wearing state of the monitored person has not changed. If the mask wearing state of the monitored person has changed (step S15, YES), the process transitions from step S15 to step S16. If the mask wearing state of the monitored person has not changed (step S15, NO), the process transitions from step S15 to step S17.

When the mask-wearing status of the monitored person changes, the recording unit 14 updates the information indicating the mask-wearing status of the monitored person stored in the management table 33 (step S16). In step S16, for example, the recording unit 14 updates the information indicating the mask-wearing status of the monitored person stored in the management table 33 so as to correspond to the mask-wearing status determined by the second determination unit 12 in step S14.

Based on the detection of the monitored person by the detection unit 15, the judgment unit 16 judges whether the monitored person is in an inaccessible area based on the area information and at least one of the information indicating the monitored person's fever state and the information indicating the monitored person's mask wearing state (step S17). In step S17, for example, the judgment unit 16 acquires at least one of the information indicating the monitored person's fever state and the information indicating the monitored person's mask wearing state from the management table 33. Here, if both the information indicating the monitored person's fever state and the information indicating the monitored person's mask wearing state are registered in the management table 33, the judgment unit 16 acquires both pieces of information. If the information indicating the monitored person's fever state or the information indicating the monitored person's mask wearing state is registered in the management table 33, the judgment unit 16 acquires one of the registered pieces of information. The judgment unit 16 judges the state or combination of states of the monitored person based on at least one of the information indicating the monitored person's fever state and the information indicating the monitored person's mask wearing state. The judgment unit 16 compares the state or combination of states of the monitored person with the area information stored in the area information table 36. The determination unit 16 determines the no-entry area for the person being monitored based on the comparison. For example, the determination unit 16 determines whether the no-entry area for the person being monitored is the first no-entry area, the second no-entry area, or the third no-entry area based on a combination of the states of the person being monitored.

The determination unit 16 determines whether or not the camera ID included in the second video analysis result is associated with a no-entry area for the monitored person in the area information table 36. The determination unit 16 determines whether or not the monitored person is in a no-entry area depending on whether or not the camera ID included in the second video analysis result is associated with a no-entry area for the monitored person. If the camera ID included in the second video analysis result is associated with a no-entry area for the monitored person, the determination unit 16 determines that the monitored person is in a no-entry area. If the camera ID included in the second video analysis result is not associated with a no-entry area for the monitored person, the determination unit 16 determines that the monitored person is not in a no-entry area.

If the person to be monitored is in a no-entry area (step S17, YES), the process transitions from step S17 to step S18. If the person to be monitored is not in a no-entry area (step S17, NO), the process ends.

The output unit 13 outputs an alert based on the determination by the determination unit 16 that the monitored person is in a no-entry area (step S18). In step S18, for example, the output unit 13 outputs an alert regarding the detection of the monitored person in a no-entry area to the output device MT2. The output unit 13 may output the alert to at least one of the output devices MT1-1 to MT1-n. For example, the output unit 13 outputs a message indicating that the monitored person is in a no-entry area as an alert. For example, the output unit 13 outputs a map indicating the position of the monitored person in the no-entry area as an alert.

The output unit 13 may output an alert according to the fever state, mask wearing state, and combination of states of the monitored person. In this example, the output unit 13 may output, as an alert, a message indicating that the monitored person with "no fever" and "no mask" is in the first no-entry area. The output unit 13 may output, as an alert, a map showing the position of the monitored person with "no fever" and "no mask" in the first no-entry area. The output unit 13 may output, as an alert, a message indicating that the monitored person with "fever" and "wearing a mask" is in the second no-entry area. The output unit 13 may output, as an alert, a map showing the position of the monitored person with "fever" and "wearing a mask" in the second no-entry area. The output unit 13 may output, as an alert, a message indicating that the monitored person with "fever" and "no mask" is in the third no-entry area. The output unit 13 may output, as an alert, a map showing the position of the monitored person with "fever" and "no mask" in the third no-entry area. The output unit 13 may also output a message or map as an alert for monitored individuals who have a fever or are not wearing a mask. The output unit 13 may increase the range of output destinations (the number of monitors who possess the output device MT2) to which the alert is output depending on the status or combination of status of the monitored individual. The output unit 13 may change the output mode of the alert depending on the combination of status of the monitored individual.

Note that the content of the alert output by the output unit 13, the range of output destinations, and the output format are not limited to those described above and can be changed as appropriate.

As described above, the Web server device SV outputs an alert based on the fact that the monitored person is determined to be in a no-entry area. This allows the Web server device SV to realize tracking according to the state or combination of states of the monitored person. The monitor can take measures according to the degree of risk of spreading infection of the monitored person determined to have entered the no-entry area. For example, the monitor can take immediate measures by increasing the number of people to ensure that the monitored person who has "fever" and "not wearing a mask" leaves the no-entry area. The monitor can take measures without increasing the number of people for the monitored person who has "fever" and "not wearing a mask" or the monitored person who has "fever" and "wears a mask". The monitor can take measures by giving priority to the monitored person who has "fever" and "not wearing a mask" over the monitored person who has "fever", the monitored person who has "not wearing a mask", the monitored person who has "fever" and "not wearing a mask", or the monitored person who has "fever" and "wears a mask".

The output unit 13 may output to the output devices MT1-1 to MT1-n an alert according to the distance from the position of the surveillance camera that acquired the second captured image data related to the detection of the monitored person to each of the output devices MT1-1 to MT1-n. The surveillance camera that acquired the second captured image data related to the detection of the monitored person is the surveillance camera that acquired the second captured image data showing the monitored person. Hereinafter, the surveillance camera that acquired the second captured image data related to the detection of the monitored person is also referred to as the surveillance camera that captured the monitored person. In this example, the output unit 13 outputs a message indicating that the monitored person is in a no-entry area as an alert to an output device installed near the position of the surveillance camera that captured the monitored person. On the other hand, the output unit 13 outputs a map indicating the position of the monitored person in the no-entry area as an alert to one or more output devices installed near the positions of one or more surveillance cameras different from the surveillance camera that captured the monitored person.

An example will be described in which the person being monitored has a fever and is not wearing a mask. The output unit 13 outputs a message indicating that the person being monitored who has a fever and is not wearing a mask is in the third no-entry area to an output device installed near the position of the surveillance camera that captured the image of the person being monitored. On the other hand, the output unit 13 outputs a map indicating the position of the person being monitored who has a fever and is not wearing a mask and is in the third no-entry area to one or more output devices installed near the positions of one or more surveillance cameras different from the surveillance camera that captured the image of the person being monitored.

That is, the output unit 13 outputs a message indicating that a monitored person who has at least a fever or is not wearing a mask is in the no-entry area to the output device among the first output device and the second output device that is closer to the position of the surveillance camera that captured the monitored person. For example, the output unit 13 outputs a message indicating that a monitored person who has a fever and is not wearing a mask is in the third no-entry area to the output device among the first output device and the second output device that is closer to the position of the surveillance camera that captured the monitored person. The first output device and the second output device are output devices among the output devices MT1-1 to MT1-n. For example, the output device closest to the position of the surveillance camera that captured the monitored person is an output device installed in the vicinity of the position of the surveillance camera that captured the monitored person. The output unit 13 outputs a map indicating the position of a monitored person who has at least a fever or is not wearing a mask in the no-entry area to the output device among the first output device and the second output device that is far from the position of the surveillance camera that captured the monitored person. For example, the output unit 13 outputs, as an alert, a map showing the location of the monitored person who has a fever and is not wearing a mask and is in the third no-entry area to the output device farthest from the surveillance camera that captured the image of the monitored person out of the first output device and the second output device. For example, the output device farthest from the surveillance camera that captured the image of the monitored person is an output device installed near the location of a surveillance camera different from the surveillance camera that captured the image of the monitored person.

According to this example, the Web server device SV can notify other people in the area in real time that a monitored person posing a risk of spreading an infectious disease is in a no-entry area. This allows people near the monitored person posing a risk of spreading an infectious disease to be wary of other monitored people posing a risk of spreading an infectious disease who are near them. The Web server device SV can notify people who are away from the location where the monitored person posing a risk of spreading an infectious disease is detected in real time of the location of the monitored person posing a risk of spreading an infectious disease. This allows people in the facility to avoid approaching the location where the monitored person posing a risk of spreading an infectious disease is detected.

The processing of steps S14 to S16 in the flowchart shown in FIG. 5 may be omitted. In this example, the web server device SV does not need to determine whether the monitored person is wearing a mask after registering the monitored person.

The information processing device may be realized as a single device as described in the above example, or may be realized as multiple devices with distributed functions.

The program may be transferred in a state where it is stored in a device, or in a state where it is not stored in a device. In the latter case, the program may be transferred via a network, or in a state where it is recorded on a recording medium. The recording medium is a non-transitory tangible medium. The recording medium is a computer-readable medium. The recording medium may be in any form, such as a CD-ROM or memory card, as long as it is capable of storing the program and is computer-readable.

Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

In short, this invention is not limited to the above-mentioned embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.
The original claims are set forth below.
[1]
a first determination unit that determines a fever state of a person based on image data from a first camera;
A second determination unit that determines whether the person is wearing a mask based on image data from a second camera;
an output unit that outputs an alert based on at least one of a first determination result regarding a fever state by the first determination unit and a second determination result regarding a mask wearing state by the second determination unit;
An information processing device comprising:
[2]
The information processing device according to [1], wherein the output unit outputs an alert corresponding to the first determination result, the second determination result, or a combination of the first determination result and the second determination result for the person.
[3]
The information processing device according to [1], wherein the output unit outputs a message indicating that at least a person with a fever or not wearing a mask has been detected as the alert to one of the first output device and the second output device that is closer to the position of the second camera.
[4]
A storage unit that stores area information regarding a no-entry area that is set based on at least one of a fever state and a mask wearing state;
a recording unit that records image data of the person in the storage unit as image data of a person to be monitored based on at least one of the first determination result and the second determination result for the person, and records at least one of information indicating a fever state of the person to be monitored and information indicating a state in which the person to be monitored is wearing a mask in the storage unit;
a detection unit that detects the person to be monitored based on image data from a plurality of second cameras including the second camera and image data of the person to be monitored stored in the storage unit;
a determination unit that determines whether the person to be monitored is in a no-entry area based on the area information stored in the storage unit and at least one of information indicating a fever state of the person to be monitored and information indicating a state in which the person to be monitored is wearing a mask, based on the detection of the person to be monitored by the detection unit;
Further equipped with
The output unit outputs an alert based on the determination by the determination unit that the monitoring target person is in a no-entry area.
The information processing device according to [1].
[5]
The information processing device described in [4], wherein the output unit outputs an alert according to the fever state, mask wearing state, and a combination of the fever state and mask wearing state of the monitored person based on the judgment unit determining that the monitored person is in a no-entry area.
[6]
The second determination unit, based on the detection of the monitoring subject by the detection unit, determines whether the monitoring subject is wearing a mask based on image data related to the detection of the monitoring subject by the detection unit;
When the second determination result by the second determination unit for the monitoring subject differs from the information stored in the storage unit that indicates whether the monitoring subject is wearing a mask, the recording unit updates the information stored in the storage unit that indicates whether the monitoring subject is wearing a mask.
The information processing device according to [4].
[7]
determining a fever state of the person based on image data from a first camera;
determining a mask-wearing state of the person based on image data from a second camera;
outputting an alert based on at least one of a first determination result regarding a fever state and a second determination result regarding a mask wearing state;
Information processing methods.
[8]
An information processing program that causes a computer to execute processing by each unit of the information processing device according to any one of [1] to [6].

SV: Web server device OT: Administrator terminal MT1-1 to MT1-n: Output device MT2: Output device NW: Network C1: Thermal camera C2-1 to C2-n: Surveillance camera VE1 to VEn: Image analysis engine 1: Control unit 2: Program storage unit 3: Data storage unit 4: Input/output I/F
5...Communication I/F
REFERENCE SIGNS 6: bus 11: first determination section 12: second determination section 13: output section 14: recording section 15: detection section 16: judgment section 31: management data 32: detection history data 33: management table 34: detection history table 35: camera information table 36: area information table

Claims (7)

A storage unit that stores area information regarding no-entry areas that are set based on a fever state and a mask wearing state;
a first determination unit that determines a fever state of a person based on image data from a first camera;
A second determination unit that determines whether the person is wearing a mask based on image data from a second camera;
a recording unit that records image data of the person in the storage unit as image data of the person to be monitored based on a first determination result regarding a fever state by the first determination unit and a second determination result regarding a mask wearing state by the second determination unit, and records information indicating the fever state of the person to be monitored and information indicating the mask wearing state of the person to be monitored in the storage unit;
a detection unit that detects the person to be monitored based on image data from a plurality of second cameras including the second camera and image data of the person to be monitored stored in the storage unit;
a determination unit that determines whether the person to be monitored is in a no-entry area based on the area information stored in the storage unit, information indicating the fever state of the person to be monitored, and information indicating whether the person to be monitored is wearing a mask, based on the detection of the person to be monitored by the detection unit; and
an output unit that outputs an alert based on the determination by the determination unit that the person to be monitored is in a no-entry area ;
An information processing device comprising: The information processing apparatus according to claim 1 , wherein the output unit outputs an alert according to a combination of the first determination result and the second determination result for the person. The information processing device according to claim 1, wherein the output unit outputs, as the alert, a message indicating that at least a person with a fever or not wearing a mask has been detected to the output device among the first output device and the second output device that is closer to the position of the second camera. The information processing device according to claim 1 , wherein the output unit outputs an alert according to a combination of the fever state and mask wearing state of the person to be monitored, based on the determination by the determination unit that the person to be monitored is in a no- entry area. The second determination unit, based on the detection of the monitoring subject by the detection unit, determines whether the monitoring subject is wearing a mask based on image data related to the detection of the monitoring subject by the detection unit;
When the second determination result by the second determination unit for the monitoring subject differs from the information stored in the storage unit that indicates whether the monitoring subject is wearing a mask, the recording unit updates the information stored in the storage unit that indicates whether the monitoring subject is wearing a mask.
The information processing device according to claim 1 . An information processing method executed by an information processing device,
determining a fever state of the person based on image data from a first camera;
determining a mask wearing state of the person based on image data from a second camera ;
recording image data of the person as image data of the person to be monitored based on a first determination result regarding the fever state and a second determination result regarding the mask wearing state in a storage unit that stores area information regarding a no-entry area set based on the fever state and the mask wearing state, and recording information indicating the fever state of the person to be monitored and information indicating the mask wearing state of the person to be monitored in the storage unit;
Detecting the person to be monitored based on image data from a plurality of second cameras including the second camera and image data of the person to be monitored stored in the storage unit;
Based on the detection of the person to be monitored, determining whether the person to be monitored is in a no-entry area based on the area information stored in the storage unit, information indicating the fever state of the person to be monitored, and information indicating the wearing state of a mask of the person to be monitored;
outputting an alert based on the determination that the person to be monitored is in a no-entry area ;
An information processing method comprising : 6. An information processing program for causing a computer to execute processing by each unit of the information processing device according to claim 1 .