JP7467304B2 - Fever tracking device, method and program - Google Patents

Description

Embodiments of the present invention relate to a fever-causing person tracking device, method, and program for tracking a fever-causing person moving around a facility, for example.

In general, in facilities where a large number of people come and go, such as shopping malls, event venues, and airports, measures are taken to prevent the spread of infectious diseases such as influenza and COVID-19 (novel coronavirus disease), such as asking people with fevers to refrain from entering the facility or guiding them to a quarantine room.

To take such measures, for example, images from a thermal camera can be used to detect people with fevers (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2-64800

However, even if a thermal camera is used to detect people with fevers, if a monitor misses the detection result or detects too many people with fevers at once and is unable to respond, and the person with a fever enters the facility, it is not possible to track them.

In particular, for those infected with infectious diseases that require disinfection procedures, such as COVID-19, the fact of infection is discovered after the time of their stay at the facility. For this reason, there is a demand for technology that can confirm the movement route of a person with a fever within a facility at a date after the time of their stay at the facility.

This invention was made with the above in mind, and its purpose is to make it possible to detect and store the movement routes of people with fever within a facility.

In order to solve the above problems, one aspect of the fever tracking device according to the present invention includes a detection unit that detects a person with a fever based on thermo data from a thermo camera installed at the entrance of a facility and detects the person with a fever from video data from multiple visible light cameras installed at the entrance and distributed throughout the facility, a known information storage unit that stores known information including at least map information of the facility and camera information showing the installation status of the multiple visible light cameras, a discrimination unit that discriminates the movement route of the person with a fever within the facility and the staying location where the person with a fever stayed for a specified time or longer based on the detection result of the detection unit and the known information, and a route storage unit that stores the movement route and the staying location of the person with a fever.

One aspect of the fever tracking device according to the present invention is a fever tracking method for tracking the movement of a fever person using video data from multiple visible light cameras, which includes the steps of: detecting the fever person based on thermo data from a thermo camera installed at the entrance of a facility; detecting the fever person from video data from the multiple visible light cameras installed at the entrance and distributed throughout the facility; determining the movement route of the fever person within the facility and the location where the fever person stayed for a specified time or longer based on known information including at least map information of the facility and camera information showing the installation status of the multiple visible light cameras stored in advance in a memory and the detection result of the fever person; and storing the movement route and the location of the fever person determined in the memory.

One aspect of the program of the present invention is a program that causes a computer to execute processing by each unit of one aspect of the fever tracking device of the present invention.

According to each aspect of the present invention, it becomes possible to detect and store the movement route of a person with a fever within a facility, and therefore it becomes possible to confirm the movement route of the person with a fever within the facility at a date later than the time the person with a fever stays at the facility.

FIG. 1 is a diagram showing an example of the overall configuration of a tracking system including a server device that operates as a fever tracking device according to a first embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the server device. FIG. 3 is a block diagram showing an example of a software configuration of the server device. FIG. 4 is a diagram showing an example of the contents of a human detection information table included in the human detection information storage unit of the server device. FIG. 5 is a diagram showing an example of the contents of a tracked target person information table included in the human detection information storage unit of the server device. FIG. 6 is a diagram showing an example of the contents of a tracing result information table provided in the tracing result storage unit of the server device. FIG. 7 is a diagram showing an example of the contents of a route information table provided in the tracing result storage unit of the server device. FIG. 8 is a flowchart illustrating an example of the overall processing procedure of the tracking process by the server device. FIG. 9 is a flowchart illustrating an example of a processing procedure of a human detection process included in the overall processing procedure illustrated in FIG. FIG. 10A is a diagram showing the first half of a flowchart illustrating an example of the processing procedure of the person tracking processing among the overall processing procedures shown in FIG. FIG. 10B is a diagram showing the latter half of the flowchart illustrating an example of the processing procedure of the person tracking processing among the overall processing procedures shown in FIG. FIG. 11 is a flowchart showing an example of a processing procedure of the output information generating and outputting process out of the overall processing procedure shown in FIG. FIG. 12 is a diagram showing an example of the contents of a registered individual information table provided in a known information storage unit of a server device operating as a fever person tracking device according to the second embodiment of the present invention. FIG. 13 is a diagram showing an example of the contents of a tracing result information table included in a tracing result storage unit of a server device in the second embodiment. FIG. 14A is a diagram illustrating the first half of a flowchart illustrating an example of a processing procedure of a person tracking process by a server device in the second embodiment. FIG. 14B illustrates the second half of the flowchart illustrating an example of the processing procedure of the person tracking process by the server device in the second embodiment. FIG. 15 is a flowchart illustrating an example of a processing procedure of an output information generating and outputting process by the server device according to the second embodiment. FIG. 16 is a diagram showing an example of the overall configuration of a tracking system including a server device that operates as a fever person tracking device according to the third embodiment of the present invention. FIG. 17 is a block diagram showing an example of a software configuration of a server device in the third embodiment. FIG. 18 is a diagram illustrating an example of the contents of a human detection information table included in a human detection information storage unit of a server device according to the third embodiment. FIG. 19 is a diagram showing an example of the contents of a tracked target human information table included in the human detection information storage unit of the server device in the third embodiment. FIG. 20 is a diagram showing an example of the contents of a registered individual information table included in the known information storage unit of the server device in the third embodiment. FIG. 21 is a diagram showing an example of the contents of a tracing result information table included in a tracing result storage unit of a server device in the third embodiment. FIG. 22 is a flowchart illustrating an example of an overall processing procedure of a tracking process by the server device according to the third embodiment. FIG. 23 is a flowchart illustrating an example of a processing procedure of a person detection process performed by the server device in the third embodiment. FIG. 24A is a diagram illustrating the first half of a flowchart illustrating an example of a processing procedure of person tracking processing by the server device in the third embodiment. FIG. 24B illustrates the second half of the flowchart illustrating an example of the processing procedure of the person tracking process by the server device in the third embodiment. FIG. 25 is a flowchart illustrating an example of a processing procedure of an output information generating and outputting process by the server device according to the third embodiment.

The following describes an embodiment of the present invention with reference to the drawings.

[First embodiment]
(Configuration example)
(1) System Fig. 1 is a diagram showing the overall configuration of a tracking system including a server device SV that operates as a fever tracking device according to the first embodiment of the present invention. The tracking system is installed in a facility used by an unspecified number of people, and tracks feverish people who enter the facility. In this specification, a facility means a section separated from the outside, such as a room, a building, a collection of buildings such as a shopping mall, an event venue, etc.

For this purpose, for example, a thermo camera SCM is placed at the entrance of the facility, and multiple surveillance cameras CM1 to CMn are distributed throughout the facility. The surveillance cameras CM1 to CMn are visible light cameras. These thermo camera SCM and surveillance cameras CM1 to CMn are installed, for example, on the ceiling or wall, and each captures a preset range. For example, the surveillance camera CM1 is installed at the entrance of the facility and captures a range common to the thermo camera SCM or a range that includes the capture range of the thermo camera SCM. The thermo camera SCM outputs thermo data, which is video data (non-visible light images) obtained by capturing. The surveillance cameras CM1 to CMn output video data obtained by capturing. Note that multiple thermo cameras SCM may be installed, and in that case, a surveillance camera, which is a visible light camera, is installed in correspondence with each thermo camera. However, some types of thermo cameras can capture visible light images at the same time as non-visible light images, and it is not necessary to install a visible light camera in correspondence with the thermo camera. Therefore, by using such a type of thermo camera for the thermo camera SCM to be placed at the entrance, it is not necessary to install a surveillance camera CM1 to capture an area common to the thermo camera SCM or an area that includes the capture area of the thermo camera SCM. In other words, the thermo camera SCM at the entrance can be used as the surveillance camera CM1.

Thermal camera SCM is connected to a display DP that displays the thermal data. Security personnel stationed at the entrance to the facility can detect people with a fever by monitoring the thermal data displayed on this display DP, and can then ask those with a fever to refrain from entering the facility or guide them to a quarantine room, etc.

The tracking system also includes a server device SV that operates as a fever tracking device. The server device SV is capable of transmitting data between the cameras SCM and CM1 to CMn and the monitoring terminal TM via the network NW.

The monitoring terminal TM is used by the monitoring personnel to monitor the situation within the facility and to make various settings required for monitoring, and is configured, for example, by a personal computer.

The network NW may be, for example, a wired LAN (Local Area Network) or a wireless LAN, but any other network may also be used.

(2) Server device SV
2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration, respectively, of the server device SV.

The server device SV has a control unit 1 having a hardware processor such as a central processing unit (CPU), and this control unit 1 is connected via a bus 5 to a storage unit having a program storage unit 2 and a data storage unit 3, and a communication interface (hereinafter abbreviated as communication I/F) 4.

Under the control of the control unit 1, the communication I/F 4 transmits various data between the thermal camera SCM, the monitoring cameras CM1 to CMn, and the monitoring terminal TM using a communication protocol defined by the network NW.

The program storage unit 2 is configured by combining, for example, a non-volatile memory such as a HDD (Hard Disk Drive) or SSD (Solid State Drive) that can be written to and read from at any time as a storage medium, and a non-volatile memory such as a ROM (Read Only Memory). In addition to middleware such as an OS (Operating System), the program storage unit 2 stores programs necessary for executing various control processes according to the first embodiment of the present invention.

The data storage unit 3 is, for example, a combination of a non-volatile memory such as an HDD or SSD, which can be written to and read from at any time, as a storage medium, and a volatile memory such as a RAM (Random Access Memory). This data storage unit 3 includes a shooting data storage unit 31, a human detection information storage unit 32, a known information storage unit 33, and a tracking result storage unit 34 as storage units necessary for implementing the first embodiment of the present invention.

The image data storage unit 31 is used to store the thermo data sent from the thermo camera SCM and the video data sent from the surveillance cameras CM1 to CMn for a certain period of time, such as one day's worth, required to track people with fevers.

The human detection information storage unit 32 is used to store information representing the detection results of people detected by the control unit 1 from the video data of the surveillance cameras CM1 to CMn. The human detection information storage unit 32 is required to store at least one day's worth of information representing the detection results of people, but if it is possible to track the past movement routes of non-feverish individuals, it is necessary to store multiple days' worth of information.

The known information storage unit 33 is used to store floor map data within the facility and camera information indicating the monitored area within the facility, which is the shooting range of the surveillance cameras CM1 to CMn. The contents stored in this known information storage unit 33 are updated when there is a change to either the floor map or the surveillance cameras.

The tracing result storage unit 34 is used to store information indicating the tracing results of the feverish person obtained by the control unit 1. It is desirable for the contents stored in the tracing result storage unit 34 to be retained for several weeks in consideration of the possibility that the feverish person may later be found to be infected with an infectious disease, making it necessary to take measures such as disinfecting the route of movement of the feverish person.

The control unit 1 includes, as processing functions according to the first embodiment of the present invention, a shooting data acquisition processing unit 11, a person detection processing unit 12, a person tracking processing unit 13, and an output information generation and output processing unit 14. All of these processing units 11 to 14 are realized by, for example, causing a hardware processor in the control unit 1 to execute a program stored in the program storage unit 2.

The image capture data acquisition processing unit 11 performs a process of receiving, via the communication I/F 4, the thermo data sent from the thermo camera SCM and transmitted via the network NW, and the video data sent from the surveillance cameras CM1 to CMn and transmitted via the network NW. The image capture data acquisition processing unit 11 further performs a process of storing the received thermo data and video data in the image capture data storage unit 31 in association with the thermo camera ID that identifies the source thermo camera SCM and the surveillance camera ID that identifies the surveillance cameras CM1 to CMn.

The human detection processing unit 12 reads video data for each surveillance camera ID of the surveillance cameras CM1 to CMn from the shooting data storage unit 31, and performs image analysis processing on the video data for each frame to extract a human image, for example a facial image, thereby detecting the image of the person. The human detection processing unit 12 then performs processing to store information representing the detection result of the image of the person in the human detection information storage unit 32.

Figure 4 is a diagram showing an example of the contents of the human detection information table 321 provided in the human detection information storage unit 32. The human detection processing unit 12 stores human detection information, which is information representing the detection result of a human image, in the human detection information table 321 in the human detection information storage unit 32. The human detection information includes, for example, a surveillance camera ID, a detection date and time, a detected person ID that is individually assigned to a facial image that is an image of a detected person, coordinate information within the frame of the detected facial image, and the detected facial image. Since images of multiple people can be detected within one frame, multiple pairs of detected person ID, coordinate information, and facial image can be stored for one pair of a surveillance camera ID and detection date and time.

The human detection processing unit 12 compares the face image extracted from the video data with face images already registered in the human detection information table 321, and if the similarity between the two exceeds a reference value, it determines that the face images are of the same person. The human detection processing unit 12 then adopts the detector ID set for the registered face image as the detector ID for the extracted face image and stores it in the human detection information table 321. Therefore, the human detection information table 321 stores human detection information including the same detector ID for one surveillance camera ID, and can also store human detection information including the same detector ID for multiple surveillance camera IDs.

Furthermore, the human detection processing unit 12 reads the thermo data with the thermo camera ID of the thermo camera SCM from the above-mentioned shooting data storage unit 31, and performs a process of detecting a person with a fever based on the thermo data. For example, the human detection processing unit 12 detects an area from the thermo data that exceeds a reference temperature (e.g., 37.5 degrees). Then, if the size of the detected area is within the range of the reference size, the human detection processing unit 12 performs a process of detecting the area as part of the person with a fever. In other words, if the size of the detected area is outside the range of the reference size, the human detection processing unit 12 does not consider the area to be part of the person with a fever. Then, the human detection processing unit 12 performs a process of storing information representing the detection result of the detected person with a fever in the human detection information storage unit 32. Here, the human detection processing unit 12 regards the detected person ID and face image of the person detected at the coordinates of the video data of the surveillance camera CM1 corresponding to the coordinates of the person with a fever in the thermo data of the thermo camera SCM as information representing the detection result of the person with a fever.

Figure 5 is a diagram showing an example of the contents of the tracked person information table 322 provided in the human detection information storage unit 32. The human detection processing unit 12 stores the tracked person information, which is information representing the detection result of a person with a fever, in the tracked person information table 322 by registering it in the tracked person information storage unit 32. The tracked person information includes, for example, a detected person ID and a facial image. Note that the human detection processing unit 12 does not newly register a detected person ID and a facial image that are already registered in the tracked person information table 322.

For each of the surveillance cameras CM2 to CMn, excluding the surveillance camera CM1 corresponding to the thermal camera SCM, the person tracking processing unit 13 performs a process of comparing the face image registered in the person detection information table 321 of the person detection information storage unit 32 with the face image of the person with a fever registered in the tracked person information table 322 of the person detection information storage unit 32, and if the similarity between the two exceeds a reference value, detects the face image registered in the person detection information table 321 as the person with a fever. The person tracking processing unit 13 also performs a process of determining a floor map position indicating the position of the detected person with a fever on the floor map, based on the floor map information and camera information stored in the known information storage unit 33. The person tracking processing unit 13 then performs a process of storing information representing the position determination result in the tracking result storage unit 34.

Figure 6 is a diagram showing an example of the contents of a tracking result information table 341 provided in the tracking result storage unit 34 of the server device. The person tracking processing unit 13 registers tracking result information, which is information indicating the position determination result of a person with a fever, in the tracking result information table 341, thereby storing it in the tracking result storage unit 34. The tracking result information includes, for example, a detected person ID, a facial image, a detection date and time, and a floor map position. The detected person ID, facial image, and detection date and time are obtained from the human detection information table 321.

Furthermore, the person tracking processing unit 13 performs a process of determining the movement route of the feverish person within the facility based on the floor map position for each detected person ID stored in the tracking result information table 341. Then, the person tracking processing unit 13 performs a process of storing information representing the movement route in the tracking result storage unit 34.

Figure 7 is a diagram showing an example of the contents of the route information table 342 provided in the tracking result storage unit 34. The person tracking processing unit 13 registers route information, which is information indicating the movement route of a person with a fever, in the route information table 342, thereby storing it in the tracking result storage unit 34. The route information includes, for example, a detected person ID, a facial image, and movement route information. The detected person ID and facial image are obtained from the tracking result information table 341. The route information table 342 stores this route information for each tracking date.

The route information may further include the stay information of the person with a fever. For each detected person ID stored in the tracking result information table 341, the person tracking processing unit 13 determines whether the person with a fever has been staying at a certain location for a specified period of time or longer based on the detection date and time and the floor map position, and if a person with a fever has been staying there, performs a process of registering the stay information including the place of stay in the route information table 342.

The output information generation and output processing unit 14 receives the search date and time sent from the monitoring terminal TM and transmitted over the network NW via the communication I/F 4, and performs a process of searching the tracing result information table 341 in the tracing result storage unit 34 for route information, which is information indicating the travel route of the person with a fever that corresponds to the search date and time. Furthermore, the output information generation and output processing unit 14 generates a route screen showing the search results as output information, and transmits it via the communication I/F 4, thereby performing a process of transmitting it to the monitoring terminal TM via the network NW.

(Example of operation)
Next, the processing operation of the server device SV configured as above will be described with reference to the flowcharts shown in FIGS.

First, the overall processing procedure of the server device SV will be explained. Figure 8 is a flowchart showing the overall processing procedure of the tracking process by this server device SV.

When the control unit 1 of the server device SV detects an instruction to start operating the system inputted on the monitoring terminal TM in step S1, first in step S2, it acquires the thermal data and video data, which are the captured data, from the thermal camera SCM and the monitoring cameras CM1 to CMn and stores them in the captured data storage unit 31. Considering that the target to be tracked is a person who moves at a slower speed than a vehicle, etc., the capture data may be acquired and stored once to several times per second. This prevents the storage capacity of the data storage unit 3 from being overloaded and ensures that time is available for the execution of the subsequent processing procedures. Of course, if there is sufficient storage capacity in the data storage unit 3 and the processing power of the hardware processor that constitutes the control unit 1, the capture data may be acquired and stored in real time.

Next, in step S3, the control unit 1 of the server device SV executes a human detection process and stores the detection result in the human detection information storage unit 32.

Furthermore, in step S4, the control unit 1 of the server device SV performs a person tracking process for the person with a fever based on the known information stored in the known information storage unit 33 and the person detection result stored in the person detection information storage unit 32, and stores the result in the tracking result storage unit 34.

The control unit 1 of the server device SV executes an output information generation and output process in parallel with the processing procedure of steps S2 to S4 above. That is, the control unit 1 acquires the search date and time from the monitoring terminal TM, generates output information based on the person tracking process result of the fever person at the search date and time stored in the tracking result storage unit 34, and outputs the output information to the monitoring terminal TM.

The processes executed in steps S2 to S5 above end when an instruction to stop operation of the system is sent from the monitoring terminal TM and this instruction is detected by the control unit 1 of the server device SV in step S6.

(1) Person Detection Processing FIG. 9 is a flowchart showing an example of the processing procedure of the person detection processing in step S3 executed by the control unit 1 of the server device SV.

Under the control of the human detection processing unit 12, the control unit 1 first selects one of the surveillance camera IDs of the surveillance cameras CM1 to CMn in step S301.

Then, in step S302, the control unit 1 reads one frame of the video data of the selected surveillance camera ID from the shooting data storage unit 31. The destination for this reading is a volatile memory such as RAM in the data storage unit 3 that the control unit 1 uses as a work area. The same applies to "reading" in the following explanation.

The control unit 1 then performs a predetermined video analysis process on the loaded video data in step S303 to extract a person's facial image. Note that the video analysis process used at this time is, for example, a process of extracting a person's facial image from a video frame using a pre-generated learning model. Note that various existing technologies can be used for the video analysis process to extract facial images, and since all of these existing technologies are well known, a description of them will be omitted.

Then, in step S304, the person detection processing unit 12 generates, for each video frame, person detection information including information representing the coordinate information and detection time within the video frame of the extracted person's face image, a detected person ID that is individually assigned to the detected face image, and the surveillance camera ID. The control unit 1 then stores the generated person detection information in the person detection information table 321 of the person detection information storage unit 32. The person detection information is generated in a volatile memory such as RAM of the data storage unit 3 that the control unit 1 uses as a work area. The same applies to "generation" in the following explanation.

Then, in step S305, the control unit 1 determines whether or not selection of all surveillance camera IDs has been completed. If unselected surveillance camera IDs remain, the control unit 1 returns to step S301 to select the next surveillance camera ID, and executes a series of human face image detection processes in steps S302 to S304. Thereafter, surveillance camera IDs are similarly selected in sequence, and the process of detecting human face images for each frame from the video data corresponding to that surveillance camera ID is repeatedly executed.

When all the surveillance camera IDs have been selected, in step S306, the control unit 1 reads the thermal data of the thermal camera ID of the thermal camera SCM from the shooting data storage unit 31.

Next, in step S307, the control unit 1 executes a detection process to detect a person with a fever exceeding a reference temperature (e.g., 37.5°C) from the read thermo data.

Then, in step S308, the control unit 1 determines whether or not a person with a fever is present as a result of this process of detecting a person with a fever. If no person with a fever is present, the control unit 1 ends this person detection process and proceeds to the process of step S4 above.

In contrast, if a person with a fever is detected, in step S309, the control unit 1 reads the human detection information of the surveillance camera ID of surveillance camera CM1, which is the surveillance camera corresponding to the thermal camera SCM, from the human detection information table 321 of the human detection information storage unit 32.

Then, in step S310, the control unit 1 generates tracked person information including the detected person ID and face image, which is information representing the result of the fever person detection, and stores it in the tracked person information table 322 of the human detection information storage unit 32. The control unit 1 then ends this human detection process and proceeds to the process of step S4 above.

(2) Person Tracking Process FIGS. 10A and 10B are flowcharts showing an example of the processing procedure of the person tracking process in step S4 executed by the control unit 1 of the server device SV.

First, in step S401, under the control of the person tracking processing unit 13, the control unit 1 reads one piece of tracking target person information from the tracking target person information table 322 in the person detection information storage unit 32.

Next, in step S402, the control unit 1 selects one of the surveillance camera IDs of the surveillance cameras CM1 to CMn.

Then, in step S403, the control unit 1 reads the human detection information of the selected surveillance camera ID from the human detection information table 321 of the human detection information storage unit 32.

In step S404, the control unit 1 compares the face image included in the read person detection information with the face image included in the tracked person information read in step S401, and determines whether the face images are similar. The control unit 1 performs the reading of the person detection information in step S403 and the face image similarity determination in step S404 for all of the person detection information for the selected surveillance camera ID.

Then, in step S405, the control unit 1 determines whether or not a similar face image is found. If a similar face image is not found, the control unit 1 proceeds to step S411, which will be described later.

In contrast, if a similar face image is found, the control unit 1 determines that the person with the similar face image corresponds to a person with a fever who is a tracking target person, and in step S406, rewrites the detector ID of the corresponding person in the tracking target person information table 322 of the human detection information storage unit 32 to the detector ID of the tracking target person information. Of course, there may be cases where the two detector IDs are the same even if no rewriting is performed.

Next, in step S407, the control unit 1 reads the tracking result information corresponding to the detected individual ID from the tracking result information table 341 of the tracking result storage unit 34. Note that if the tracking result information corresponding to the detected individual ID has not yet been stored in the tracking result information table 341, nothing is read in this step S407.

In addition, in step S408, the control unit 1 reads floor map information of the facility and camera information of the selected surveillance camera ID from the known information storage unit 33.

Then, in step S409, the control unit 1 converts the coordinate information of the relevant person in the person detection information into a floor position based on the read floor map information and camera information.

Then, in step S410, the control unit 1 adds the detection date and time and the floor map position to the tracking result information read in step S407, and stores it in the tracking result information table 341. Note that if there is no tracking result information stored in step S407 and it cannot be read, the control unit 1 creates new tracking result information, registers the detected person ID and face image in addition to the detection date and time and floor map position, and stores it in the tracking result information table 341.

The control unit 1 then determines in step S411 whether or not selection of all surveillance camera IDs has been completed. If unselected surveillance camera IDs remain, the control unit 1 returns to step S402 to select the next surveillance camera ID, and executes a series of processes for generating tracking result information from steps S402 to S410. Thereafter, surveillance camera IDs are similarly selected in sequence, and a process is repeatedly executed in which a person corresponding to a person registered in the tracking target person information as a person with a fever is detected from the person detection information corresponding to that surveillance camera ID, and the position of the detected person within the facility is stored as tracking result information.

When all the surveillance camera IDs have been selected, the control unit 1 determines in step S412 whether or not the processing of all the tracking target person information has been completed. If unprocessed tracking target person information remains, the control unit 1 returns to step S401 to read the next tracking target person information, and executes a series of processes for generating tracking result information from step S402 to step S411.

When processing of all tracked person information has been completed in this manner, the control unit 1 determines in step S413 whether it is time to perform daily report processing. This daily report processing can be performed multiple times a day, and needs to be performed at least once a day. For example, daily report processing can be performed when there are no visitors to the facility, such as between closing time and opening time the next day in the case of a shopping mall. If it is not time to perform daily report processing, the control unit 1 ends this person tracking processing and proceeds to the processing of step S6 above.

On the other hand, if it is time to perform daily report processing, the control unit 1 selects one of the detector IDs of the tracking result information in the tracking result information table 341 of the tracking result storage unit 34 in step S414. Then, the control unit 1 reads the tracking result information of the selected detector ID from the tracking result information table 341 in step S415.

In step S416, the control unit 1 generates route information including the detected person ID and face image contained in the read tracking result information.

Next, in step S417, the control unit 1 calculates the movement route within the facility of the person who is a person with a fever and is identified by the detected person ID, based on the multiple floor map positions included in the read tracking result information. Then, in step S418, the control unit 1 adds this calculated movement route as movement route information to the route information generated in the above step S416.

Then, in step S419, the control unit 1 determines the location within the facility where the person stayed for a specified time or more, for example 15 minutes or more, based on the multiple detection dates and times and floor map locations included in the loaded tracking result information. For example, it can be determined that the person stayed when they are captured by one surveillance camera for a specified time or more. In this case, even if they are not captured by one surveillance camera continuously, it is determined based on the total time they are captured by one camera in total for one day. Then, in step S420, the control unit 1 adds stay information about the locations where the person stayed for a specified time or more based on the determination result to the route information. If there is no location where the person stayed for a specified time or more, stay information with a value of "no stay location" is added.

In step S421, the control unit 1 stores the route information thus created, including the detected person ID, face image, movement route information, and stay information, in the route information table 342 of the tracking result storage unit 34 in association with the date of its creation.

Then, in step S422, the control unit 1 determines whether or not selection of all detected person IDs has been completed. If unselected detected person IDs remain, the control unit 1 returns to step S414 to select the next detected person ID, and executes a series of route information generation processes from steps S415 to S421. Thereafter, detected person IDs are similarly selected in sequence, and the process of generating and storing route information from the tracking result information corresponding to that detected person ID is repeatedly executed.

When all detected person IDs have been selected, the control unit 1 ends this person tracking process and proceeds to step S6.

(3) Generation and Output Processing of Output Information FIG. 11 is a flowchart showing the procedure of the output information generation and output processing of step S5 executed by the control unit 1 of the server device SV.

Under the control of the output information generation and output processing unit 14, the control unit 1 first determines in step S501 whether or not a search date and time has been input from the monitoring terminal TM. If a search date and time has not been input from the monitoring terminal TM, the control unit 1 ends this output information generation and output process and proceeds to step S6.

In response to this, when a search date and time is input from the monitoring terminal TM, the control unit 1 searches for the corresponding tracking date from the route information stored in the route information table 342 of the tracking result storage unit 34 using the date and time specified as the search date and time in step S502. Then, in step S503, the control unit 1 generates a search result screen that lists, for example, the detected person ID and face image on the searched tracking date, and outputs it to the monitoring terminal TM. Thereafter, the control unit 1 waits for a predetermined time for the monitoring terminal TM to specify a person to be tracked. In other words, it waits for the monitor operating the monitoring terminal TM to view the face images listed on the search result screen and specify from among them a person with a fever whose movement route should be confirmed.

In step S504, the control unit 1 determines whether or not a tracking target has been specified from the monitoring terminal TM. If a tracking target has not been specified, the control unit 1 ends this output information generation and output process and proceeds to step S6.

In contrast, if a tracking target has been specified, in step S505, the control unit 1 reads the movement route information and stay information from the route information of the detector ID of the specified face image stored in the route information table 342.

Then, in step S506, the control unit 1 creates a route screen in which the movement route of the person within the facility is mapped on a map indicated by the floor map information stored in the known information storage unit 33 based on the movement route information. At this time, if the stay information records a location where the person stayed for a specified time or longer, a mark indicating that location, for example a key disinfection location mark, is added to the map on the route screen. The control unit 1 then outputs this created route screen to the monitoring terminal TM. The control unit 1 then ends this output information generation and output process, and proceeds to step S6 above.

(Action and Effects)
As described above, in the server device SV operating as the fever-inducing person tracking device according to the first embodiment of the present invention, the control unit 1 detects a fever-inducing person based on thermo data from a thermo camera SCM installed at the entrance of the facility, and detects a fever-inducing person based on video data from multiple surveillance cameras CM1 to CMn, which are visible light cameras installed at the entrance and distributed throughout the facility. The server device SV also stores known information in a known information storage unit 33, including at least floor map information of the facility and camera information showing the installation status of the multiple surveillance cameras CM1 to CMn. Then, based on the detection result of the fever-inducing person and the known information, the control unit 1 determines the movement route of the fever-inducing person within the facility and the staying place where the fever-inducing person stayed for a specified time or more, and stores the determined movement route and staying place of the fever-inducing person in a tracking result storage unit 34.
Therefore, it becomes possible to detect and store the movement route of a person with a fever within the facility.

Furthermore, the control unit 1 of the server device SV receives from the monitoring terminal TM the designation of a feverish person to be output, searches the tracing result storage unit 34 for the movement route and staying locations of the feverish person, and outputs the results to the monitoring terminal TM.
Therefore, it will be possible to confirm the movement route of the person with a fever within the facility and the location of the person's stay, such as areas that require priority disinfection, at a date later than the time the person with a fever stays at the facility.

[Second embodiment]
Next, a server device SV that operates as a fever tracking device according to a second embodiment of the present invention will be described. This second embodiment is intended for facilities used by a specific number of people, such as schools and companies, in addition to facilities used by an unspecified number of people. Here, only the configuration example, operation example, and effects and advantages that differ from the first embodiment will be described, and the description of the similar parts will be omitted.

(Configuration example)
In the second embodiment, the known information storage unit 33 of the data storage unit 3 stores, in addition to floor map information and camera information, information on specific people staying within the facility, such as store clerks, students, and office workers.

Figure 12 is a diagram showing an example of the contents of the registered individual information table 331 provided in the known information storage unit 33 of the server device SV. The registered individual information table 331 stores registrant information, which is information about specific individuals staying in the facility. The registrant information includes a registrant ID uniquely set for each registrant and their facial image, as well as descriptive information about the registrant, such as the name and affiliation of the registrant.

In the second embodiment, the tracking result information table 341 stored in the tracking result storage unit 34 differs from that in the first embodiment. FIG. 13 is a diagram showing an example of the contents of the tracking result information table 341 in the second embodiment. As shown in the figure, the tracking result information stored in the tracking result information table 341 may include the registrant ID at the floor map position for each detection date and time for each detected person ID. There may also be cases where the registrant ID is not included. The registrant ID in this tracking result information table 341 is used to identify a person with whom the feverish person indicated by the detected person ID has been in contact for a specified period of time or longer at the position indicated by the floor map position.

In the second embodiment, the tracing result storage unit 34 does not store route information, i.e., the route information table 342.

(Example of operation)
In the second embodiment, the details of the person tracking process in step S4 and the output information generating and output process in step S5 are different from those in the first embodiment.

14A and 14B are flowcharts showing an example of the procedure of the person tracking process in step S4 executed by the control unit 1 of the server device SV. Here, the processes in steps S401 to S409 and the processes in steps S411 and S412 are similar to those in the first embodiment, and therefore the description thereof will be omitted.

Under the control of the person tracking processing unit 13, the control unit 1 stores the tracking result information in the tracking result information table 341 in step S410. At this time, the registrant ID has not yet been stored in the tracking result information.

Then, in step S431, the control unit 1 determines whether the person with the detected person ID has been staying at one location in the facility for a specified time or more, for example, 15 minutes or more, based on the multiple detection dates and times and floor map positions included in the tracking result information. If the person has not been staying at a location in the facility for the specified time or more, the control unit 1 proceeds to the processing of step S411.

On the other hand, if the person has stayed for longer than the specified time, the control unit 1 reads the registered person information from the registered person information table 331 provided in the known information storage unit 33 in step S432.

Next, in step S433, the control unit 1 determines similar facial images of persons other than the tracking target. That is, the control unit 1 detects similar facial images by comparing the facial images in the person detection information loaded in step S403 with the facial images included in the registered person information loaded in step S432. However, rather than targeting all facial images in the loaded person detection information, the control unit 1 excludes facial images of detected person IDs corresponding to tracking targets, and further targets facial images corresponding to detection dates and times that correspond to a stay period equal to or longer than the specified time.

Then, in step S434, the control unit 1 determines whether or not a similar face image is found. If a similar face image is not found, the control unit 1 proceeds to step S411.

In contrast, if similar facial images are found, the control unit 1 calculates the presence time of those similar facial images during the stay period that is equal to or longer than the above-mentioned specified time in step S435. Then, the control unit 1 determines in step S436 whether the calculated presence time is equal to or longer than the above-mentioned specified time. If there is no facial image that has been present for the specified time or longer, the control unit 1 proceeds to the processing of step S411.

If there is a face image that exists for more than the specified time, the control unit 1 determines the distance between the tracking target person and the person in the similar face image in step S437. The distance between the two can be determined, for example, based on the coordinate information in the person detection information read in step S403 and the camera information read in step S408.

Then, in step S438, the control unit 1 determines whether or not the tracking target is within a specified distance, for example, 2 m, from the person in the similar face image based on the determined distance. If the tracking target is not within the specified distance, the control unit 1 proceeds to step S411.

In contrast, if the tracking target person exists within the specified distance from the person in the similar face image, the control unit 1 calculates the time during which the person in the similar face image existed within the specified distance from the tracking target person in step S439. Then, in step S440, the control unit 1 determines whether the calculated presence time is equal to or longer than the specified time. If there is no face image that exists for the specified time or longer, the control unit 1 proceeds to the processing of step S411.

If there is a face image that exists for more than the specified time, the control unit 1 identifies the registrant ID of the person in the similar face image in step S441. Then, in step S442, the control unit 1 adds the identified registrant ID to the tracking result information and stores it in the tracking result information table 341. The control unit 1 then proceeds to step S411.

In this manner, the tracking result information table 341 can store tracking result information such as that shown in FIG. 13.

(2) Generation and Output Processing of Output Information FIG. 15 is a flowchart showing the procedure of the output information generation and output processing of step S5 executed by the control unit 1 of the server device SV.

Under the control of the output information generation and output processing unit 14, the control unit 1 first determines in step S501 whether or not a search date and time has been input from the monitoring terminal TM. If a search date and time has not been input from the monitoring terminal TM, the control unit 1 ends this output information generation and output process and proceeds to step S6.

In response to this, when a search date and time is input from the monitoring terminal TM, the control unit 1 searches for the corresponding detection date and time from the tracking result information stored in the tracking result information table 341 of the tracking result storage unit 34 using the date and time specified as the search date and time in step S511. Then, in step S503, the control unit 1 generates a search result screen that lists, for example, the detected person ID and face image at the searched detection date and time, and outputs it to the monitoring terminal TM. Thereafter, the control unit 1 waits for a predetermined time for the monitoring terminal TM to specify a person to be tracked. In other words, it waits for the monitor operating the monitoring terminal TM to view the face images listed on the search result screen and specify from among them a person with a fever whose movement route should be confirmed.

If a tracking target is specified from the monitoring terminal TM in step S504, the control unit 1 reads in step S512 the tracking result information of the detector ID of the specified face image stored in the tracking result information table 341. In addition, the control unit 1 reads in step S513 the floor map information stored in the known information storage unit 33.

Then, in step S514, the control unit 1 creates a route screen that maps the person's movement route within the facility on the map indicated by the floor map information based on the tracking result information.

Then, in step S515, the control unit 1 determines the location where the person with the detected person ID stayed for a specified period of time or longer. This location is the floor map location where the registered person ID is stored in the tracking result information.

Then, in step S516, the control unit 1 adds a mark indicating the location where the user stayed for a specified period of time or longer, such as a key disinfection location mark, to the corresponding location on the map on the route screen.

Furthermore, in step S517, the control unit 1 identifies the registrant ID corresponding to the relevant position from the tracking result information. Next, in step S518, the control unit 1 reads the facial image and explanatory information corresponding to the registrant ID from the registered person information table 331 provided in the known information storage unit 33. Then, in step S519, the control unit 1 adds the read facial image and explanatory information to the tracking screen.

In step S520, the control unit 1 outputs the route screen thus created to the monitoring terminal TM. After that, the control unit 1 ends this output information generation and output process and proceeds to step S6.

(Action and Effects)
As described above, in the server device SV operating as a fever-causing person tracking device according to the second embodiment of the present invention, registrant information for a plurality of registrants is further stored as known information in the known information storage unit 33, and the control unit 1 detects people other than the person with a fever from video data from the plurality of monitoring cameras CM1 to CMn, and, based on the detection results and the known information, determines people other than the person with a fever who stayed within a specified distance from the person with a fever for a specified time or longer at a location where the person with a fever stayed for a specified time or longer. The control unit 1 then stores in the tracking result storage unit 34 the person other than the person with a fever as a person who came into contact with the person with a fever, in addition to the movement route and location of the person with a fever.
This makes it possible to identify close contacts of people with a fever.

In addition, when the control unit 1 receives designation of a person with a fever as the output target from the monitoring terminal TM, it searches the tracking result memory unit 34 for the person with a fever's movement route, places of stay, and people other than the person with a fever, and outputs the results to the monitoring terminal TM.
This makes it possible to confirm the route of movement within the facility of a person with a fever and the locations where the person is staying, such as areas that require priority disinfection, and also makes it possible to report people who have had prolonged contact with the person with a fever.

[Third embodiment]
Next, a server device SV that operates as a fever tracking device according to a third embodiment of the present invention will be described. The third embodiment is designed to track fever individuals using whole-body images in addition to facial images. Here, only the configuration, operation, and effects that differ from the second embodiment will be described, and descriptions of similar parts will be omitted.

(Configuration example)
FIG. 16 is a diagram showing an example of the overall configuration of a tracking system including a server device SV that operates as a fever person tracking device according to the third embodiment of the present invention.

In the third embodiment, the video analysis engine VE0 is disposed between the thermo camera SCM and the server device SV, and the video analysis engines VE1 to VEn are disposed between the surveillance cameras CM1 to CMn and the server device SV. Note that the video analysis engines VE1 to VEn are not disposed one-to-one with the surveillance cameras CM1 to CMn, but a smaller number of video analysis engines may be disposed for a plurality of cameras, and the video data of the plurality of surveillance cameras may be processed collectively by the smaller number of video analysis engines. Furthermore, if the thermo camera SCM and the surveillance cameras CM1 to CMn have a computing device, the thermo camera SCM and the surveillance cameras CM1 to CMn themselves may be equipped with a video analysis engine to perform the processing described below. Alternatively, the video analysis engines VE0 to VEn may be provided as a function of the server device SV.

The video analysis engine VE0 analyzes thermo data, which is video data (invisible light images) from the thermo camera SCM, and the video analysis engines VE1 to VEn analyze video data from the surveillance cameras CM1 to CMn. For example, the video analysis engines VE1 to VEn detect face images and/or whole-body images from each image frame included in the video data output from the corresponding surveillance cameras CM1 to CMn, for example, using a learning model generated in advance by a learning process, and output a video analysis result including the face image and/or the whole-body image. That is, since a person in an image frame does not necessarily show both the face and the whole body, there may be cases where only one of the face image and the whole-body image can be detected. In addition to the face image and/or the whole-body image, the video analysis result can include the coordinates of the face image and/or the coordinates of the whole-body image, image feature amounts of the face image and/or the image feature amounts of the whole-body image, etc. Also, the video analysis engine VE0 detects temperature-specific areas from each image frame included in the video data output from the corresponding thermo camera SCM, and outputs a video analysis result including information on the temperature-specific areas.

Figure 17 is a block diagram showing an example of the software configuration of the server device SV in the third embodiment. Compared to the configuration example of the server device SV in the first and second embodiments shown in Figure 3, the control unit 1 has an analysis data acquisition processing unit 15 as a processing function instead of the shooting data acquisition processing unit 11, and does not have a human detection processing unit 12. In addition, the data storage unit 3 does not have a shooting data storage unit 31.

The analysis data acquisition processing unit 15 performs a process of receiving, via the communication I/F 4, thermoanalysis data, which is the result of video analysis including information on temperature-specific regions, sent from the video analysis engine VE0 and transmitted via the network NW. The analysis data acquisition processing unit 15 also performs a process of receiving, via the communication I/F 4, video analysis data, which is the result of video analysis including face images and/or whole-body images, sent from the video analysis engines VE1 to VEn and transmitted via the network NW. The shooting data acquisition processing unit 11 further performs a process of detecting a person with a fever by detecting an area exceeding a reference temperature from the thermoanalysis data from the video analysis engine VE0. The shooting data acquisition processing unit 11 then generates human detection information based on the video analysis data, generates tracking target person information based on this human detection information and the detection result of the person with a fever, and performs a process of storing the human detection information and the tracking target person information in the human detection information storage unit 32.

Figure 18 is a diagram showing an example of the contents of the human detection information table 321 provided in the human detection information storage unit 32 in the third embodiment. In the human detection information table 321 in the first and second embodiments shown in Figure 4, a face image corresponding to a detected person ID is stored, but in the human detection information table 321 in the third embodiment, face information and whole body information are stored instead. The face information includes a face image, image features of the face image, and coordinates of the face image. The whole body information includes a whole body image, image features of the whole body image, and coordinates of the whole body image. Note that both face information and whole body information are not necessarily stored for a detected person ID, and there may be cases where only one of them is stored.

Figure 19 is a diagram showing an example of the contents of the tracked person information table 322 provided in the human detection information storage unit 32 in the third embodiment. In the tracked person information table 322 in the first and second embodiments shown in Figure 5, a face image corresponding to the detected person ID is stored, but in the tracked person information table 322 in the third embodiment, face information including a face image and image features of the face image, and whole body information including a whole body image and image features of the whole body image are stored instead. Note that both face information and whole body information are not necessarily stored for a detected person ID, and there may be cases where only one of them is stored.

Figure 20 is a diagram showing an example of the contents of the registered personal information table 331 provided in the known information storage unit 33 in the third embodiment. In the registered personal information table 331 in the second embodiment shown in Figure 12, a face image corresponding to a registrant ID is stored, but in the registered personal information table 331 in the third embodiment, face information including a face image and image features of the face image, and whole body information including a whole body image and image features of the whole body image are stored instead.

Figure 21 is a diagram showing an example of the contents of a tracking result information table 341 provided in the tracking result storage unit 34 in the third embodiment. In the tracking result information table 341 in the second embodiment shown in Figure 13, a face image corresponding to the detected person ID is stored, but in the tracking result information table 341 in the third embodiment, face information including a face image and image features of the face image, and whole body information including a whole body image and image features of the whole body image are stored instead. Note that both face information and whole body information are not necessarily stored for a detected person ID, and there may be cases where only one of them is stored.

(Example of operation)
Next, the processing operation of the server device SV in the third embodiment configured as above will be described with reference to the flowcharts shown in FIGS.

Figure 22 is a flowchart showing an example of the overall processing procedure of the tracking process by the server device SV in the third embodiment. In the third embodiment, the processing of step S2 is not executed. The details of the processing procedures in each of the person detection processing of step S3, the person tracking processing of step S4, and the output information generation and output processing of step S5 are different from those in the second embodiment.

(1) Person Detection Processing FIG. 23 is a flowchart showing an example of a processing procedure of the person detection processing in step S3 executed by the control unit 1 of the server device SV in the third embodiment.

Under the control of the analysis data acquisition processing unit 15, the control unit 1 first selects one of the surveillance camera IDs of the surveillance cameras CM1 to CMn in step S301.

Then, in step S321, the control unit 1 receives video analysis data from the video analysis engine of the selected surveillance camera ID.

In step S304, the control unit 1 generates person detection information and stores the generated person detection information in the person detection information table 321 of the person detection information storage unit 32. However, in this embodiment, the control unit 1 generates person detection information based on the video analysis data received from the video analysis engine. That is, the control unit 1 generates person detection information including face information and/or whole-body information included in the video analysis data, a detected person ID that is individually assigned to the face image and/or whole-body image, the above-mentioned surveillance camera ID, and information indicating the detection time, which is the current time.

Then, in step S305, the control unit 1 determines whether or not selection of all surveillance camera IDs has been completed. If unselected surveillance camera IDs remain, the control unit 1 returns to step S301 to select the next surveillance camera ID, and executes a series of human detection information generation processes in steps S321 and S304. Thereafter, surveillance camera IDs are similarly selected in sequence, and the process of generating human detection information based on the video analysis data from the video analysis engine corresponding to that surveillance camera ID is repeatedly executed.

Then, when selection of all surveillance camera IDs has been completed, in step S322, the control unit 1 receives thermal analysis data from the video analysis engine VE0 of the thermal camera ID of the thermal camera SCM.

The control unit 1 then executes a detection process in step S307 to detect individuals with a fever exceeding a reference temperature (e.g., 37.5°C). However, in this embodiment, the control unit 1 detects individuals with a fever based on the thermoanalysis data received from the video analysis engine VE0.

The subsequent processing of steps S308 to S310 is the same as that of the first and second embodiments, except that the person detection information read in step S309 and the tracked person information generated in step S310 include face information and/or whole-body information. Therefore, a description thereof will be omitted.

(2) Person Tracking Process FIGS. 24A and 24B are flowcharts showing an example of the processing procedure of the person tracking process in step S4 executed by the control unit 1 of the server device SV in the third embodiment.

First, in step S401, under the control of the person tracking processing unit 13, the control unit 1 reads one piece of tracked person information from the tracked person information table 322 of the person detection information storage unit 32. The tracked person information includes face information and/or whole body information.

Next, in step S402, the control unit 1 selects one of the surveillance camera IDs of the surveillance cameras CM1 to CMn.

Then, in step S403, the control unit 1 reads the human detection information of the selected surveillance camera ID from the human detection information table 321 of the human detection information storage unit 32. The human detection information includes face information and/or whole-body information.

In step S404, the control unit 1 compares the loaded person detection information with the tracked person information loaded in step S401 to determine whether the facial images are similar. In the first and second embodiments, facial images are compared, and in this embodiment, facial images may also be compared. Alternatively, in this embodiment, the similarity of the facial images may be determined by comparing image features of the facial images included in the person detection information and the tracked person information.

Then, in step S405, the control unit 1 determines whether or not a similar face image is found. If a similar face image is found, the control unit 1 proceeds to step S406, and if not, the control unit 1 proceeds to step S451.

In step S451, the control unit 1 compares the person detection information read in step S403 with the tracked person information read in step S401 to determine whether the whole-body images are similar. That is, the control unit 1 determines the similarity of the whole-body images by comparing the whole-body images included in the person detection information and the tracked person information with each other, or the image features of the whole-body images with each other. The control unit 1 performs the reading of the person detection information in step S403, the similarity determination of the face images in step S404, and the similarity determination of the whole-body images in step S451 for all of the person detection information of the selected surveillance camera ID.

Then, in step S452, the control unit 1 determines whether or not a similar whole-body image is found. If a similar whole-body image is found, the control unit 1 proceeds to step S406, and if not, the control unit 1 proceeds to step S411.

The processing of steps S406 to S412 is the same as in the first and second embodiments. However, in this embodiment, the floor map position stored in the tracking result information is converted from the coordinate information of the face image and/or the whole body image in the face information and/or the whole body information of the person detection information. When the person detection information includes the coordinate information of both the face image and the whole body image, for example, the coordinate information of the face image is used preferentially. Also, if there is no tracking result information stored in step S407 and it cannot be read, the control unit 1 creates new tracking result information and registers the detected person ID and the face information or the whole body information in addition to the detection date and time and the floor map position, and stores them in the tracking result information table 341.

In addition, if it is determined in step S431 that the person with the detected person ID has been staying at one location within the facility for a specified time or longer, the control unit 1 reads the registrant information from the registered person information table 331 provided in the known information storage unit 33 in step S432. The registrant information includes both face information including a face image and its image features, and whole-body information including a whole-body image and its image features.

Next, in step S433, the control unit 1 determines similar face images of persons other than the tracking target. That is, the control unit 1 detects similar face images by comparing face information included in the registered user information read in step S432 with face information at detection dates and times corresponding to a stay period equal to or longer than the specified time, excluding face information of the detected person ID corresponding to the tracking target, from among the person detection information read in step S403. Specifically, the control unit 1 determines the similarity of face images by comparing face images included in the face information with each other, or by comparing image feature amounts of face images with each other.

Then, in step S434, the control unit 1 determines whether or not a similar face image is found. If a similar face image is found, the control unit 1 proceeds to step S437; if not, the control unit 1 proceeds to step S453.

The processing of steps S437 to S442 is the same as in the second embodiment. However, in determining the distance between the tracking target and the person in the similar face image in step S437, the coordinate information of the face image included in the face information in the person detection information read in step S403 is used. In addition, in this embodiment, if it is determined in step S436 that there is no face image that exists for a specified time or more, if it is determined in step S438 that the tracking target is not present within a specified distance from the person in the similar face image, or if it is determined in step S440 that there is no face image that exists for a specified time or more, the control unit 1 proceeds to step S453.

In step S453, the control unit 1 determines similar whole-body images other than that of the tracking target. That is, the control unit 1 detects similar whole-body images by comparing the whole-body information included in the registered person information read in step S432 with the whole-body information of the detection date and time corresponding to the stay period of the specified time or more among the human detection information read in step S403, excluding the whole-body information of the detected person ID corresponding to the tracking target. Specifically, the control unit 1 determines the similarity of the whole-body images by comparing the whole-body images included in the whole-body information with each other, or the image features of the whole-body images with each other.

Then, in step S454, the control unit 1 determines whether or not a similar whole-body image is found. If a similar whole-body image is found, the control unit 1 proceeds to step S455; if not, the control unit 1 proceeds to step S411.

In contrast, if there are similar whole-body images, the control unit 1 calculates the presence time of those similar whole-body images during the stay period that is equal to or longer than the above-mentioned specified time in step S455. Then, the control unit 1 determines whether the calculated presence time is equal to or longer than the above-mentioned specified time in step S456. If there are no whole-body images that have been present for the specified time or longer, the control unit 1 proceeds to the processing of the above-mentioned step S411.

If there is a full-body image that exists for a specified time or longer, in step S457, the control unit 1 determines the distance between the tracking target person and the person in the similar full-body image. The distance between the two can be determined, for example, based on the coordinate information of the full-body image included in the full-body information in the person detection information read in step S403 above, and the camera information read in step S408 above.

Then, in step S458, the control unit 1 determines whether the tracking target is within a specified distance, for example, 2 m, from the person in the similar full-body image based on the determined distance. If the tracking target is not within the specified distance, the control unit 1 proceeds to step S411.

In contrast, if the tracking target person exists within the specified distance from the person in the similar full-body image, the control unit 1 calculates the time during which the person in the similar full-body image existed within the specified distance from the tracking target person in step S459. Then, in step S460, the control unit 1 determines whether the calculated presence time is equal to or longer than the specified time. If there is no full-body image that exists for the specified time or longer, the control unit 1 proceeds to the processing of step S411.

If there is a full-body image that has existed for more than the specified time, the control unit 1 identifies the registered user ID of the person in the similar full-body image in step S461.

Then, in step S462, the control unit 1 adds the identified registrant ID to the tracking result information, as in step S442 above, and stores it in the tracking result information table 341.

(3) Generation and Output Processing of Output Information FIG. 25 is a flowchart showing the procedure of the output information generation and output processing of step S5 executed by the control unit 1 of the server device SV in the third embodiment.

Under the control of the output information generation and output processing unit 14, the control unit 1 first determines in step S501 whether or not a search date and time has been input from the monitoring terminal TM. If a search date and time has not been input from the monitoring terminal TM, the control unit 1 ends this output information generation and output process and proceeds to step S6.

In response to this, when a search date and time is input from the monitoring terminal TM, the control unit 1 searches for the corresponding detection date and time from the tracking result information stored in the tracking result information table 341 of the tracking result storage unit 34 using the date and time specified as the search date and time in step S511. Then, in step S503, the control unit 1 generates a search result screen that lists, for example, the detected person ID at the searched detection date and time and the face image or whole body image contained in the face information or whole body information, and outputs it to the monitoring terminal TM. Thereafter, the control unit 1 waits for a predetermined time for the designation of a person to be tracked from the monitoring terminal TM. That is, it waits for the monitor operating the monitoring terminal TM to view the face images or whole body images listed on the search result screen and designate from among them a person with a fever whose movement route should be confirmed.

If a tracking target is specified from the monitoring terminal TM in step S504, the control unit 1 reads in step S512 the tracking result information of the detector ID of the face information or whole body information including the specified face image or whole body image stored in the tracking result information table 341.

The subsequent processing from step S513 to step S517 is the same as in the second embodiment.

In step S531, the control unit 1 reads the facial image and full-body image and explanatory information contained in the facial information and full-body information corresponding to the registrant ID from the registered person information table 331 provided in the known information storage unit 33. Then, in step S532, the control unit 1 adds the read facial image, full-body image, and explanatory information to the tracking screen.

In step S520, the control unit 1 outputs the route screen thus created to the monitoring terminal TM. After that, the control unit 1 ends this output information generation and output process and proceeds to step S6.

(Action and Effects)
As described above, in the server device SV operating as a fever person tracking device according to the third embodiment of the present invention, when the control unit 1 detects a fever person based on thermo data from a thermo camera SCM installed at the entrance of a facility, it acquires a first image which is a facial image and/or a whole body image of the fever person from the video data of a surveillance camera CM1 installed at the entrance, and also acquires a second image which is a facial image and/or a whole body image of the person from the video data of a plurality of surveillance cameras CM2 to CMn installed in a dispersed manner within the facility, and acquires an image from these second images which corresponds to the first image as an image of the fever person.The control unit 1 then determines the movement route of the fever person within the facility based on the installation status of the surveillance camera that acquired the second image which is an image of the fever person.
In this way, by detecting individuals with a fever using whole-body images as well, it is possible to determine the movement route of an individual with a fever even if the individual moves around the facility without being able to obtain a facial image.

The server SV also utilizes the image analysis results of the image analysis engines VE1 to VEn which perform image analysis on the image data from the surveillance cameras CM1 to CMn.
This allows the control unit 1 to speed up processing since it is not necessary to detect a face image or a whole-body image of a person from video data. Alternatively, it allows the control unit 1 to use an inexpensive product that does not require a large processing capacity as a hardware processor.

Furthermore, the control unit 1 performs a process of determining similarity of images based on image feature amounts of a face image or a whole-body image.
This reduces the time required for the similarity determination process.

[Other embodiments]
In each embodiment, in the above step S310 of the person detection process, tracked person information including the detected person ID and a face image and/or a whole-body image is generated. When generating this tracked person information, the registered person ID may be used instead of the detected person ID by referring to the registered person information. Furthermore, since it is also possible to obtain a face image and/or a whole-body image by referring to the registered person information in the output information generation and output process of the above step S5, if the detected person is a registered person, the tracked person information does not need to include a face image and/or a whole-body image. By not storing image information in this way, it is possible to prevent pressure on the storage capacity of the data storage unit 3.

In the first and second embodiments, in step S404 of the person tracking process, the face image detected from the video data from each of the monitoring cameras CM1 to CMn is compared with the face image of the person with a fever. In the first and second embodiments, in the person detection process of step S3, image features are obtained from the detected face image, and the image features are stored in addition to the face image in the person detection information and the tracked person information, and when making a comparison in step S404, a comparison may be made between these image features.

In addition, in the output information generation and output process of step S5 in the third embodiment, the route screen generated in step S520 may include only the face image of the contact person, i.e., no whole-body image, as in the second embodiment.

The order of the processing steps shown in the flowcharts of Figures 8 to 11, 14A to 15, and 22 to 25 is merely an example, and is not limited to this order. For example, the order of the processing of step S407 and the processing of step S408 in Figure 10A may be reversed. In this way, the processing order of each processing step may be changed as long as there is no discrepancy with the preceding or subsequent processing step.

Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

In short, this invention is not limited to the above-mentioned embodiments as they are, and in the implementation stage, the components can be modified to the extent that does not deviate from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiments. For example, some components may be deleted from all the components shown in each embodiment. Furthermore, components from different embodiments may be appropriately combined.

1: Control unit 2: Program storage unit 3: Data storage unit 4: Communication interface (communication I/F)
Description of the Related Art 5... Bus 11... Shooting data acquisition processing section 12... Human detection processing section 13... Person tracking processing section 14... Output information generation and output processing section 15... Analysis data acquisition processing section 31... Shooting data storage section 32... Human detection information storage section 33... Known information storage section 34... Tracking result storage section 321... Human detection information table 322... Tracking target person information table 331... Registered person information table 341... Tracking result information table 342... Route information table CM1 to CMn... Surveillance camera DP... Display NW... Network SCM... Thermo camera SV... Server device TM... Surveillance terminal VE0, VE1 to VEn... Video analysis engine

Claims (7)

a detection unit that detects a person with a fever based on thermographic data from a thermo camera installed at an entrance of a facility, and detects the person with a fever from image data from a plurality of visible light cameras installed at the entrance and dispersed throughout the facility;
a known information storage unit configured to store known information including at least map information of the facility and camera information indicating an installation status of the plurality of visible light cameras;
A discrimination unit that discriminates a movement route of the person with a fever within the facility and a place where the person with a fever stayed for a specified period of time or longer based on the detection result of the detection unit and the known information;
A route storage unit that stores the movement route and the stay location of the person with a fever;
Equipped with
The known information stored in the known information storage unit further includes registrant information for a plurality of registrants,
The detection unit detects people other than the person having a fever from the image data from the plurality of visible light cameras,
the discrimination unit discriminates a person other than the person with a fever who stayed within a specified distance from the person with a fever for a specified time or longer at the staying location where the person with a fever stayed for the specified time or longer based on the detection result of the detection unit and the known information;
The route storage unit stores, in addition to the movement route and the stay location of the person with a fever, a person other than the person with a fever as a person who has come into contact with the person with a fever.
Fever tracking device. The fever-causing person tracking device of claim 1, further comprising an output unit that, upon receiving designation of the fever-causing person to be output, searches the route memory unit for the movement route of the fever-causing person, the place of stay, and people other than the fever-causing person, and outputs them . a detection unit that detects a person with a fever based on thermographic data from a thermo camera installed at an entrance of a facility, and detects the person with a fever from image data from a plurality of visible light cameras installed at the entrance and dispersed throughout the facility;
a known information storage unit configured to store known information including at least map information of the facility and camera information indicating an installation status of the plurality of visible light cameras;
A discrimination unit that discriminates a movement route of the person with a fever within the facility and a place where the person with a fever stayed for a specified period of time or longer based on the detection result of the detection unit and the known information;
A route storage unit that stores the movement route and the stay location of the person with a fever;
Equipped with
The detection unit is
When the person with a fever is detected based on the thermo data from the thermo camera installed at the entrance of the facility, a first image is obtained from the video data of the visible light camera installed at the entrance, which is a facial image and/or a whole-body image of the person with a fever;
acquiring a second image, which is a face image and/or a whole-body image of a person, from the video data from the plurality of visible light cameras installed in a distributed manner within the facility, and acquiring an image corresponding to the first image from among the second images as an image of the person with a fever;
the discrimination unit discriminates the movement route of the person with a fever within the facility based on the installation status of the visible light camera that was used to acquire the second image, which is an image of the person with a fever.
Fever tracking device. The fever-causing person tracking device according to claim 1 , further comprising an output unit that, upon receiving designation of the fever-causing person to be output, searches the route memory unit for the movement route and the staying places of the fever-causing person and outputs the same . A method for tracking the movement of a person with a fever using image data from a plurality of visible light cameras, comprising:
The computer
detecting the person with a fever based on thermographic data from a thermo camera installed at the entrance of the facility;
detecting the person with a fever from image data from the plurality of visible light cameras installed at the entrance and distributed throughout the facility;
a step of determining a movement route of the person with a fever within the facility and a place where the person with a fever stayed for a specified period of time or longer based on known information including at least map information of the facility and camera information indicating the installation status of the plurality of visible light cameras, which are stored in advance in a memory, and the detection result of the person with a fever;
storing the movement route and the stay location of the determined fever person in the memory;
Equipped with
The known information stored in the memory further includes registrant information for a plurality of registrants;
The computer,
detecting people other than the person with a fever from image data from the plurality of visible light cameras;
a step of determining, based on the detection result and the known information, a person other than the person with a fever who stayed within a specified distance from the person with a fever for a specified period of time or longer at the location where the person with a fever stayed for the specified period of time or longer;
A process of storing a person other than the person with a fever as a contact of the person with a fever, in addition to the movement route and the place of stay of the person with a fever;
A fever tracking method comprising: A method for tracking the movement of a person with a fever using image data from a plurality of visible light cameras, comprising:
The computer
detecting the person with a fever based on thermographic data from a thermo camera installed at the entrance of the facility;
detecting the person with a fever from image data from the plurality of visible light cameras installed at the entrance and distributed throughout the facility;
a step of determining a movement route of the person with a fever within the facility and a place where the person with a fever stayed for a specified period of time or longer based on known information including at least map information of the facility and camera information indicating the installation status of the plurality of visible light cameras, which are stored in advance in a memory, and the detection result of the person with a fever;
storing the movement route and the stay location of the determined fever person in the memory;
a step of acquiring a first image, which is a facial image and/or a whole-body image of the person with a fever, from the video data of the visible light camera installed at the entrance of the facility when the person with a fever is detected based on the thermographic data from the thermo camera installed at the entrance of the facility;
acquiring a second image, which is a face image and/or a whole-body image of a person, from the video data from the plurality of visible light cameras installed in a distributed manner within the facility, and acquiring an image corresponding to the first image from among the second images as an image of the person with a fever;
determining the movement route of the person with a fever within the facility based on the installation status of the visible light camera that was used to acquire the second image, which is an image of the person with a fever;
A fever tracking method comprising: A program that causes a computer to execute processing by each unit of the fever tracking device according to any one of claims 1 to 4 .