JP7492226B2 - Moving object tracking device - Google Patents

Description

This invention relates to tracking people, moving machines, etc. in a target area such as a workplace, facility, etc.

In workplaces, facilities, etc., the locations of people and mobile machines are tracked. By tracking, it is possible to understand their behavior and issue warnings about dangerous areas, etc.

Patent document 1 discloses a system in which a person is fitted with a portable transceiver and multiple location identification devices capable of communicating with the portable transceiver are provided to identify and track the person's location.

Patent document 2 discloses a system that uses a camera to capture and identify people, and determines and records the movement trajectory of the person based on the captured image. With this system, it is possible to obtain the movement trajectory without requiring people to carry a transceiver.

Patent document 3 discloses a system in which a GPS receiver is attached to a work machine or the like, and the movement trajectory of the work machine is tracked. This makes it possible to remotely grasp the operating status of the work machine or the like.

International Publication No. 2010/029918 Patent Publication 2018-93283 JP2003-34954

However, the above-mentioned conventional technology has the following problems:

In Patent Documents 1 and 3, tracking is not possible unless a transceiver is installed on the person or machine, and a transceiver must be attached to each person or machine, which is costly.

In Patent Document 2, there is no need to attach a transmitter or receiver to a person or machine, and the method can be easily implemented. However, the person is identified based on the captured face image, and the accuracy of identifying the person is insufficient in cases where the face is not captured, for example.

The purpose of this invention is to solve the problems described above and provide a system that is simple in configuration but can accurately identify target people or machines.

Some features of this invention that can be applied independently are listed below.

(1)(3)The moving object tracking device according to the present invention is a moving object tracking device that includes one or more target area cameras that capture an image of a target area, a recognition means that recognizes a unique identification code displayed on a moving object moving in the target area based on an image captured by the target area camera, an image position acquisition means that acquires the position of the moving object in the target area based on the captured image, and a position processing means that determines the identity of the moving object based on the recognized identification code and records the temporal change in the position of the moving object in a recording unit, and the recognition means includes a first estimation means that distinguishes between a first identification code and other identification codes and calculates an estimated probability that the moving object is the first identification code, a second estimation means that distinguishes between a second identification code and other identification codes and calculates an estimated probability that the moving object is the second identification code, an nth estimation means that distinguishes between an nth identification code and other identification codes and calculates an estimated probability that the moving object is the nth identification code, and an integration means that recognizes the identification code based on the estimation results of the first to nth estimation means.

This allows the movement of moving objects to be tracked with high accuracy.

(2)(4) The moving object tracking device according to the present invention further comprises an entry time recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by the entry time camera, a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entry time camera, and a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other, wherein the position processing means acquires the moving object ID of the moving object by referring to the recording unit based on the identification code recognized from the image of the moving object captured by the target area camera.

Therefore, the moving object ID and identification code can be automatically associated upon entry into the target area.

(5) The moving object tracking device according to the present invention is characterized in that a plurality of identification codes are displayed on the moving object, and the recognition means recognizes the identification codes using a machine learning trained model that has been trained based on images of the identification codes displayed on the moving object captured from different angles.

Therefore, the identification code can be correctly recognized regardless of the imaging angle.

(6) The moving object tracking device of the present invention is characterized in that an identification code is displayed on an attachment attached to the moving object.

Therefore, the identification code can be linked to the wearer by wearing the accessory.

(7) The moving object tracking device according to the present invention is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the integration means determines the identification code with the highest estimated probability as the recognition result.

Therefore, accurate estimation results can be obtained.

(8) The moving object tracking device according to the present invention is characterized in that, if the highest estimation probability among the first identification code to the nth identification code is equal to or less than a predetermined exclusion probability, the integration means determines that the identification code is the n+1th identification code, performs learning based on the captured image, distinguishes between the n+1th identification code and other identification codes, and performs a learning process to generate an n+1th estimation means that calculates the estimation probability that the identification code is the n+1th identification code, or suggests the learning process.

Therefore, it is possible to learn to recognize new moving objects.

(9) The moving object tracking device according to the present invention is characterized in that, when there are multiple identification codes, among the first identification code to the nth identification code, that have estimated probabilities that exceed a predetermined exclusion probability and are equal to or lower than a predetermined confirmation probability, the integration means displays moving object images captured by the camera and allows the operator to select one.

This allows for more accurate estimation results.

(10) The moving object tracking device according to the present invention is characterized in that the moving object ID acquisition means acquires the moving object ID from a medium on which the moving object ID is recorded and which is carried by a person who is a moving object, by short-range communication or by an input operation by the person.

This allows the mobile unit ID to be linked to the identification code.

(11) The moving object tracking device according to the present invention is characterized in that the position processing means refers to the no-entry zone recorded for each moving object based on the position of the moving object acquired by the image position acquisition means, and includes a warning means for outputting a warning when the moving object enters the no-entry zone.

This ensures worker safety and confidentiality.

(12) The moving object tracking device of the present invention is characterized in that the no-entry zone changes dynamically according to the movement of a potentially dangerous moving object or according to the date and time.

Therefore, it is possible to respond to ever-changing dangerous situations and issue warnings.

(13) The moving object tracking device according to the present invention is characterized in that the recognition means includes a learning means for learning the recognition means based on an image captured of only one moving object with an attachment attached thereto in a state in which the moving object can be identified.

Therefore, additional learning can be carried out to enable more accurate estimation.

(14) The moving object tracking device according to the present invention is characterized in that the identification code has an individual code of multiple digits, and the display form of the individual code or the display form of the background is configured to change according to the digits, and the recognition means recognizes which digit the individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code.

Therefore, it is easy to recognize which digit an individual code belongs to.

(15) The moving object tracking device according to the present invention is characterized in that the identification code has an individual code of multiple digits, and the recognition means recognizes the identification code based on the recognition result of the individual code of each digit and the recognition result of the entire multiple digits.

This allows for improved estimation accuracy.

(16) The moving object tracking device according to the present invention is characterized in that the individual codes for each digit of the identification code are low in similarity to each other.

This allows for improved estimation accuracy.

(17)(18) A moving object tracking device according to the present invention is a moving object identification device comprising one or more target area cameras which capture an image of a target area, an entrance camera which captures an image of a moving object which is about to enter the target area, an entrance time recognition means which recognizes a unique identification code displayed on a moving object which is about to enter the target area based on an image captured by the entrance camera, a moving object ID acquisition means which acquires a moving object ID of the moving object when the moving object is imaged by the entrance time camera, a correspondence recording means which records in a recording unit the acquired moving object ID and the recognized identification code in correspondence with each other, a recognition means which recognizes a unique identification code displayed on a moving object moving in the target area based on an image captured by the target area camera, and an identification means which identifies the moving object imaged by the target area camera based on the identification code recognized by the recognition means and the correspondence between the identification code recorded in the recording unit and the moving object ID,
The recognition means comprises a first estimation means for distinguishing between a first identification code and other identification codes and calculating an estimated probability that the code is the first identification code, a second estimation means for distinguishing between a second identification code and other identification codes and calculating an estimated probability that the code is the second identification code, an nth estimation means for distinguishing between an nth identification code and other identification codes and calculating an estimated probability that the code is the nth identification code, and an integration means for recognizing the identification code based on the estimation results of the first to nth estimation means.

This allows the movement of moving objects to be tracked with high accuracy.

(19) The moving object tracking device according to the present invention is characterized in that a plurality of identification codes are displayed on the moving object, and the recognition means recognizes the identification codes using a machine learning trained model that has been trained based on images of the identification codes displayed on the moving object captured from different angles.

Therefore, the identification code can be correctly recognized regardless of the imaging angle.

(20) The moving object tracking device according to the present invention is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the integration means determines the identification code with the highest estimated probability as the recognition result.

Therefore, accurate estimation results can be obtained.

(21) The moving object tracking device according to the present invention is characterized in that, if the highest estimation probability among the first identification code to the nth identification code is equal to or less than a predetermined exclusion probability, the integration means determines that the identification code is the n+1th identification code, performs learning based on the captured image, distinguishes between the n+1th identification code and other identification codes, and performs a learning process to generate the n+1th estimation means that calculates the estimation probability that the identification code is the n+1th identification code, or suggests the learning process.

Therefore, it is possible to learn to recognize new moving objects.

(22) The moving object tracking device according to the present invention is characterized in that, when there are multiple identification codes, among the first identification code to the nth identification code, that have estimated probabilities that exceed a predetermined exclusion probability and are equal to or lower than a predetermined confirmation probability, the integration means displays moving object images captured by the camera and allows the operator to select one.

This allows for more accurate estimation results.

(23) The moving object tracking device according to the present invention is characterized in that the identification code has an individual code of multiple digits, and the display form of the individual code or the display form of the background is configured to change according to the digits, and the recognition means recognizes which digit the individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code.

This makes it possible to improve the accuracy of recognizing which digit an individual code belongs to.

(24) The moving object tracking device according to the present invention is characterized in that the identification code has an individual code of multiple digits, and the recognition means recognizes the identification code based on the recognition result of the individual code of each digit and the recognition result of the entire multiple digits.

This allows for improved recognition accuracy.

(25) The moving object tracking device according to the present invention is characterized in that the individual codes for each digit of the identification code are low in similarity to each other.

This allows for improved recognition accuracy.

(26)(27) The recognition device according to the present invention is a recognition device that recognizes an object based on an image captured by a camera, and is characterized in that it comprises a first estimation means for distinguishing between a first identification code and other identification codes and calculating an estimated probability that the code is the first identification code, a second estimation means for distinguishing between a second identification code and other identification codes and calculating an estimated probability that the code is the second identification code, an nth estimation means for distinguishing between an nth identification code and other identification codes and calculating an estimated probability that the code is the nth identification code, and an integration means for taking the identification code with the highest estimated probability as the recognition result if the highest estimated probability of the first object to the nth object estimated by the first to nth estimation means is equal to or higher than a predetermined final probability.

This allows for more accurate recognition of objects.

(28) The recognition device according to the present invention is characterized in that the identification code has an individual code of multiple digits, and the display form of the individual code or the display form of the background is changed according to the digits, and the recognition means recognizes which digit the individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code.

This allows for a high degree of accuracy in estimating which digit an individual code belongs to.

(29) The recognition device according to the present invention is characterized in that the identification code has individual codes of multiple digits, and the recognition means recognizes the identification code based on the recognition results of the individual codes of each digit and the recognition results of the entire multiple digits.

This allows for improved recognition accuracy.

(30) The recognition device according to the present invention is characterized in that the individual codes for each digit of the identification code are low in similarity to each other.

This allows for improved recognition accuracy.

(31) The recognition device according to the present invention is characterized in that, when the first to nth identification codes to be estimated are the target identification codes and the other identification codes are the non-target identification codes, each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same.

This allows for proper learning.

(32)(36)(48)(51)The recognition device according to the present invention is a moving object tracking device including one or more target area cameras for capturing an image of a target area, recognition means for recognizing a unique identification code displayed on a moving object moving in the target area based on an image captured by the target area camera, image position acquisition means for acquiring the position of the moving object in the target area based on the captured image, and position processing means for determining the identity of the moving object based on the recognized identification code and recording the change in the position of the moving object over time in a recording unit, wherein the identification code has an individual code of multiple digits, and the display form of the individual code or the display form of the background is configured to change according to the digits, and the recognition means recognizes which digit the individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code.

Therefore, it is easy to estimate which digit an individual code is, and tracking can be performed with high estimation accuracy.

(33)(37)(49)(52) The moving object tracking device according to the present invention is characterized in that the recognition means recognizes the identification code based on the recognition results of the individual codes of each digit and the recognition results of the entire set of digits.

This allows for improved estimation accuracy.

(34)(38)(50)(53)The object tracking device according to the present invention is a moving object tracking device including one or more object area cameras for capturing an image of a target area, a recognition means for recognizing a unique identification code displayed on a moving object moving in the target area based on an image captured by the object area camera, an image position acquisition means for acquiring a position of the moving object in the target area based on the captured image, and a position processing means for determining the identity of the moving object based on the recognized identification code and recording a temporal change in the position of the moving object in a recording unit, wherein the identification code has an individual code of multiple digits, and the recognition means recognizes the identification code based on the recognition result of the individual code of each digit and the recognition result of the entire multiple digits.

This allows for accurate estimation and tracking of the target object.

(35)(39) The object tracking device according to the present invention is characterized in that the cameras include an entry camera, and further includes an entry recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by the entry camera, a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entry camera, and a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other, and the position processing means acquires the moving object ID of the moving object by referring to the recording unit based on the identification code recognized from the image of the moving object captured by the target area camera.

Therefore, matching can be done using the image taken at the time of entry.

(40)(54) The object tracking device according to the present invention is characterized in that the individual codes for each digit of the identification code are low in similarity to each other.

This allows for improved estimation accuracy.

(41) The object tracking device according to the present invention is characterized in that the moving object is displayed with a plurality of identification codes, and the recognition means recognizes the identification codes using a machine learning trained model that has been trained based on images of the identification codes displayed on the moving object captured from different angles.

This allows for improved estimation accuracy.

(42) The object tracking device according to the present invention is characterized in that the identification code is displayed on an attachment attached to the moving object.

Therefore, tracking can be done using the identification code on the person's helmet or other equipment.

(43) The object tracking device according to the present invention is characterized in that the moving object ID acquisition means acquires the moving object ID from a medium on which the moving object ID is recorded and which is carried by a person who is a moving object, by short-range communication or by an input operation by the person.

This allows appropriate association of people with moving object IDs.

(44) The object moving device according to the present invention is characterized in that the position processing means refers to the no-entry zone recorded for each moving object based on the position of the moving object acquired by the image position acquisition means, and includes a warning means for outputting a warning when the moving object enters the no-entry zone.

So warnings can help ensure safety.

(45) The object moving device of the present invention is characterized in that the no-entry zone changes dynamically according to the movement of a potentially dangerous moving object or according to the date and time.

This allows appropriate warnings to be given depending on the situation.

(46) The object movement device according to the present invention is characterized in that the recognition means includes a learning means for learning the recognition means based on an image captured of only one moving object with an attachment attached thereto in a state in which the moving object can be identified.

This allows for reliable matching.

(47) The object moving device according to the present invention is characterized in that, when the first to nth identification codes to be estimated are the target identification codes and the other identification codes are the non-target identification codes, each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same.

This allows for proper learning.

(55)(58) The recognition device according to the present invention is a recognition device that recognizes an identification code based on an image captured by a camera, the identification code having an individual code of multiple digits, and configured to change the display form of the individual code or the display form of the background according to the digits, and is characterized in that it recognizes which digit the individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code.

Therefore, it is easy to recognize which digit an individual code belongs to.

(56)(59) The recognition device according to the present invention is characterized in that it recognizes an identification code based on the recognition results of the individual codes of each digit and the recognition results of the entire set of digits.

This allows for improved recognition accuracy.

(57)(60) The recognition device according to the present invention is a recognition device that recognizes an identification code based on an image captured by a camera, and the identification code has individual codes of multiple digits, and is characterized in that the recognition device recognizes the identification code based on the recognition results of the individual codes of each digit and the recognition results of the entire set of digits.

This allows for improved recognition accuracy.

(61) The recognition device according to the present invention is characterized in that the individual codes for each digit of the identification code are low in similarity to each other.

This allows for improved recognition accuracy.

(62)(63) The warning device according to the present invention includes a position acquisition means for acquiring the positions of a plurality of moving objects moving within a target area, and a warning means for referring to a no-entry zone recorded for each moving object based on the position of the moving object acquired by the position acquisition means, and outputting a warning when the moving object enters the no-entry zone.

Therefore, it is possible to issue a warning to moving objects about no-entry areas.

(64) The warning device of this invention is characterized in that the no-entry zone changes dynamically according to the movement of a potentially dangerous moving object or according to the date and time.

This allows warnings to be issued according to ever-changing dangerous situations.

In the embodiment, the "recognition means" corresponds to steps S26 to S35, S54 to S63, etc.

In this embodiment, step S22 corresponds to the "moving object ID acquisition means."

In this embodiment, step S46 corresponds to the "corresponding recording means."

In this embodiment, step S67 corresponds to the "position recording means."

The term "program" refers not only to programs that can be executed directly by the CPU or GPU (or both), but also to source-format programs, compressed programs, encrypted programs, etc.

1 is a functional block diagram of a moving object tracking device according to an embodiment of the present invention. This is an example of a system configuration in which a moving object tracking device is installed at a work site 28. This shows the hardware configuration of PC24. 13 is an example of an identification code attached to the helmet 4. 13 is a flowchart of learning data generation. FIG. 13 is a diagram showing image processing for generating learning data. FIG. 1 illustrates a learning dataset. 13 is a flowchart of an entry process. 13 is a flowchart of an entry process. FIG. 13 is a diagram showing image processing in the entry process. 1 is an example of an estimated probability. 13 is a correspondence table between worker IDs and identification codes. 13 is an example of a screen for selecting an identification code. 13 is an example of a screen for inputting a new identification code. 13 is a flowchart of a tracking process. 13 is a flowchart of a tracking process. 11A and 11B are diagrams for explaining image processing in tracking processing. 1 is an example of an estimated probability. 1 is an example of tracking data. FIG. 13 illustrates another tracking technique. FIG. 13 is a configuration diagram of a system according to another embodiment. This is a table that records whether or not entry into dangerous areas is permitted. FIG. 11 is a system diagram of a moving object tracking device according to another embodiment. FIG. 11 is a functional block diagram of a moving object tracking device according to a second embodiment. This is an example of a system configuration in which a moving object tracking device is installed at a work site 28. 13 is a flowchart of an additional learning process. 11A and 11B are diagrams for explaining image processing in the additional learning process. FIG. 11 is a functional block diagram of a moving object tracking device according to a third embodiment. FIG. 1 shows a helmet with symbols. FIG. 13 is a diagram showing an example of selected characters to be used. 13 is a flowchart of learning data generation. FIG. 13 is a diagram showing an image for learning. 13 is a flowchart of an entry process. 13 is a flowchart of an entry process. 1A and 1B are diagrams showing an image of an identification code and an image of an individual code extracted from a captured image. FIG. 13 illustrates the estimated probability of the overall code. FIG. 13 illustrates the estimated probabilities of individual codes. 13A and 13B are diagrams illustrating the estimation results of individual codes and the estimation results of an overall code. FIG. 13 is a diagram showing an estimation result of an individual code and an estimation result of an overall code. FIG. 13 is a diagram showing an estimation result of an individual code and an estimation result of an overall code. 13 is a flowchart of a tracking process. FIG. FIG. 13 is a diagram showing correspondence between a person image and a helmet image.

1. First embodiment
1 shows the functional configuration of a person tracking device according to a first embodiment of the present invention. In this embodiment, tracking of the behavior of a worker at a work site will be described as an example.

When a worker 2 enters a work site, he or she puts on a helmet 4, which is an accessory. An identification code 6 is printed or otherwise attached to the helmet 4 in multiple locations, such as the front, back, left and right sides, and top. The same identification code is attached to multiple locations on the same helmet 4, and each helmet 4 is assigned a unique identification code. For example, the identification codes are "0," "1," ... "9."

When a worker enters, the worker 2 wearing the helmet 4 is captured by the entry camera 7. The identification code in the captured image is recognized by the recognition means 10. The recognition means 10 includes a first estimation means E1, a second estimation means E2, ..., and an n-th estimation means En.

The first estimation means E1 estimates whether the imaged identification code is a "0" or a value other than "0" (whether it is a "1"..."9"). The second estimation means E2 estimates whether the imaged identification code is a "1" or a value other than "1" (whether it is a "0", "2",..."9"). Similarly, the nth estimation means En estimates whether the imaged identification code is a "9" or a value other than "9" (whether it is a "0"..."8").

The integration means 12 integrates the estimation results of the first estimation means E1 to the nth estimation means En to recognize the identification code captured in the image.

When the entrance camera 7 captures an image at the time of entry, the ID acquisition means 14 acquires the worker ID from the worker 2. For example, the worker can be asked to hold an ID card with a short-range communication function over a reader to read the worker ID.

The correspondence recording means 16 records the acquired worker ID in association with the estimated identification code. This makes it possible to identify the worker who is wearing the helmet with the identification code.

A worker 2 who enters the work area is imaged by the work area camera 8. The identification code of the helmet 4 that is imaged is recognized by the recognition means 10. The image position acquisition means 18 estimates and acquires the position of the worker 2 wearing the helmet 4 based on the captured image of the worker 2.

The position processing means 20 identifies the corresponding worker ID by the recognized identification code and records the worker's position in the recording unit in association with the worker ID.

By providing multiple cameras 8, it is possible to record the movement trajectory of the worker over a wide range.

1.2 System Configuration Figure 2 shows the system configuration of the moving object tracking device. An entrance area 26 is provided at a work site 28. When a worker enters the work site 28, he or she wears a helmet and, in the entrance area 26, holds an ID card over a card reader 22 for authentication. The ID card records the worker ID of each worker, which is read by the card reader 22 and transmitted via PC 21 to a PC 24 provided at a site office or the like. A short-distance communication system such as FeliCa (trademark) can be used as the ID card.

An entrance camera C1 is installed in the entrance area 26, and captures an image of a worker wearing a helmet with an identification code attached. The captured image is sent to the PC 24. The PC 24 recognizes the identification code based on this image. It then records this in association with the worker ID received from the card reader 22.

In the figure, it is shown that workers W1 to W5 have already entered the work site. Work site cameras C2 to C5 are installed at the work site 28 and capture images of the work site 28. These captured images are sent to the PC 24, which recognizes the identification codes on the helmets of each worker W1 to W5. The positions of the workers are also ascertained based on the images. Therefore, the positions of each worker W1 to W5 can be known and their movement trajectories can be recorded.

Note that construction machine M1 is also present at the work site 28. This construction machine M1 is also given an identification code. Therefore, it can be photographed by work site cameras C2 to C5 and its movement trajectory recorded. Note that in this embodiment, the construction machine M1 is given a predetermined identification code. This differs from the worker, whose identification code changes depending on the helmet he or she wears.

Figure 3 shows the hardware configuration of PC 24. Connected to CPU 30 are memory 32, display 34, hard disk 36, DVD-ROM drive 38, keyboard/mouse 40, and communication circuit 42. Communication circuit 42 is for connecting to a network or the Internet.

The hard disk 36 stores an operating system 44 and a moving object tracking program 46. The moving object tracking program 46 performs its functions in cooperation with the operating system 44. These programs were recorded on a DVD-ROM 48 and installed on the hard disk 36 via the DVD-ROM drive 38.

Fig. 4 shows a helmet 4 to be worn by a worker. Each helmet 4 is assigned a unique identification code. In the figure, the identification codes are "0" to "9". Furthermore, the same identification code is assigned to the front, rear, left and right sides, and top of each helmet 4. This is to ensure that the identification code is included in the image even when the image is captured from various angles.

1.3 Tracking Process The process of tracking the behavior of a worker wearing the helmet 4 will now be described.

(1) Learning Data Generation Process and Trained Model Generation In this embodiment, an image of an identification code is recognized by a trained model of deep learning. First, the learning data generation process will be described.

Figure 5 shows a flowchart of the learning data generation process of the moving object tracking program 46. A large number of images of the helmet 4 in Figure 4 taken from various angles are recorded in advance on the hard disk 36.

The CPU 30 reads these images one by one from the hard disk 36 (step S2). An example of the read image is shown in FIG. 6A. In the figure, the image is shown in monochrome, but in this embodiment, a color image is used (a monochrome image may also be used).

Next, the CPU 30 extracts only the helmet 4 from this image (step S3). In this embodiment, the hue, saturation, and brightness are specified using HSV to extract a predetermined portion of the helmet 4. If the helmet 4 is yellow, the hue, saturation, and brightness are specified with a predetermined width for the yellow color to extract the helmet portion. The extracted image is shown in FIG. 6B. It is also possible to extract the helmet based on other color representations such as RGB.

Next, the CPU 30 performs an expansion/contraction process on the yellow color of the helmet (for example, a closing process). This removes the identification code contained in the helmet (by painting it yellow), and the area of the entire helmet can be extracted. Note that the area of the helmet may also be recognized by a method such as contour recognition.

The CPU 30 then refers to the recognized area of the entire helmet and extracts the identification code from the original image (step S4). The specified color (black in this embodiment) used in the identification code is extracted. The extracted identification code is shown in FIG. 6C. In this embodiment, identification codes are attached to multiple locations on the helmet 4. Therefore, as shown in FIG. 6C, the extracted image also contains multiple identification codes, including identification codes that are only partially captured.

Next, the CPU 30 displays the image of the extracted identification code on the display 34, and prompts the operator to input a class (here, the value of the identification code shown by the image) as teaching data (step S5). In response, the operator inputs the class ("0" in the example of FIG. 6C) using the keyboard/mouse 40. The CPU 30 records the input class in association with the image of the identification code in FIG. 6C.

In addition, the image may be recorded in advance with a class assigned in the state shown in Figure 6A.

When the processing for one image is completed in this manner, the same process is carried out for the next recorded image to extract an identification code and assign a class (steps S1 to S6).

After extracting the identification codes and assigning classes to all images, the CPU 30 generates a learning data set for the identification code "0" class (step S7). As shown in FIG. 7A, a learning data set for "0" is generated from images with identification codes assigned the class "0" and images with identification codes assigned classes other than "0" (here, images assigned classes "1" to "9").

Next, the CPU 30 generates a learning data set for the class of the identification code "1" (step S8). As shown in FIG. 7B, the learning data set for "1" is generated from images of identification codes assigned the class "1" and images of identification codes assigned classes other than "1" (here, images assigned the classes "0", "2" to "9").

The CPU 30 then generates learning data sets for the identification codes "3," "4," "5," "6," "7," "8," and "9" in the same manner.

If there are only a few images of the identification codes with classes generated in steps S1 to S6, the data in the learning data set will be small, and learning effects cannot be obtained. Therefore, in order to increase the amount of data in the learning data set, the images of the identification codes with classes generated in steps S1 to S6 may be processed, such as by deformation or movement, to obtain multiple images from a single image.

Once the learning data sets for each of "0" to "9" have been generated in the above manner, the CPU 30 trains a deep learning program based on each learning data set. This enables the CPU 30 to obtain a trained model for estimating "0", a trained model for estimating "1", ... and a trained model for estimating "9". These models are stored on the hard disk 36.

(2) Corresponding Process Between Identification Code and Worker ID When it is not decided which helmet 4 the worker 2 will wear, it is necessary to correlate the worker ID with the identification code of the helmet 4. In this embodiment, this is done when the worker enters the work site.

Figures 8 and 9 show a flowchart of the entry processing of the moving object tracking program 46. When entering the entry area 26, the worker 2 holds his/her ID card over the card reader 22. The worker ID read from the ID card is sent to PC 24 via PC 21.

The CPU 30 reads the ID card and determines whether the worker ID has been transmitted (step S21). If the worker ID has been transmitted, the CPU 30 records the worker ID in the hard disk 36 as a worker who has clocked in for the day (step S22).

The CPU 30 then acquires an image of the worker 2 captured by the camera C1 installed in the entrance area 26 (step S23). An example of the acquired image is shown in FIG. 10A. The CPU 10 extracts an image of the worker 2 from this image (step S24). In this embodiment, an image of the person (worker 2) is extracted using a deep learning estimation program using Dark Net (trademark). Note that if the background at the time the entrance area 26 is captured is constant, the image of the worker 2 may be extracted using a background subtraction method. An example of the extracted image is shown in FIG. 10B.

Furthermore, the CPU 30 extracts an image of the helmet 4 from the person image in FIG. 10B in the same manner as steps S3 and S4 in FIG. 5, and extracts the identification code (step S25). An image of the extracted identification code is shown in FIG. 10C.

Next, the CPU 30 performs a process to recognize the identification code captured in the image of the identification code. This estimation process is performed using the model learned by the deep learning described above.

In this embodiment, instead of performing a single estimation process to determine which identification code an image of an identification code is, the estimation process is performed multiple times and the results are integrated to make a final estimation.

First, the CPU 30 estimates whether the image shown in FIG. 10C is a "0" and outputs the estimated probability (step S26). At this time, a trained model for "0" is used. As described above, the trained model for "0" is trained using data that distinguishes between "0" and non-"0".

Next, the CPU 30 uses the trained model for "1" to estimate whether the image shown in FIG. 10C is "1" and outputs the estimated probability (step S27).

Then, in the same manner, it estimates whether the images shown in Figure 10C are "2," "3," "4," ... "9," and outputs the estimated probability.

Figure 11 shows an example of the calculated estimated probability. Next, the CPU 30 selects the highest estimated probability and determines whether this is equal to or greater than a predetermined confirmed probability (step S41). In Figure 11, the highest estimated probability is 97.7% for "2". The CPU 30 determines whether this is equal to or greater than a confirmed probability (a reliable predetermined probability, for example 95%).

Here, since the probability is equal to or greater than the predetermined probability, the identification code is recognized according to this estimated probability (step S43). That is, the identification code is recognized to be "2." The CPU 30 records the recognized identification code in association with the worker ID obtained in step S22.

This process is performed each time a worker enters the facility, so that the correspondence between each worker's worker ID and the identification code attached to the helmet 4 worn by the worker is recorded, as shown in FIG. 12. This correspondence is updated every day.

If the highest estimated probability is lower than the confirmed probability in step S41, the CPU 30 determines whether the highest estimated probability is lower than the excluded probability (a predetermined unreliable probability, for example 10%) (step S42).

If it is equal to or greater than the exclusion probability (i.e., less than the confirmed probability but equal to or greater than the exclusion probability), the CPU 30 selects all identification codes with estimated probabilities equal to or greater than the exclusion probability (or a predetermined number of the top estimated probabilities) and displays these on the display 34 together with the identification code image of step S25 (step S44).

Figure 13 shows an example of the screen. An identification code image is displayed at the top, and below that, radio buttons and options "9", "8", and "none" are displayed. When the operator selects "9" or "8", the identification code is recognized according to this selection (step S45).

In this case, the CPU 30 may further learn the learned model using the identification code image, using the operator's selection as training data.

If the operator selects "None of the above," the operator is prompted to input the correct identification code. In response, recognition is performed according to the input identification code. In this case, re-learning may also be performed as described above.

Also, in step S42, if the highest estimated probability is less than the exclusion probability, the CPU 30 determines that the item does not match any of the identification codes and asks the operator to confirm this. An example of the screen is shown in FIG. 14.

If the numbers can be recognized from the screen, the operator inputs the correct identification code. The CPU 30 performs recognition accordingly (step S47). Therefore, processing can be performed even for identification codes for which no learning model is prepared.

When a new identification code is input a predetermined number of times and its images and training data are accumulated, the CPU 30 generates a new learning model for this identification code. It also re-learns the learning models for other identification codes.

If the operator cannot recognize the numbers on the screen, he or she clicks the "Cannot read numbers" radio button. This causes the CPU 30 to determine that the worker is not wearing a helmet. Regarding the presence or absence of a helmet, if the area of the color region corresponding to the helmet is equal to or smaller than a predetermined value in step S3, it may be determined that the helmet is not being worn and subsequent processing may not be performed.

In this way, a correspondence table like that shown in FIG. 12 can be obtained. This correspondence table is updated daily, because the helmet worn by each worker 2 at the work site may change every day.

(3) Tracking Process When a worker 2 enters the work site 28, he or she is photographed as a video by the cameras C2 to C5, and the trajectory of the worker's movements is recorded.

The CPU 30 acquires images from each of the cameras C2 to C5 (step S51). From the acquired images, a person image is extracted (step S52). Figure 17A shows the image from camera C2, and Figure 17B shows the extracted person image.

In this embodiment, a model trained through deep learning is used to extract images of people from within an image. For example, Dark Net (trademark) can be used. Dark Net (trademark) can distinguish between people, construction machinery, etc., and extract them as rectangles that surround them.

Next, the CPU 30 extracts a helmet image from the person image to obtain an identification code image (step S53). This extraction method is preferably the same as that in step S25 of FIG. 8. The extracted helmet image is shown in FIG. 17C, and the identification code image is shown in FIG. 17D.

Next, the CPU 30 performs a process to recognize the identification code captured in the image of the identification code. This estimation process is performed using the model learned by the deep learning described above.

First, the CPU 30 estimates whether the image shown in FIG. 17D is a "0" and outputs the estimated probability (step S54). At this time, a trained model for "0" is used. As described above, the trained model for "0" is trained using data that distinguishes between "0" and non-"0".

Next, the CPU 30 uses the trained model for "1" to estimate whether the image shown in FIG. 17D is "1" and outputs the estimated probability (step S55).

Then, in the same manner, it estimates whether the images shown in Figure 17D are "2," "3," "4," ... "9," and outputs the estimated probability.

Figure 18 shows an example of the calculated estimated probability. Next, the CPU 30 selects the highest estimated probability and determines whether this is equal to or greater than a predetermined confirmed probability (step S64). In Figure 18, the highest estimated probability is 96.2% for "5". The CPU 30 determines whether this is equal to or greater than a confirmed probability (a reliable predetermined probability, for example 95%).

Here, since the probability is equal to or greater than the predetermined probability, the identification code is recognized according to this estimated probability (step S65). That is, the identification code is recognized to be "5". Furthermore, the CPU 30 refers to the table in FIG. 12 and obtains the worker ID corresponding to the identification code "5".

Next, the CPU 30 identifies the position of the worker based on the image in FIG. 17A (step S66). As shown in FIG. 17A, the center point 62 of the bottom side of the frame 60 used to extract the person is identified as the position of the worker.

The imaging range of camera C2 is fixed. In this embodiment, feature points F1 to F4 in the captured image (points with landmarks, points where a wall meets the ground, etc.) are determined in advance, and the coordinate positions of each are recorded. Therefore, CPU 30 calculates the coordinates of center point 62, which indicates the worker's position, based on the relative positional relationship with feature points F1 to F4 (projective transformation). In this way, the worker's position coordinates can be obtained.

Next, the CPU 30 records the coordinate position in association with the worker ID (step S67). At this time, the time of capturing the image in FIG. 17A (which is attached to the image) is also recorded. If multiple workers are captured in FIG. 17A, the identification code of each worker is recognized and the coordinate position is identified.

The CPU 30 also performs the above process on the images from the other cameras C3 to C5. In this way, the time and position as shown in FIG. 19 are recorded for each worker.

Therefore, the movement trajectory of each worker can be recorded over time. In addition, the change in position over time can be displayed as an image on a floor plan of the work site.

Although an example of tracking the trajectory of a worker has been shown above, the trajectory of the construction machine M1 can also be recorded in a similar manner. It is preferable to directly assign an identification code to the construction machine M1. Therefore, the correspondence of the identification code can be recorded in advance for each construction machine M1.

1.4 Other
(1) In the above embodiment, the location of the worker is determined and recorded in real time. However, images for a predetermined period of time (for example, one day's worth) may be recorded, and the location of the worker may be determined and recorded all at once.

(2) In the above embodiment, the worker is identified and his/her position is determined without considering the image correlation between each frame of the captured video. However, the worker may be identified by considering the correlation between frames.

For example, in a captured video, the first frame is FIG. 20A, the second frame is FIG. 20B, the third frame is FIG. 20C, and the fourth frame is FIG. 20D. At this time, the CPU 30 first recognizes the identification code on the worker's helmet in the first frame and identifies the worker (for example, the worker is identified as Worker A). The position is identified by the center point 62a.

Next, the CPU 30 extracts a person image within a specified range around the center point 62a in the second frame (Fig. 20B). It recognizes the identification code of this worker image and identifies the worker. Its position is identified by the center point 62b.

Next, the CPU 30 extracts a person image within a specified range around the center point 62b in the third frame (Fig. 20C). It recognizes the identification code of this worker image and identifies the worker. Its position is identified by the center point 62c.

By repeating the above process, it is easy to track the trajectory of a specific worker. Note that if the worker recognized by the identification code is different from the previous and next frames, it may be possible to match the worker with the previous and next frames. For example, if worker A is recognized in the frames of Figures 20A, 20B, and 20D, and worker B is recognized only in the frame of Figure 20C, the worker will also be recognized as A in the frame of Figure 20C.

(3) In the above embodiment, the worker ID is obtained from an ID card. However, it may also be possible to read a QR code or the like held by the worker using a barcode reader or the like. It may also be possible to obtain the ID from a smartphone held by the worker.

For example, the smartphone's location information (from a built-in GPS receiver) and a pre-recorded worker ID (or a machine ID linked to the same, etc.) are transmitted to PC 24 via the Internet. This transmission process is performed by an app (program) installed on the smartphone. This enables PC 24 to detect that a worker has entered entrance area 26 and obtain the worker ID.

Alternatively, the operator may input the information via a keyboard, etc.

(4) In the above embodiment, numbers are used as the identification code. However, letters, symbols, patterns, etc. may also be used. Furthermore, multiple identification codes may be randomly generated as patterns, and a combination of multiple patterns that is highly likely to be correctly estimated by the trained estimation means may be used.

(5) In the above embodiment, the trajectory of the worker is recorded. However, instead of this, or in addition to this, a warning or the like may be issued based on the acquired position of the worker.

As shown in FIG. 21, assume that a danger area 70 has been set up at a work site. For example, an area into which no one should enter due to the risk of a building collapsing is set as the danger area 70, and the coordinates of its four corners are recorded in advance on the hard disk 36.

After identifying the location of each worker, the CPU 30 judges whether any of the workers have entered the danger area 70. If it judges that any of the workers have entered the danger area 70, the CPU 30 makes a call to the smartphone of the worker and sends a warning message.

To carry out these processes, the phone number of each worker's smartphone is pre-recorded on the hard disk 36. Also, a warning voice message is pre-recorded on the hard disk 36.

In the above, the warning voice message was sent by telephone, but it is also possible to install a dedicated application on a smartphone in advance and control the application via the Internet, etc. to emit a warning sound.

In some cases, only authorized personnel may enter the dangerous area 70. In this case, whether each worker is permitted to enter the dangerous area 70 can be recorded in advance as a table. An example is shown in FIG. 22. In this figure, a worker with a worker ID of 1328 is permitted to enter the dangerous area 70.

The CPU 30 determines whether a worker who has entered the danger area 70 is permitted to reenter, and issues a warning if the worker is not permitted to reenter.

Whether or not an area can be entered may be determined based on the duration of work that the worker will be doing that day. For example, areas may be set up that workers who are working long hours cannot enter.

You could also set up areas where access is prohibited in the morning but is permitted in the afternoon. By recording this, you could issue warnings according to the time.

In the above embodiment, the danger area 70 was fixed. However, as shown by the danger area 72 in FIG. 21, it may change dynamically. In other words, as the construction machine M1 moves, the danger area 72 within a specified range around it also moves. The CPU 30 is also aware of the position of the construction machine M1, so it can update the danger area 72 from moment to moment.

(6) In the above embodiment, cameras C2 to C5 capture images of workers and construction machinery to identify their locations. However, it is also possible to have workers carry GPS receivers (or to install a GPS receiver on construction machinery) to obtain the location of each worker and construction machinery. The location of each worker and construction machinery can be identified by the unique ID of the GPS receiver.

However, as described above, position determination using a GPS receiver cannot be used indoors or in other places where radio waves cannot reach. Therefore, for places where GPS radio waves cannot reach, such as indoors, position determination using a camera according to the above embodiment can be used in addition.

Figure 23 shows a work site 28 that has an area 90 where GPS radio waves cannot reach. When a worker enters, the identification code on the helmet is read and linked to the worker ID. The helmet is equipped with a GPS receiver. Therefore, the receiver ID of this GPS receiver is also linked to the worker ID. Note that the GPS receiver may be attached to the worker's body instead of the helmet. In this case, the linking can be done based on the time of the worker's entry and the time and position measured by the GPS receiver.

In the work site 28 other than the area 90 where radio waves cannot reach, the position of each worker is identified by a GPS receiver and their trajectory is tracked. In the area 90 where radio waves cannot reach, the worker is identified and his/her position is determined by the image captured by the camera C7.

(7) In the above embodiment, the tracking of moving objects at the work site 28 has been described. However, the present invention can also be applied to tracking moving objects outside of work sites. For example, the present invention can be applied to facilities such as libraries, theme parks, parks, and equipment in factories.

(8) In the above embodiment, an identification code is attached to the helmet. However, an identification code may be attached to an employee ID card, an ID card, clothing, a wristband, etc.

(9) In the above embodiment, the worker is identified by the identification code on the helmet worn by the worker. However, the worker may also be identified based on an image of the worker's face or body.

(10) In the above embodiment, estimation is performed for each identification code (for each digit from "0" to "9"), and the estimation results are integrated.

However, it is also possible to provide a single estimation means for estimating which identification code the identification code image represents.

(11) In the above, the trajectory of each individual worker is tracked.

However, it would also be possible to assign different helmet identification codes to each group of workers (each job, such as plasterer, scaffolder, etc.) and track the trajectories of workers belonging to the group without identifying individuals.

(12) In the above embodiment, the identification code is recognized to identify the worker, and the worker is tracked based on this. However, the process of identifying the worker by recognizing the identification code can be used in cases other than tracking. For example, it can be used to identify the worker itself, or to determine whether the worker is at the work site.

(13) In the above embodiment, the worker is tracked and then a warning is issued for intrusion into a dangerous area. However, it is also possible to only issue a warning for intrusion without tracking.

(14) The above embodiment and its variations can be applied to other embodiments.

2. Second embodiment
2.1 Overall Configuration Fig. 24 shows the functional configuration of a person tracking device according to a second embodiment of the present invention. In this embodiment, tracking of the behavior of a worker at a work site is described as an example. The same tracking process as in the first embodiment can be used. This embodiment relates to a mechanism for additional learning of the recognition means 10.

When a worker 2 enters a work site, he or she puts on a helmet 4, which is an accessory. An identification code 6 is printed or otherwise attached to the helmet 4 in multiple locations, such as the front, back, left and right sides, and top. The same identification code is attached to multiple locations on the same helmet 4, and each helmet 4 is assigned a unique identification code. For example, the identification codes are "0," "1," ... "9."

As with the first embodiment, the worker ID is matched to the identification code upon entry.

After entering the work site, let's say a worker purchases a drink from a vending machine. At this work site, each worker can purchase a product by holding their ID card over the vending machine. Therefore, when purchasing a product, the worker ID can be obtained and it is possible to identify the worker standing in front of the vending machine. That is, the ID obtaining means 14 obtains the worker ID.

The vending machine is equipped with a camera 9, which is capable of capturing an image of the worker. The learning means 13 additionally learns the first estimation means E1 to En based on the identified worker ID and the captured image of the worker.

This makes it possible to improve the estimation accuracy of the first estimating means E1 to En.

2.2 System Configuration Figure 25 shows the system configuration of the moving object tracking device. An entrance area 26 is provided at a work site 28. When a worker enters the work site 28, he or she wears a helmet and is authenticated by holding an ID card over a card reader 22 at the entrance area 26. The ID card records the worker ID of each worker, which is read by the card reader 22 and transmitted via PC 21 to a PC 24 provided at a site office or the like. A short-distance communication system such as FeliCa (trademark) can be used as the ID card.

An entrance camera C1 is installed in the entrance area 26, and captures an image of a worker wearing a helmet with an identification code attached. The captured image is sent to the PC 24. The PC 24 recognizes the identification code based on this image. It then records this in association with the worker ID received from the card reader 22.

In the figure, it is shown that workers W1 to W5 have already entered the work site. Work site cameras C2 to C5 are installed at the work site 28, and capture images of the work site 28. These captured images are sent to the PC 24, which recognizes the identification codes on the helmets of each worker W1 to W5. In addition, their positions are ascertained based on the images. Therefore, the positions of each worker W1 to W5 can be known, and their movement trajectories can be recorded.

At the work site 28, a vending machine 94 for beverages and the like is installed. This vending machine 94 has a built-in function for reading the worker ID on the ID card held by each worker. This allows for settlement at the time of product purchase. The worker ID read by the vending machine 94 is sent to the PC 24.

The vending machine 94 is also equipped with a camera W8, which is capable of capturing an image of the worker W3 who has purchased a product. The captured image is sent to the PC 24 together with the worker ID described above.

The PC 24 additionally learns a program that recognizes the identification code based on the worker ID and image of the worker that were sent.

The hardware configuration of the PC 24 is similar to that shown in FIG.

2.3 Additional Learning Process The process of tracking a worker is the same as that in the first embodiment. The additional learning process will be described below.

Figure 26 shows a flowchart of the additional learning process in the moving object tracking program 46.

The CPU 30 determines whether an operator ID has been transmitted from the vending machine 94 (i.e., whether the operator has purchased a product from the vending machine) (step S91). When the operator ID is transmitted, the CPU 30 records the operator ID on the hard disk 36 (step S92).

Next, the CPU 30 acquires and records an image of the operator at the time of purchase captured by the vending machine 94 (step S93). An example of the acquired image is shown in FIG. 27A.

Next, the CPU 30 extracts the person from this worker image, and further extracts the helmet, and extracts an image of the identification code (step S94). This process is as described in the first embodiment. An example of the extracted identification code image is shown in FIG. 27B.

The CPU 30 then acquires an identification code associated with the acquired worker ID (step S95). That is, the CPU 30 acquires the identification code recorded in step S46 of the entry processing in Figures 8 and 9.

The CPU 30 performs additional learning of the estimation means for each of the digits "0" to "9" for the identification code image obtained above, using the acquired identification code as training data (steps S96 to S105).

In this manner, the estimation accuracy of the estimation means can be improved.

2.4 Other
(1) In the above embodiment, additional learning is performed using an image captured after the worker ID is acquired and the worker is identified.

However, additional learning may be performed based on images in which the worker can be identified.

For example, additional learning may be performed based on images captured by a worker carrying a GPS receiver alone. In this case, the worker's position is identified, so the worker in the image can be identified.

Alternatively, additional learning can be done based on images of workers who are determined to be in a specific location at a specific time. In this case too, it is possible to identify the worker whose image is being captured.

(2) The above embodiment and its modifications can be applied to other embodiments.

3. Third embodiment
3.1 Overall Configuration Fig. 28 shows the functional configuration of a person tracking device according to the third embodiment of the present invention. The basic configuration is the same as that of the first embodiment.

However, the identification code 6 attached to the helmet 4 is different. In this embodiment, the identification code 6 has multiple digits, and each digit has a different color. This allows the identification code 6 to be recognized more accurately, taking the color into consideration.

3.2 System Configuration The system configuration is the same as that of the first embodiment described with reference to FIGS.

Figure 29 shows helmets 4 to be worn by workers. Each helmet 4 has a sticker 7 with a unique identification code attached to it. The sticker 7 is preferably made of a material with low reflection or a surface treatment with low reflection. For example, matte paper can be used. In this embodiment, the sticker 7 is formed by printing a colored identification code with a black background on white matte paper.

The sticker 7 has a background color of black (a specific color) on which an identification code of a specified number of digits (three digits in this example) is written. For example, the identification code of the helmet 4 at the top left is made up of a series of individual codes "J", "1", and "K". The same identification code is also given to the front, top, left and right sides. This is to ensure that the identification code is included in the image even when it is photographed from various angles.

The color of the individual code is also changed for each digit. Although not shown in FIG. 29, in this embodiment, the first digit is displayed in pink, the second digit in yellow, and the third digit in blue. Any color combination may be used as long as the colors are distinguishable, but it is preferable to set the RGB values of each color to be significantly different.

It is also preferable that the font used for the symbols in each digit has a uniform line width (such as Gothic).

Furthermore, the symbols used for each digit are designed to have low similarity to each other. In this embodiment, the numbers and alphabets marked with circles as shown in Figure 30 are used.

The selection of codes with low similarity may be performed as follows. A predetermined number of codes to be judged are given to the recognition means 10, which has been trained on multiple types of candidate codes (for example, 0-9 and A-Z), and the recognition means 10 is made to judge them. Based on the results of this judgment, an F-value (F-scale) is calculated for each code. This F-value is calculated using the following formula:

F-score = (2 * precision * recall) / (precision + recall)
For example, the precision rate for the code "1" is calculated as follows: If there are 15 numbers that the recognition means 10 judges to be "1" and 10 of them give the value "1", the precision rate is 10/15. In other words, it is calculated based on how many of the numbers recognized as the code are valid.

The recall rate for the code "1" is calculated as follows. If 12 code "1"s are given, and the recognition means 10 correctly judges 10 of them to be "1", then the recall rate is 10/12. In other words, when a given number of codes are given, the recall rate is calculated based on how many of them are correct.

In this manner, the F value is calculated for each candidate code, and a predetermined number of codes having the highest F value are selected as the codes to be used.

3.3 Tracking Process The process of tracking the behavior of a worker wearing the helmet 4 will now be described.

(1) Learning Data Generation Process and Trained Model Generation In this embodiment, an image of an identification code is recognized by a trained model of deep learning. First, the learning data generation process will be described.

Figure 31 shows a flowchart of the learning data generation process of the moving object tracking program 46. A large number of images of the helmet 4 in Figure 29 taken from various angles are recorded in advance on the hard disk 36.

The CPU 30 reads these images one by one from the hard disk 36 (step S2). An example of the read image is shown in FIG. 32A. In the figure, the images are shown in monochrome, but in this embodiment, color images are used.

Next, the CPU 30 extracts only the helmet 4 from this image (step S103). In this embodiment, the helmet is extracted using a trained model that has been deep-trained on the helmet image. For example, YOLO (https://pjreddie.com/darknet/yolo/) can be used. As shown in FIG. 32A, the area of the helmet 4 is extracted as a rectangular boundary box.

Next, the CPU 30 extracts the entire sticker 7 from the extracted image of the helmet 4 based on the color of the sticker 4 (black in this embodiment) (step S104). An example of the extracted image is shown in FIG. 32B. Note that in FIG. 32A, sticker 7 is shown in three places. In this embodiment, one image of the sticker 7 with the largest surface area is selected. Also, an image of sticker 7 that includes all three digits may be selected based on the color of each digit. Note that the image of sticker 4 may be selected by the operator using a mouse or the like.

The identification code shown on sticker 7 is colored, but CPU 30 converts the identification code portion of the extracted image of sticker 7 to white, resulting in an image like that shown in Figure 32B.

The CPU 30 displays the image of the sticker 7 thus extracted (i.e., the overall code) on the display. The operator looks at this image and inputs the class of the overall code ("J1K" in the example of FIG. 32B) (step S105). The CPU 30 records the image of the overall code in FIG. 32B in association with the input class "J1K".

Then, the CPU 30 separates the image of the overall code (image of sticker 7) into digits based on their color (step S106). The first digit is pink, the second digit is yellow, and the third digit is blue, so the individual codes for each digit are separated accordingly. The CPU 30 converts the recognized parts of each individual code to white and the other parts to black, obtaining an image separated into digits as shown in FIG. 32C.

The CPU 30 displays the image of the individual code extracted in this manner on the display. The operator looks at this image and inputs the class of the individual code (in the example of FIG. 32B, "J", "1", or "K") (step S107). The CPU 30 records the input classes "J", "1", and "K" in association with each individual image.

When the processing for one image is completed in this manner, the same processing is carried out for the next recorded image to extract the overall code and assign a class, and to extract individual codes and assign a class (steps S101 to S108).

After extracting the overall code and individual code and assigning classes to all images, the CPU 30 generates a learning data set for the class of the identification code "J1K" (step S7). The learning data set for "J1K" is generated from images of the overall code assigned the class "J1K" and images of the overall code assigned a class other than "J1K".

Note that the number of "J1K" data sets generated in steps S101 to S108 in FIG. 31 is usually not equal to the number of data sets other than "J1K". For example, if there are 10 classes of the overall code, the ratio is usually 1:9.

However, a learning dataset in which the number of "J1K" datasets and the number of datasets other than "J1K" are significantly different is not desirable for learning. Therefore, in this embodiment, data other than "J1K" is picked up and used as the learning dataset, so that the number of "J1K" datasets and the number of datasets other than "J1K" are approximately equal (for example, in a range not exceeding 1:1.5 (preferably, not exceeding 1:1.1)).

The CPU 30 performs similar processing to generate learning data sets for each of the classes "BW2", "RJ6", "LNU", "YHX", "4UC", "STD", "TL1", "CKF", and "F2Y" (steps S109 to S110).

Next, the CPU 30 generates a learning data set for the class of the individual code "1" (step S111). The learning data set for "1" is generated from images of individual codes assigned the class "1" and images of individual codes assigned a class other than "1".

Note that the number of "1" data sets generated in steps S101 to S108 in FIG. 31 is usually not equal to the number of data sets other than "1". For example, if there are 20 classes of individual codes, the ratio is usually 1:19.

However, a training dataset in which the number of "1" data sets and the number of non-"1" data sets differ significantly is not desirable for training. Therefore, in this embodiment, data other than "1" is picked up and used as the training dataset, so that the number of "1" data sets and the number of non-"1" data sets are approximately equal (for example, in a range not exceeding 1:1.5 (preferably, not exceeding 1:1.1)).

The CPU 30 performs similar processing to generate learning data sets for each of the classes "2", "4", "6", "B", "C", "D", "F", "H", "J", "K", "L", "N", "R", "S", "T", "U", "W", "X", and "Y" (steps S111 to S112).

After generating the learning data sets for the overall code and the individual codes in the above manner, the CPU 30 trains a deep learning program based on the respective learning data sets. This allows the CPU 30 to obtain trained models for estimating the overall codes "J1K", "BW2", ... "CKF", "F2Y", and trained models for estimating the individual codes "1", "2", ... "X", "Y". These models are stored on the hard disk 36.

(2) Corresponding Process Between Identification Code and Worker ID When it is not decided which helmet 4 the worker 2 will wear, it is necessary to correlate the worker ID with the identification code of the helmet 4. In this embodiment, this is done when the worker enters the work site.

Figures 33 and 34 show a flowchart of the entry processing of the moving object tracking program 46. When entering the entry area 26, the worker 2 holds his/her ID card over the card reader 22. The worker ID read from the ID card is sent to PC 24 via PC 21.

The CPU 30 reads the ID card and determines whether the worker ID has been transmitted (step S121). If the worker ID has been transmitted, the CPU 30 records the worker ID in the hard disk 36 as a worker who has clocked in for the day (step S122).

The CPU 30 then acquires an image of the worker 2 captured by the camera C1 installed in the entrance area 26 (step S123). An example of the acquired image is shown in FIG. 35A. The CPU 10 extracts an image of the helmet 4 from this image (step S124). In this embodiment, an estimation program using deep learning is used to extract the image of the helmet 4. For example, YOLO can be used. As shown in FIG. 35A, the image of the helmet 4 is extracted as a rectangular area (boundary box).

Furthermore, the CPU 30 extracts an image of the sticker 7 bearing the identification code based on the background color of the sticker 7 (black in this embodiment) (step S125). In FIG. 35A, three images of stickers 7 are shown within the image of the helmet 4. The CPU 30 selects the image of the sticker 7 with the largest surface area. Also, an image of the sticker 7 containing all three digits may be selected based on the color of each digit. The extracted image of the sticker 7 is shown in FIG. 35B.

The identification code shown on sticker 7 is colored, but CPU 30 converts the extracted part of sticker 7 that contains the identification code to white, resulting in an image like that shown in Figure 35C.

Next, the CPU 30 extracts the individual code for each digit based on the color image in FIG. 35B (step S126). As shown in FIG. 35D, the CPU 30 converts the identification code portion of the extracted individual code to white and the other portions to black.

Next, the CPU 30 performs a process to recognize the symbols captured in the image of the overall symbol extracted above and the individual symbols of each digit. This estimation process is performed using the model learned by the deep learning described above.

First, the CPU 30 estimates whether the image shown in FIG. 35C is "J1K" and outputs the estimated probability (step S127). At this time, a trained model for "J1K" is used. As described above, the trained model for "J1K" is trained using data that distinguishes between "J1K" and non-"J1K".

Next, the CPU 30 uses the trained model for "BW2" to estimate whether the image shown in FIG. 35C is "BW2" and outputs the estimated probability.

In the same manner, it is estimated whether the image shown in FIG. 35C is "RJ6", "LNU", "YXH", ... "F2Y", and the estimated probability is output. An example of the estimated probability calculated for the entire code is shown in FIG. 36.

Next, the CPU 30 estimates whether the image of the first digit shown in FIG. 35D is a "1" and outputs the estimated probability (step S131). At this time, a trained model for "1" is used. As described above, the trained model for "1" is trained using data that distinguishes between "1" and non-"1".

Furthermore, the CPU 30 uses the trained model for "2" to estimate whether the image of the first digit shown in FIG. 35D is a "2" and outputs the estimated probability.

In the same manner, it is estimated whether the image of the first digit shown in Fig. 35D is "4", "6", ... "X", or "Y", and the estimated probability is output (step S132). In this way, the estimated probability of which individual code the first digit corresponds to is calculated.

Next, the CPU 30 performs the same process as above to calculate the estimated probability that the image of the second digit and the image of the third digit in FIG. 35D correspond to each of them. FIG. 37 shows an example of the estimated probability calculated for the individual codes.

The CPU 30 estimates the identification code based on the estimated overall code and the estimated individual code (step S134). For example, the CPU 30 estimates the identification code as follows.

The CPU 30 first selects the individual code with the highest estimated probability for each digit. In the example of FIG. 37, "R" would be selected as the first digit individual code, "J" as the second digit individual code, and "6" as the third digit individual code.

Next, the CPU 30 compares each digit between the individual codes "R", "J", and "6" selected in this way and all the overall codes in Figure 36, and calculates the Hamming distance (the sum of "0" if there is a match for each digit, and "1" if there is no match). Figure 38 shows the calculated Hamming distance. The Hamming distance is calculated as "3" for the overall code "J1K", "3" for "BW2", "0" for "RJ6", etc.

The CPU 30 determines the overall code corresponding to the smallest of these calculated Hamming distances as the first identification code candidate. In the case of FIG. 38, the overall code "RJ6" is determined as the first identification code candidate.

Next, the CPU 30 selects the overall code with the highest estimated probability as the second identification code candidate. In the case shown in FIG. 36, the overall code "RJ6" is determined to be the second identification code candidate.

When the first and second identification code candidates are the same, the CPU 30 determines the overall code as the identification code. In the case of FIG. 38, the identification code is determined to be "RJ6."

If the first identification code candidate and the second identification code candidate are different, the CPU 30 selects one of them to determine the identification code. For example, as shown in FIG. 39, a case will be described in which the first identification code candidate is selected as "RJ6" and the second identification code candidate is selected as "YXH". In FIG. 39, the estimated probability for the overall code is highest for "YXH" (0.931), so this will be selected as the second identification code candidate.

First, the CPU 30 calculates the representative estimation probability for the individual codes "R", "J", and "6" in the first identification code candidate "RJ6" that match the codes in each digit. Here, the individual codes "R", "J", and "6" match the codes in all digits of the overall code "RJ6". Therefore, the CPU 30 calculates the average value (or the maximum value) of the estimation probabilities of these individual codes as the representative estimation probability. If the estimation probabilities of the individual codes "R", "J", and "6" are "0.988", "0.975", and "0.964", respectively, the representative estimation probability (average value) is 0.975.

The CPU 30 compares the representative estimated probability (0.975) of the first identification code candidate calculated in this manner with the estimated probability (0.931) of the second identification code candidate. The CPU 30 then determines the one with the higher estimated probability as the identification code. In the example of Figure 39, the first identification code candidate is selected as the identification code.

As described above, the CPU 30 estimates the final identification code by taking into account both the individual code estimation results and the overall code estimation results.

In determining the first identification code candidate, in FIG. 38, there was only one with the smallest Hamming distance. However, as shown in FIG. 40, there are cases where multiple candidates with the smallest Hamming distance are found. In FIG. 40, three candidates with a Hamming distance of "2" are selected: "BW2", "RJ6", and "LNU".

In this case, the CPU 30 selects the first identification code candidate as follows. Among the digits represented in the above three overall codes, the estimated probability of the individual code with a matching code is obtained. In the example of Figure 40, for the overall code "BW2", the estimated probability of the individual code "B" is obtained; for the overall code "RJ6", the estimated probability of the individual code "6" is obtained; and for the overall code "LNU", the estimated probability of the individual code "N" is obtained; these are set as the representative estimated probabilities. Here, it is assumed that "1.000", "0.990", and "0.985" are obtained, respectively, as shown in Figure 40.

When there are multiple individual codes that match one overall code, the average (or maximum) of the estimated probabilities is calculated as the representative estimated probability.

The CPU 30 selects the overall code corresponding to the highest representative estimation probability obtained above, and sets this as the first identification code candidate. In the example of Figure 40, the overall code "BW2" is selected and set as the first identification code candidate.

In this way, even if there are multiple overall codes with the smallest Hamming distance, one first identification code candidate can be selected.

The CPU 30 estimates an identification code based on the first identification code candidate and the second identification code candidate as described above. Furthermore, the CPU 30 records the estimated identification code in association with the worker ID (step S135).

In this way, a correspondence table can be obtained. This correspondence table is updated daily, because the helmet worn by each worker 2 at the work site may change every day.

(3) Tracking Process The camera C2 to C5 capture the image of the worker 2 who has entered the work site 28 as a moving image, and record the movement trajectory of the worker 2. A flowchart of the tracking process is shown in FIG.

The CPU 30 acquires images from each of the cameras C2 to C5 (step S151). From the acquired images, a person image 60 and a helmet image 61 are extracted (steps S152, S153). The person image 60 and the helmet image 61 are recorded in correspondence with each other. This is to clarify the correspondence when multiple people are captured.

Figure 43 shows a frame 60a of a person image, a frame 61a of a helmet image, a frame 60b of another person image, and a frame 61b of the helmet image when multiple people are captured overlapping each other.

The CPU 30 calculates the degree of overlap of the helmet image frames 61a, 61b for each of the person image frames 60a, 60b. The degree of overlap is calculated as 100% if the helmet image frame 61 is completely contained within the person image frame 60, and 0% if there is no overlap at all. The CPU 30 selects a helmet image frame 61 such that the degree of overlap between the person image frame 60 of interest and the helmet image frame 61 exceeds a predetermined percentage (for example, 10%). In the example of Figure 43, for the person image frame 60a, the helmet image frames 61a, 61b are selected. For the person image frame 60b, only the helmet image frame 61b is selected.

When one helmet image frame is selected from a person image frame, the CPU 30 associates this helmet image frame with the person image frame. In the example of Figure 43, helmet image frame 61b is associated with person image frame 60b.

When multiple helmet image frames are selected for a person image frame, the helmet image frame with the uppermost top edge is selected and associated with the person image frame. In the example of Figure 43, helmet image frame 61a is associated with person image frame 60a.

In this way, the first correspondence candidates (60a and 61a, 60b and 61b) are obtained. These may be used as the correspondence as is, but in this embodiment, to increase the accuracy, the second correspondence candidates are also calculated as follows.

For each helmet image frame 61a, 61b, the CPU 30 selects person image frames 60a, 60b whose overlapping degree is equal to or greater than a predetermined value. In the example of FIG. 43, for the helmet image frame 61a, only the person image frame 60a is selected. For the helmet image frame 61b, the person image frames 60a, 60b are selected.

When a person image frame is selected for a helmet image frame, the CPU 30 associates this person image frame with the helmet image frame. In the example of FIG. 43, the person image frame 60a is associated with the helmet image frame 61a.

When multiple person image frames are selected for a helmet image frame, the one with the lowest upper edge is selected and associated with the helmet image frame. In the example of Figure 43, person image frame 60b is associated with helmet image frame 61b.

In this way, the second correspondence candidates (60a and 61a, 60b and 61b) are obtained.

The CPU 30 uses the first and second correspondence candidates that match as the correspondence. In the above example, all of the correspondences match, so all of the correspondences are used and recorded.

Matchings that do not match will not be adopted. Therefore, for people who are not adopted as a match, the position estimation described below will not be performed for that frame. Note that it is also possible to match a person who could not be matched with a helmet using information from previous and following frames.

Figure 42A shows an image from camera C2, and Figure 42B shows an extracted helmet image 61. A person image 60 is extracted in the same way.

In this embodiment, a trained model based on deep learning is used to extract images of people and helmets from an image. For example, YOLO can be used.

Then, the CPU 30 extracts the sticker portion from the helmet image. The extracted sticker image is shown in FIG. 42C. Furthermore, as shown in FIG. 42D, images of the overall code and individual code are obtained. From the images of the overall code and individual code extracted in this manner, an identification code is estimated (step S154). This process is the same as steps S125 to S134 in FIG. 33 and FIG. 34.

Next, the CPU 30 identifies the position of the worker based on the image in FIG. 42A (step S155). As shown in FIG. 42A, the center point 62 of the bottom side of the frame 60 used when extracting the person is identified as the position of the worker. Note that the center of gravity of the frame 60 or another point may be used as the position of the worker.

The imaging range of camera C2 is fixed. In this embodiment, feature points F1 to F4 in the captured image (points with landmark marks, edges of tiles, points where walls meet the ground, etc.) are determined in advance, and the coordinate positions of each are recorded. Therefore, CPU 30 calculates the coordinates of center point 62, which indicates the worker's position, based on the relative positional relationship with feature points F1 to F4 (projective transformation). In this way, the worker's position coordinates can be obtained.

Next, the CPU 30 records the coordinate position in association with the worker ID (step S156). At this time, the image capture time (added to the image) of the image in FIG. 42A is also recorded. If multiple workers are captured in the image of FIG. 42A, the identification code of each worker is recognized and the coordinate position is identified.

The CPU 30 also performs the above process on the images from the other cameras C3 to C5. In this way, the time and position as shown in FIG. 19 are recorded for each worker.

Therefore, the movement trajectory of each worker can be recorded over time. In addition, the change in position over time can be displayed as an image on a floor plan of the work site.

Although an example of tracking the trajectory of a worker has been shown above, the trajectory of the construction machine M1 can also be recorded in a similar manner. It is preferable to directly assign an identification code to the construction machine M1. Therefore, the correspondence of the identification code can be recorded in advance for each construction machine M1.

3.4 Other
(1) In the above embodiment, the individual codes are differentiated by changing the color of the individual codes. However, the individual codes may be differentiated by changing the display form, such as the pattern of the individual codes. Also, the individual codes may be differentiated by changing the display form of the background, such as the color or pattern of the background.

(2) In the above embodiment, the individual code is estimated using only images of the sticker 7 displayed on the helmet that show all three digits. However, in addition to this, the individual code may also be estimated by taking into consideration images of the sticker 7 that show only two digits or one digit. For example, when an individual code estimate based on a sticker image that shows all three digits differs from an individual code estimate based on a sticker image that shows only two digits or one digit, it is advisable to select the one with the higher estimation probability.

(3) In the above embodiment, the identification code is estimated using the Hamming distance. However, the identification code may be set so that different individual codes are used for the same digit, and the identification code may be estimated based on the numbering conditions.

For example, suppose the individual codes for each digit have the highest estimated probability of being "B," "1," and "K." Here, the only overall code with "B" in the first digit is "BW2." The only overall code with "1" in the second digit is "J1K." The only overall code with "K" in the third digit is "J1K." As a result, one "BW2" and two "J1K"s are found, so the most common "J1K" can be determined as the first identification code candidate based on the individual codes for each digit. If it matches the second identification code candidate, it is determined to be the identification code. If they differ, the process is as explained above.

In addition, there may be multiple codes with the highest estimated probability for each digit. In such cases, the first identification code candidate can be determined using the numbering conditions described above.

For example, suppose that the individual codes for each digit have the highest probability of being guessed as "B," "1," and "H or K." In other words, suppose that for the individual code for the third digit, "H" and "K" have the highest probability of being guessed, and no ranking can be determined.

Here, the overall code with "B" in the first digit is "BW2". The overall code with "1" in the second digit is "J1K". The overall code with "K" in the third digit is "J1K", and the overall code with "H" is "YXH". As a result, one "BW2", one "YXH", and two "J1K"s were found, so the most common "J1K" can be determined as the first identification code candidate based on the individual codes of each digit.

(4) In the above embodiment, a three-digit identification code is used, but an identification code of two or less digits or four or more digits may also be used.

(5) In the above embodiment, a first identification code candidate is calculated based on an estimation of the individual code, a second identification code candidate is calculated based on an estimation of the overall code, and a final identification code is estimated based on the first identification code candidate and the second identification code candidate.

However, the final identification code may be estimated based on only the first identification code candidate or only the second identification code candidate. For example, when judging based only on the first identification code candidate, even if there is no sticker 7 with all three digits of the code on the worker's helmet in the image, the identification code can be estimated by integrating the codes displayed on multiple stickers 7. For example, suppose that three stickers 7a, 7b, and 7c are displayed in an image of a worker (e.g., FIG. 32A). Sticker 7a includes the first and second digits, sticker 7b includes the second and third digits, and sticker 7c includes only the third digit. The CPU 30 can determine this based on the color of the codes.

In the above case, by combining the estimation of the individual code based on the image of sticker 7a, the estimation of the individual code based on the image of sticker 7b, and the estimation of the individual code based on the image of sticker 7c (the estimation with the higher probability should be used for the second and third digits), it is possible to estimate the individual code for each digit and thus the identification code.

It may also be possible to determine whether to use only the first identification code candidate or to also use the second identification code candidate based on the Hamming distance. For example, if the Hamming distance for the first identification code candidate is "0", the first identification code candidate is determined as the identification code without calculating the second identification code candidate. If the Hamming distance for the first identification code candidate is not "0", the second identification code candidate is calculated, and the identification code is determined taking both into consideration.

(6) In the above embodiment, the worker is identified and his/her position is determined without considering the image correlation between each frame of the captured video. However, the worker may be identified by considering the correlation between frames.

For example, in a captured video, the first frame is FIG. 20A, the second frame is FIG. 20B, the third frame is FIG. 20C, and the fourth frame is FIG. 20D. At this time, the CPU 30 first recognizes the identification code on the worker's helmet in the first frame and identifies the worker (for example, the worker is identified as Worker A). The position is identified by the center point 62a.

Next, the CPU 30 extracts a person image within a specified range around the center point 62a in the second frame (Fig. 20B). It recognizes the identification code of this worker image and identifies the worker. Its position is identified by the center point 62b.

Next, the CPU 30 extracts a person image within a specified range around the center point 62b in the third frame (Fig. 20C). It recognizes the identification code of this worker image and identifies the worker. Its position is identified by the center point 62c.

By repeating the above process, it is easy to track the trajectory of a specific worker. Note that if the worker recognized by the identification code is different from the previous and next frames, it may be possible to match the worker with the previous and next frames. For example, if worker A is recognized in the frames of Figures 20A, 20B, and 20D, and worker B is recognized only in the frame of Figure 20C, the worker will also be recognized as A in the frame of Figure 20C.

(7) In the above embodiment, the position of the worker is determined using a person frame in step S155. However, it is also possible to extract only the person image using a background subtraction method or the like and binarize it, and use the coordinates of the midpoint of two minimal pixels (corresponding to the two toes) near the bottom of the binarized person image as the person position. Also, if the two toes cannot be recognized in the binarized image (if the toes are grouped together), the coordinates of the bottommost pixel may be used as the person position.

Furthermore, the calculation may be performed as follows. A trained model is prepared by providing a large number of person images and the corresponding person position coordinates (such as the coordinates of a point at the feet) as training data. A person image may be provided to this trained model to calculate local coordinates within the person frame, which may then be combined with the position of the person frame to determine the person position coordinates.

(8) The above embodiment and its modifications can be applied to other embodiments.

Claims (55)

one or more target area cameras for imaging a target area;
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by the target area camera;
an image position acquisition means for acquiring a position of the moving object within a target area based on the captured image;
a position processing means for determining the identity of the moving object based on the recognized identification code and for recording a time-dependent change in the position of the moving object in a recording unit;
A moving object tracking device comprising:
The recognition means includes:
a first estimation means for distinguishing the first identification code from other identification codes and calculating an estimated probability that the code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to nth estimation means.
The moving object tracking device is characterized in that, if the highest estimation probability among the first identification code to the nth identification code is equal to or lower than a predetermined exclusion probability, the integration means determines that identification code to be the n+1th identification code, performs learning based on a captured image, distinguishes between the n+1th identification code and other identification codes, and performs a learning process to generate the n+1th estimation means that calculates the estimation probability that the identification code is the n+1th identification code, or suggests the learning process. A moving object tracking program for implementing a moving object tracking device by a computer, the moving object tracking program comprising:
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by a target area camera;
an image position acquisition means for acquiring a position of the moving object within a target area based on the captured image;
A moving object tracking program for causing the program to function as a position processing means for determining the identity of a moving object based on the recognized identification code and recording a time-dependent change in the position of the moving object in a recording unit,
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to n-th estimation means,
The moving object tracking program is characterized in that, if the highest estimation probability among the first identification code to the nth identification code is equal to or lower than a predetermined exclusion probability, the integration means determines that identification code to be the n+1th identification code, performs learning based on a captured image, distinguishes between the n+1th identification code and other identification codes, and performs a learning process to generate an n+1th estimation means that calculates the estimation probability that the identification code is the n+1th identification code, or suggests the learning process. In the moving object tracking program according to claim 2,
The moving object tracking program causes the computer to further function as an entry recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by an entry camera , a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entry camera , and a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other,
A moving object tracking program characterized in that the position processing means obtains a moving object ID of the moving object by referring to the recording unit based on an identification code recognized from an image of the moving object captured by the target area camera. In the moving object tracking program according to claim 2 or 3,
A plurality of identification codes are displayed on the moving object,
A moving object tracking program characterized in that the recognition means recognizes the identification code displayed on the moving object using a machine learning trained model that has been trained based on images captured from different angles of the identification code displayed on the moving object. In the moving object tracking program according to any one of claims 2 to 4,
A moving object tracking program, wherein the identification code is displayed on an attachment attached to the moving object. In the moving object tracking program according to any one of claims 2 to 5,
The moving object tracking program is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the identification code with the highest estimated probability is determined as the recognition result. In the moving object tracking program according to any one of claims 2 to 6,
The moving object tracking program is characterized in that, when there are multiple estimated probabilities among the first identification code to the nth identification code that exceed a predetermined exclusion probability and are equal to or less than a predetermined confirmation probability, the integration means displays moving object images captured by the target area camera and allows an operator to select one. In the moving object tracking program according to claim 3 ,
A moving object tracking program characterized in that the moving object ID acquisition means acquires the moving object ID from a medium on which a moving object ID carried by a person who is a moving object is recorded by short-range communication or by an input operation by the person. In the moving object tracking program according to any one of claims 2 to 8,
A moving object tracking program characterized in that the position processing means refers to a no-entry area recorded for each moving object based on the position of the moving object acquired by the image position acquisition means, and is equipped with a warning means for outputting a warning if the moving object enters the no-entry area. 10. The moving object tracking program according to claim 9,
A moving object tracking program characterized in that the no-entry zone changes dynamically with the movement of a dangerous moving object or with the date and time. In any one of claims 2 to 10, the moving object tracking program further comprises:
A moving object tracking program characterized in that the recognition means includes a learning means that learns the recognition means based on an image captured of only one moving object wearing an attachment in a state in which the moving object can be identified. In any one of claims 2 to 11, the moving object tracking program further comprises:
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The recognition means recognizes which digit an individual code is, based on the display form of the identification code or the display form of the background, thereby recognizing the identification code. In the moving object tracking program according to any one of claims 2 to 12,
The identification code has a multi-digit individual code,
The recognition means recognizes an identification code based on a recognition result of an individual code of each digit and a recognition result of the entire set of digits. The moving object tracking program according to claim 12 or 13,
A moving object tracking program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other. one or more target area cameras for imaging a target area;
An entrance camera that captures an image of a moving object that is about to enter a target area;
an entrance recognition means for recognizing a unique identification code displayed on a moving object that is about to enter a target area based on an image captured by the entrance camera;
a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entrance camera ;
a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other;
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by the target area camera;
an identification means for identifying a moving object captured by the target area camera based on an identification code recognized by the recognition means and a correspondence between the identification code and a moving object ID recorded in a recording unit;
A moving object identification device comprising:
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to nth estimation means.
The integration means performs a learning process to generate an n+1th estimation means that calculates the estimated probability of being the n+1st identification code by distinguishing between the n+1st identification code and other identification codes and determining whether the highest estimation probability among the first identification code to the nth identification code is equal to or lower than a predetermined exclusion probability, or by suggesting the learning process. A moving object identification program for implementing a moving object identification device by a computer, the moving object identification program comprising:
an entrance recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by an entrance camera that captures an image of the moving object attempting to enter the target area;
a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entrance camera ;
a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other;
A recognition means for recognizing a unique identification code displayed on a moving object moving in a target area based on an image captured by one or more target area cameras that capture an image of the target area;
A moving object identification program for causing the device to function as an identification means for identifying a moving object captured by the target area camera based on an identification code recognized by the recognition means and a correspondence between the identification code and a moving object ID recorded in a recording unit,
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to n-th estimation means,
The moving object identification program is characterized in that, if the highest estimation probability among the first identification code to the nth identification code is equal to or lower than a predetermined exclusion probability, the integration means determines that identification code to be the n+1th identification code, performs learning based on a captured image, distinguishes between the n+1th identification code and other identification codes, and performs a learning process to generate an n+1th estimation means that calculates the estimation probability that the identification code is the n+1th identification code, or suggests the learning process. The moving object identification program according to claim 16,
A plurality of identification codes are displayed on the moving object,
The moving object identification program is characterized in that the recognition means recognizes the identification code using a machine learning trained model that has been trained based on images of the identification code displayed on the moving object taken from different angles. 18. The moving object identification program according to claim 16,
The moving object identification program is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the identification code with the highest estimated probability is determined as the recognition result. In any one of claims 16 to 18, the moving object identification program further comprises:
The moving object identification program is characterized in that, when there are multiple estimated probabilities among the first identification code to the nth identification code that exceed a predetermined exclusion probability and are equal to or less than a predetermined confirmation probability, the integration means displays moving object images captured by the target area camera and allows an operator to select one. In the moving object identification program according to any one of claims 16 to 19,
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The moving object identification program, characterized in that the recognition means recognizes which digit an individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code. In the moving object identification program according to any one of claims 16 to 20,
The identification code has a multi-digit individual code,
The moving object identification program, wherein the recognition means recognizes an identification code based on a recognition result of an individual code of each digit and a recognition result of the entire set of digits. 22. The moving object identification program according to claim 20,
A moving object identification program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other. A recognition device that recognizes an object based on an image captured by a camera,
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
and an integration means for, if the highest estimation probability among the first object to the nth object estimated by the first to nth estimation means is equal to or higher than a predetermined final probability, determining the identification code with the highest estimation probability as a recognition result;
the integration means, if the highest estimation probability among the first identification code to the nth identification code is equal to or less than a predetermined exclusion probability, determines that identification code to be the n+1th identification code, performs learning based on the captured image, distinguishes the n+1th identification code from other identification codes, and performs a learning process to generate an n+1th estimation means for calculating the estimation probability that the identification code is the n+1th identification code, or suggests the learning process;
A recognition device comprising: A recognition program for implementing a recognition device that recognizes an object based on an image captured by a camera, the program comprising:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
A recognition program for causing an identification code having the highest estimation probability to function as an integration means for determining an identification code having the highest estimation probability as a recognition result if the highest estimation probability of a first object to an n-th object estimated by the first to n-th estimation means is equal to or higher than a predetermined final probability,
the integration means, if the highest estimation probability among the first identification code to the nth identification code is equal to or less than a predetermined exclusion probability, determines that identification code to be the n+1th identification code, performs learning based on the captured image, distinguishes the n+1th identification code from other identification codes, and performs a learning process to generate an n+1th estimation means for calculating the estimation probability that the identification code is the n+1th identification code, or suggests the learning process;
A recognition program featuring: 25. The recognition program of claim 24,
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The recognition program is characterized in that the recognition device recognizes which digit an individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code. 26. The moving object identification program according to claim 24,
The identification code has a multi-digit individual code,
The moving object identification program is characterized in that the recognition device recognizes an identification code based on the recognition results of individual codes of each digit and the recognition results of all the digits. 27. The moving object identification program according to claim 25,
A moving object identification program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other. In any one of claims 1 to 27, the moving object identification program further comprises:
a moving object identification program characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same when the first to nth identification codes to be estimated are target identification codes and the other identification codes are non-target identification codes. one or more target area cameras for imaging a target area;
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by the target area camera;
an image position acquisition means for acquiring a position of the moving object within a target area based on the captured image;
a position processing means for determining the identity of the moving object based on the recognized identification code and for recording a change in the position of the moving object over time in a recording unit;
A moving object tracking device comprising:
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to n-th estimation means,
A moving object tracking device characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items used for learning for the target identification codes and the non-target identification codes are substantially the same when the first to nth identification codes to be estimated are target identification codes and the other identification codes are non-target identification codes. A moving object tracking program for implementing a moving object tracking device by a computer, the moving object tracking program comprising:
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by a target area camera;
an image position acquisition means for acquiring a position of the moving object within a target area based on the captured image;
A moving object tracking program for causing the program to function as a position processing means for determining the identity of a moving object based on the recognized identification code and recording a time-dependent change in the position of the moving object in a recording unit,
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to n-th estimation means,
a moving object tracking program characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same when the first to nth identification codes to be estimated are target identification codes and the other identification codes are non-target identification codes. The moving object tracking program according to claim 30,
The moving object tracking program causes the computer to further function as an entry recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by an entry camera , a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entry camera , and a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other,
A moving object tracking program characterized in that the position processing means obtains a moving object ID of the moving object by referring to the recording unit based on an identification code recognized from an image of the moving object captured by the target area camera. In the moving object tracking program according to claim 30 or 31,
A plurality of identification codes are displayed on the moving object,
A moving object tracking program characterized in that the recognition means recognizes the identification code displayed on the moving object using a machine learning trained model that has been trained based on images captured from different angles of the identification code displayed on the moving object. In the moving object tracking program according to any one of claims 30 to 32,
A moving object tracking program, wherein the identification code is displayed on an attachment attached to the moving object. In the moving object tracking program according to any one of claims 30 to 33,
The moving object tracking program is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the identification code with the highest estimated probability is determined as the recognition result. In the moving object tracking program according to any one of claims 30 to 34,
The moving object tracking program is characterized in that, when there are multiple estimated probabilities among the first identification code to the nth identification code that exceed a predetermined exclusion probability and are equal to or less than a predetermined confirmation probability, the integration means displays moving object images captured by the target area camera and allows an operator to select one. The moving object tracking program according to claim 31 ,
A moving object tracking program characterized in that the moving object ID acquisition means acquires the moving object ID from a medium on which a moving object ID carried by a person who is a moving object is recorded by short-range communication or by an input operation by the person. In the moving object tracking program according to any one of claims 30 to 36,
A moving object tracking program characterized in that the position processing means refers to a no-entry area recorded for each moving object based on the position of the moving object acquired by the image position acquisition means, and is equipped with a warning means for outputting a warning if the moving object enters the no-entry area. The moving object tracking program according to claim 37,
A moving object tracking program characterized in that the no-entry zone changes dynamically with the movement of a dangerous moving object or with the date and time. In the moving object tracking program according to any one of claims 30 to 38,
A moving object tracking program characterized in that the recognition means includes a learning means that learns the recognition means based on an image captured of only one moving object wearing an attachment in a state in which the moving object can be identified. In the moving object tracking program according to any one of claims 30 to 39,
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The recognition means recognizes which digit an individual code is, based on the display form of the identification code or the display form of the background, thereby recognizing the identification code. In the moving object tracking program according to any one of claims 30 to 40,
The identification code has a multi-digit individual code,
The recognition means recognizes an identification code based on a recognition result of an individual code of each digit and a recognition result of the entire set of digits. In the moving object tracking program according to claim 40 or 41,
A moving object tracking program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other. one or more target area cameras for imaging a target area;
An entrance camera that captures an image of a moving object that is about to enter a target area;
an entrance recognition means for recognizing a unique identification code displayed on a moving object that is about to enter a target area based on an image captured by the entrance camera;
a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entrance camera ;
a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other;
A recognition means for recognizing a unique identification code displayed on a moving object moving within a target area based on an image captured by the target area camera;
an identification means for identifying a moving object captured by the target area camera based on an identification code recognized by the recognition means and a correspondence between the identification code and a moving object ID recorded in a recording unit;
A moving object identification device comprising:
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to nth estimation means.
A moving object identification device characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same when the first to nth identification codes to be estimated are target identification codes and the other identification codes are non-target identification codes. A moving object identification program for implementing a moving object identification device by a computer, the moving object identification program comprising:
an entrance recognition means for recognizing a unique identification code displayed on a moving object attempting to enter a target area based on an image captured by an entrance camera that captures an image of the moving object attempting to enter the target area;
a moving object ID acquisition means for acquiring a moving object ID of the moving object when the moving object is imaged by the entrance camera ;
a correspondence recording means for recording the acquired moving object ID and the recognized identification code in a recording unit in association with each other;
A recognition means for recognizing a unique identification code displayed on a moving object moving in a target area based on an image captured by one or more target area cameras that capture images of the target area;
A moving object identification program for causing the device to function as an identification means for identifying a moving object captured by the target area camera based on an identification code recognized by the recognition means and a correspondence between the identification code and a moving object ID recorded in a recording unit,
The recognition means includes:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
and an integration means for recognizing an identification code based on the estimation results of the first to n-th estimation means,
a moving object identification program characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items for the target identification codes and the non-target identification codes used for learning are substantially the same when the first to nth identification codes to be estimated are target identification codes and the other identification codes are non-target identification codes. The moving object identification program according to claim 44,
A plurality of identification codes are displayed on the moving object,
The moving object identification program is characterized in that the recognition means recognizes the identification code using a machine learning trained model that has been trained based on images of the identification code displayed on the moving object taken from different angles. 46. The moving object identification program according to claim 44 or 45,
The moving object identification program is characterized in that, if the highest estimated probability among the first identification code to the nth identification code is equal to or higher than a predetermined confirmed probability, the identification code with the highest estimated probability is determined as the recognition result. In any one of claims 44 to 46, the moving object identification program further comprises:
The moving object identification program is characterized in that, when there are multiple estimated probabilities among the first identification code to the nth identification code that exceed a predetermined exclusion probability and are equal to or less than a predetermined confirmation probability, the integration means displays moving object images captured by the target area camera and allows an operator to select one. In any one of claims 44 to 47, the moving object identification program further comprises:
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The moving object identification program, characterized in that the recognition means recognizes which digit an individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code. In any one of claims 44 to 48, the moving object identification program further comprises:
The identification code has a multi-digit individual code,
The moving object identification program, wherein the recognition means recognizes an identification code based on a recognition result of an individual code of each digit and a recognition result of the entire set of digits. 50. The moving object identification program according to claim 48 or 49,
A moving object identification program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other. A recognition device that recognizes an object based on an image captured by a camera,
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the identification code is the n-th identification code;
and an integration means for, if the highest estimation probability among the first object to the nth object estimated by the first to nth estimation means is equal to or higher than a predetermined final probability, determining the identification code with the highest estimation probability as a recognition result,
The recognition device is characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items of the target identification codes and the non-target identification codes used for learning are substantially the same, when the first to nth identification codes to be estimated are set as target identification codes and the other identification codes are set as non-target identification codes. A recognition program for implementing a recognition device that recognizes an object based on an image captured by a camera, the program comprising:
a first estimation means for distinguishing between the first identification code and other identification codes and calculating an estimated probability that the identification code is the first identification code;
a second estimation means for distinguishing the second identification code from other identification codes and calculating an estimated probability that the code is the second identification code;
...
an n-th estimation means for distinguishing the n-th identification code from other identification codes and calculating an estimated probability that the code is the n-th identification code;
A recognition program for causing an identification code having the highest estimation probability to function as an integration means for determining an identification code having the highest estimation probability as a recognition result if the highest estimation probability of a first object to an n-th object estimated by the first to n-th estimation means is equal to or higher than a predetermined final probability,
a recognition program characterized in that each of the first to nth estimation means is trained with learning data in which the number of data items of the target identification codes and the non-target identification codes used for learning are substantially the same, when the first to nth identification codes to be estimated are set as target identification codes and the other identification codes are set as non-target identification codes. 53. The recognition program of claim 52,
the identification code has an individual code of multiple digits, and is configured such that a display form of the individual code or a display form of a background of the individual code is changed according to the digits;
The recognition program is characterized in that the recognition device recognizes which digit an individual code is based on the display form of the identification code or the display form of the background, and recognizes the identification code. In the moving object identification program according to claim 52 or 53,
The identification code has a multi-digit individual code,
The moving object identification program is characterized in that the recognition device recognizes an identification code based on the recognition results of individual codes of each digit and the recognition results of all the digits. 55. The moving object identification program according to claim 53 or 54,
A moving object identification program, characterized in that the individual codes for each digit of the identification code are low in similarity to each other.