JP6953292B2 - Object identification device and method - Google Patents

Description

The present invention relates to an object identification device and a method for rapidly and robustly recognizing and identifying an object from images taken by a plurality of cameras and displaying the result.

Conventionally, a technique has been proposed in which some object represented by a person is extracted and identified from an image taken by a single camera. For this identification, for example, there is a means of recognizing the athlete's uniform number and face in the case of a person, and the license plate number in the case of a car, and identifying and identifying an object based on the number. This technology recognizes an image obtained by cutting out an object by some means.

As a means of recognition, a technique using deep learning is disclosed in Non-Patent Document 1. For example, when targeting a player in a sports image, this recognition result can be used for identifying an object when tracking the object, or can be applied to tactical analysis or the like.

However, it is rare that the object to be recognized is always visible in the image. For example, in the case of a uniform number, it is difficult to always correctly recognize the uniform number due to problems such as the angle at which the athlete stands with respect to the camera and the overlap between the athletes, and in the case of a car license plate, the license plate is used. The viewing angle is limited. Therefore, in order to recognize an arbitrary frame in a moving image with high accuracy, it is conceivable to add a process of tracking the recognized result as well as the recognition result for each frame, or to use a plurality of cameras. Therefore, there has been an existing technique for acquiring the relative directional relationship between images taken by a plurality of cameras and identifying a person based on the characteristics obtained from the directional relationship.

Japanese Unexamined Patent Publication No. 2016-001447

Sebastian Gerke; Karsten Muller; Ralf Schafer, "Soccer Jersey Number Recognition Using Convolutional Neural Networks," The IEEE International Conference on Computer Vision (ICCV) Workshops, pp. 17-24, 2015.

Patent Document 1 is a technique for improving the accuracy of person identification by using the orientation of a face obtained from the relative orientation relationship of a plurality of cameras, but it is necessary to clearly extract a candidate region image. It is also stated in Non-Patent Document 1 that it is technically difficult to cut out only the face and the uniform number completely automatically, and if the cutting fails, it causes erroneous recognition. It was difficult to apply.

Regarding the above-mentioned problem that it is difficult to obtain high accuracy, as an approach to improve recognition accuracy, it is said to apply a method such as deep learning that can obtain a certain recognition accuracy even if the image cropping method is not uniform. There is a means.

Non-Patent Document 1 recognizes the obtained image of the uniform number using a deep learning model. However, it only shows the recognition accuracy for the image in which the uniform number can be seen, and the image in which the uniform number cannot be seen is manually removed.

However, it is reported that the recognition accuracy is about 83% even if such manual processing is added. Non-Patent Document 1 does not assume that a plurality of cameras are used, and when recognition is performed at a stadium or the like using a plurality of cameras, there are many cameras whose uniform numbers cannot be seen for a certain person. It is conceivable that if this method is applied to such a scene, many misrecognitions may occur. Further, when recognition is performed using a method such as machine learning with high accuracy, the calculation cost of the recognition itself is large, so that a large problem also arises in terms of processing time as the number of cameras increases.

An object of the present invention is to solve the above technical problems and to provide an object identification device and a method for recognizing and identifying an object at high speed and robustly from images taken by a plurality of cameras and displaying the result. ..

In order to achieve the above object, the present invention is characterized in that the object identification device that identifies each object based on the ID attached to the object has the following configuration.

(1) Multiple cameras that shoot each object from different viewpoints for each frame, means for estimating the position of each object, means for estimating the orientation of each object, and for ID recognition based on the orientation of each object. A means for selecting a camera, a means for extracting an object image from the image of each selected camera, a means for performing ID recognition on the object image, and a means for integrating the ID recognition result of the object to identify the ID. , A means for outputting the position of the object and the ID identification result in association with each other is provided.

(2) The means for selecting the camera is to calculate the direction of the ID for each object based on its orientation and the position of the ID on the object, and for each object, the direction of the camera based on the direction of the ID. The selection range was calculated, and the cameras located within the selection range were selected.

(3) The means for selecting the cameras is such that each camera is weighted based on the past selection results.

(4) The means for identifying the ID is to identify the ID of the object based on a majority decision of a plurality of ID recognition results obtained for each object.

(5) The means for identifying the ID is such that each recognition result of the ID is weighted based on the recognition probability at the time of ID recognition and reflected in the majority vote.

(6) The means for identifying the ID includes a means for setting the reliability of the camera that captured each object image and a means for weighting each recognition result of the ID based on the reliability.

(7) The means for setting the reliability is to set a higher reliability for a camera located closer to the directivity direction of the ID.

(8) The means for setting the reliability is to set a higher reliability for a camera having a history in which the probability of correct answer of the recognition result for the ID identification result is higher.

(9) A tracking means for tracking each object based on the distance between each object between frames is further provided, and the means for estimating the orientation of the object determines the orientation of each object based on the tracking result of the object. I tried to estimate.

(10) A tracking result integration means for integrating the tracking result by the tracking means into the ID identification result is further provided.

(11) If the ID of the object extracted in the past frame cannot be identified in the frame this time, a vanishing ID allocation means for assigning the object whose identification result is the ID to the position of the object extracted in the past frame or its vicinity is further provided.

According to the present invention, the following effects are achieved.

(1) Since the object images that perform ID recognition with a high processing load are limited to the object images extracted from the cameras selected in advance based on the orientation of the object or the orientation direction of the ID, it is estimated that the ID recognition probability is low. The object image to be created can be excluded in advance. Therefore, it is possible to improve the ID identification accuracy while reducing the processing load associated with ID recognition.

(2) When selecting a camera to extract an object image for ID recognition, each camera is weighted based on the past selection results, so something is different from the direction of the object and the direction of the ID. For this reason, it will be possible to select cameras with potentially high recognition rates without omission.

(3) When integrating the results of multiple ID recognitions obtained for each object and finally identifying the ID, give priority to cameras that are closer to the directivity direction of the ID and cameras that have a high history of correct answers. Since each recognition result is weighted, the accuracy of ID identification can be improved.

(4) A means for tracking objects between frames is provided, and the tracking result between frames and the ID recognition result obtained for each frame are integrated to identify the ID, so that the accuracy of ID identification is further improved. Can be improved.

(5) Since the ID lost due to tracking and recognition failure is assigned to the lost position or its vicinity, continuous output of the ID identification result becomes possible even if occlusion occurs between objects.

It is a functional block diagram which showed the structure of the main part of the object identification apparatus which concerns on one Embodiment of this invention. It is a figure explaining the method of estimating the orientation of an object. It is a figure (the 1) which explained the selection method of a camera. It is a figure (2) explaining the selection method of a camera. It is a figure which showed the recognition result of the ID for each camera. It is a figure explaining an example of machine learning. It is a figure explaining the setting method of the camera reliability. It is a figure explaining the identification method of ID based on the recognition result of ID. It is a figure which showed the display example of the identification result. It is a functional block diagram which showed the structure of the main part of the object identification apparatus which concerns on 2nd Embodiment of this invention. It is a functional block diagram which showed the structure of the main part of the object identification apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining the allocation method of the disappearance ID.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration of a main part of an object identification device according to an embodiment of the present invention. In this embodiment, an identifier (ID) such as a license plate, a number or a bib is displayed on the front (front), rear (rear) and / or side surface, for example, in a car, motorcycle or sports athlete. The object with is the recognition target.

The positions and orientations of the plurality of cameras 10 (Ca1, Ca2 ...) Are known, and a plurality of objects are photographed from different viewpoints and fields of view, and the camera images are output in frame units.

The object position estimation unit 11 extracts an object from each camera image and estimates its position coordinates. For the position coordinates of each object, for example, a 3D model (for example, Visual Hull) is generated based on a plurality of mask images extracted for each object from each camera image, arranged in a three-dimensional space, and the position of the center of gravity is obtained. It can be the position coordinates of each object.

Alternatively, a device device such as a positioning sensor may be mounted on each object, and the position coordinates of each object may be obtained based on the position signal positioned by each device device. The position coordinates of each object may be either a two-dimensional coordinate position or a three-dimensional coordinate position including the height.

The object tracking unit 12 tracks each object based on the position coordinates of each object extracted from the camera image of the past frame and the position coordinates of each object extracted from the camera image of the current frame. That is, tracking is realized by finding out which object each object extracted from the frame this time matches the object extracted from the past frame and assigning the same ID to the matched object.

In the present embodiment, the maximum tracking distance D _T that allows tracking is set in advance, and the past positions of all the objects extracted from the previous frame with respect to the current position coordinates of all the objects extracted from the frame this time. The distance to the coordinates is calculated, and the object pair that gives the smallest distance below _{the maximum tracking distance D T is estimated to be the same object.}

At this time, as the color information from each object, for example, if the object is a soccer player, uniform or hair color information is acquired, and objects having significantly different color information are not determined to be the same object even if the distance is small. May be added to improve the accuracy.

As another tracking method, a method of tracking by applying a time series filter such as a Kalman filter or a particle filter can also be applied. In the tracking process of a person object, it is necessary to record information such as a position extracted from a past frame and an ID associated with the information. This is a database for object identification by the result recording unit 20 described later. This can be achieved by recording to 21.

The object orientation estimation unit 13 estimates the orientation of an object based on, for example, the tracking result by the object tracking unit 12. In the present embodiment, as shown in FIG. 2, a movement vector is obtained for each object based on the position coordinate P0 in the previous frame and the position coordinate P1 in the current frame, and the movement direction is set to the front (front part). It can be estimated that the opposite direction is the back surface (rear part) and the direction orthogonal to the moving direction is the side surface.

Alternatively, an acceleration sensor may be mounted on each object, the output thereof may be applied to a known posture estimation algorithm to estimate the posture of each object, and the orientation of the object may be estimated based on the estimation result.

The feature position acquisition unit 14 acquires information on the feature position of the object that should be prioritized when performing ID recognition from the object identification database 21. If the object is a soccer player and its ID is a uniform number, information that the side opposite to the direction of the object (back side) is the feature position is acquired. Similarly, if the object is a car and its ID is a license plate, the information that the front side and the rear side of the object are the feature positions is acquired.

The camera selection unit 15 includes an ID direction direction estimation unit 151 and a range setting unit 152, and executes ID recognition for each object based on the estimation result of the orientation of each object, the feature position, and the position coordinates of each object. Select a camera.

The ID directivity direction estimation unit 151 estimates the directivity direction of the ID attached to the object. For example, if the object is a soccer player, information that the feature position is the back (number) can be acquired, so that the back direction of the object can be estimated as the direction of the ID.

The range setting unit 152 estimates that a camera located within a predetermined selection range based on the directivity direction of the ID is a group of cameras suitable for ID recognition. If the object is a soccer player, as shown in FIG. 3, the angle range of ± θa with respect to the directivity direction (backward direction) of the ID is set as the camera selection range.

Further, if the object is an automobile, it is possible to acquire information that the feature positions are the front and the rear of the vehicle body, so that not only the back direction of the object but also the front direction can be estimated as the direction of the ID. Therefore, as shown in FIG. 4, the angle range of ± θa with respect to the front direction in addition to the back direction is set as the camera selection range.

The camera selection unit 15 selects a camera located within the selection range as a camera for ID recognition related to the object. The camera identifiers selected for each object are stored in the database 21.

It is not necessary for the camera selection unit 15 to select all the cameras located within a predetermined angle range with respect to the directivity direction of the ID, and each camera is weighted and selected based on the angle and the past selection results. You may try to do. For example, the weight value may be increased as the camera is closer to the directivity direction of the ID, while the weight value may be increased as the camera has a large number of achievements selected in the past, and the top N best camera having a large sum of weight values may be selected. ..

In addition, the above camera selection does not have to be performed for all frames. When the total number of cameras is n, all n cameras are selected once every several to several tens of frames, and m (<n) otherwise. You may choose only one camera. The number of selected cameras is not limited to a fixed value, and all cameras satisfying a predetermined condition may be selected.

The object image extraction unit 16 extracts an object image from the camera image of the camera selected for each object. The object image may be output in a predetermined image format, or may be extracted as a mere luminance value, a depth image other than a color image, or the like.

The object image extraction unit 16 further collates the extraction position of each object image with the position estimated by the object position estimation unit 11 to manage the image and the position of the same object in association with each other.

The ID acquisition unit 17 acquires the object ID by executing ID recognition on the object image extracted from the selected camera image.

FIG. 5 is a diagram showing the relationship between the position of the cameras and the recognition result when ID recognition is executed by the eight cameras Ca1 to Ca8 for the object with the uniform number "24" as the ID. Yes, the correct recognition results are obtained with the cameras Ca6 and Ca7 located in the direction of the ID, but the incorrect recognition results are obtained with the cameras Ca3 and Ca8 that are not in the direction of the ID. Note that "recognition failure" indicates a case where a recognition result cannot be obtained because the recognition probability is extremely low. According to this embodiment, it can be seen that high recognition results can be obtained with the cameras Ca6 and Ca7 located in the directivity direction of the ID. The recognition result is accumulated in the object identification database 21 together with the recognition probability.

Machine learning can be used for ID recognition in the ID acquisition unit 17. For example, if machine learning is adopted when recognizing a uniform number as an ID, when an object image showing the uniform number is input, the predicted recognition result (estimation result of what the uniform number is) is acquired. Since it is necessary to create a model that can be used, a model for number recognition is first generated using the training image.

In such model creation, for example, a large number of training images are prepared, and a model for uniform number recognition is created using a convolutional neural network. Regarding the generation of the learning image, a large number of images showing the uniform number may be prepared and the correct answer label may be manually assigned, but as shown in FIG. 6, the ID is represented in an arbitrary background image. The learning image may be artificially generated by superimposing fonts and the like.

According to the latter method, since the learning image with the correct answer label can be automatically generated, it is not necessary to manually assign the correct answer label, which is efficient. Also, since various learning images can be generated by rotating, distorting, and adjusting the size of the font from the beginning, the ID is slightly diagonal on the object image, or it is not cut out neatly. However, highly accurate recognition becomes possible.

In addition, as a model generation method, a convolutional neural network is taken up in the example, but any learning method may be used as long as ID recognition is possible. Then, using the generated model for ID recognition, the ID is estimated and output by recognizing the object image extracted from the camera image. At this time, the probability of the recognition result (recognition probability) may be output in addition to the ID recognition result.

In the case of machine learning, the recognition probability can be calculated by using the softmax function as the activation function of the output layer of the ID recognition model. Further, when the results from all the cameras are determined to be "recognition failure", the ID itself here can be regarded as "recognition failure" and the result can be passed to the subsequent stage.

The recognition result integration unit 18 includes a recognition success / failure determination unit 181 and a camera reliability setting unit 182, and identifies IDs by integrating the results of ID recognition executed on a plurality of camera images selected for each object. do.

When the recognition success / failure determination unit 181 obtains a result in which the recognition probability falls below a predetermined reference value or a recognition result that cannot be actually expected, the recognition result with low accuracy is ID by classifying these as "recognition failure". Prevent it from being reflected in the identification.

For example, in a case where a large number of erroneous recognition results are obtained, it cannot be denied that the erroneous recognition results may be regarded as identification results even if the individual recognition probabilities are low. Also, in the case of competitions such as baseball and soccer, a list of appearing IDs (numbers and bibs) can be obtained in advance, so recognition results that are not on the list can be classified as "recognition failure". Can be reduced as an identification result.

The camera reliability setting unit 182 sets the reliability for each camera selected for each object based on the angle difference between the ID directivity direction and the camera direction.

FIG. 7 is a diagram showing an example of a method of setting the camera reliability. It is obtained how much the direction of the ID and the direction of the camera are angularly separated, and the smaller the angle difference, the higher the reliability. Is set high.

In the illustrated example, the angle difference θ7 between the direction of the ID and the direction of the camera Ca7 is compared with the angle difference θ6 between the direction of the ID and the direction of the camera Ca6. Here, θ7 <θ6, so the reliability of the camera Ca7 Set the degree higher than the reliability of the camera Ca6. If θ7 <θ6, then cosθ7> cosθ6. Therefore, in this embodiment, the cos value of the angle between the directivity direction of the ID and the direction of the camera is adopted as the camera reliability.

Further, a higher reliability may be set for a camera having a history in which the probability of correct answer of the recognition result for the ID identification result is higher.

The recognition result integration unit 18 identifies the maximum likelihood ID for each recognition result based on the determination of success or failure of recognition, the recognition probability in ID recognition, and the camera reliability.

FIG. 8 is a diagram schematically showing an ID identification method by the recognition result integration unit 18. Here, attention is paid to one object, and an example is obtained in which eight cameras Cc1 to Cc8 are selected for the object. I will explain.

In the present embodiment, the cameras Ca1, Ca2, and Ca4 are excluded from the integration target because they are determined to be recognition failures by the recognition success / failure determination unit 181. For other cameras, the product of the recognition probability and the camera reliability is used as the evaluation value. In the present embodiment, the camera Ca3 has a camera reliability of "0", so the evaluation value is "0". The camera Ca5 has a recognition probability of "0.8" and a camera reliability of "0.4", so its evaluation value is "0.32".

Similarly, the cameras Ca6, Ca7, and Ca8 have recognition probabilities of "0.8", "1.0", and "0.8", respectively, and camera reliability of "0.7", "0.9", and "0.6", respectively. The values are "0.56", "0.9", and "0.48".

The recognition result integration unit 18 further obtains the sum of the evaluation values for each recognition result, and adopts the recognition result that gives the maximum value as the result of the ID identification. In the present embodiment, the total evaluation value of the recognition result "26" is 0, the total evaluation value of the recognition result "24" is 1.78, and the total evaluation value of the recognition result "28" is 0.48. Therefore, the recognition result "24" Is the ID identification result.

The result display unit 19 displays the identification result of the position coordinates and the ID of each object extracted from the frame this time. There are various ways to display the result, and it is sufficient to display the position coordinates and ID as numerical values on the console. However, as an example of graphically expressing the result, as shown in FIG. 9, the position of the object. There is a plane map that superimposes the ID on.

In FIG. 9, a round colored (round line type in the figure) markers based on the position coordinates of each player are arranged on a background imitating one side of a soccer court. The number displayed on top of the marker is the ID indicating the uniform number.

It is also possible to output such a plane map for each frame of the camera image and move it in a moving manner. Further, at the time of this display, for example, by acquiring the uniform color information from the image, the team to which the player belongs may be determined, and the color of the marker may be changed and reflected in the plane map as a result. At this time, regarding the color of the uniform, the person who is judged to be the color of the referee may be judged not to be a player and excluded from the display of the result.

The result recording unit 20 aims to use the result recording unit 20 for estimating the ID in the next frame and thereafter by recording the position coordinates of the object extracted from the frame this time, the final ID identification result, and the like in the database 21.

In the present embodiment, the position coordinates of the object obtained in the current frame and the identification result of the ID corresponding to the object are saved. This is indispensable information when performing the tracking process in the next frame.

Further, when introducing the reliability in order to determine the optimum ID by using the reliability in the "tracking result integration unit 22" and the "disappearing ID allocation unit 23" described later, the reliability is linked to the ID. Saved in the object identification database 21 and added a function to take over the reliability of the current frame as it is (or to take over by discounting the reliability according to the passage of time) when tracking or disappearance ID assignment is successful after the next frame. You may.

According to the present embodiment, the object image that executes ID recognition with a high processing load is limited to the object image extracted from the camera selected in advance based on the orientation of the object or the orientation direction of the ID. Therefore, the ID recognition probability. Object images that are presumed to have a low value can be excluded in advance. Therefore, it is possible to improve the ID identification accuracy while reducing the processing load associated with ID recognition.

Further, according to the present embodiment, when selecting a camera for extracting an object image for ID recognition, each camera is weighted based on the past selection results, so that the orientation of the object and the orientation of the ID Apart from the direction, it will be possible to select a camera with a potentially high recognition rate for some reason without omission.

Further, according to the present embodiment, when a plurality of ID recognition results obtained for each object are integrated to finally identify the ID, a camera closer to the direction in which the ID is directed and a history of correct answers are high. Since the priority of the camera is raised and each recognition result is weighted, the accuracy of ID identification can be improved.

FIG. 10 is a functional block diagram showing the configuration of the object identification device according to the second embodiment of the present invention, and the same reference numerals as those described above represent the same or equivalent parts. This embodiment is characterized in that a tracking result integration unit 22 is further provided after the recognition result integration unit 18.

The tracking result integration unit 22 further integrates the integration result by the recognition result integration unit 18 and the tracking result by the object tracking unit 12, and reflects the tracking result in the integration result to further improve the accuracy of ID identification. ..

In the present embodiment, the case where the ID tracking result and the identification result match is recognized as the tracking success and the number of times is stored. After that, when the tracking result and the identification result do not match, the tracking is successful. Priority is given to either the tracking result or the identification result according to the number of times or the ratio.

For example, for an object whose tracking result and identification result match at "21" over multiple frames, if only the frame happens to be recognized as "27" this time, if the number of successful tracking is greater than or equal to a predetermined threshold. On the other hand, if the number of successful tracking is less than a predetermined threshold value, the identification result can be preferentially identified as "27" regardless of the tracking result.

By implementing such an implementation, it is possible to improve the accuracy. The ID output here may be a form in which one ID is output for the position information, or an implementation in which a plurality of IDs are output in association with the reliability.

According to the present embodiment, since the ID is identified by integrating the tracking result between frames and the ID recognition result obtained for each frame, the accuracy of ID identification can be further improved.

FIG. 11 is a functional block diagram showing the configuration of the object identification device according to the third embodiment of the present invention, and the same reference numerals as those described above represent the same or equivalent parts. The present embodiment is characterized in that the disappearance ID allocation unit 23 is further provided in the subsequent stage of the tracking result integration unit 22 (or the subsequent stage of the recognition result integration unit 18 in the first embodiment).

When the disappeared ID allocation unit 23 can no longer recognize the ID that could be recognized up to the previous frame in the previous frame, in this frame, the object could be extracted but the ID could not be identified at the end. An object located near the ID recognition position in the frame before the previous time is selected, and the last recognized ID is assigned to the object regardless of the recognition result this time.

FIG. 12 is a diagram showing the function of the disappearance ID allocation unit 23. Here, since the ID “10” that could be identified in the previous frame could not be identified in the previous frame, the ID “10” could not be identified in the previous frame. An example is shown in which an object whose identification result is the lost ID "10" is assigned regardless of the recognition result.

In a scene where objects approach or overlap from the camera's point of view, the two objects are judged to be one, or they overlap and become invisible, so the object that existed in the previous frame is now in the frame. It may be treated as if it disappeared on the camera image of.

In such a case, if the density and overlap of the objects are resolved with the passage of time, the objects will be recognized in the vicinity again, but since the objects appear to appear suddenly, it is necessary to redo the recognition and tracking.

Normally, when such a situation occurs, the position where the object reappears is often in the vicinity of the disappearance position (the position of the last estimated object: FIG. 12 (a)). Therefore, in the present embodiment may be recorded in advance disappearance position of the object, location and estimated in the past frame was estimated in the current frame object, to calculate the distance between the recorded erasure position, the maximum loss distance D _V The lost ID is assigned to the object at the closest position among the objects below.

Maximum loss distance D _V is, the user may set in advance as the unchanged parameters may be changed depending on the situation. When changing according to the situation, the number of other objects existing in a certain area is counted around the target object, and it is calculated as the density. Then, it is conceivable to dynamically change the maximum loss distance D _V for each object based on the density.

Generally, the more if the situation in which the object is dense, because the possibility of erroneous ID is allocated by the disappearance ID allocation is increased, avoiding the allocation of wrong ID by reducing the maximum loss distance D _V It becomes possible to do.

When the vanishing ID allocation unit 23 is provided, the result recording unit 20 compares the ID identified in the current frame with the ID identified in the previous frame between the previous frame and the current frame. Find the lost ID and record its position (lost position). This is recorded because the disappearance position is required in the disappearance ID allocation after the next frame. If an ID that was supposed to be lost appears in the current frame, the lost position information related to that ID is deleted.

In addition, when recording the disappearance ID, if the team to which the team belongs can be analyzed from information such as the color of the uniform, such additional information is recorded in association with the disappearance position, and the disappearance ID assignment unit 23 It may be helpful when making an assignment at.

In addition, when displaying the ID identification result using this device, recognition and tracking, and further disappearance ID assignment are not performed, and there may be a case where a significant ID does not exist. In that case, "no recognition result" may be set, or some random ID may be assigned.

According to the present embodiment, the ID lost due to the failure of tracking and recognition can be assigned to the lost position or its vicinity, so that the continuous output of the ID identification result can be obtained even if occlusion occurs between the objects. It will be possible.

10 ... camera, 11 ... object position estimation unit, 12 ... object tracking unit, 13 ... object orientation estimation unit, 14 ... feature position acquisition unit, 15 ... camera selection unit, 16 ... object image extraction unit, 17 ... ID acquisition unit, 18 ... Recognition result integration unit, 19 ... Result display unit, 20 ... Result recording unit, 21 ... Object identification database, 22 ... Tracking result integration unit, 23 ... Loss ID allocation unit, 151 ... ID direction direction estimation unit, 152 ... Range setting unit, 181 ... Recognition success / failure judgment unit, 182 ... Camera reliability setting unit

Claims (12)

In an object identification device that identifies each object based on the ID attached to the object
With multiple cameras that shoot objects from different perspectives,
A means of estimating the position of each object,
A means of estimating the orientation of each object,
A means of selecting a camera for ID recognition based on the orientation of each object,
A means for extracting an object image from the images of the selected cameras, and
A means to perform ID recognition for each object image,
A means of integrating the ID recognition results of objects to identify IDs,
It is provided with a means for outputting the position of the object and the identification result of the ID in association with each other .
The means for selecting a camera is an object identification device characterized in that each camera is weighted and selected based on a past selection record. The object identification device according to claim 1, wherein the means for estimating the orientation of the object is to estimate the orientation of the object based on a change in the position of each object between frames. It also has a means of retrieving information about the location of the ID on the object.
The means for selecting the camera is
For each object, a means of calculating the direction of the ID based on its orientation and the position of the ID,
Each object is provided with a means for obtaining a camera selection range based on the directivity direction of the ID.
The object identification device according to claim 1 or 2, wherein a camera located within the selection range is selected. The object identification device according to claim 3 , wherein the means for identifying the ID is to identify the ID of the object based on a majority decision of a plurality of ID recognition results obtained for each object. The object identification device according to claim 4 , wherein the means for identifying the ID is such that each recognition result of the ID is weighted based on the recognition probability at the time of ID recognition and reflected in the majority vote. The means for identifying the ID is
A means to set the reliability of the camera that took each object image,
The object identification device according to claim 4 or 5 , further comprising a means for weighting each recognition result of an ID based on the reliability. The object identification device according to claim 6 , wherein the means for setting the reliability is to set a higher reliability for a camera located closer to the directivity direction of the ID. The object identification device according to claim 6 , wherein the means for setting the reliability is to set a high reliability by a camera having a history in which the probability of correct answer of the recognition result with respect to the ID identification result is higher. Further equipped with a tracking means for tracking each object based on the distance between each object between frames,
The object identification device according to any one of claims 1 to 8 , wherein the means for estimating the orientation of the object estimates the orientation of each object based on the tracking result of the object. The object identification device according to claim 9 , further comprising a tracking result integrating means for integrating the tracking result by the tracking means into the identification result of the ID. If the ID of the object extracted in the past frame cannot be identified in the frame this time, the vanishing ID allocation means for assigning the object whose identification result is the ID to the position of the object extracted in the past frame or its vicinity is further provided. The object identification device according to claim 9 or 10. In the object identification method that identifies each object based on the ID attached to the object,
Shoot objects with multiple cameras from different viewpoints,
Estimate the position of each object
Estimate the orientation of each object
For each object, select a camera for ID recognition based on its orientation,
An object image is extracted from the images of each of the selected cameras.
ID recognition is executed on the object image,
Identifies the ID by integrating the ID recognition results of the object,
Outputs the object position and ID identification result in association with each other .
An object identification method characterized in that when selecting a camera, each camera is weighted and selected based on a past selection record.

Images