JP6929241B2 - Selection device, selection method and selection program - Google Patents

Description

The present invention relates to a selection device, a selection method, and a selection program capable of selecting an imaging device that generates an image used for detecting an object.

Conventionally, in generating a flow line indicating a movement trajectory of an object in a predetermined area, the object is detected based on a plurality of images acquired from a plurality of imaging devices (see, for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2016-139949

When the imaging ranges of the plurality of imaging devices are installed so as to overlap, one object may be duplicated in the images captured by the plurality of imaging devices. In order to accurately detect the object, it is preferable to be able to select an imaging device that captures an image capable of accurately detecting the object from among a plurality of imaging devices.

Therefore, the present invention has been made in view of these points, and provides a selection device, a selection method, and a selection program capable of selecting an imaging device that captures an image capable of accurately detecting an object. With the goal.

The selection device according to the first aspect of the present invention is a case where a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating a predetermined area. The first region showing the area of the image showing the three-dimensional model included in the first image captured by the first imaging device that images the predetermined area, and the second imaging device that images the predetermined area. 2 Indicates an area specifying unit that specifies an area of an image showing the three-dimensional model included in the image and a second area indicating an area of the image, and the reliability of the detection when the object is detected based on the first area. A reliability value specifying unit that specifies a first reliability value and a second reliability value indicating a reliability value of the detection when the object is detected based on the second region, and the first reliability value. A selection unit for selecting an image pickup device to be used for detecting an object at a predetermined position among the first image pickup device and the second image pickup device based on the ratio with the second confidence value is provided.

The reliability value specifying unit may use the area of the first region for specifying the first reliability value and the area of the second region for specifying the second reliability value.
The first imaging device or the second imaging device is an imaging device that captures an omnidirectional image, and the reliability value specifying unit determines the position of the first region and the distance from the center of the first image. It may be used to specify the first confidence value, and the position of the second region and the distance from the center of the second image may be used to specify the second confidence value.

In the reliability value specifying unit, when the three-dimensional model and the second three-dimensional model imitating the second object that is not the target to be detected are virtually arranged, the first region is the first. The ratio hidden by the image region showing the second three-dimensional model included in one image is used for specifying the first confidence value, and the second region is the second three-dimensional included in the second image. The ratio hidden by the image region showing the model may be used to specify the second confidence value.

The selection device has information relating the position in the image of the first image and the first confidence value corresponding to the position, the position in the image of the second image, and the second corresponding to the position. A storage unit for storing information associated with the reliability value is further provided, and the reliability value identification unit refers to the storage unit and is associated with a position of an image showing the three-dimensional model included in the first image. By acquiring the first confidence value and the second confidence value associated with the position of the image showing the three-dimensional model included in the second image, the first confidence value and the first confidence value are obtained. The second confidence value may be specified.

The selection unit serves as an imaging device used for detecting the object at the predetermined position when the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is equal to or more than the first threshold value. The first imaging device is selected and used to detect the object at the predetermined position when the ratio of the second confidence value to the first confidence value corresponding to the predetermined position is equal to or greater than the second threshold value. The second imaging device may be selected as the imaging device.

In the selection unit, the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is less than the first threshold value, and the selection unit has the said to the first confidence value corresponding to the predetermined position. When the ratio of the second confidence value is less than the second threshold value, the first image pickup device and the second image pickup device may be selected as the image pickup device used for detecting the object at the predetermined position.

When the predetermined position is within the first distance from the first imaging device, the selection unit is the object at the predetermined position regardless of the ratio of the first reliability value and the second reliability value. When the first imaging device is selected as the imaging device used for the detection of the above and the predetermined position is within the second distance from the second imaging device, the first reliability value and the second reliability value are Regardless of the ratio, the second imaging device may be selected as the imaging device used for detecting the object at the predetermined position.

The reliability value specifying unit identifies the first reliability value and the second reliability value when the three-dimensional model is virtually arranged for each of a plurality of positions in the three-dimensional space, and the selection unit. May select an imaging device to be used for detecting the object based on the ratio of the first confidence value to the second confidence value for each of the plurality of positions in the three-dimensional space.

The selection device further includes a storage control unit that stores the device identification information for identifying the image pickup device and the position information indicating the position where the image pickup device is selected to be used for detecting the object in the storage unit. You may prepare.

The reliability value specifying unit includes the first reliability value and the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position for each of the plurality of shapes corresponding to the object at the predetermined position. The second confidence value is specified, and the selection unit determines the object based on the ratio of the first confidence value and the second confidence value for each of the plurality of shapes corresponding to the object at the predetermined position. The imaging device used for the detection of is may be selected.

The reliability value specifying unit is the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position for each of the one or more shapes that the object can take at the predetermined position. The second confidence value may be specified.

The selection device stores the device identification information for identifying the image pickup device, the shape of the object, and the position information indicating the position where the image pickup device is selected to be used for detecting the object in a storage unit in association with each other. A memory control unit may be further provided.

The selection method according to the second aspect of the present invention virtually creates a three-dimensional model that imitates an object to be detected in the predetermined area at a predetermined position in a three-dimensional space indicating a predetermined area, which is executed by a computer. The first area showing the area of the image showing the three-dimensional model included in the first image captured by the first imaging device that images the predetermined area and the second imaging that images the predetermined area when arranged in The step of specifying the second region indicating the region of the image showing the three-dimensional model included in the second image captured by the apparatus, and the reliability of the detection when the object is detected based on the first region. A step of specifying a first reliability value indicating the property and a second reliability value indicating the reliability value of the detection when the object is detected based on the second region, and the first reliability value. A step of selecting an image pickup device to be used for detecting an object at a predetermined position among the first image pickup device and the second image pickup device based on the ratio with the second confidence value is provided.

In the selection program according to the third aspect of the present invention, the computer virtually arranges a three-dimensional model imitating an object to be detected in the predetermined area at a predetermined position in a three-dimensional space indicating a predetermined area. In the case of A region identification unit that identifies a second region that indicates a region of an image that indicates the three-dimensional model included in the captured second image, and reliability of the detection when the object is detected based on the first region. A reliability value specifying unit that specifies a first reliability value indicating the above, and a second reliability value indicating the reliability value of the detection when the object is detected based on the second region, and the first Functions as a selection unit for selecting an image pickup device to be used for detecting an object at a predetermined position among the first image pickup device and the second image pickup device based on the ratio of the reliability value and the second confidence value. Let me.

According to the present invention, it is possible to select an imaging device that captures an image capable of accurately detecting an object.

It is a figure which shows the structure of the object tracking system which concerns on this embodiment. It is a figure which shows the outline of the selection apparatus which concerns on this embodiment. It is a figure which shows the structure of the selection apparatus which concerns on this embodiment. It is a figure which shows the image pickup apparatus selected at each of a plurality of positions in a three-dimensional space. It is a figure which shows the structure of the object tracking apparatus which concerns on this embodiment. It is a flowchart which shows the flow of processing in the selection apparatus which concerns on this embodiment. It is a flowchart which shows the flow of processing in the object tracking apparatus which concerns on this embodiment.

[Overview of Object Tracking System S]
FIG. 1 is a diagram showing a configuration of an object tracking system S according to the present embodiment. The object tracking system S includes a first imaging device 1A, a second imaging device 1B, a selection device 2, and an object tracking device 3.

The first imaging device 1A and the second imaging device 1B are imaging devices that image a predetermined area. Here, the predetermined area is, for example, the floor of a store. In the predetermined area, a person such as a clerk exists as an object for tracking the trajectory of movement in the object tracking system S. Each of the first imaging device 1A and the second imaging device 1B captures an omnidirectional image including at least a part of a predetermined area. Here, it is assumed that the range in which the first imaging device 1A can take an image and the range in which the second imaging device 1B can take an image overlap at least in part.

In the present embodiment, each of the first imaging device 1A and the second imaging device 1B is intended to capture an omnidirectional image, but the present invention is not limited to this, and an image other than the omnidirectional image may be captured. ..

The omnidirectional image captured by the first imaging device 1A and the second imaging device 1B may include the same object. In this case, the object can be tracked in any of the omnidirectional images, but depending on the size of the object image, the object may be tracked by the omnidirectional image captured by the first imaging device 1A. There may be a difference in tracking accuracy between the case where the object is tracked and the case where the object is tracked by the omnidirectional image captured by the second imaging device 1B.

Therefore, for each of the plurality of positions included in the predetermined area, the selection device 2 selects an image pickup device having high object tracking accuracy among the image pickup devices used for identifying the object existing at the position, as the first image pickup device 1A. , Select from the second imaging device 1B.

FIG. 2 is a diagram showing an outline of the selection device 2 according to the present embodiment. As shown in FIG. 2, the selection device 2 virtualizes a three-dimensional model M that imitates an object to be detected in a predetermined area at a predetermined position in a three-dimensional space A indicating a predetermined area among a plurality of positions. The first region indicating the region of the image showing the three-dimensional model M included in the first image captured by the first imaging device 1A in the case of being specifically arranged is specified. Further, the selection device 2 is a second image captured by the second image pickup device 1B when the three-dimensional model M imitating the object is virtually arranged at a predetermined position in the three-dimensional space A indicating a predetermined area. The second region showing the region of the image showing the three-dimensional model M included in is specified.

The selection device 2 has a first confidence value indicating the reliability of the detection when the object is detected based on the first region, and the reliability of the detection when the object is detected based on the second region. The second confidence value indicating the sex is specified. The selection device 2 selects an image pickup device to be used for detecting an object at a predetermined position from the first image pickup device 1A and the second image pickup device 1B based on the ratio of the first confidence value and the second confidence value. By doing so, the selection device 2 can select an imaging device that captures an image capable of accurately detecting an object at a predetermined position.

The selection device 2 selects an imaging device that performs the above-mentioned processing at each of a plurality of positions in a predetermined area and captures an image capable of accurately detecting an object. The selection device 2 arranges the device identification information for identifying the image pickup device based on the selection result, the position information indicating the position where the image pickup device is selected to be used for detecting the object, and the object at the position. When this is done, the area in charge information associated with the information indicating the area of the image indicating the object included in the image captured by the image pickup apparatus is generated.

The explanation is returned to FIG. The object tracking device 3 acquires an omnidirectional image from the first imaging device 1A and an omnidirectional image from the second imaging device 1B. The object tracking device 3 identifies an area indicating an object included in the omnidirectional image by analyzing the acquired omnidirectional image.

Here, the omnidirectional image acquired from the first imaging device 1A and the omnidirectional image acquired from the second imaging device 1B may include a region showing the same object. The object tracking device 3 selects the area of the object used for tracking the object from the area of the specified object based on the area of the specified object and the area information in charge generated by the selection device 2. Then, the object tracking device 3 tracks the object based on the position associated with the area of the selected object in the area in charge information. By doing so, the object tracking device 3 can track the object with high accuracy.

[Configuration of selection device 2]
Subsequently, the configuration of the selection device 2 will be described. FIG. 3 is a diagram showing the configuration of the selection device 2 according to the present embodiment. As shown in FIG. 3, the selection device 2 includes a storage unit 21 and a control unit 22.

The storage unit 21 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The storage unit 21 stores a program executed by the control unit 22. For example, the storage unit 21 stores a selection program that causes the selection device 2 to function as the area identification unit 221, the reliability value identification unit 222, the selection unit 223, and the storage control unit 224.

The control unit 22 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The control unit 22 functions as an area identification unit 221, a reliability value identification unit 222, a selection unit 223, and a storage control unit 224 by executing a selection program stored in the storage unit 21.

The area specifying unit 221 captures a predetermined area when a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area. 1 The first region indicating the region of the image showing the three-dimensional model included in the first image captured by the imaging device 1A is specified.

Similarly, the area specifying unit 221 is a second imaging device 1B that images a predetermined area when a three-dimensional model imitating the object is virtually arranged at a predetermined position in a three-dimensional space indicating a predetermined area. A second region showing a region of an image showing the three-dimensional model included in the captured second image is specified.

Specifically, the area identification unit 221 constructs a virtual three-dimensional space indicating a predetermined area. Then, the area specifying unit 221 generates an omnidirectional image at a position corresponding to the installation position of the first imaging device 1A in a predetermined area in a virtual three-dimensional space as in the case of the first imaging device 1A. Place the object. Further, the area specifying unit 221 generates an omnidirectional image at a position corresponding to the installation position of the second imaging device 1B in a predetermined area in a virtual three-dimensional space, similarly to the second imaging device 1B. Place the object. Here, the position in the virtual three-dimensional space is represented by, for example, the three-dimensional coordinates (X, Y, 0) of the global coordinate system.

The area specifying unit 221 causes the first camera object corresponding to the first imaging device 1A to generate an omnidirectional image when the three-dimensional model is arranged at a predetermined position in a virtual three-dimensional space indicating a predetermined area. At the same time, the second camera object corresponding to the second image pickup apparatus 1B is made to generate an omnidirectional image. Here, the shape of the three-dimensional model is preferably the shape of the object (person) itself, but it may be a shape that simply imitates the object by combining cylinders and the like.

The area specifying unit 221 analyzes the omnidirectional image generated by the first camera object, and identifies the first area indicating the area of the image showing the three-dimensional model included in the omnidirectional image. For example, the area specifying unit 221 specifies the contour of the three-dimensional model included in the omnidirectional image generated by the first camera object, and specifies the information indicating the region surrounding the contour to specify the first region. Identify.

Here, as the information indicating the region, a polygon which is a three-dimensional model of an arbitrary shape composed of a group of contour points or an circumscribing rectangle of the polygon can be used. Here, the circumscribed rectangle is a rectangle that circumscribes a polygon and has two sides parallel to a predetermined axis on an omnidirectional image. The information indicating the rectangular area includes two-dimensional coordinates (x, y) on the image indicating the center point of the rectangular area in the local coordinate system defined for the omnidirectional image, and two vertices adjacent to the vertex. It is defined by information (x, y, w, h, θ) including the distances w and h of the above and the inclination θ with respect to a predetermined axis. When θ is zero degrees, it is called an upright rectangle, and when it is other than zero degrees, it is called a rotating rectangle.

Similarly, the area specifying unit 221 analyzes the omnidirectional image generated by the second camera object and identifies the second area indicating the area of the image showing the three-dimensional model included in the omnidirectional image.

Here, the area specifying unit 221 has a first reliability value and a second reliability value when a three-dimensional model imitating the shape is arranged at a predetermined position for each of a plurality of shapes corresponding to the objects at a predetermined position. And identify. Since the object is a person, the shape of the object is, for example, the shape of a person when standing upright, the shape of a person when sitting on a chair, the shape of a person indicating an adult, and the shape of a person indicating a child. Is.

In addition, when a chair or the like is not provided at a predetermined position and only a person is supposed to stand, the first reliability value and the second reliability corresponding to the shape of the person when sitting on the chair There is less need to identify the value. Therefore, when the object can take a plurality of shapes at a predetermined position, the area specifying unit 221 arranges a three-dimensional model imitating the shape at a predetermined position for each of the plurality of shapes that the object can take. The first region and the second region in the case of the above are specified.

For example, the storage unit 21 stores shape-specific position information in which each of a plurality of shapes that an object can take is associated with a position where the object can take the shape. The area specifying unit 221 refers to the position information for each shape and specifies the shape of the object that can be taken at a predetermined position. The area specifying unit 221 specifies a first area and a second area when a three-dimensional model imitating the shape is arranged at a predetermined position for each of a plurality of shapes that the object can take at a predetermined position. ..

The area specifying unit 221 performs the processing of the above-mentioned procedure for each of a plurality of positions in the three-dimensional space indicating a predetermined area, and at each of the plurality of positions, a tertiary that imitates a shape that an object can take at that position. The first region and the second region when the original model is virtually arranged are specified.

The trust value specifying unit 222 has a first trust value indicating the reliability of the detection when the object is detected based on the first region, and a trust of the detection when the object is detected based on the second region. The second confidence value indicating the sex is specified.

The reliability value specifying unit 222 sets the first reliability value and the second reliability value when a three-dimensional model imitating the shape is arranged at a predetermined position for each of one or more shapes that the object can take at a predetermined position. Identify.

The reliability value specifying unit 222 uses the area of the first region for specifying the first reliability value and the area of the second region for specifying the second reliability value. For example, the reliability value specifying unit 222 increases the first reliability value as the area of the first region is larger, and increases the second reliability value as the area of the second region is larger.

Further, the reliability value specifying unit 222 uses the distance between the position of the first region and the center of the first image for specifying the first reliability value, and determines the distance between the position of the second region and the center of the second image. It is used to specify the second confidence value. For example, in the reliability value specifying unit 222, the longer the distance between the position of the first region and the center of the first image, the smaller the first reliability value, and the position of the second region and the center of the second image. The longer the distance, the smaller the second confidence value.

The area specifying unit 221 may virtually arrange the three-dimensional model and the second three-dimensional model that imitates the second object that is not the object to be detected in the three-dimensional space. Here, the second three-dimensional model is a three-dimensional model corresponding to static objects such as chairs, tables, and shelves arranged in a predetermined area.

In the reliability value specifying unit 222, when the three-dimensional model and the second three-dimensional model are virtually arranged in the three-dimensional space indicating a predetermined area, the first region is included in the first image. The ratio hidden by the image region showing the three-dimensional model may be used to specify the first confidence value. Similarly, the confidence value specifying unit 222 may use the ratio in which the second region is hidden by the image region showing the second three-dimensional model included in the second image to specify the second confidence value.

For example, in the reliability value specifying unit 222, the smaller the ratio of the first region hidden by the image region showing the second three-dimensional model included in the first image, the larger the first reliability value and the second region. However, the smaller the proportion hidden by the image region showing the second three-dimensional model included in the second image, the larger the second confidence value.

Further, in the storage unit 21, the first trust value information in which the position in the image of the first image and the first trust value corresponding to the position are associated with each other, the position in the image of the second image, and the position are stored. The second confidence value information associated with the second confidence value corresponding to may be stored. The reliability value specifying unit 222 may specify the position of the three-dimensional model in the first image and may also specify the position of the three-dimensional model in the second image. Then, the reliability value specifying unit 222 refers to the first reliability value information stored in the storage unit 21, and sets the first reliability value associated with the position of the image indicating the three-dimensional model included in the first image. get. Similarly, the confidence value specifying unit 222 refers to the second reliability value information stored in the storage unit 21, and the second reliability value associated with the position of the image indicating the three-dimensional model included in the second image. To identify. By doing so, the reliability value specifying unit 222 can easily specify the first reliability value and the second reliability value corresponding to the positions of the three-dimensional model virtually arranged in the three-dimensional space.

The reliability value specifying unit 222 performs the processing of the above-mentioned procedure for each of a plurality of positions in the three-dimensional space indicating a predetermined area. The plurality of positions may be defined by, for example, each grid point by dividing a predetermined area into grids at regular intervals. Then, the trust value specifying unit 222 specifies the first trust value and the second trust value of each of the plurality of shapes that the object can take, corresponding to each of the plurality of positions.

The selection unit 223 detects an object at a predetermined position among the first imaging device 1A and the second imaging device 1B based on the ratio of the first reliability value and the second reliability value specified by the reliability value specifying unit 222. Select the imaging device to be used for.

Specifically, the selection unit 223 is an imaging device used to detect an object at a predetermined position when the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is equal to or greater than the first threshold value. The first imaging device 1A is selected as.

Further, the selection unit 223 is a second imaging device used for detecting an object at a predetermined position when the ratio of the second confidence value to the first confidence value corresponding to the predetermined position is equal to or greater than the second threshold value. Select the image pickup device 1B. Here, in the present embodiment, it is assumed that the first threshold value and the second threshold value are the same threshold value (predetermined threshold value), but even if the first threshold value and the second threshold value are different values. good. By doing so, the selection device 2 can select an image pickup device corresponding to an image having a relatively high reliability value.

Further, in the selection unit 223, the ratio of the first reliability value to the second reliability value corresponding to the predetermined position is less than the first threshold value, and the second reliability value to the first reliability value corresponding to the predetermined position is set. When the ratio of is less than the second threshold value, the first image pickup device 1A and the second image pickup device 1B are selected as the image pickup devices used for detecting the object at a predetermined position. By doing so, when the difference between the reliability values of the two imaging devices is not large, the selection device 2 selects the two imaging devices and detects the object using the two imaging devices. can do.

The selection unit 223 is used to detect an object based on the ratio of the first trust value and the second trust value corresponding to each of the plurality of shapes for each of the plurality of shapes that the object can take at a predetermined position. Select an imaging device. Further, the selection unit 223 selects an imaging device to be used for detecting an object based on the ratio of the first reliability value and the second reliability value for each of the plurality of positions in the three-dimensional space.

Even if the area of the object is the same at a predetermined position, if the object exists in the vicinity of each imaging device, it is better to use an imaging device in the vicinity of the object as the imaging device used for detection. In many cases, the position of can be detected with high accuracy. Therefore, when the predetermined position is within the first distance from the first imaging device 1A, the selection unit 223 detects the object at the predetermined position regardless of the ratio of the first reliability value and the second reliability value. The first imaging device 1A may be selected as the imaging device to be used. Further, when the predetermined position is within the second distance from the second imaging device 1B, the selection unit 223 detects the object at the predetermined position regardless of the ratio of the first reliability value and the second reliability value. The second imaging device 1B may be selected as the imaging device to be used. By doing so, the selection device 2 can select an imaging device that can detect the position of the object with high accuracy.

The memory control unit 224 includes device identification information for identifying the imaging device, information indicating the shape of the object, position information indicating the position where the imaging device is selected to be used for detecting the object, and the position at the position. When an object is arranged, it is stored in the storage unit 21 as the area in charge information in association with the information indicating the area of the image indicating the object included in the image captured by the image pickup apparatus.

Here, the position selected to be used for detecting the object is represented by the position coordinates in the global coordinate system. Further, the information indicating the area of the image showing the object is the information indicating the upright rectangular area, and is the first area or the second area. Note that the information indicating the area of the image showing the object is expressed by the information indicating the upright rectangle area when matching with the upright rectangle in the object tracking process in the subsequent stage, but it can be expressed as a rotating rectangle or a polygon of an arbitrary shape. When matching is performed, it is expressed by the same information. The storage control unit 224 stores the responsible area information corresponding to each of the plurality of shapes that the object can take in the storage unit 21. The area in charge information is transmitted to the object tracking device 3 by, for example, the control unit 22. The area in charge information may be provided separately for each of the plurality of shapes, or may be provided only for each of the plurality of shapes.

FIG. 4 is a diagram showing an imaging device selected at each of a plurality of positions in a three-dimensional space. FIG. 4A is a diagram showing an imaging device selected at each of a plurality of positions when the shape of the object is a shape indicating an upright state. FIG. 4B is a diagram showing an imaging device selected at each of a plurality of positions when the shape of the object is a shape indicating a seated state.

The area A1 shown in FIG. 4 is the area in which the first image pickup device 1A is selected as the image pickup device used for detecting the object, and the area A2 is the first image pickup device 1A and the second image pickup device used as the image pickup device for detecting the object. The area where the device 1B is selected, the area A3, is the area where the second image pickup device 1B is selected as the image pickup device used for detecting the object.

As shown in FIGS. 4A and 4B, it can be confirmed that the area in charge is different depending on whether the object is in the upright state or in the seated state. Further, in FIG. 4A, the first imaging device 1A and the second imaging device 1B are selected as the imaging devices used for detecting the object in the vicinity of the first imaging device 1A. This is because if the three-dimensional model is too close to the first imaging device 1A, only the upper part of the three-dimensional model M is shown in the first image captured by the first imaging device 1A, and the area of the first region becomes small. be.

That is, from the viewpoint of the size of the area, the superiority or inferiority of the first imaging device 1A and the second imaging device 1B cannot be given as the imaging device used for detecting the object existing directly under the first imaging device 1A. However, even if the area on the image is the same, if the object exists in the vicinity of each image pickup device, the position of the object can be detected with higher accuracy by using the image pickup device as the image pickup device used for detection. In many cases. Therefore, the selection unit 223 determines the area in charge as shown in FIGS. 4 (a) and 4 (b) according to the size of the area, and then a certain distance from the area in the vicinity of each image pickup device, for example, the position directly below the image pickup device. Within, the area in charge may be redefined (overwritten) so that the imaging device is selected.
Further, in FIG. 4B, in the action area A, there is an area that is not set as the areas A1 to A3, but this area is an area in which the object does not take a seated state.

[Configuration of object tracking device 3]
Subsequently, the configuration of the object tracking device 3 will be described. FIG. 5 is a diagram showing the configuration of the object tracking device 3 according to the present embodiment. As shown in FIG. 5, the object tracking device 3 includes a storage unit 31 and a control unit 32.

The storage unit 31 is a storage medium including a ROM, a RAM, and the like. The storage unit 31 stores a program executed by the control unit 32. For example, the storage unit 31 stores an object tracking program that causes the object tracking device 3 to function as an image acquisition unit 321, an object detection unit 322, and an object tracking unit 323.
Further, it is assumed that the storage unit 31 stores the responsible area information generated by the selection device 2 in advance.

The control unit 32 is, for example, a CPU. The control unit 32 functions as an image acquisition unit 321, an object detection unit 322, and an object tracking unit 323 by executing the object tracking program stored in the storage unit 31.

The image acquisition unit 321 acquires the omnidirectional image captured by the first imaging device 1A, and also acquires the omnidirectional image captured by the second imaging device 1B. The omnidirectional image contains information indicating the imaging time.

The object detection unit 322 detects an object included in the omnidirectional image by analyzing the omnidirectional image acquired by the image acquisition unit 321. For example, the object detection unit 322 specifies a contour region indicating the contour of an object included in the omnidirectional image. Then, the object detection unit 322 detects the object by specifying the information indicating the upright rectangular area based on the specified contour area.

The object detection unit 322 matches the information indicating the specified upright rectangular area with the information indicating the upright rectangular area included in the area in charge information. For example, the object detection unit 322 has an overlap rate (IOU) between the upright rectangular area indicated by the information indicating the specified upright rectangular area and the upright rectangular area indicated by the information indicating the plurality of upright rectangular areas included in the responsible area information. : Intersection over Union) is calculated. Then, the object detection unit 322 identifies the information having the highest duplication rate among the information indicating the upright rectangular area included in the area in charge information.

The object detection unit 322 identifies the position information associated with the information indicating the upright rectangular area having the highest overlap rate, and sets the position indicated by the position information as the position of the object. Then, the object detection unit 322 associates the information indicating the imaging time included in the omnidirectional image including the object with the position information, and stores the object position information in the storage unit 31. The object detection unit 322 identifies the position of the object for each of the acquired omnidirectional images based on the above procedure, and stores the object position information in the storage unit 31.

The object tracking unit 323 tracks the object based on the object position information stored in the storage unit 31. For example, the object tracking unit 323 tracks the objects by setting the objects whose positions indicated by the position information are within a predetermined distance as the same object at two consecutive times.

[Process flow in selection device 2]
Subsequently, the flow of processing in the selection device 2 will be described. FIG. 6 is a flowchart showing a processing flow in the selection device 2 according to the present embodiment.

First, the area specifying unit 221 selects an unselected position from a plurality of positions included in the action area (S1).
Subsequently, the area specifying unit 221 selects an unselected shape from the shapes that the object can take at the selected position (S2).

Subsequently, the area specifying unit 221 specifies the first area and the second area when the three-dimensional model corresponding to the shape selected in S2 is arranged at the position selected in S1 (S3).
Subsequently, the reliability value specifying unit 222 specifies the first reliability value based on the first region and specifies the second reliability value based on the second region (S4). Here, the reliability value specifying unit 222 may specify the first reliability value and the second reliability value based on the information other than the first region and the second region.

Subsequently, the selection unit 223 determines whether or not the ratio of the first confidence value to the second confidence value is equal to or greater than the first threshold value (S5). If the selection unit 223 determines that the threshold value is equal to or higher than the first threshold value, the process is transferred to S6, and if the selection unit 223 determines that the threshold value is lower than the first threshold value, the process is transferred to S7.

In S6, the selection unit 223 selects the first imaging device 1A as the imaging device used to detect the object corresponding to the shape selected in S2 at the position selected in S1. When the processing of S6 is completed, the selection unit 223 shifts the processing to S10.

Subsequently, the selection unit 223 determines whether or not the ratio of the second confidence value to the first confidence value is equal to or greater than the second threshold value (S7). When the selection unit 223 determines that the value is equal to or higher than the second threshold value, the process is transferred to S8, and when the selection unit 223 determines that the value is less than or equal to the second threshold value, the process is transferred to S9.

In S8, the selection unit 223 selects the second imaging device 1B as the imaging device used to detect the object corresponding to the shape selected in S2 at the position selected in S1. When the processing of S8 is completed, the selection unit 223 shifts the processing to S10.

In S9, the selection unit 223 selects the first image pickup device 1A and the second image pickup device 1B as the image pickup device used to detect the object corresponding to the shape selected in S2 at the position selected in S1.

Subsequently, the storage control unit 224 includes position information indicating the position selected in S1, shape information indicating the shape selected in S2, device identification information for identifying the imaging device selected by the selection unit 223, and the like. When an object is placed at a position, it is stored in the storage unit 21 as the area in charge information in association with the information indicating the area of the image indicating the object (information indicating the upright rectangle) included in the image captured by the imaging device. (S10).

Subsequently, the area specifying unit 221 determines whether or not all possible shapes of the object have been selected (S11). When the area specifying unit 221 determines that all the shapes have been selected, the process is transferred to S12, and when it is determined that all the shapes have not been selected, the area specifying unit 221 transfers the process to S2.

Subsequently, the area specifying unit 221 determines whether or not all the positions included in the action area have been selected (S12). When the area specifying unit 221 determines that all the positions have been selected, it ends the process according to this flowchart, and when it determines that all the positions have not been selected, it shifts the process to S1.

[Process flow in object tracking device 3]
Subsequently, the processing flow in the object tracking device 3 will be described. FIG. 7 is a flowchart showing a processing flow in the object tracking device 3 according to the present embodiment. At the start of this flowchart, it is assumed that the storage unit 31 of the object tracking device 3 stores the area information in charge generated by the selection device 2.

First, the image acquisition unit 321 acquires a plurality of omnidirectional images from the first image pickup device 1A and the second image pickup device 1B (S21). Here, it is assumed that the imaging times included in the omnidirectional images captured by the first imaging device 1A and the second imaging device 1B are synchronized.

Subsequently, the object detection unit 322 selects the omnidirectional image having the oldest imaging time among the unselected omnidirectional images (S22). Here, the object detection unit 322 selects two omnidirectional images captured by the first imaging device 1A and the second imaging device 1B as the omnidirectional images having the oldest imaging time.

Subsequently, the object detection unit 322 performs image analysis of the selected omnidirectional image and identifies an upright rectangular region of the object included in the omnidirectional image (S23).
Subsequently, the object detection unit 322 overlaps the upright rectangular area of the specified object with the upright rectangular area indicated by each of the plurality of upright rectangular area information included in the responsible area information stored in the storage unit 31. Calculate the rate (S24).

Subsequently, the object detection unit 322 specifies the position indicated by the position information associated with the upright rectangular area having the highest overlap rate in the area information in charge to the position of the object (S25).

Subsequently, the object tracking unit 323 tracks the object based on the position of the object specified in S25 (S26).

Subsequently, the object detection unit 322 determines whether or not all omnidirectional images have been selected (S27). When the object detection unit 322 determines that all the omnidirectional images have been selected, it ends the process according to this flowchart, and when it determines that all the omnidirectional images have not been selected, it shifts the process to S22.

[Effect in this embodiment]
As described above, the selection device 2 according to the present embodiment virtually arranges a three-dimensional model imitating an object to be detected in a predetermined area at a predetermined position in a three-dimensional space indicating a predetermined area. In this case, the first region showing the area of the image showing the three-dimensional model included in the first image captured by the first imaging device 1A that images the predetermined area and the second imaging device 1B that images the predetermined area are imaged. A first reliability indicating the reliability of the detection when the second area indicating the area of the image showing the three-dimensional model included in the second image is specified and the object is detected based on the first area. The value and the second confidence value indicating the reliability value of the detection when the object is detected based on the second region are specified. Then, the selection device 2 uses an imaging device used to detect an object at a predetermined position among the first imaging device 1A and the second imaging device 1B based on the ratio of the first reliability value and the second reliability value. select. By doing so, the selection device 2 can select an imaging device that captures an image capable of accurately detecting an object.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments.

Further, in particular, the specific embodiment of the distribution / integration of the apparatus is not limited to those shown above, and all or a part thereof may be arbitrarily added according to various additions or functional loads. It can be functionally or physically distributed / integrated in units.

For example, in the above embodiment, the selection device 2 and the object tracking device 3 are different devices, but the device is not limited to this. The selection device 2 and the object tracking device 3 may be realized by one device.

1A ... 1st imaging device, 1B ... 2nd imaging device, 2 ... selection device, 21 ... storage unit, 22 ... control unit, 221 ... area identification unit, 222 ... -Reliability value identification unit, 223 ... selection unit, 224 ... storage control unit, 3 ... object tracking device, 31 ... storage unit, 32 ... control unit, 321 ... image acquisition unit 322 ... Object detection unit, 323 ... Object tracking unit, S ... Object tracking system

Claims (15)

When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that images the predetermined area takes an image. The first region showing the region of the image showing the three-dimensional model included in the first image and the image showing the three-dimensional model included in the second image captured by the second imaging device that images the predetermined area. An area specifying part that specifies a second area indicating an area,
A first reliability value indicating the reliability of the detection when the object is detected based on the first region, and a reliability value of the detection when the object is detected based on the second region. The trust value specifying part that specifies the second trust value indicating
A selection unit that selects an imaging device to be used for detecting an object at a predetermined position among the first imaging device and the second imaging device based on the ratio of the first reliability value to the second reliability value. When,
A selection device comprising. The reliability value specifying unit uses the area of the first region to specify the first reliability value, and uses the area of the second region to specify the second reliability value.
The selection device according to claim 1. The first imaging device or the second imaging device is an imaging device that captures an omnidirectional image.
The reliability value specifying unit uses the distance between the position of the first region and the center of the first image to specify the first reliability value, and the position of the second region and the center of the second image. The distance from and is used to specify the second confidence value.
The selection device according to claim 1 or 2. In the reliability value specifying unit, when the three-dimensional model and the second three-dimensional model imitating the second object that is not the target to be detected are virtually arranged, the first region is the first. The ratio hidden by the image region showing the second three-dimensional model included in one image is used for specifying the first confidence value, and the second region is the second three-dimensional included in the second image. The ratio hidden by the image area showing the model is used to identify the second confidence value.
The selection device according to any one of claims 1 to 3. The information relating the position in the image of the first image and the first trust value corresponding to the position is associated with the position in the image of the second image and the second trust value corresponding to the position. It also has a storage unit that stores the information
The reliability value specifying unit refers to the storage unit, acquires the first reliability value associated with the position of the image showing the three-dimensional model included in the first image, and obtains the first reliability value and obtains the second image. The first confidence value and the second confidence value are specified by acquiring the second confidence value associated with the position of the image showing the included three-dimensional model.
The selection device according to claim 1. The selection unit serves as an imaging device used for detecting the object at the predetermined position when the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is equal to or more than the first threshold value. The first imaging device is selected and used to detect the object at the predetermined position when the ratio of the second confidence value to the first confidence value corresponding to the predetermined position is equal to or greater than the second threshold value. The second imaging device is selected as the imaging device to be used.
The selection device according to any one of claims 1 to 5. In the selection unit, the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is less than the first threshold value, and the selection unit has the said to the first confidence value corresponding to the predetermined position. When the ratio of the second confidence value is less than the second threshold value, the first image pickup device and the second image pickup device are selected as the image pickup device used for detecting the object at the predetermined position.
The selection device according to claim 6. When the predetermined position is within the first distance from the first imaging device, the selection unit is the object at the predetermined position regardless of the ratio of the first reliability value and the second reliability value. When the first imaging device is selected as the imaging device used for the detection of the above and the predetermined position is within the second distance from the second imaging device, the first reliability value and the second reliability value are Regardless of the ratio, the second imaging device is selected as the imaging device used to detect the object at the predetermined position.
The selection device according to any one of claims 1 to 7. The reliability value specifying unit identifies the first reliability value and the second reliability value when the three-dimensional model is virtually arranged for each of the plurality of positions in the three-dimensional space.
The selection unit selects an imaging device to be used for detecting the object based on the ratio of the first reliability value and the second reliability value for each of the plurality of positions in the three-dimensional space.
The selection device according to any one of claims 1 to 8. A storage control unit further includes a storage control unit that stores the device identification information for identifying the image pickup device and the position information indicating the position where the image pickup device is selected to be used for detecting the object in the storage unit.
The selection device according to any one of claims 1 to 9. The reliability value specifying unit includes the first reliability value and the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position for each of the plurality of shapes corresponding to the object at the predetermined position. Identify the second confidence value and
The selection unit selects an imaging device to be used for detecting the object based on the ratio of the first reliability value and the second reliability value for each of the plurality of shapes corresponding to the object at the predetermined position. do,
The selection device according to any one of claims 1 to 10. The reliability value specifying unit is the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position for each of the one or more shapes that the object can take at the predetermined position. Identifying the second confidence value,
The selection device according to claim 11. A storage control unit that stores the device identification information that identifies the image pickup device, the shape of the object, and the position information indicating the position where the image pickup device is selected to be used for detecting the object in the storage unit in association with each other. Further prepare
The selection device according to claim 11 or 12. Computer runs,
When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that images the predetermined area takes an image. The first region showing the region of the image showing the three-dimensional model included in the first image and the image showing the three-dimensional model included in the second image captured by the second imaging device that images the predetermined area. The step of identifying the second area indicating the area and
A first reliability value indicating the reliability of the detection when the object is detected based on the first region, and a reliability value of the detection when the object is detected based on the second region. And the step of identifying the second confidence value that indicates
A step of selecting an image pickup device to be used for detecting an object at a predetermined position among the first image pickup device and the second image pickup device based on the ratio of the first reliability value and the second confidence value. ,
Selection method with. Computer,
When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that images the predetermined area takes an image. The first region showing the region of the image showing the three-dimensional model included in the first image and the image showing the three-dimensional model included in the second image captured by the second imaging device that images the predetermined area. Area identification unit that specifies the second area indicating the area,
A first reliability value indicating the reliability of the detection when the object is detected based on the first region, and a reliability value of the detection when the object is detected based on the second region. The trust value specifying part that specifies the second trust value indicating
A selection unit that selects an imaging device to be used for detecting an object at a predetermined position among the first imaging device and the second imaging device based on the ratio of the first reliability value to the second reliability value. ,
A selection program that works as.

Images