JP7435230B2 - Object tracking device, object tracking method, and object tracking program - Google Patents

Description

The present invention relates to technology for tracking the movement of objects (particularly people).

Conventionally, there has been a radio wave radar device that detects a moving object by emitting radio waves such as millimeter waves or microwaves as probe waves and receiving reflected waves thereof (see Patent Document 1, etc.). Furthermore, the radio radar device can obtain the speed of a detected moving object (person, vehicle, etc.) based on the difference between the frequency of emitted radio waves and the frequency of received radio waves.

Japanese Patent Application Publication No. 2006-329912

However, the radio radar device is configured to detect a moving object using the Doppler effect. Therefore, when the moving speed of a moving object becomes approximately 0 (that is, when the moving object stops), the radio wave radar device cannot detect the moving object in distinction from buildings, installed objects, and the like.

For example, when a moving object whose movement is being tracked stops within a target area, a radio wave radar device cannot detect the moving object in distinction from a building, an installation, etc., and may lose sight of the moving object. As a result, for a moving object that has temporarily stopped within the target area, it may not be possible to correlate the tracking data before and after the stopping. That is, tracking data before and after a moving object stops may be obtained as tracking data for different moving objects.

An object of the present invention is to provide a technique that can appropriately acquire tracking data of a moving object within a target area, regardless of whether the moving object whose movement is being tracked within the target area has stopped.

In order to achieve the above object, the object tracking device of the present invention is configured as shown below.

Object detection data including the speed and position of an object detected by repeatedly scanning the target area with a probe wave is input to the detection data input section. Further, a moving image of the target area is input to the image input unit.

The detection unit detects, as an observation target object, an object whose state has changed from a moving state to a stopped state within the target area based on the object detection data input to the detection data input unit. Further, the stopped state monitoring section processes frame images of the moving image input to the image input section, and monitors the stopped state of the observation target object.

Then, the tracking data generation unit generates tracking data of the observation target object within the target area using the object detection data input to the detection data input unit and the monitoring result of the observation target object by the stop state monitoring unit.

In this configuration, when a moving object that is being tracked using radio waves such as millimeter waves or microwaves stops within the target area, the image input section This is done by processing frame images of moving images input to the . The moving object may be, for example, a person, a bicycle, or a vehicle. The type of moving object can be determined by its size and moving speed. Thereby, when the monitored moving body changes state from a stopped state to a moving state, this state change can be appropriately detected. Therefore, even if the moving object temporarily stops within the target area, tracking data of the moving object within the target area can be appropriately acquired. In other words, regardless of whether or not the moving body whose movement is being tracked within the target area has stopped, tracking data of this moving body within the target area can be appropriately acquired.

Further, the stop state monitoring unit is configured to monitor the stop state of the observation target object using, for example, feature quantities of the observation target object extracted by processing frame images related to the moving image input to the image input unit. You may also do so. Specifically, it is sufficient to detect whether the state of the observed object has changed from a stopped state to a moving state based on whether the feature amount of the observed object has changed on the frame image.

In addition, the tracking data generation unit may generate, for example, object detection data in a first period until the observed object changes state from a moving state to a stopped state within the target area, and object detection data when the observed object changes state from a stopped state to a moving state. The object detection data in the second period after the detection may be correlated using the monitoring result of the object to be observed by the stop state monitoring unit.

Further, for example, when it is detected that the observation target object changes state from a stopped state to a moving state, it may be configured to include an output unit that outputs a notification to that effect.

Further, the stop state monitoring unit may be configured to, for example, estimate the posture of a person who is an object to be observed, and determine the degree of risk based on the estimated posture.

With this configuration, it is possible to estimate whether the person stopped, crouched down, fell, etc., and determine the degree of danger of the person.

The target area includes, for example, an area where a person crossing a vehicle travel path stops before the vehicle travel path. The vehicle travel path referred to herein is not limited to roads on which automobiles travel, but also includes tracks on which trains travel.

According to the present invention, regardless of whether the moving object whose movement is being tracked within the target area has stopped, it is possible to appropriately obtain tracking data of the moving object within the target area.

FIG. 2 is a schematic diagram showing a target area in which a moving person is tracked by an object tracking device. FIG. 2 is a diagram showing connections between an object tracking device, a radio wave radar device, and an imaging device. 1 is a diagram showing the configuration of main parts of an object tracking device. FIG. 3 is a diagram showing tracking data of pedestrians who did not stop within the target area. Figure 5 (A) shows tracking data of a pedestrian before stopping within the target area, and Figure 5 (B) shows tracking data of a pedestrian after restarting walking after stopping within the target area. It is a diagram. It is a flowchart showing the operation of the radio wave radar device. 3 is a flowchart showing the operation of the object tracking device. It is a flowchart which shows monitoring processing.

Embodiments of this invention will be described below.

<1. Application example>
FIG. 1 is a schematic diagram showing a target area for tracking a moving person by an object tracking device according to this example. FIG. 2 is a diagram showing connections among the object tracking device, radio wave radar device, and imaging device according to this example.

As shown in FIG. 2, a radio wave radar device 5 and an imaging device 6 are connected to the object tracking device 1 according to this example. The object tracking device 1 may be connected to the radio wave radar device 5 and the imaging device 6 by wire, or may be connected wirelessly. Furthermore, as will be described later, the radio wave radar device 5 is installed at a position where it can scan the target area 100 with radio waves, which are probe waves. Further, the imaging device 6 is installed so that the imaging area includes the target area. The radio wave radar device 5 and the imaging device 6 will be installed near the target area 100, but the object tracking device 1 may be installed near the target area 100 or at a control center far from the target area 100. , a data collection center, etc.

As shown in FIG. 1, in this example, the target area 100 includes the entire crosswalk where a pedestrian 111 crosses a roadway on which a vehicle 110 travels. In this example, the object tracking device 1 tracks a pedestrian 111 crossing a crosswalk, but does not track a vehicle 110 running on a roadway. When pedestrians cross a crosswalk, they may stop on the sidewalk in front of the crosswalk. For example, when the pedestrian 111 checks the running condition of the vehicle 110 on the roadway (when checking whether it is safe to cross the road), when the pedestrian 111 checks the driving condition of the vehicle 110 on the roadway (when checking whether it is safe to cross the road), when the pedestrian 111 has a traffic light installed, the pedestrian 111 may check the running condition of the vehicle 110 on the roadway (when checking whether the vehicle 110 can cross the road safely), or when a traffic light is installed, when the light is red, the pedestrian 111 may Stop on the sidewalk.

The road shown in Figure 1 has one lane on each side, but it may be one-way with one lane or multiple lanes on one side (the number of opposite lanes may be the same or different). ).

The radio wave radar device 5 is installed at a position where it can scan the target area 100 with a search wave. Further, the imaging device 6 is attached at an angle such that the target area 100 fits within the imaging area. The radio wave radar device 5 and the imaging device 6 are attached to, for example, a pole (not shown) installed on the roadside of a roadway or an arm (not shown) attached to this pole.

The radio wave radar device 5 scans the target area 100 shown in FIG. 1 with radio waves (corresponding to search waves in the present invention) and detects the position and speed of the pedestrian 111. The radio wave radar device 5 repeatedly scans the target area 100 using radio waves at set time intervals (for example, at 100 msec intervals).

Furthermore, the radio wave radar device 5 may be configured to generate tracking data for tracking the pedestrian 111 within the target area 100. In this example, the radio wave radar device 5 will be described as having a configuration for generating this tracking data, but the configuration for generating this tracking data may be provided on the object tracking device 1 side. The configuration for generating tracking data is a configuration that associates the pedestrian 111 detected in the previous scan of the target area 100 by the radio wave radar device 5 with the pedestrian 111 detected in the current scan of the target area 100. . Every time the radio radar device 5 scans the target area 100, it outputs pedestrian detection data that associates the ID, position, and speed of the pedestrian 111 detected in the current scan.

The imaging device 6 is a known video camera. The frame rate of the imaging device 6 is several tens of frames/sec (for example, 30 frames/sec). The imaging device 6 is installed so that the imaging area includes the target area 100 as described above.

As shown in FIG. 2, the object tracking device 1 according to this example receives the above-mentioned pedestrian detection data from the radio radar device 5, and also receives a moving image captured by the imaging device 6 of the target area 100. Ru.

The object tracking device 1 acquires tracking data of a pedestrian 111 who passed through the target area 100 based on the input pedestrian detection data. The tracking data referred to here is data indicating changes in the position and speed of the pedestrian 111 over time.

Further, the object tracking device 1 does not detect a moving object whose detected speed exceeds a preset threshold speed (for example, 10 km/h) as not being a pedestrian 111. In addition, the object tracking device 1 detects a pedestrian 111 whose detected speed has approached approximately 0 (for example, a moving object whose speed has become less than 1 km/h), and a pedestrian 111 who stops (an observation target in this invention). object).

The object tracking device 1 also processes frame images related to the moving image input from the imaging device 6, detects a stopped pedestrian 111 captured in the frame image, and monitors the pedestrian 111. perform monitoring processing. The monitoring of the pedestrian 111 referred to here means monitoring the stopped pedestrian 111 to start (resume) walking (movement).

When the speed component of the pedestrian 111 becomes approximately 0, the radio wave radar device 5 is more likely to lose sight of the pedestrian 111. When the radio wave radar device 5 loses sight of the stopped pedestrian, the object tracking device 1 does not receive pedestrian detection data regarding the stopped pedestrian 111 from the radio wave radar device 5 .

Furthermore, when the stopped pedestrian 111 resumes walking, the radio wave radar device 5 detects the pedestrian 111 again when the pedestrian 111 exceeds a certain speed. The radio wave radar device 5 outputs pedestrian detection data of the re-detected pedestrian 111 to the object tracking device 1.

As can be seen from the above description, there is a difference between the position where the radio radar device 5 loses sight of the observed object, the stopped pedestrian 111, and the position where the pedestrian 111, who has resumed walking, is redetected. Therefore, the radio wave radar device 5 cannot correlate the pedestrian detection data of the temporarily stopped pedestrian 111 before and after the pedestrian 111 stops. Specifically, the radio wave radar device 5 detects, for the pedestrian 111 who has temporarily stopped, a period before the stop (corresponding to the first period in the present invention) and a period after resuming walking ( Corresponding to the second period in the present invention), different IDs are assigned (the IDs included in the pedestrian detection data are different).

The object tracking device 1 processes the frame images of the moving image input from the imaging device 6 before the radio wave radar device 5 detects the pedestrian 111 who has resumed walking. It is detected that the pedestrian 111 has resumed walking. When the object tracking device 1 detects that the pedestrian 111 has resumed walking, the object tracking device 1 determines that the ID of the pedestrian detection data input from the radio wave radar device 5 regarding the pedestrian 111 who has resumed walking is the same as before the pedestrian 111 stopped. It is determined whether they are different (it is determined whether the radio wave radar device 5 has lost sight of the stopped pedestrian 111).

If the ID of the pedestrian detection data inputted from the radio wave radar device 5 regarding the pedestrian 111 is different before and after the pedestrian 111 stopped walking, the object tracking device 1 associates these IDs. That is, the object tracking device 1 associates pedestrian detection data before and after the temporary stop. Therefore, even if the pedestrian 111 stops within the target area 100, the object tracking device 1 can appropriately acquire tracking data (pedestrian detection data) that tracks the movement of the pedestrian 111.

<2. Configuration example>
FIG. 3 is a diagram showing the configuration of the main parts of the object tracking device according to this example. The object tracking device 1 according to this example includes a control unit 11, a detection data input section 12, an image input section 13, and an output section 14.

The control unit 11 controls the operation of each part of the object tracking device 1 main body. The control unit 11 also includes a detection section 21, an image processing section 22, and a tracking data generation section 23. The detection section 21, image processing section 22, and tracking data generation section 23 included in the control unit 11 will be described later.

A radio wave radar device 5 is connected to the detected data input section 12 . Pedestrian detection data is input to the detection data input section 12 from the radio wave radar device 5 . The pedestrian detection data is data in which the ID, position, and speed of the pedestrian 111 are associated with each detected pedestrian. Every time the radio wave radar device 5 scans the target area 100 with radio waves, it outputs pedestrian detection data of each pedestrian 111 detected in the scan to the object tracking device 1.

An imaging device 6 is connected to the image input section 13 . A moving image of the target area 100 captured by the imaging device 6 is input to the image input unit 13 .

The output unit 14 outputs tracking data of each pedestrian in the target area 100 to a higher-level device (not shown) such as a control center.

Next, the detection section 21, image processing section 22, and tracking data generation section 23 included in the control unit 11 will be explained.

Based on the pedestrian detection data input from the radio wave radar device 5, the detection unit 21 detects a pedestrian 111 whose speed has approached approximately 0 (for example, a pedestrian whose speed has become less than 2 km/h), a pedestrian who has stopped, 111 (observation target object in this invention).

The image processing unit 22 selects a frame image to be processed from among the frame images of the moving image input from the imaging device 6. The image processing unit 22 processes the selected frame image to be processed and monitors the movement of the pedestrian 111 to be observed, which is captured in the frame image to be processed. The pedestrian 111 to be observed is a pedestrian 111 whose speed approaches approximately 0, which is detected based on pedestrian detection data.

The control unit 11 stores parameters for associating a position on a frame image of a moving image captured by the imaging device 6 with a position detected by the radio wave radar device 5 in a memory (not shown). For example, this parameter is a parameter that associates a position on a frame image of a moving image captured by the imaging device 6 with a position in real space. The radio wave radar device 5 detects the position of the pedestrian 111 in the real space based on the installed position in the real space.

The image processing unit 22 uses the above parameters to identify the position on the frame image where the pedestrian 111 to be observed detected by the detection unit 21 is captured. The image processing unit 22 processes the frame image and extracts the feature amount of the pedestrian 111 to be observed. The image processing unit 22 uses the extracted feature amount to monitor whether the pedestrian 111 to be observed resumes walking after stopping. Specifically, the image processing unit 22 monitors a change in the position of the frame image in which the feature amount of the pedestrian 111 to be observed is detected on the frame image. The image processing section 22 corresponds to a stop state monitoring section in the present invention.

In addition, in monitoring the pedestrian 111 to be observed, the image processing unit 22 may select frame images related to the input moving image one frame at a time in the order in which they are arranged as processing target frame images, or The frame images may be selected as processing target frame images at several frame intervals (2 to 10 frame intervals) in the temporal order.

The tracking data generation unit 23 processes the pedestrian detection data input from the radio wave radar device 5 to generate tracking data that tracks the movement of the pedestrian 111 within the target area 100. FIG. 4 is a diagram showing tracking data of pedestrians who did not stop within the target area. FIG. 5 is a diagram showing tracking data of a pedestrian stopped within the target area. Figure 5(A) shows the tracking data of the pedestrian before stopping within the target area, and Figure 5(B) shows the tracking data of the pedestrian after restarting walking after stopping within the target area. ing. The example shown in FIG. 5 is an example in which the radio wave radar device 5 loses sight of a pedestrian 111 who has temporarily stopped within the target area.

As shown in FIGS. 4 and 5, the tracking data of the pedestrian 111 is data in which an ID for identifying the pedestrian 111 is associated with the speed and position of the pedestrian at each detection time. In this example, the position of the pedestrian 111 is the distance from the first reference position in the cross direction in which the pedestrian crosses the roadway, and the distance from the second reference position in the width direction of the crosswalk. The first reference position is, for example, the installation position of the radio wave radar device 5 in the transverse direction. Further, the second reference position is, for example, the center position of the crosswalk in the width direction, and the right side is a positive value and the left side is a negative value.

For pedestrians 111 who have passed through target area 100 without stopping, the ID included in the pedestrian detection data input from radio wave radar device 5 does not change. Therefore, the tracking data generation unit 23 can generate tracking data of the pedestrian 111 who passed through the target area 100 without stopping by classifying and totaling the input pedestrian detection data by ID.

Furthermore, if the pedestrian 111 temporarily stops within the target area 100 and the radio wave radar device 5 loses sight of the pedestrian 111, the tracking data of the pedestrian 111 before the stop (see FIG. The IDs are different between the pedestrian tracking data (see FIG. 5(B)) and the pedestrian tracking data after resuming walking after stopping (see FIG. 5(B)). Therefore, if the tracking data generation unit 23 simply categorizes and totals the input pedestrian detection data by ID, it can only separately generate tracking data before the vehicle stops and tracking data after the vehicle stops. The tracking data generation unit 23 associates the tracking data of the pedestrian 111 before and after stopping, based on the monitoring result of the pedestrian 111 to be observed by the image processing unit 22.

The control unit 11 of the object tracking device 1 includes a hardware CPU, memory, and other electronic circuits. When the hardware CPU executes the object tracking program according to the present invention, it operates as the detection section 21, the image processing section 22, and the tracking data generation section 23. The memory also has an area for developing the object tracking program according to the present invention, and an area for temporarily storing data generated when the object tracking program is executed. The control unit 11 may be an LSI integrated with a hardware CPU, memory, and the like. Further, the hardware CPU is a computer that executes the object tracking method according to the present invention. Furthermore, the control unit 11 may have a configuration in which the hardware CPU is replaced with a hardware GPU, or may include a hardware CPU and a hardware GPU (in this case, the hardware GPU is The object tracking program according to the invention is executed, and the hardware GPU executes the object tracking method according to the invention).

The memory also stores parameters for associating positions on the frame image with positions detected by the radio wave radar device 5. For example, this parameter is a parameter that associates a position on a frame image of a moving image captured by the imaging device 6 with a position in real space. In this case, the radio wave radar device 5 detects the position of the pedestrian 111 in the real space based on the installed position in the real space.

Note that the above parameters stored in the memory may be in any format as long as they are parameters that can associate the position on the frame image with the position detected by the radio wave radar device 5.

Furthermore, since the radio wave radar device 5 and the imaging device 6 are known devices that have already been put into practical use, detailed explanations of their configurations will be omitted here.

<3. Operation example>
FIG. 6 is a flowchart showing the operation of the radio wave radar device. When the scanning timing of the target area 100 comes (s1), the radio wave radar device 5 scans the target area 100 with radio waves, which are search waves, and detects the reflected waves, thereby detecting the pedestrians located within the target area 100. A detection process is performed to detect the person 111 (s2). In s2, the radio wave radar device 5 detects the time until detecting the reflected wave (radio wave flight time) and the frequency of the reflected wave for each radio wave irradiation direction, and detects objects that reflected the radio wave (pedestrian 111, The position of the vehicle 110, road surface, etc.) and the speed of this object are detected.

In this example, the radio wave radar device 5 does not detect a moving object whose detected speed exceeds a threshold speed (for example, 10 Km/h), as it is not the pedestrian 111 (it is the vehicle 110).

When the detection processing in s2 is completed, the radio wave radar device 5 creates an association between the pedestrian 111 detected in the previous scan of the target area 100 and the pedestrian 111 detected in the current scan of the target area 100. Processing is performed (s3). The matching process in s3 is performed using the position and speed of the pedestrian 111. The radio wave radar device 5 assigns the pedestrian 111 that was able to be matched to the pedestrian 111 detected in the previous scan (the pedestrian 111 detected in the current scan) to the matched pedestrian 111. Assign an ID. On the other hand, the radio radar device 5 generates a new ID for the pedestrian 111 that could not be associated with the pedestrian 111 detected in the previous scan (the pedestrian 111 detected in the current scan), and Assign.

The radio wave radar device 5 may lose sight of the pedestrian 111 because the pedestrian 111 has temporarily stopped. In this case, the radio wave radar device 5 cannot associate the pedestrian detection data from before the pedestrian 111 stopped and the pedestrian detection data detected after the pedestrian resumed walking. That is, for the pedestrian 111 who has temporarily stopped, the ID of the pedestrian detection data before the stop is different from the ID of the pedestrian detection data detected after resuming walking.

The radio radar device 5 generates pedestrian detection data that associates ID, speed, and position for each pedestrian 111 detected in the current scan, and outputs the pedestrian detection data generated here to the object tracking device 1. (s4) and return to s1.

In this example, the radio wave radar device 5 scans the target area 100 with radio waves at intervals of 100 msec, and outputs pedestrian detection data regarding the detected pedestrian 111 to the object tracking device 1. That is, pedestrian detection data is input to the object tracking device 1 at intervals of 100 msec.

Further, by performing the above processing, the radio wave radar device 5 can obtain tracking data (see FIGS. 4 and 5) of the pedestrian 111 who passed through the target area 100. However, if the radio wave radar device 5 loses sight of the pedestrian 111 who has temporarily stopped within the target area 100, it associates the tracking data of the pedestrian 111 before the stop with the tracking data after resuming walking. I can't.

Next, the operation of the object tracking device 1 according to this example will be explained. FIG. 7 is a flowchart showing the operation of the object tracking device. When pedestrian detection data is input to the detection data input unit 12 (s11), the object tracking device 1 detects a pedestrian 111 whose speed has approached approximately 0 (for example, 2 km) based on the input pedestrian detection data. /h) is detected as the pedestrian 111 to be observed (s12). The detection unit 21 executes the process related to s12.

The object tracking device 1 determines whether there is a pedestrian 111 detected for the first time among the pedestrians 111 to be observed detected in s12 (s13). The process related to s13 is provided to prevent multiple executions of the monitoring process described below for the same pedestrian 111. The detection unit 21 executes the process related to s13.

If there is a pedestrian 111 to be observed that has been extracted for the first time this time, the object tracking device 1 starts monitoring processing for this pedestrian 111 (s14). The monitoring process started in s14 will be described later.

If the object tracking device 1 determines in s13 that there is no pedestrian 111 to be observed, which is extracted for the first time this time, or if it starts monitoring processing for the pedestrian 111 to be observed, which is extracted for the first time in s14, The detected pedestrian detection data is classified and totaled by ID (s15), and the process returns to s11. The tracking data generation unit 23 executes the process related to s15.

The object tracking device 1 generates the tracking data shown in FIG. 4 or 5 described above by executing the process shown in FIG.

Next, the monitoring process that starts at s14 will be explained. FIG. 8 is a flowchart showing this monitoring process. The image processing unit 22 calculates the detection time of the pedestrian detection data input this time (pedestrian detection data from which the pedestrian 111 to be observed is extracted) from among the frame images related to the moving image input to the image input unit 13. , the frame image whose imaging time is temporally closest is selected (s21). The image processing unit 22 detects the observation target pedestrian 111 captured in the frame image selected in s21, and extracts the feature amount of this pedestrian 111 (s22). In s22, the ID of this pedestrian 111 and the extracted feature amount are stored in association with each other.

The image processing unit 22 can obtain the position in real space of the currently detected pedestrian 111 to be observed from the pedestrian detection data. Furthermore, the object tracking device 1 stores in the memory of the control unit 11 a parameter that associates a position in real space with a position in a frame image. Therefore, the image processing unit 22 can determine the position on the frame image where the pedestrian 111 to be observed is captured. In s22, the image processing unit 22 extracts not only the features related to the external shape of the pedestrian 111 to be observed, but also the color of clothing and the like.

The image processing unit 22 selects a processing target frame image for monitoring whether the pedestrian 111 to be observed resumes walking (s23), and determines whether the pedestrian 111 resumes walking (s24). The image processing unit 22 repeats the processes in s23 and s24 until it is confirmed that the pedestrian 111 to be observed has resumed walking. In s23, the frame image is captured at a later timing than the frame image selected in s21. In addition, in s23, frame images related to the moving image input to the image input unit 13 may be selected one frame at a time in the order of time, or several frames may be selected in the order of time. Frame images to be processed may be selected at intervals (2 to 10 frame intervals). Further, in s24, the image processing unit 22 monitors changes in the position on the frame image from which the feature amount of the pedestrian 111 to be observed can be extracted.

When the image processing unit 22 confirms that the pedestrian 111 to be observed has resumed walking in s24, it waits for the pedestrian detection data to be input to the detection data input unit 12 (s25), and then performs the pedestrian detection input this time. It is determined whether the data includes pedestrian detection data of the observed pedestrian 111 who has confirmed to have resumed walking (s26). In s26, it is determined whether there is pedestrian detection data to which a new ID has been assigned near the stop position of the pedestrian 111 to be observed. Near the stop position of the observed pedestrian 111, the pedestrian detection data to which a new ID is assigned this time is the pedestrian detection data of the observed pedestrian 111 who has resumed walking.

The image processing unit 22 repeats the processing in s25 and s26 until it determines in s26 that the pedestrian detection data of the pedestrian who has been confirmed to have resumed walking is included. When the image processing unit 22 determines in s26 that the pedestrian detection data of the pedestrian 111 who has confirmed that he has resumed walking is included, the image processing unit 22 stores the ID assigned to the pedestrian detection data of this pedestrian 111 and the ID assigned to the pedestrian detection data of the pedestrian 111 in s22. The extracted feature quantity of the pedestrian 111 to be observed is stored in association with the ID (s27), and the present process is ended.

In s27, for the pedestrian 111 who has temporarily stopped within the target area 100, the ID assigned by the radio radar device 5 before the pedestrian stopped is associated with the ID assigned after the pedestrian resumed walking. Therefore, the tracking data generation unit 23 generates tracking data of the pedestrian 111 that the radio wave radar device 5 temporarily loses sight of due to the pedestrian 111 stopping within the target area 100 before the stop shown in FIG. Even if the tracking data of the pedestrian 111 within the target area 100 is divided into after the resumption of walking shown in B), the tracking data of the pedestrian 111 within the target area 100 can be appropriately acquired.

Note that the processes shown in FIGS. 7 and 8 are just examples, and the order of some of the processing steps may be changed or some of the processing steps may be replaced with other processing steps that yield similar processing results. It's okay.

<4. Modified example>
The object tracking device 1 of this modification also has the configuration shown in FIGS. 2 and 3. The object tracking device 1 of this modification has a configuration in which the image processing unit 22 estimates the posture of the pedestrian 111 to be observed, and determines the degree of risk based on the estimated posture. Specifically, while monitoring the pedestrian 111 who has stopped in s23 and s24, the image processing unit 22 determines whether the pedestrian 111 is standing still, crouching down, falling down, etc. presume.

The image processing unit 22 determines the risk level of the pedestrian 111 based on the estimated posture of the pedestrian 111. For example, if the pedestrian 111 is standing still, the risk level is determined to be low, if the pedestrian 111 is crouching down, the risk level is determined to be medium, and if the pedestrian 111 is falling, the risk level is determined to be high.

Further, when the determined degree of risk is high, the object tracking device 1 may output a notification to that effect to the host device connected to the output unit 14.

The image processing unit 22 may also determine the degree of danger of the pedestrian 111 based on the posture of the pedestrian 111 and the position of the pedestrian 111 (for example, in the middle of the roadway, in front of a crosswalk, etc.). good. For example, if there is a pedestrian 111 who has fallen down in the middle of the road, it may be determined that the risk is extremely high.

Further, when detecting that the pedestrian 111 is crossing the road, the object tracking device 1 notifies the driver of the vehicle 110 traveling on the road that the pedestrian 111 is crossing the road. It may be configured to notify. For example, the object tracking device 1 outputs a signal indicating the presence or absence of a pedestrian crossing the roadway to a host device connected to the output unit 14. In response to a signal from the object tracking device 1 indicating the presence or absence of a pedestrian crossing the road, this host device displays a road guidance display board (not shown) for the driver of the vehicle 110 that indicates the pedestrian crossing the road. A message will be displayed indicating the presence or absence of the person. Thereby, the driver of the vehicle 110 can be informed of the presence of the pedestrian 111 who is crossing the street, and traffic accidents can be suppressed. In particular, traffic accidents can be effectively suppressed at crosswalks where no traffic lights are installed.

In this case, when the object tracking device 1 detects a pedestrian 111 who has fallen on the roadway, a pedestrian 111 who has crouched down on the roadway, etc., the object tracking device 1 may notify the driver of the vehicle 110 including the state of the pedestrian 111. .

Furthermore, the radio wave radar device 5 described in the above example may be replaced with a laser radar device that uses laser light as a search wave.

Furthermore, the object tracking device 1 may track not only the pedestrian 111 crossing a crosswalk but also the pedestrian 111 crossing a railroad crossing or the like (the target area 100 may be a railroad crossing).
Further, in the above description, the object tracking device 1 tracks the pedestrian 111, but it may track other types of moving objects such as bicycles and vehicles.

It should be noted that the present invention is not limited to the above-described embodiments as they are, but can be embodied by modifying the constituent elements within the scope of the invention at the implementation stage. Moreover, various inventions can be formed by appropriately combining the plurality of components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components from different embodiments may be combined as appropriate.

Furthermore, the correspondence between the configuration according to the present invention and the configuration according to the above-described embodiments can be described as in the following supplementary notes.
<Additional notes>
a detection data input unit (12) into which object detection data including the velocity and position of an object detected by repeatedly scanning the target area (100) with a search wave is input;
an image input unit (13) into which a moving image of the target area (100) is input;
a detection unit (111) that detects an object whose state has changed from a moving state to a stopped state within the target area (100) as an observation target object (111) based on the object detection data input to the detection data input unit (12); 21) and
a stop state monitoring unit (22) that processes frame images of the moving image input to the image input unit (13) and monitors a stop state of the observation target object (111);
The tracking data of the observation target object (111) in the target area (100) is combined with the object detection data input to the detection data input unit (12) and the observation target by the stop state monitoring unit (22). An object tracking device (1) comprising a tracking data generation unit (23) that generates data using monitoring results of an object.

1...Object tracking device 5...Radio wave radar device 6...Imaging device 11...Control unit 12...Detected data input section 13...Image input section 14...Output section 21...Detection section 22...Image processing section 23...Tracking data generation section 100... Target area 110...vehicles 111...pedestrians

Claims (9)

a detection data input unit into which object detection data including the speed and position of an object detected by repeatedly scanning the target area with a search wave is input;
an image input unit into which a moving image of the target area is input;
a detection unit that detects an object whose state has changed from a moving state to a stopped state within the target area as an observation target object based on the object detection data input to the detection data input unit;
a stop state monitoring unit that processes frame images of the moving image input to the image input unit and monitors a stop state of the observation target object;
Tracking data generation that generates tracking data of the observation target object in the target area using the object detection data input to the detection data input unit and the monitoring result of the observation target object by the stop state monitoring unit. An object tracking device comprising a section and. The stopped state monitoring unit monitors the stopped state of the observed object using feature quantities of the observed object extracted by processing frame images of the moving image input to the image input unit. Item 1. The object tracking device according to item 1. The tracking data generation unit includes the object detection data in a first period until the observed object changes from a moving state to a stopped state within the target area, and the object detection data when the observed object changes from a stopped state to a moving state. 3. The object tracking device according to claim 1, wherein the object detection data in a second period after a state change is associated with the object detection data using a monitoring result of the observed object by the stopped state monitoring unit. The object tracking device according to any one of claims 1 to 3, further comprising an output section that outputs a notification when detecting that the observed object changes state from a stopped state to a moving state. The object tracking device according to claim 1, wherein the observation target object is a person. The object tracking device according to claim 5, wherein the stop state monitoring unit estimates the posture of the person who is the observation target object, and determines the degree of risk based on the estimated posture. 7. The object tracking device according to claim 5, wherein the target area includes an area where a person crossing a vehicle travel path stops before the vehicle travel path. Based on the object detection data including the speed and position of the object detected by repeatedly scanning the target area with a search wave, which is input to the detection data input section, the state changes from a moving state to a stopped state within the target area. a detection step of detecting the object as an observation target object;
a stop state monitoring step of processing a frame image related to a moving image of the target area input to an image input unit and monitoring a stop state of the observation target object;
Tracking data generation that generates tracking data of the observation target object in the target area using the object detection data input to the detection data input unit and the monitoring result of the observation target object in the stop state monitoring step. A method for tracking objects performed by a computer. Based on the object detection data including the speed and position of the object detected by repeatedly scanning the target area with a search wave, which is input to the detection data input section, the state changes from a moving state to a stopped state within the target area. a detection step of detecting the object as an observation target object;
a stop state monitoring step of processing a frame image related to a moving image of the target area input to an image input unit and monitoring a stop state of the observation target object;
Tracking data generation that generates tracking data of the observation target object in the target area using the object detection data input to the detection data input unit and the monitoring result of the observation target object in the stop state monitoring step. An object tracking program that causes a computer to perform steps and.

Images