JP7797008B2 - Observation information processing method, observation information processing device, and recording medium - Google Patents

Description

Article 30, Paragraph 2 of the Patent Act applies. On May 21, 2021, the proceedings of the conference were posted on the website of the Society of Automotive Engineers of Japan (https://tech.jsae.or.jp/paperinfo/ja/content/p202101.097/). On May 26, 2021, the proceedings were published at the Society of Automotive Engineers of Japan's 2021 Spring Meeting Academic Lectures (https://www.jsae.or.jp/2021haru/). On August 16, 2021, the Society of Automotive Engineers of Japan paper was published on the J-STAGE website (https://www.jstage.jst.go.jp/article/jsaeronbun/52/5/52_20214640/_pdf/-char/ja).

The present invention relates to an observation information processing method, an observation information processing device, and a recording medium.

Object recognition using LiDAR (Laser Imaging Detection and Ranging) uses a technique called EOT (Extended Object Tracking), which predicts the shape of a target object from a cloud of observation points and tracks the target object. Patent Document 1 discloses a tracking device that reduces tracking loss of a target object using EOT.

International Publication No. 2021/075581

However, the technology disclosed in Patent Document 1 is unable to prevent errors from occurring in tracking during EOT when the size of the target object as seen by the observer changes due to a change in the relative positional relationship between the observer and the target. For example, this occurs when a target object that was partially hidden behind an object emerges from the shadow, or when the target approaches a stationary observer.

The present invention therefore provides an observation information processing method and the like that can suppress errors in tracking in EOT, even when the size of the target object as seen by the observer changes due to a change in the relative positional relationship between the observer and the target.

An observation information processing method according to one aspect of the present invention is an observation information processing method using extended object tracking, and includes an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is the subject observing the target object; an adjustment step of generating a third contour based on the first contour and the second contour so that a size of a first contour, which is the contour of the target object estimated from the first observation information, and a size of a second contour, which is the contour estimated from the second observation information, are the same; and a projection step of projecting the first point of interest on the first contour as the second point of interest onto the third contour so that the relative position of the first point of interest on the first side, which is the side of the first contour on which the first point of interest is located, is equal to the relative position of the second point of interest on the second side, which is the side of the third contour on which the second point of interest is located, and projecting the second point of interest on the third contour as the third point of interest onto the second contour so that the relative position of the second point of interest on the second side, which is the side of the third contour on which the third point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour on which the third point of interest is located.

An observation information processing device according to one aspect of the present invention is an observation information processing method using extended object tracking, which includes an acquisition step of acquiring first observation information (observation information) acquired at a first point in time and second observation information acquired at a second point in time different from the first point in time, the first observation information being obtained by observing a target object to be tracked from an observation body that is the subject observing the target object, and a step of adjusting the first contour and the previous contour so that the size of the first contour, which is the contour of the target object estimated from the first observation information, and the size of the second contour, which is the contour estimated from the second observation information, are the same. The method includes an adjustment step of generating the second contour into a third contour, and a projection step of projecting the first focus point on the first contour as the second focus point onto the third contour so that the relative position of the first focus point on a first side of the first contour on which the first focus point is located is equal to the relative position of the second focus point on a second side of the third contour on which the second focus point is located, and in the projection step, the third contour onto which the second focus point is projected is changed to the size of the second contour while maintaining the relative position of the second focus point on the third contour.

A recording medium according to one aspect of the present invention is a non-transitory computer-readable recording medium on which a program for causing a computer to execute an observation information processing method according to one aspect of the present invention is recorded.

An observation information processing method according to one aspect of the present invention can suppress tracking errors in EOT even when the size of the target object changes as seen by the observer.

FIG. 1 is a block diagram of an observation information processing apparatus according to an embodiment. FIG. 2 is a diagram showing an example of an observation object equipped with LiDAR. FIG. 3 is a diagram showing the concept of contour prediction in EOT. FIG. 4 is a diagram showing an example of prediction of the contour of a target object in EOT. FIG. 5 is a diagram showing an example of the visible range of a target object by LiDAR. FIG. 6 is a diagram showing an example of a change in the shape of the contour of a target object in EOT. FIG. 7 is a diagram showing an example of projection of a point of interest in a conventional EOT and an example of projection of a point of interest in an ideal EOT. FIG. 8 is a diagram showing a method for projecting a point of interest in a conventional EOT and a method for projecting a point of interest according to the embodiment. FIG. 9 is a diagram showing another method of projecting a point of interest according to an embodiment. FIG. 10 is a diagram showing an example of projection of a point of interest in the EOT according to the embodiment. FIG. 11 is a flowchart showing an observation information processing method according to an embodiment.

The following describes the embodiments in detail, with reference to the drawings.

The embodiments described below each represent a preferred example of the present invention. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not recited in the independent claims that represent the highest concept of the present invention will be described as optional components that constitute a more preferred embodiment. Note that identical components will be assigned the same reference numerals, and descriptions may be omitted.

(Embodiment)
Below, we will explain an observation information processing method, observation information processing device, and recording medium that can suppress errors in tracking in EOT even when the size of the target object as seen by the observing subject changes due to a change in the relative positional relationship between the observing subject and the target, even though the size of the target object does not actually change.

[Outline of observation information processing device]
First, an observation information processing device 1 that realizes an observation information processing method according to an embodiment will be described. Fig. 1 is a block diagram of the observation information processing device 1 according to an embodiment.

The observation information processing device 1 includes a control unit 11 and a memory unit 12. The control unit 11 is implemented by a processor such as a CPU (Central Processing Unit) and memory. The memory unit 12 is implemented by memory such as ROM, RAM, and a hard disk. In the observation information processing device 1, the control unit 11 executes programs stored in the memory unit 12, thereby functioning as an acquisition unit 11a, an adjustment unit 11b, and a projection unit 11c. The operations of the acquisition unit 11a, adjustment unit 11b, and projection unit 11c provided in the control unit 11, which will be described below, are realized by the control unit 11 using the memory and processor.

The acquisition unit 11 acquires data acquired by LiDAR. LiDAR measures the distance between a target object and an observation body equipped with LiDAR equipment by irradiating the target object with laser light and acquiring the laser light reflected from the target object. The data acquired by LiDAR includes first observation information acquired at a first point in time and second observation information acquired at a second point in time different from the first point in time. Here, the first observation information and second observation information include observation points obtained by observing the target object to be tracked in EOT from the observation body that is the entity observing the target object. An "observation point" is a point on the target object obtained by observing the target object. Here, the acquisition unit 11 acquires data from LiDAR, but this is not limited to this. Data may be acquired from any device that can measure the distance to the target object, such as a laser sensor other than LiDAR, an ultrasonic sensor, or a camera.

The adjustment unit 12 generates third contours 300ac and 300bc (see FIG. 8 ; details will be described later) by changing the sizes of the first contour 300a (see FIG. 8 ; details will be described later), which is the contour of the target object estimated from the observation points included in the first observation information, and the second contour 300b (see FIG. 8 ; details will be described later), which is the contour estimated from the observation points included in the second image, so that these contours are the same. In other words, the adjustment unit 12 changes the sizes of the first contour 300a and the second contour 300b to the sizes of the third contours 300ac and 300bc that are different from the sizes of the first contour 300a and the second contour 300b. Note that the adjustment unit 12 may change the size of the first contour 300a to the size of the second contour 300b, or may change the size of the second contour 300b to the size of the first contour 300a. In addition, the size of the first contour 300a and the second contour 300b may be the same as the size of the third contour 300ac and 300bc.

The adjustment unit 12 may estimate the first contour 300a and the second contour 300b from the first image and the second image acquired by the acquisition unit 11, respectively. The above estimation may be performed using a general EOT method.

The projection unit 13 projects the first attention point as a second attention point onto the third contour 300ac so that the relative position of the first attention point on the first side, which is the side of the first contour 300a on which the first attention point is located, is equal to the relative position of the second attention point on the second side, which is the side of the third contour 300ac on which the second attention point is located. The projection unit 13 then projects the second attention point as a third attention point onto the second contour 300b so that the relative position of the second attention point on the second side, which is the side of the third contour 300bc on which the second attention point is located, is equal to the relative position of the third attention point on the third side, which is the side of the second contour 300b on which the third attention point is located. Note that a "attention point" is a point located at a characteristic location on the target object that is useful for reproducing the contour. It is preferable to select characteristic points from areas other than blind spots, and to increase the accuracy of contour reproduction, it is preferable to select at least one of the following: the nearest point closest to the observation body, one or more edge points of the visible area visible from the observation body, the midpoint between the nearest point and one or more edge points of the visible area, or a point near said midpoint. In particular, if only one point is used as a characteristic point, the nearest point is preferable.

Note that "projecting" means determining the position of a point of interest on one contour that corresponds to a point of interest on another contour.

Specifically, the projection unit 13 may project the first point of interest onto the third contour 300ac as the second point of interest so that the ratio of the length from the end of the first side (the side of the first contour 300a on which the first point of interest is located) to the first point of interest to the length of the first side is equal to the ratio of the length from the end of the second side (the side of the third contour 300ac on which the second point of interest is located) to the second point of interest to the length of the second side. The projection unit 13 may then project the second point of interest onto the second contour 300b as the third point of interest so that the ratio of the length from the end of the second side (the side of the third contour 300ac on which the second point of interest is located) to the second point of interest to the length of the second side is equal to the ratio of the length from the end of the third side (the side of the third contour 300b on which the third point of interest is located) to the third point of interest to the length of the third side.

The memory unit 15 stores data acquired or generated by the acquisition unit 11, the adjustment unit 12, and the projection unit 13. For example, the memory unit 15 may store data on the first and second images acquired by the acquisition unit 11, data on the third contour 300ac etc. generated by the adjustment unit 12, and data on the second attention point on the third contour 300ac and the third attention point on the second contour 300b projected by the projection unit 13.

The observation information processing device 1 acquires data observed by the observation body 100 and processes the observation information. Figure 2 is a diagram showing an example of an observation body 100 equipped with LiDAR. The observation body 100 is, for example, a four-wheeled vehicle. The observation body 100 is equipped with LiDAR on its upper surface. While traveling, the observation body 100 uses the LiDAR to measure the distance to the target object being observed.

Note that the observation body 100 is not limited to a four-wheeled vehicle as shown in Figure 2. It may also be a small four-wheeled vehicle or a large four-wheeled vehicle such as a truck. The observation body 100 may also be a two-wheeled vehicle, a three-wheeled vehicle, or a robot. Alternatively, the observation body 100 may also be a ship. The observation body 100 does not have to be a moving object, but may also be an object that is fixed in one place, such as a support pole.

[Object Recognition by EOT]
Next, object recognition using EOT will be described. FIG. 3 is a diagram showing the concept of predicting a contour 300 in EOT. In EOT, an observation body 100 (not shown) observes a target object 200 and acquires observation points on the target object 200. The observation information processing device 1 then predicts (i.e., generates) a contour 300 of the target object 200 from the observation points on the target object 200. Note that the contour 300 is a contour in a bird's-eye view of the target object 200 viewed from above. For example, the observation information processing device 1 performs contour prediction of the target object 200 as shown in FIG. 4.

Figure 4 is a diagram showing an example of a prediction of the contour 300 (300a, 300b) of a target object 200 (not shown) in EOT. The observed object 100 is, for example, a four-wheeled vehicle traveling on a road, and the target object 200 is an oncoming vehicle. As the relative position of the observed object 100 and the target object 200 changes, the shape of the target object 200 as seen from the observed object 100 changes. The first contour 300a is the contour of the target object 200 at time t, and the contour of the target object 200 at time t', which is a time after time t, is the second contour 300b. In this way, the contour 300 of the target object 200 changes as the relative position of the observed object 100 and the target object 200 changes. In EOT, the changing shape of the target object 200 as seen from the observed object 100 is tracked as the contour 300 of the same object.

Figure 5 is a diagram showing an example of the visible range of a target object using LiDAR. It shows how the visible range of the target object 200 from the observation body 100 changes as the relative position of the observation body 100 and the target object 200 changes. (a) of Figure 5 shows how only a portion of the target object 200 (the portion drawn with a solid line) can be seen from the observation body 100. (b) of Figure 5 shows how the entire length of the target object 200 can be seen from the observation body 100 as the distance between the observation body 100 and the target object 200 decreases. Therefore, as shown in (a) and (b) of Figure 5, when the relative position of the observation body 100 and the target object 200 changes, the shape of the target object 200 as seen from the observation body 100 changes.

Figure 6 is a diagram showing an example of the shape change of the contour 300 of the target object 200 in EOT. Areas that are blind spots from the perspective of the observation body 100 (not shown), such as areas that are obscured by buildings, are shown as hatched areas in Figure 6. As the target object 200 moves from an area that is blind spots from the perspective of the observation body 100 to an area that is not blind spots from the perspective of the observation body 100, the size of the contour 300 of the target object 200 as seen by the observation body 100 expands as shown in Figures 6(a) and 6(b). Specifically, in Figure 6(a), a blind spot of approximately half of the target object 200 occurs, so only a portion (approximately half) of the target object 200 is visible from the observation body 100. Therefore, the contour 300 of the target object 200 becomes approximately half the contour of the target object 200. Then, for example, a focus point 400 is placed at the upper left corner of the contour 300. In Figure 6(b), the range of the target object 200 visible from the observed body 100 expands, and the shape of the contour 300 becomes larger than in Figure 6(a). In this case, the position of the attention point 400 (Figure 6(a)) at a predetermined position on the contour 300, which was set on the contour 300 before the size as seen from the observed body 100 changed, must also be set on the contour 300 (Figure 6(b)) after the size as seen from the observed body 100 has changed, so that it indicates the same characteristic of the contour, for example, a point on a corner.

[Problems with conventional EOT]
Next, problems with the conventional EOT will be described. Fig. 7 is a diagram showing an example of projection of a point of interest 400 in the conventional EOT and an example of projection of the point of interest 400 in the ideal EOT. (a) and (b) of Fig. 7 show an example of projection of a point of interest in the conventional EOT, and (c) of Fig. 7 shows an example of projection of the point of interest 400 in the ideal EOT.

The attention point 400 is a point that indicates a characteristic location on the contour 300 (not shown), and it is desirable that it be projected onto the changed contour 300 as seen from the observation body 100 so that it indicates the same characteristic location on the contour 300 as seen from the observation body 100, even if the shape of the contour 300 as seen from the observation body 100 changes.

However, in the past, for example, the position of the center line CL of the first contour 300a at time t (as shown in FIG. 7(a)) was aligned with the position of the center line CL' of the second contour 300b at time t' (as shown in FIG. 7(b)), and the point of interest on the first contour 300a was projected onto the second contour 300b at the same position as on the first contour 300a. Specifically, the distance from the center line CL of the point of interest 400 on the first contour 300a is the same as the distance from the center line CL' of the point of interest 400 on the second contour 300b. In other words, in the past, EOT projected the point of interest to a position from the center line CL' of the second contour 300b the same distance as the distance from the center line CL of the first contour 300a to the point of interest. As a result, the position of the point of interest 400, which should be projected onto a characteristic location (here, the upper left corner) as shown in FIG. 7(c), was conventionally projected onto the position shown in FIG. 7(b). In this case, the estimated shape changes depending on the visible area from the observation object, so when reproducing the contour, the target object may be reproduced smaller than its actual size or the shape may be reproduced incorrectly.

In other words, the conventional attention point 400 shown in Figure 7(b) is not located in a characteristic part of the target object 200 (here, the upper left corner), and therefore is not suitable as an attention point that is useful for reproducing the visible contour of the target object 200. As a result, in conventional observation information processing devices, the attention point is not located in a characteristic part of the target object 200, which causes problems such as an inaccurate contour reproduction, and the target object cannot be tracked stably.

[Point of interest projection method]
Therefore, in the observation information processing method according to the embodiment, the attention point 400 is projected in the following manner. The observation information processing method according to the embodiment is realized by an observation information processing device 1 according to the embodiment. Fig. 8 is a diagram showing a method of projecting the attention point 400 in a conventional EOT (Fig. 8(a)) and a method of projecting the attention point according to the embodiment (Fig. 8(b)).

Figure 8(a) shows a method for projecting a point of interest in a conventional EOT. As shown in Figure 8(a), in the conventional method for projecting a point of interest in an EOT, the center line of the first contour 300a at time t is aligned with the center line of the second contour 300b at time t', and the position of the point of interest on the first contour 300a at time t is projected directly onto the second contour 300b at time t'.

In contrast, the projection method of the attention point 400 according to the embodiment shown in Figure 8(b) is processed as follows. Here, the first contour 300a shown in Figure 8(b) is an observed contour that is an observed value at the current time t', and the second contour 300b is a predicted contour at the current time t' predicted from an estimated contour (described below) that is an estimated value at a time t prior to the current time t'.

First, the adjustment unit 12 normalizes the size of the first contour 300a at time t and the size of the second contour 300b at time t' to generate the third contour 300ac and the third contour 300bc, respectively (see (b-1) to (b-2) in Figure 8). Here, "normalizing" means setting the size of the first contour 300a and the second contour 300b to a default size. In this embodiment, the third contours 300ac and 300bc are set to squares with a 1:1 ratio. This normalization process allows the projection unit 13 to project a point of interest at a characteristic location on the first contour 300a onto the corresponding characteristic location on the second contour 300b.

At this time, the projection unit 13 projects the position of the attention point 400 on the first contour 300a onto the third contour 300ac (i.e., the upper right corner) while maintaining the attention point 400 on the first contour 300a located at a characteristic point (here, the upper right corner) ((b-2) of Figure 8). Here, the third contours 300ac and 300bc are contours generated by changing the size of the first contour 300a and the second contour 300b themselves. However, the third contours 300ac and 300bc may also be contours generated separately from the first contour 300a and the second contour 300b.

On the other hand, with regard to the point of interest on the second contour 300b, the projection unit 13 normalizes the second contour 300b and then determines the point at the same position as the point of interest 400 on the third contour 300ac, which is the observed contour, as the position of the point of interest 400 on the third contour 300bc ((b-3) in Figure 8). Then, while maintaining the relative position of the point of interest 400 on the third contour 300bc, the size of the third contour 300bc is enlarged or reduced to change it to the size before normalization, i.e., the second contour 300b ((b-4) in Figure 8). Here, the second contour 300b that has been changed to the size before normalization after the normalization process is referred to as the estimated contour.

When the third contours 300ac and 300bc are generated separately from the first contour 300a and the second contour 300b, the focus point 400 is projected onto the original second contour 300b (i.e., the upper right corner) while maintaining the relative position of the focus point 400 on the third contour 300bc (maintaining the characteristic points).

In this way, in this embodiment, the second contour 300b, which is an estimated value at the current time t', is determined from the third contour 300ac ((b-2) in Figure 8), which is obtained by normalizing the first contour 300a, which is the observed value at the current time t', and the second contour 300b (estimated contour) ((b-4) in Figure 8), which is the predicted value at the current time t' predicted from the previous time t, normalized and then returned to its pre-normalization size.

In the above embodiment, the second contour 300b is normalized before projecting the attention point 400, but this is not limited to this embodiment.

For example, the projection unit 13 first projects the first contour 300a onto the second contour 300b at the same position as the position of the attention point 400 on the first contour 300a, with the center lines of the first contour 300a and the second contour 300b aligned, as in the conventional case (Figure 8(a)).

After performing the normalization process described above, the projection unit 13 projects the position of the attention point 400 on the third contour 300ac (i.e., the upper right corner) onto the third contour 300bc (i.e., the upper right corner) ((b-3) in Figure 8). At this time, the attention point 400 that was on the third contour 300bc before the projection is deleted.

This allows the projection unit 13 to set the point of interest at a position on the predicted contour that reflects the relative position of the point of interest on the observed contour.

Finally, the adjustment unit 12 changes (i.e., restores) the size of the third contour 300bc to the size of the second contour 300b, and the projection unit 13 projects the attention point 400 on the third contour 300bc onto the attention point 400 on the second contour 300b while maintaining the position of the attention point 400 projected onto the third contour 300bc at the characteristic location (i.e., the upper right corner) ((b-4) in Figure 8).

While the above example employs a point of interest at a corner of a contour as a characteristic location, the point of interest may also be a point on a side that constitutes the contour. In this case, the projection unit 13 projects the point of interest onto the third contour 300bc so that the ratio of the length from the end of the side of the first contour 300a before normalization on which the point of interest is located to the length of the side is equal to the ratio of the length from the end of the side of the third contour 300ac on which the point of interest is located to the length of the side. The projection unit 13 may then project the point of interest onto the second contour so that the ratio of the length from the end of the side of the third contour on which the point of interest is located to the length of the side of the third contour 300bc on which the point of interest is located to the length of the side of the second contour 300b on which the point of interest is located to the length of the side of the second contour is equal. As a result, the point of interest on the pre-normalized contour is projected onto the third contour 300bc at the same relative position to the contour as it was before normalization (i.e., while remaining located at the same characteristic point).

Figure 9 is a diagram showing another method of projecting a point of interest according to an embodiment. The projection unit 13 may determine the distance from the center point of the pre-normalized contour to a location on the side of the pre-normalized contour and the angle in a predetermined direction from the center point by enlarging or reducing the pre-normalized contour by a factor by which the pre-normalized contour is enlarged or reduced to the third contour 300ac, thereby determining the distance from the center point of the pre-normalized contour to a location on the side of the third contour 300bc and the angle in a predetermined direction from the center point. The projection unit 13 may then determine the distance from the center point of the pre-normalized contour to a location on the side of the third contour 300bc and the angle in a predetermined direction from the center point by enlarging or reducing the pre-normalized contour to a location on the side of the third contour 300ac by a factor by which the third contour 300ac is enlarged or reduced to the pre-normalized contour.

Specifically, the point of interest 400 on the first contour 300a is projected onto the third contour 300ac at a distance d2 and angle α2 obtained by magnifying the distance d1 from the center point 500 of the first contour 300a to the location of the point of interest 400 on the first contour 300a and the angle α1 in a predetermined direction as viewed from the center point 500 by the magnification factor at which the first contour 300a is magnified onto the third contour 300ac ((a-1) to (a-2) in Figure 9).

Then, the second contour 300b is reduced to the size of the third contour 300bc, which is the same size as the third contour 300ac, and the point of interest 400 on the contour 300ac is projected onto the contour 300bc (see (a-2) of Figure 9). Then, the size of the third contour 300bc is enlarged to the size of the second contour 300b, and the point of interest 400 is projected onto the second contour 300b at a distance d3 and angle α3 from the center point 500 (see (a-3) of Figure 9).

That is, first, the distance from the center point 500 of the first contour 300a to the location of the point of interest on the first contour 300a is defined as the first distance d1, and the angle between a line L1 extending in a predetermined direction as viewed from the center point 500 and a line L2 extending in the X-axis direction is defined as the first angle α1. The point of interest 400 on the first contour 300a is projected onto a position on the third contour 300ac determined by the second distance d2 and second angle α2 obtained by changing the first distance d1 and first angle α1 by the magnification used to change the first contour 300a to the third contour 300ac.

Then, the point of interest 400 is projected onto a position on the third contour 300bc, which is obtained by normalizing the second contour 300b, determined by the second angle α2 between a line L1 extending in a predetermined direction as viewed from the center point 500 and a line L2 extending in the X-axis direction. Next, the point of interest 400 is projected onto a position on the second contour 300b determined by the third distance d3 and third angle α3, which are obtained by enlarging or reducing the second distance d2 and second angle α2 by a magnification factor that changes the size of the third contour 300b to that of the second contour 300b.

As a result, the point of interest on the pre-normalized contour is projected onto the third contour 300bc at the same relative position as it was relative to the pre-normalized contour (i.e., while remaining located at the same characteristic point). In other words, the position of the point of interest 400 on the contour 300 is determined.

An example of the above-mentioned method for projecting points of interest will now be described. Figure 10 is a diagram showing an example of projecting points of interest in EOT according to an embodiment. This shows an example in which the shape of a target object as seen from an observer at a past time is enlarged to match the shape of the target object as seen from an observer at the current time.

(a) of Figure 10 shows the shape of the target object 200 as seen from the observation body 100 (not shown) at a past time. Assume that a point of interest 400 is located in the upper left corner of the target object 200. (c) of Figure 10 shows the shape of the target object 200 as seen from the observation body 100 at the current time. (b) of Figure 10 shows an enlarged view of the shape of the target object 200 as seen from the observation body 100 at a past time ((a) of Figure 10) to match the shape of the target object 200 as seen from the observation body 100 at the current time ((c) of Figure 10). In this case, as shown in (c) of Figure 10, the position of the point of interest 400 at the upper left corner of the target object 200 on the enlarged target object 200 is determined in accordance with changes in the shape of the target object 200 as seen from the observation body 100, without changing its relative position on the target object 200. In other words, even when the shape is enlarged in this way, the point of interest 400 remains located at the same characteristic location. This prevents the relative position of the attention point 400 with respect to the target object 200 from changing, suppresses errors when reproducing the contour of the target object 200 using the attention point 400, and as a result, the target object 200 can be tracked stably.

Next, the observation information processing method according to the embodiment will be described in detail. Figure 11 is a flowchart showing the observation information processing method according to the embodiment.

First, the acquisition unit 11 acquires the first observation information and the second observation information (step S10). Specifically, the acquisition unit 11 acquires, from the storage unit 12 or the like, the observation information generated from data obtained by an observation body 100 equipped with LiDAR observing the target object 200.

The first observation information is observation information at a first time point and includes an observation point on the target object 200 as seen from the observing body 100 at the first time point. The second observation information is observation information at a time point different from the first time point and includes an observation point on the target object 200 as seen from the observing body 100 at the second time point. The observation point is a point on the target object 200 that the observing body 100 recognizes by observing the target object 200. For example, the observation point is a point at which laser light emitted from the LiDAR is reflected on the target object 200.

Next, adjustment unit 12 generates third contours 300ac and 300bc so that first contour 300a, which is the contour of target object 200 estimated from the observation points included in the first observation information, and second contour 300b, which is the contour of target object 200 estimated from the observation points included in the second observation information, are the same in size (step S11). Specifically, adjustment unit 12 generates first contour 300a, which is the contour 300 of target object 200, from the observation points included in the first observation information, and generates second contour 300b, which is the contour 300 of target object 200, from the observation points included in the second observation information. Then, adjustment unit 12 changes the sizes of first contour 300a and second contour 300b so that the generated first contour 300a and second contour 300b are the same in size, thereby generating third contours 300ac and 300bc. In other words, the third contours 300ac and 300bc are contours generated separately and at different sizes from the first contour 300a and the second contour 300b. Note that the third contours 300ac and 300bc may be the same size as the first contour 300a or the second contour 300b.

Next, the projection unit 13 projects the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side of the first contour 300a is equal to the relative position of the second point of interest on the second side of the third contour 300bc that corresponds to the first side, i.e., so that the first and second points of interest are located at the same characteristic location on the contour (step S12). Specifically, the projection unit 13 projects the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located is equal to the relative position of the second point of interest on the second side of the third contour 300bc that corresponds to the second point of interest on which the second point of interest is located. For example, the projection unit 13 may project the first point of interest on the first contour 300a as the second point of interest onto the third contour 300bc so that the ratio of the length from the end of the first side to the position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located to the length of the first side is equal to the ratio of the length from the end of the second side to the position of the second point of interest on the second side of the corresponding side of the third contour 300bc on which the second point of interest is located to the length of the second side.

The projection unit 13 then projects the second point of interest on the third contour 300bc onto the second contour 300b as the third point of interest so that the relative position of the second point of interest on the second side of the third contour 300bc is equal to the relative position of the third point of interest on the third side of the second contour 300b, i.e., so that the second and third points of interest are located at the same characteristic location on the contour (step S13). Specifically, the projection unit 13 projects the second point of interest on the third contour 300bc onto the second contour 300b as the third point of interest so that the relative position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located is equal to the relative position of the third point of interest on the third side of the second contour 300b on which the third point of interest is located. For example, the projection unit 13 may project the second point of interest on the third contour 300bc as the third point of interest onto the second contour 300b so that the ratio of the length from the end of the second side, which is the side of the third contour 300bc on which the second point of interest is located, to the length of the second side is equal to the ratio of the length from the end of the third side, which is the side of the second contour 300b on which the third point of interest is located, to the length of the third side.

Furthermore, when the third contours 300ac and 300bc are the same size as the first contour 300a or the second contour 300b, the projection unit 13 may project the first point of interest on the first contour 300a as the third point of interest onto the second contour 300b so that the relative position of the first point of interest on the first side, which is the side of the first contour 300a on which the first point of interest is located, is equal to the relative position of the second point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located.

Specifically, the projection unit 13 may project the first point of interest on the first contour 300a as the third point of interest onto the second contour 300b so that the ratio of the length from the end of the first side to the position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located to the length of the first side is equal to the ratio of the length from the end of the third side to the position of the third point of interest on the third side of the second contour 300b on which the third point of interest is located to the length of the third side.

Alternatively, the projection unit 13 may project the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the ratio of the angle and distance from the center point of the first contour 300a to the position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located, to the angle and distance from the center point of the third contour 300bc to the position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located, is equal to the ratio of the lengths of the first side and the second side.

The projection unit 13 may then project the second point of interest on the third contour 300bc onto the second contour 300b as the third point of interest so that the ratio of the angle and distance from the X-axis from the center point of the third contour 300bc to the position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located, to the angle and distance from the center point of the second contour 300b to the position of the third point of interest on the third side of the second contour 300b on which the third point of interest is located, is equal to the ratio of the lengths of the second side and the third side.

[Reducing the number of observation points used]
Next, a reduction in the number of observation points will be described as a technique useful for improving the real-time performance of processing. The observation information processing device 1 uses a second number of observation points, which is less than the first number, from among a first number of observation points obtained by observing the target object 200. In other words, the observation information processing device 1 acquires multiple observation points on the target object 200 using LiDAR, but uses a fewer number of observation points than the acquired observation points for predicting the contour 300 in the first observation information and second observation information acquired by the acquisition unit 11.

This allows the observation information processing device 1 to reduce the processing load and shorten the processing time. According to a simulation, the observation information processing device 1 can reduce the average processing time by 42.9%. In this simulation, the number of observation points is selected from 1, 3, or 5, depending on the size of the contour of the observed object. The observation points are extracted from the points that make up the contour and are within the range observable from the observation body 100. Specifically, of the points on the contour observed from the observation body, the points closest to the observation body 100, the points at both ends of the contour, and the midpoints between them are selected. In other words, the observation points are selected evenly from positions on the contour observed from the observation body 100. Note that the larger the contour of the observed object, the greater the number of observation points selected.

The simulation is performed on a PC equipped with a 4-core CPU with 4.2 GHz each and 32 GB of memory. The LiDAR frame rate is 10 Hz, and each frame is processed within 100 msec. There are approximately 100 objects to be tracked, and the simulation is performed using data measured by an observation body 100 traveling through an actual urban area (for example, Odaiba in Tokyo).

Furthermore, when the observation information processing device 1 observes multiple target objects 200 and obtains multiple observation points, the third number of observation points used to determine which of the multiple target objects 200 each observation point corresponds to may be fewer than the fourth number of observation points used in the process of projecting each of the multiple third contour attention points on the third contour 300bc onto the second contour 300b as multiple second contour attention points.

For example, in the process of determining (association) which of the multiple target objects 200 multiple observation points on multiple target objects 200 acquired by LiDAR correspond to, the observation information processing device 1 may use, for example, only one of the multiple observation points. Note that the number of observation points used may be any number as long as it is less than the number of observation points used for updating, which will be described later. Specifically, when determining which of the multiple observed contours the predicted contour corresponds to, the observation information processing device 1 may associate one point on the observed contour with one point on the predicted contour, thereby associating one of the multiple observed contours with the predicted contour.

The observation information processing device 1 may use, for example, five of the multiple observation points in the process of updating (updating) the estimated value of the contour 300 at the current time from the associated predicted contour and observed contour. Note that the number of observation points used may be any number greater than or equal to the number of observation points used for the correspondence.

Specifically, when the observation information processing device 1 estimates the contour 300 at the current time from the observed contour and the predicted contour, it may associate five points on the observed contour with five points on the predicted contour.

This allows the observation information processing device 1 to reduce the processing load and shorten the processing time. According to a simulation, the observation information processing device 1 can reduce the average processing time by 29.7%. In this simulation, of the observation points observed from the observation body 100, only one nearest point is used as the point of interest, and correspondence with points on the predicted contour is performed. Note that two or more points of interest may be used for correspondence with points on the predicted contour.

The simulation is performed on a PC equipped with a 4-core CPU with 4.2 GHz each and 32 GB of memory. The LiDAR frame rate is 10 Hz, and each frame is processed within 100 msec. There are approximately 100 objects to be tracked, and the simulation is performed using data measured by an observation body 100 traveling through an actual urban area (for example, Odaiba in Tokyo).

Furthermore, in the process of matching and updating observation points, observation points that are not observed by the observation body 100 are not used. This makes it possible to exclude observation points where observations are unstable and may contain noise from the prediction process.

By reducing the number of observation points used as described above, the observation information processing device 1 enables real-time processing even while a vehicle, etc., is traveling on a public road. For example, it can stably and accurately reproduce the contours of a moving object and track it dozens of times per second or more.

[Effects, etc.]
An observation information processing method using extended object tracking, comprising: an acquisition step of acquiring first observation information acquired at a first time point, the first observation information including observation points obtained by observing a target object (200) to be tracked from an observation body (100) that is a subject observing the target object (200), and second observation information acquired at a second time point different from the first time point, the second observation information including the observation points; and generating a third contour (300bc) by changing the sizes of first contour (300a), which is a contour (300) of the target object (200) estimated from the observation points included in the first observation information, and second contour (300b), which is a contour estimated from the observation points included in the second observation information, so that the sizes of first contour (300a) and second contour (300b) are the same. and a projection step of projecting the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side, which is the side of the first contour 300a on which the first point of interest is located, is equal to the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, and projecting the second point of interest on the third contour 300bc as a third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located.

As a result, even if the size of the contour 300 of the target object 200 changes due to a change in the relative positional relationship between the observing subject and the target, the observation information processing method according to the embodiment can project the point of interest, which is useful for reproducing the contour, onto the changed contour 300 without changing the relative position of the point of interest with respect to the contour 300 before and after the change in contour 300, i.e., while maintaining the point of interest located at the same characteristic location. Therefore, according to the observation information processing method according to the embodiment, even if the size of the target object 200 changes as seen by the observing subject, the contour of the target object 200 can be accurately reproduced, errors in tracking in EOT can be suppressed, and the target object can be stably tracked. Therefore, by applying this method to automobile driving control, accurate and robust autonomous driving and following driving can be achieved.

Also, for example, in the observation information processing method according to the embodiment, the first point of interest is a point corresponding to an observation point included in the first observation information.

As a result, the observation information processing method according to the embodiment can use points of interest from among the observation points of the target object 200 that are useful for reproducing the contours of the target object.

Furthermore, for example, in an observation information processing method according to an embodiment, the first observation information includes a plurality of observation points, each of which is a plurality of first contour attention points including a first attention point projected onto the first contour 300a, and the relative position of the first attention point on the first side, which is the side of the first contour 300a on which the first attention point is located, is equal to the relative position of the second attention point on the second side, which is the side of the third contour 300bc on which the second attention point is located. The first point of interest on the first contour 300a is projected onto the third contour 300bc as a second point of interest, and the second point of interest on the third contour 300bc is projected onto the second contour 300b as a third point of interest so that the relative position of the second point of interest on the second side, which is the side of the third contour 300bc on which the second point of interest is located, is equal to the relative position of the third point of interest on the third side, which is the side of the second contour 300b on which the third point of interest is located.

As a result, the observation information processing method according to the embodiment can accurately track the target object 200 using multiple points of interest. Therefore, the observation information processing method according to the embodiment can track the target object 200 with higher accuracy.

Furthermore, for example, in the observation information processing method according to the embodiment, the multiple observation points include, among the observation points, a point located closest to the observation body 100, a point located at the rightmost point as seen from the observation body 100, a point located at the leftmost point as seen from the observation body 100, and a point located midway between the rightmost and leftmost points.

As a result, the observation information processing method according to the embodiment can use points of interest located at distinctive locations on the target object 200. Therefore, the observation information processing method according to the embodiment can reproduce accurate contours and track the target object 200 with greater accuracy.

Furthermore, for example, the observation information processing method according to the embodiment uses a second number of observation points, which is less than the first number, from among a first number of observation points obtained by observing the target object 200.

As a result, the observation information processing method according to the embodiment can speed up processing and improve real-time performance by using a smaller number of observation points than conventional methods to track the target object 200.

Furthermore, for example, in the observation information processing method according to the embodiment, when multiple target objects 200 are observed and multiple observation points are obtained, the third number of observation points used in the process of determining which of the multiple target objects 200 each observation point corresponds to is less than the fourth number of observation points used in the process of projecting each of the multiple third contour attention points on the third contour 300bc onto the second contour 300b as multiple second contour attention points.

As a result, the observation information processing method according to the embodiment can speed up processing and improve real-time performance by using a smaller number of observation points than conventional methods to track the target object 200.

Furthermore, for example, an observation information processing device 1 according to an embodiment is an observation information processing device using extended object tracking, and includes an acquisition unit 11 that acquires first observation information acquired at a first time point, including observation points obtained by observing a target object 200 to be tracked from an observation body 100 that is the subject observing the target object 200, and second observation information acquired at a second time point different from the first time point, including the observation points; and a processing unit 12 that changes the sizes of the first contour 300a, which is the contour of the target object 200 estimated from the observation points included in the first observation information, and the second contour 300b, which is the contour estimated from the observation points included in the second observation information, so that the sizes of the first contour 300a and the second contour 300b are the same. The system includes an adjustment unit 12 that generates a third contour 300bc, and a projection unit 13 that projects the first point of interest on the first contour 300a as a second point of interest onto the third contour 300bc so that the relative position of the first point of interest on the first side of the first contour 300a on which the first point of interest is located is equal to the relative position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located, and projects the second point of interest on the third contour 300bc as a third point of interest onto the second contour 300b so that the relative position of the second point of interest on the second side of the third contour 300bc on which the second point of interest is located is equal to the relative position of the third point of interest on the third side of the second contour 300b on which the third point of interest is located.

As a result, the observation information processing device 1 according to the embodiment can achieve the same effects as the observation information processing method described above.

(others)
Although the embodiments have been described above, the present invention is not limited to the above embodiments. For example, in the above embodiments, the processing performed by a specific processing unit may be performed by another processing unit. Furthermore, the order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

Furthermore, in the above-described embodiments, each component may be realized by executing a software program appropriate for that component using various hardware resources. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Furthermore, each component may be realized by hardware that does not include a software program. For example, each component may be a circuit (or integrated circuit). These circuits may be configured as a single circuit as a whole, or may each be configured as a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

Furthermore, general or specific aspects of the present invention may be realized as a system, device, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM. Furthermore, general or specific aspects of the present invention may be realized as any combination of a system, device, method, integrated circuit, computer program, and recording medium.

For example, the present invention may be realized as a program for causing a computer to execute the observation information processing method of the above-described embodiment. The present invention may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

In addition, the present invention also includes forms obtained by applying various modifications to each embodiment that would occur to those skilled in the art, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the spirit of the present invention.

REFERENCE SIGNS LIST 1 observation information processing device 11a acquisition unit 11b adjustment unit 11c projection unit 11 control unit 12 storage unit 100 observation body 200 target object 300 contour 300a first contour 300b second contour 300ac, 300bc third contour

Claims (9)

1. A method for processing observation information using extended object tracking, comprising:
an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object;
an adjustment step of generating a third contour based on the first contour and the second contour so that a size of a first contour, which is a contour of the target object estimated from the first observation information, and a size of a second contour, which is a contour estimated from the second observation information, are the same;
projecting the first attention point on the first contour as the second attention point onto the third contour so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located;
a projection step of projecting the second attention point on the third contour as the third attention point onto the second contour so that a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located, is equal to a relative position of the third attention point on a third side, which is a side of the second contour on which the third attention point is located,
Observation information processing methods. the first point of interest is a point corresponding to an observation point included in the first observation information;
The observation information processing method according to claim 1 . the first observation information includes a plurality of observation points;
the plurality of observation points are a plurality of first contour attention points including the first attention point, which are projected onto the first contour;
projecting the first and second attention points onto the third contour so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located;
projecting the second attention point onto the second contour as the third attention point so that a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located, is equal to a relative position of the third attention point on a third side, which is a side of the second contour on which the third attention point is located;
3. The observation information processing method according to claim 1 or 2. The plurality of observation points include, among the observation points, a point closest to the observation body, a point at the right end as seen from the observation body, a point at the left end as seen from the observation body, and a point at the center between the right end point and the left end point.
The observation information processing method according to claim 3. using a second number of observation points, which is less than the first number, from the first number of observation points obtained by observing the target object;
3. The observation information processing method according to claim 1 or 2. When a plurality of target objects are observed and a plurality of observation points are obtained, the third number of observation points used in the process of determining which of the plurality of target objects each observation point corresponds to is less than the fourth number of observation points used in the process of projecting, onto the second contour, each of the plurality of third contour attention points on the third contour as a plurality of second contour attention points.
The observation information processing method according to claim 3. 1. A method for processing observation information using extended object tracking, comprising:
an acquisition step of acquiring first observation information acquired at a first time point and second observation information acquired at a second time point different from the first time point, the first observation information being obtained by observing a target object to be tracked from an observation body that is a subject observing the target object;
an adjustment step of generating a third contour from the first contour, which is a contour of the target object estimated from the first observation information, so that a size of the first contour and a size of the second contour, which is a contour estimated from the second observation information, are the same;
a projection step of projecting the first attention point on the first contour onto the third contour as the second attention point so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located,
In the projecting step, the third contour onto which the second attention point is projected is changed to a size of the second contour while maintaining a relative position of the second attention point on the third contour.
Observation information processing methods. An observation information processing device using extended object tracking,
a processor;
Memory and
an acquisition unit that acquires first observation information acquired at a first time point, the first observation information including an observation point obtained by observing a target object to be tracked from an observation body that is a subject observing the target object, and second observation information acquired at a second time point different from the first time point, the second observation information including the observation point;
an adjustment unit that generates a third contour by changing the sizes of the first contour and the second contour so that a size of a first contour, which is a contour of the target object inferred from the observation points included in the first observation information, and a size of a second contour, which is a contour inferred from the observation points included in the second observation information, are the same;
projecting the first attention point onto the third contour as the second attention point so that a relative position of the first attention point on a first side, which is a side of the first contour on which the first attention point is located, is equal to a relative position of the second attention point on a second side, which is a side of the third contour on which the second attention point is located;
a projection unit that projects the second attention point onto the second contour as the third attention point so that a relative position of the second attention point on a second side that is a side of the third contour on which the second attention point is located is equal to a relative position of the third attention point on a third side that is a side of the second contour on which the third attention point is located,
Observation information processing equipment. A non-transitory computer-readable recording medium on which a program for causing a computer to execute the observation information processing method described in claim 1 is recorded.