JP7701101B2 - Map generation device, map generation method, and program - Google Patents

Description

The present disclosure relates to a map generation device and a map generation method for generating an environmental map constructed from point cloud data, and further to a program for implementing these.

In recent years, with the improvement of sensing technology, SLAM (Simultaneous Localization and Mapping) has been attracting attention. SLAM is a technology in which a mobile object equipped with a sensor simultaneously estimates its own position and creates an environmental map (see, for example, Patent Document 1). With SLAM, a mobile object such as an autonomous robot does not need to move randomly, but can move along an autonomous driving map obtained from an environmental map, thereby improving the efficiency of its movement.

Specifically, the mobile body acquires image data output from the camera for each frame, searches for feature points in the latest frame that correspond to feature points extracted from past frames, and extracts pairs of feature points consisting of corresponding feature points. The mobile body then calculates a camera matrix for the latest frame using the pairs of feature points, and calculates, for each pair of feature points, the three-dimensional coordinates of the feature points using the camera matrix and the two-dimensional coordinates of the feature points in the frame.

After calculating the camera matrix, the mobile body calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the mobile body generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. The mobile body then generates a map for autonomous driving by converting the 3D environmental map into 2D, and performs autonomous driving using the map for autonomous driving.

In addition to cameras, sensors that can measure the distance (depth) to an object for each pixel of image data are also used. By using a depth sensor, the density of the point cloud that constitutes the environmental map can be increased compared to when a camera is used, enabling highly accurate autonomous driving. A specific example of a depth sensor is a LiDAR (light detection and ranging) sensor (see, for example, Patent Document 2).

Patent Document 2 discloses a three-dimensional LiDAR sensor. The three-dimensional LiDAR sensor disclosed in Patent Document 2 includes a light source, a light receiver, a mirror, and a control device. Laser light emitted from the light source is reflected by the mirror and then irradiated onto the object. The laser light reflected by the object then enters the light receiver. The control device calculates the depth to the object from the time it takes for the laser light to be emitted from the light source and to enter the light receiver.

In addition, the 3D LiDAR sensor disclosed in Patent Document 2 is a rotary 3D LiDAR sensor and also includes a rotation mechanism for rotating a mirror. Therefore, by using the 3D LiDAR sensor disclosed in Patent Document 2, scanning (searching for feature points) is possible in a range of 360 degrees or a range close to that.

JP 2020-76714 A JP 2021-124496 A

The above-mentioned rotating 3D LiDAR sensor is capable of scanning over a wide area, but it requires a rotation mechanism, which has the drawback of high manufacturing costs. For this reason, 3D LiDAR with a fixed angle of view that does not have a rotation mechanism (hereinafter referred to as "fixed angle of view 3D LiDAR") has also been proposed for SLAM.

However, with fixed-angle 3D LiDAR, because the angle of view is fixed, it is possible that in SLAM, parts of the latest frame do not overlap with previous frames. When this happens, an incorrect set of feature points is extracted, and an environment map with low accuracy is generated. This results in a significant decrease in the accuracy of autonomous driving.

One example of the objectives of this disclosure is to improve the accuracy of environmental maps when using a depth sensor with a fixed angle of view.

In order to achieve the above object, a map generating device according to one aspect of the present disclosure includes:
A data acquisition unit that acquires image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction unit that extracts feature points from the image data for each frame;
a filtering unit that determines, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, removes the feature points extracted from the non-overlapping portion;
an environment map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map composed of a collection of the feature points identified as pairs using a calculation result;
Equipped with
It is characterized by:

In order to achieve the above object, a map generating method according to one aspect of the present disclosure includes:
A data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction step of extracting feature points from the image data for each frame;
a filtering step of determining, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, eliminating the feature points extracted from the non-overlapping portion;
an environmental map generating step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map constituted by a set of the feature points identified as pairs using the calculation results;
having
It is characterized by:

Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure includes:
On the computer,
A data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction step of extracting feature points from the image data for each frame;
a filtering step of determining, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, eliminating the feature points extracted from the non-overlapping portion;
an environmental map generating step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map constituted by a set of the feature points identified as pairs using the calculation results;
The present invention is characterized in that:

As described above, according to the present disclosure, the accuracy of the environmental map can be improved when a depth sensor with a fixed angle of view is used.

FIG. 1 is a diagram showing a schematic configuration of a map generating device. FIG. 2 is a diagram showing a specific configuration of the map generating device. FIG. 3 is an explanatory diagram for explaining the angle of view of the depth sensor. FIG. 4 is a diagram showing a determination process in the filtering unit of the map generating device. FIG. 5 is a diagram showing an example of a three-dimensional environment map. FIG. 6 is a flow diagram showing the operation of the map generating device. FIG. 7 is a block diagram showing an example of a computer that realizes the map generating device.

(Embodiment)
A map generating device, a map generating method, and a program according to an embodiment will be described below with reference to FIGS.

[Device configuration]
First, the schematic configuration of the map generating device will be described with reference to Fig. 1. Fig. 1 is a diagram showing the schematic configuration of the map generating device.

The map generating device 10 shown in Fig. 1 is a device for generating an environmental map constructed from point cloud data. As shown in Fig. 1, the map generating device 10 includes a data acquisition unit 11, a feature point extraction unit 12, a filtering unit 13, and an environmental map generating unit 14.

The data acquisition unit 11 acquires image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis. The feature point extraction unit 12 extracts feature points from the image data acquired by the data acquisition unit 11 for each frame.

The filtering unit 13 determines, for each frame, whether or not there is a portion in the frame that does not overlap with other frames other than the frame itself. If the result of the determination indicates that there is such a portion, the filtering unit 13 eliminates the feature points extracted from the non-overlapping portion.

The environment map generating unit 14 identifies pairs of corresponding feature points between frames, and further calculates the three-dimensional coordinates of the feature points identified as pairs.The environment map generating unit 14 then uses the calculation results to generate an environment map composed of a collection of the feature points identified as pairs.

Thus, in the embodiment, the map generating device 10 excludes extracted feature points in parts of each frame that do not overlap with other frames. This avoids the extraction of an incorrect set of feature points, which occurs when the angle of view of the depth sensor is fixed. In other words, according to the embodiment, the accuracy of the environmental map can be improved when a depth sensor with a fixed angle of view is used.

Next, the configuration and functions of the map generating device in the embodiment will be specifically described with reference to Figures 2 and 3. Figure 2 is a configuration diagram specifically showing the configuration of the map generating device. Figure 3 is an explanatory diagram for explaining the angle of view of the depth sensor.

As shown in FIG. 2, in the embodiment, the map generating device 10 is mounted on a mobile body 100 capable of autonomous driving, such as a robot. In addition to the map generating device 10, the mobile body 100 includes a depth sensor 20, a control device 30, a steering device 40, and a power train 50.

In the embodiment, the depth sensor 20 is attached to the moving body 100 so as to capture an image in the direction of travel. The depth sensor 20 is a sensor capable of capturing images with depth and outputs the depth-capped image data at a set frame rate. In the example of FIG. 2, a fixed-angle 3D LiDAR is used as the depth sensor 20.

As shown in Figure 3, the angle of view of the depth sensor 20 is fixed, and its shooting area 21 changes depending on the orientation of the moving body 100. In the example of Figure 3, the change in the shooting area 21 from time (t-2) to time (t) is shown.

The control device 30 is constructed by a computer mounted on the moving body 100. The control device 30 controls the traveling direction and moving speed of the moving body 100 using the environmental map generated by the map generating device 10. For example, the control device 30 uses the environmental map to set a route from the present to the destination, determines the traveling direction and moving speed of the moving body 100 so that the moving body 100 moves along the set route, and controls the power train 50 and the steering device 40.

The powertrain 50 is composed of an electric motor for driving, a power transmission mechanism, etc. Tires, caterpillar tracks, etc. are connected to the powertrain 50. The powertrain 50 rotates the tires, caterpillar tracks, etc. in response to instructions from the control device 30.

The steering device 40 is equipped with a mechanism for controlling the direction of the steering wheels of the moving body 100. The steering device 40 determines the direction of the steering wheels in response to instructions from the control device 30. The steering device 40 may also be equipped with a mechanism for controlling the direction of movement by controlling the torque of the left and right drive wheels.

In an embodiment, the map generating device 10 is constructed on a computer mounted on the mobile body 100 by a program in an embodiment described below. In addition, the map generating device 10 may be constructed by a device (e.g., an electronic circuit, etc.) separate from the computer mounted on the mobile body 100.

In the embodiment, the data acquisition unit 11 acquires depth-based image data on a frame-by-frame basis from the depth sensor 20. The data acquisition unit 11 inputs the acquired image data (frames) sequentially to the feature point extraction unit 12.

In the embodiment, when a frame is input, the feature point extraction unit 12 extracts feature points of an object for each frame using, for example, a general FAST algorithm. Then, the feature point extraction unit 12 inputs information that identifies the feature points extracted for each frame to the filtering unit 13.

In an embodiment, when feature points for each frame are input, the filtering unit 13 first determines the moving direction and position of the moving body for each input frame based on the information of the moving body 100. Specifically, the filtering unit 13 determines the position of the moving body 100 for each frame based on the information of the moving body 100 at the time the frame was output, and further determines the absolute attitude as the moving direction. The absolute attitude is expressed by the translation vector T (x, y, z) and rotation components R (roll, pitch, yaw) of the moving body 100.

The information of the moving body 100 includes information indicating the position information and the traveling direction of the moving body 100. The position information may be obtained from a GPS (Global Positioning System) receiver mounted on the moving body 100, or may be obtained by self-position estimation, which will be described later. In addition, the information indicating the traveling direction can be obtained, for example, from an angular velocity sensor mounted on the moving body 100.

Next, the filtering unit 13 uses the traveling direction and position of the moving body 100 to determine, for each frame, whether or not there is a portion of the frame that does not overlap with other frames. In the embodiment, the filtering unit 13 uses the traveling direction and position of the moving body 100 to determine, for each frame, whether or not there is another frame in the vicinity of the frame. Then, if there is no other frame in the vicinity of the frame, the filtering unit 13 determines that there is a portion of the frame that does not overlap with other frames.

Specifically, the filtering unit 13 identifies a frame located at the edge on the right, left, top, or bottom side of the moving object. In the example of FIG. 3, the frame output at time t is the frame located at the edge. A frame located at the edge may have a portion that does not overlap with another frame.

Next, filtering unit 13 calculates the amount of change in the attitude of moving body 100 between the frame located at the edge and the frame outputted before that. Specifically, the time when the frame located at the edge is outputted is t, the time when the previous frame is outputted is t-1, and the rotation components at the respective times are R _t and R _t-1 . In this case, the amount of change γ in the attitude of the moving body is calculated by the following equation 1.

(Equation 1)
γ=R _t −R _t−1

Next, the filtering unit 13 determines whether or not there is a portion of the frame located at the edge that does not overlap with another frame based on the calculated posture change amount γ, and if the result of the determination indicates that there is, eliminates the feature points extracted from the non-overlapping portion. Here, the process of determining whether frames overlap will be explained using Figure 4. Figure 4 is a diagram showing the determination process in the filtering unit of the map generating device.

In Figure 4, 21 indicates the shooting area when a frame located at the edge is shot (time t). In Figure 4, the depth direction is the z-axis direction, the horizontal direction of the image is the x-axis direction, and the vertical direction of the image is the y-direction. Furthermore, the angle of view of the depth sensor 20 is H, the extracted feature point is P(x,y,z), and the angle in the zx plane between the normal line passing through the center of the imaging surface of the depth sensor 20 and the line extending from the feature point P that also passes through the center is θ.

In this case, if the angle θ of feature point P is greater than the angle ((H/2) - abs(γ)) obtained by subtracting the amount of change in posture γ from half the value of the angle of view H, feature point P is located in an area where the frames do not overlap.

Therefore, the filtering unit 13 judges whether or not each of the feature points extracted from the frame at time t satisfies the following equation 2. If there is a feature point that satisfies the following equation 2, the filtering unit 13 judges that the frame located at the edge has a portion that does not overlap with another frame.

(Equation 2)
(H/2)-abs(γ)>θ

Then, the filtering unit 13 eliminates feature points that exist in the corresponding parts of the frame that are determined not to overlap with another frame. Specifically, the filtering unit 13 eliminates feature points (feature points that satisfy the above formula 2) that exist in the parts of the frame located at the edge that do not overlap with another frame.

The environmental map generation unit 14 first uses the feature points of each frame that were not excluded by the filtering unit 13 to match, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points.

Next, for each frame, the environment map generating unit 14 calculates a camera matrix for that frame using the set of feature points identified in that frame. Then, the environment map generating unit 14 calculates the three-dimensional coordinates of the feature points using the calculated camera matrix and the two-dimensional coordinates of the feature points in the frame.

Next, after calculating the camera matrix, the environmental map generation unit 14 calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the moving body generates or updates an environmental map composed of a three-dimensional point cloud (hereinafter referred to as a "three-dimensional environmental map") using the feature points whose three-dimensional coordinates have been calculated. Figure 5 is a diagram showing an example of a three-dimensional environmental map.

The environmental map generating unit 14 also inputs the generated three-dimensional environmental map to the control device 30. Furthermore, the environmental map generating unit 14 can also generate a two-dimensional environmental map by converting the three-dimensional environmental map into two dimensions. In this case, the environmental map generating unit 14 passes the generated two-dimensional environmental map to the control device 30. The control device 30 then uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

In the above example, a 3D LiDAR capable of measuring depth in a 3D space is used as the depth sensor 20, but in the embodiment, the depth sensor 20 is not limited to this. In the embodiment, other examples of the depth sensor 20 include a 2D LiDAR with a fixed angle of view that measures depth in a specific plane, and a TOF (Time Of Flight) camera.

[Device operation]
Next, the operation of the map generation device 10 will be described with reference to FIG. 6. FIG. 6 is a flow diagram showing the operation of the map generation device. In the following description, reference will be made to FIGS. 1 to 5 as appropriate. In addition, in the embodiment, a map generation method is implemented by operating the map generation device 10. Therefore, the description of the map generation method in the embodiment will be replaced by the following description of the operation of the map generation device 10.

6, first, when image data with depth is output from the depth sensor 20 on a frame-by-frame basis, the data acquisition unit 11 acquires the output image data (step A1). Furthermore, when the data acquisition unit 11 acquires image data for the set number of frames, it inputs the acquired image data to the feature point extraction unit 12.

Next, when the image data acquired in step A1 is input, the feature point extraction unit 12 extracts feature points for each frame (step A2). The feature point extraction unit 12 also outputs the extracted feature points for each frame to the filtering unit 13.

Next, when the feature points for each frame are input in step A2, the filtering unit 13 determines the traveling direction and position of the moving body 100 for each input frame based on the information of the moving body 100 (step A3).

Next, the filtering unit 13 uses the direction of travel and position of the moving body 100 to determine, for each frame, whether there is any part of that frame that does not overlap with other frames (step A4).

Specifically, in step A4, the filtering unit 13 identifies a frame located at the edge on the right, left, top, or bottom side of the moving body. Furthermore, the filtering unit 13 uses the above formula 1 to calculate the amount of change in the posture of the moving body 100 between the frame located at the edge and the frame output previously thereto.

Then, the filtering unit 13 uses the calculated posture change amount γ to determine whether or not each of the feature points extracted from the frame located at the edge satisfies the above formula 2. If there is a feature point that satisfies the above formula 2, the filtering unit 13 determines that the frame located at the edge has a portion that does not overlap with another frame.

Next, in a frame that has a portion that does not overlap with another frame, the filtering unit 13 eliminates feature points in that portion (step A5). Specifically, in A5, the filtering unit 13 eliminates feature points that satisfy the above equation 2 from frames located at the edges.

Next, the environmental map generation unit 14 uses the feature points that were not excluded in step A5 to match, for each frame, the feature points extracted from it with the feature points extracted from previous frames, and identifies pairs of feature points (step A6).

Next, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points identified as groups in step A6, and uses the calculation results to generate an environmental map composed of a collection of the feature points identified as groups (step A7).

Thereafter, the environmental map generating unit 14 inputs the environmental map generated in step A7 to the control device 30. As a result, the control device 30 uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

As described above, in the embodiment, even if the angle of view of the depth sensor 20 is fixed, feature points are excluded from portions of a frame located at the edge that do not overlap with other frames, thereby avoiding the extraction of an incorrect set of feature points. Therefore, according to the embodiment, the accuracy of the environmental map can be improved when a depth sensor with a fixed angle of view is used.

[program]
The program in the embodiment may be a program that causes a computer to execute steps A1 to A7 shown in Fig. 6. By installing and executing this program in a computer, the map generating device 10 and the map generating method in the embodiment can be realized. In this case, the processor of the computer functions as the data acquisition unit 11, the feature point extraction unit 12, the filtering unit 13, and the environmental map generating unit 14 and performs the processing.

Another example of a computer is a computer installed in the mobile object 100. Other examples of computers include general-purpose PCs, smartphones, and tablet terminal devices.

In addition, the program in the embodiment may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as any one of the data acquisition unit 11, the feature point extraction unit 12, the filtering unit 13, and the environmental map generation unit 14.

[Physical configuration]
A computer that realizes the map generating device 10 by executing a program in the embodiment will now be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of a computer that realizes the map generating device.

7, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader/writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this embodiment, the GPU or FPGA can execute the program in the embodiment.

The CPU 111 loads the program in the embodiment, which is composed of a group of codes stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

In addition, the program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

The data reader/writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes the results of processing in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

Specific examples of recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory).

Note that the map generating device 10 in the embodiment can be realized not by a computer with a program installed, but by using hardware corresponding to each part, for example, electronic circuits. Furthermore, the map generating device 10 may be realized in part by a program and the remaining part by hardware. In the embodiment, the computer is not limited to the computer shown in FIG. 7.

Some or all of the above-described embodiments can be expressed by (Appendix 1) to (Appendix 9) described below, but are not limited to the following descriptions.

(Appendix 1)
A data acquisition unit that acquires image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction unit that extracts feature points from the image data for each frame;
a filtering unit that determines, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, removes the feature points extracted from the non-overlapping portion;
an environment map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map composed of a collection of the feature points identified as pairs using a calculation result;
Equipped with
A map generating device comprising:

(Appendix 2)
The depth sensor is mounted on a moving body,
the filtering unit determines, for each frame, a moving direction and a position of the moving body based on information about the moving body, and determines, using the determined moving direction and position of the moving body for each frame, whether or not a portion of the frame does not overlap with another frame other than the frame;
2. A map generating device as described in claim 1.

(Appendix 3)
the filtering unit determines, for each frame, whether or not another frame exists around the frame using the orientation and position of the moving object obtained for each frame, and if no other frame exists around the frame, determines that the frame has a portion that does not overlap with any other frame other than the frame itself;
3. A map generating device as described in claim 2.

(Appendix 4)
A data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction step of extracting feature points from the image data for each frame;
a filtering step of determining, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, eliminating the feature points extracted from the non-overlapping portion;
an environmental map generating step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map constituted by a set of the feature points identified as pairs using the calculation results;
having
A map generating method comprising:

(Appendix 5)
The depth sensor is mounted on a moving body,
in the filtering step, determining a moving direction and a position of the moving object for each frame based on information about the moving object, and using the determined moving direction and position of the moving object for each frame, determining whether or not the frame has a portion that does not overlap with another frame other than the frame;
5. A map generating method as described in claim 4.

(Appendix 6)
In the filtering step, for each frame, using the orientation and position of the moving object determined for each frame, it is determined whether or not another frame exists around the frame, and if no other frame exists around the frame, it is determined that the frame has a portion that does not overlap with other frames other than the frame.
6. A map generating method as claimed in claim 5.

(Appendix 7)
On the computer,
A data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction step of extracting feature points from the image data for each frame;
a filtering step of determining, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, eliminating the feature points extracted from the non-overlapping portion;
an environmental map generating step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map constituted by a set of the feature points identified as pairs using the calculation results;
A program to execute.

(Appendix 8)
The depth sensor is mounted on a moving body,
in the filtering step, determining a moving direction and a position of the moving object for each frame based on information about the moving object, and using the determined moving direction and position of the moving object for each frame, determining whether or not the frame has a portion that does not overlap with another frame other than the frame;
8. The program according to claim 7.

(Appendix 9)
In the filtering step, for each frame, using the orientation and position of the moving object determined for each frame, it is determined whether or not another frame exists around the frame, and if no other frame exists around the frame, it is determined that the frame has a portion that does not overlap with other frames other than the frame.
9. The program according to claim 8.

Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2022-91128, filed on June 3, 2022, the disclosure of which is incorporated herein in its entirety.

As described above, according to the present disclosure, it is possible to improve the accuracy of an environmental map when using a depth sensor with a fixed angle of view. The present disclosure is useful in fields where SLAM is used.

REFERENCE SIGNS LIST 10 Map generating device 11 Data acquisition unit 12 Feature point extraction unit 13 Filtering unit 14 Environmental map generating unit 20 Depth sensor 30 Control device 40 Steering device 50 Power train 100 Mobile body 110 Computer 111 CPU
112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader/writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims (9)

A data acquisition unit that acquires image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis;
A feature point extraction unit that extracts feature points from the image data for each frame;
a filtering unit that determines, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists , removes the feature points extracted from the non-overlapping portion;
an environment map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map composed of a collection of the feature points identified as pairs using a calculation result;
Equipped with
A map generating device comprising: The depth sensor is mounted on a moving body,
the filtering unit determines, for each frame, a moving direction and a position of the moving body based on information about the moving body, and determines, using the determined moving direction and position of the moving body for each frame, whether or not a portion of the frame does not overlap with another frame other than the frame;
The map generating device according to claim 1 . the filtering unit determines, for each frame, whether or not another frame exists around the frame using the orientation and position of the moving object obtained for each frame, and if no other frame exists around the frame, determines that the frame has a portion that does not overlap with any other frame other than the frame itself;
The map generating device according to claim 2 . Image data output from a depth sensor with a fixed angle of view is acquired frame by frame.
Extracting feature points from the image data for each frame;
determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and if the result of the determination indicates that there is a portion, excluding the feature points extracted from the non-overlapping portion;
Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environment map composed of the sets of the feature points identified as pairs using the calculation results.
A map generating method comprising: The depth sensor is mounted on a moving body,
In the determination, a moving direction and a position of the moving body are obtained for each frame based on information of the moving body, and using the obtained moving direction and position of the moving body for each frame, it is determined whether or not a portion of the frame does not overlap with another frame other than the frame.
The map generating method according to claim 4. In the determination, for each frame, using the orientation and position of the moving object determined for each frame, it is determined whether or not another frame exists around the frame, and if no other frame exists around the frame, it is determined that the frame has a portion that does not overlap with any other frame other than the frame.
The map generating method according to claim 5. On the computer,
Image data output from a depth sensor with a fixed angle of view is acquired frame by frame,
extracting feature points from the image data for each frame;
determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and, if the result of the determination indicates that there is a portion, excluding the feature points extracted from the non-overlapping portion;
Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map composed of a set of the feature points identified as pairs using the calculation results.
program . The depth sensor is mounted on a moving body,
In the determination, a moving direction and a position of the moving body are obtained for each frame based on information of the moving body, and using the obtained moving direction and position of the moving body for each frame, it is determined whether or not a portion of the frame does not overlap with another frame other than the frame.
The program according to claim 7. In the determination, for each frame, the orientation and position of the moving body obtained for each frame are used to determine whether or not another frame exists around the frame, and if no other frame exists around the frame, the frame is determined to have a portion that does not overlap with any other frame other than the frame itself.
The program according to claim 8.

Images