JP7793567B2 - LiDAR analysis device, LiDAR analysis method, and LiDAR analysis program - Google Patents

Description

The present invention relates to a LiDAR analysis device, a LiDAR analysis method, and a LiDAR analysis program, and in particular to a LiDAR analysis device, method, and program for estimating measurement-missing areas known as black spots that occur during LiDAR measurements.

To achieve automated driving and autonomous driving of passenger cars and other vehicles, it is necessary to detect and recognize the external environment while the vehicle is traveling in order to obtain information for driving control. Currently, LiDAR (Light Detection and Ranging) is an example of a sensor with excellent recognition capabilities. For example, one technology and method that uses LiDAR to detect the position and orientation of floating objects involves measuring the target object with LiDAR and converting the three-dimensional point cloud data into a grayscale two-dimensional depth image. This two-dimensional depth image is compared with target object data stored in a model database and used to both predict the target object's initial position and orientation, and more accurately predict its position and orientation (see Patent Document 1).

It has been found that LiDAR measurements can result in areas where the measurement is missing, known as black spots. Because LiDAR uses optical detection, there are areas where detection problems occur due to the reflection and absorption of light rays, resulting in partial measurement errors or even being unable to measure. However, the method disclosed in the aforementioned Patent Document 1 does not necessarily effectively address the problem of areas where the measurement is missing during LiDAR measurements.

Japanese Patent Application Laid-Open No. 2015-201192

The present invention has been developed in consideration of the above points, and provides a LiDAR analysis device that estimates areas that are difficult to measure due to the characteristics of LiDAR measurement, detects conditions that will result in poor measurement during LiDAR measurement, and leads to improved analytical capabilities for LiDAR measurements. It also provides a LiDAR analysis method and a LiDAR analysis program.

That is, the LiDAR analysis device of the embodiment is characterized by comprising an acquisition unit that acquires point cloud data consisting of multiple detection points, each including its position and optical intensity, by LiDAR measurement of the detection target; an extraction unit that extracts the position and optical intensity of each detection point in the point cloud data; a generation unit that converts the optical intensity of each detection point in the point cloud data to luminance and generates a two-dimensional grayscale image based on the converted luminance; a division unit that divides the two-dimensional grayscale image into specified regions to generate a plurality of divided regions; and a determination unit that determines whether a defect has occurred in the LiDAR measurement based on the luminance value contained in each of the divided regions.

Furthermore, the extraction unit and generation unit of the LiDAR analysis device may be equipped with an editing unit that edits each detected point in the point cloud data at the same time.

Furthermore, the generation unit of the LiDAR analysis device may be equipped with a video generation unit that animates the two-dimensional grayscale image to generate a two-dimensional grayscale video.

Furthermore, the division unit of the LiDAR analysis device may generate the two-dimensional grayscale video as still image frames and divide the still image frames into multiple divided regions.

Furthermore, the judgment unit of the LiDAR analysis device may be equipped with a video composition unit that composes a divided area containing brightness values when a LiDAR measurement error occurs as a confirmation video of a predetermined duration.

Furthermore, the judgment unit of the LiDAR analysis device may determine whether a LiDAR measurement error has occurred based on the ratio of detection points that exceed a predetermined brightness value to detection points that do not exceed the predetermined brightness value for each detection point included in the divided area.

The LiDAR analysis device of the present invention includes an acquisition unit that acquires point cloud data consisting of multiple detection points, each containing position and optical intensity, by LiDAR measurement of the detection target; an extraction unit that extracts the position and optical intensity of each detection point in the point cloud data; a generation unit that converts the optical intensity of each detection point in the point cloud data to luminance and generates a two-dimensional grayscale image based on the converted luminance; a division unit that divides the two-dimensional grayscale image into specified regions to generate multiple divided regions; and a determination unit that determines whether a LiDAR measurement has failed based on the luminance values contained in each of the multiple divided regions. Therefore, due to the characteristics of LiDAR measurement, it is possible to estimate areas that are difficult to measure and detect conditions that will result in poor LiDAR measurement, thereby improving the analytical capabilities of LiDAR measurements. Similar effects can also be achieved in LiDAR analysis methods and LiDAR analysis programs.

FIG. 1 is a schematic diagram illustrating a usage situation of a LiDAR analysis device according to an embodiment. FIG. 1 is a block diagram showing the configuration of a LiDAR analysis device. 1 is a first flowchart illustrating processing in a LiDAR analysis device. 2 is a second flowchart illustrating processing in the LiDAR analysis device. 3 is a third flowchart illustrating processing in the LiDAR analysis device. 10 is an example of a two-dimensional grayscale image. This is an example of a still image extracted from a video captured from the driver's perspective in a vehicle equipped with a LiDAR measurement device. 10 is an example image showing a plurality of divided regions. 1 is a first flowchart showing the control of the LiDAR analysis device. 2 is a second flowchart showing the control of the LiDAR analysis device. 3 is a third flowchart showing the control of the LiDAR analysis device. 4 is a fourth flowchart showing the control of the LiDAR analysis device.

Conventional LiDAR (Light Detection and Ranging) measurements typically involve converting three-dimensional point cloud data into a two-dimensional grayscale image. However, the component corresponding to three-dimensional depth (the distance from the position of the person measuring (vehicle) to the target) is represented by the brightness of the two-dimensional grayscale image (two-dimensional point cloud). However, this two-dimensional grayscale image does not include the reflection intensity during LiDAR measurement. As a result, the brightness of the image represents the distance between the target and the person measuring (vehicle), making it impossible to distinguish between a nearby target with low reflection intensity and one that is simply far away. Furthermore, when identifying missing areas in LiDAR measurements, known as black spots, methods that utilize training data, such as machine learning, increase the amount of processing required.

The measurement gaps known as black spots that inevitably occur during LiDAR measurements are not proportional to the distance from the position of the person taking the measurements (vehicle) to the object, nor is the optical reflection intensity proportional to the distance. Therefore, the LiDAR analysis device of this embodiment addresses these unique issues without increasing the amount of data processing required. It acquires and processes three-dimensional point cloud data acquired from a LiDAR sensor while a passenger vehicle or other vehicle is traveling, to detect measurement gaps known as black spots that occur during LiDAR measurements. This analysis device is then used to analyze the tendency for unavoidable gaps to appear during LiDAR measurements. The analysis device also includes an analysis method and analysis program executed by the analysis device.

Figure 1 is a schematic diagram showing one usage scenario of a LiDAR analysis device according to an embodiment. Specifically, a LiDAR measurement device 1 is installed on a vehicle 2, such as a passenger car, in an automated driving scenario. Laser light 3 is emitted from the LiDAR measurement device 1 toward the front of the vehicle 2, and the reflected light is detected by the LiDAR measurement device 1. In the illustration, the vehicle 2 is traveling on a road surface 4, and for simplicity of illustration, the direction of irradiation of the laser light 3 is shown as being toward the front of the vehicle 2. Another vehicle 5 is traveling ahead of the vehicle 2. In addition, the laser light may be irradiated all around the vehicle 2. The LiDAR analysis device 10 is connected to the LiDAR measurement device 1 by wire, wirelessly, or via an internet line, etc.

The block diagram in Figure 2 shows a LiDAR analysis device 10 according to an embodiment, connected to the LiDAR measurement device 1. The LiDAR measurement device 1 is equipped with a laser irradiation unit 11, a sensor 12 that receives reflected laser light, and is appropriately equipped with a communication unit 13 that communicates with the LiDAR analysis device 10, a memory unit 14 that stores data acquired by the sensor 12, and a control unit 15 that controls the LiDAR measurement device 1. Figure 2 shows an example in which the LiDAR analysis device 10 is directly connected to the LiDAR measurement device 1 to acquire point cloud data. Alternatively, the point cloud data may be acquired by the LiDAR analysis device 10 after data measured by the LiDAR measurement device 1 is stored in an appropriate storage medium, or it may be acquired via wireless communication or the like via another device or network.

The LiDAR analysis device 10 (processing unit, computer) is composed of hardware elements such as a CPU, GPU, etc., and storage elements such as RAM, ROM, HDD, and SSD. It also has various interfaces for receiving signals from the LiDAR measurement device 1 and for inputting and outputting data from and to the outside of the LiDAR analysis device 10. In terms of software, each function is realized by a program loaded into main memory. The recording medium that stores this program can be a "non-transitory tangible medium," such as a CD, DVD, semiconductor memory, or programmable logic circuit. The program may also be supplied to the LiDAR analysis device 10 (computer) via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves).

The LiDAR analysis device 10 (processing unit, computer) in the block diagram of Figure 2 shows a schematic representation of the device's functional units, and the device 10 includes functional units such as an acquisition unit 110, extraction unit 120, editing unit 121, generation unit 130, editing unit 131, video generation unit 132, division unit 140, determination unit 150, and video composition unit 151. Each functional unit will be explained below using the flowcharts of Figures 3 to 5.

The acquisition unit 110 acquires point cloud data consisting of multiple detection points, each containing position and optical intensity, by LiDAR measurement of the detection target. A characteristic of LiDAR measurement is that each irradiation and reflection of laser light can be obtained as a point (detection point) containing position and optical intensity information. Position information is expressed as data in a coordinate system in three-dimensional space. Optical intensity is the reflection intensity when laser light emitted from the LiDAR measurement device 1 is reflected by a detection target, such as a vehicle traveling in front of the vehicle, a pedestrian, or a roadside tree, and received by the LiDAR measurement device 1. Obviously, light may be absorbed by the detection target, so the intensity is lower when received than when emitted.

The laser light irradiation range extends to the surrounding area, including the front of the vehicle 2 equipped with the LiDAR measurement device 1, and although the distance varies depending on the product, it is typically within a range of 20 to 50 meters from the vehicle, or even several hundred meters. Therefore, LiDAR measurement generates point cloud data, which is a collection of points containing information on position and optical intensity, and this point cloud data is acquired. In the case of the LiDAR measurement device 1 of this embodiment, approximately 1 million detection points are detected in one second of measurement, meaning that approximately 100,000 detection points are included per sampling period.

The extraction unit 120 extracts the position and optical intensity of each detection point in the point cloud data. As mentioned above, point cloud data contains approximately 1 million detection points per second of measurement, and approximately 100,000 detection points per sampling period. Therefore, for each detection point, the position in three-dimensional coordinates and optical intensity (reflection intensity) of the detection point are extracted individually. Extraction is then performed for the entire point cloud.

The extraction unit 120 is equipped with an editing unit 121, which edits each detection point of the point cloud data for each identical time. As an editing process, the editing unit 121 executes a process of arranging each detection point of the point cloud data in chronological order for each identical time. "Same time" here means that the sampling time recorded in the measurement is the same. Each detection point included in the point cloud data has a position and optical intensity measured by LiDAR measurement. Therefore, by arranging each detection point in chronological order, it becomes easier to understand the changes in the detection points over time, especially as a video, as described below.

The processing up to this point corresponds to data acquisition (S1), time series processing (S2), reflection intensity and coordinate extraction (S3), and processing all point clouds (S4) in Figure 3. Data acquisition (S1) corresponds to the processing of the acquisition unit 110, time series processing (S2) corresponds to the processing of the editing unit 121, and reflection intensity and coordinate extraction (S3) corresponds to the processing of the extraction unit 120. Processing all point clouds (S4) functions in conjunction with processing all point clouds (S6), described below, and is a repetitive process in which brightness conversion (S5) between them is applied to all of the detected points in the point cloud data. In Figures 3, 4, and 5, the "S" symbols such as S1 indicate processing, and the "D" symbols such as D13 indicate the products (still images, videos, etc.) generated by the processing.

The generation unit 130 converts the optical intensity of each detection point in the point cloud data into brightness, and generates a two-dimensional grayscale image based on the converted brightness. Each detection point in the point cloud data includes a position and optical intensity (reflection intensity). The optical intensity (reflection intensity) of each detection point is then converted into a brightness value. The amount of information is then compressed to two dimensions based on the magnitude relationship of the brightness values. Of the detection points in the point cloud data, only the optical intensity (reflection intensity) is used and position information is removed, so each detection point is reduced to information representing the degree of brightness. As a result, a two-dimensional, flat grayscale image is generated from the magnitude relationship of the brightness values, with no sense of distance and only high and low brightness values being expressed.

Through processing by the generation unit 130, for example, the situation around the vehicle 2 equipped with the LiDAR measurement device 1 of Figure 1 can be converted into a planar image. Furthermore, the LiDAR analysis device 10 of the embodiment aims to detect black spots (environmental conditions that are prone to black spots, detection of targets), and performs this detection using image statistics calculated from planar images. This eliminates the need to represent the position of each detected point in the point cloud data using three-dimensional coordinates, making it possible to compress the data to the optically required level.

The generation unit 130 is equipped with an editing unit 131, which edits each detection point in the point cloud data for each identical time. The editing unit 131 executes a process of arranging each detection point converted into a brightness value in chronological order for each identical time.

The generation unit 130 also includes a video generation unit 132, which animates the two-dimensional grayscale image to generate a two-dimensional grayscale video. The LiDAR analysis device 10 of this embodiment aims to detect black spots (detect conditions and locations where black spots are likely to occur). For this reason, by generating a two-dimensional grayscale video, the surrounding situation when the vehicle 2 equipped with the LiDAR measurement device 1 is traveling becomes a continuous image, making it easier to confirm the situation when a black spot appears.

Figure 6 shows image Zd, a cut-out scene from a two-dimensional grayscale video. Normally, the higher the brightness value, the darker the image. In the two-dimensional grayscale video of this embodiment, the shades are reversed, and detected points with low brightness values are displayed darker. Because this is a two-dimensional grayscale video (image Zd) based solely on the relationship between brightness values, the overall image has a reduced sense of three-dimensionality.

More specifically, the generation unit 130 executes the sampling at a predetermined interval, such as every 1/1000 seconds. The reflection intensity of each detection point in the point cloud data is converted into a grayscale value with 255 gradations. To create an image with a white background, the maximum and minimum values are inverted and linearly converted so that the maximum reflectance is 0 on the grayscale and the minimum reflectance is 255 on the grayscale.

The processing up to this point corresponds to the brightness conversion (S5), processing all point clouds (S6), point cloud reconstruction (S7), viewpoint movement (S8), 2D grayscale image generation (S9), 2D grayscale image (D10), time series processing (S11), 2D grayscale video generation (S12), and 2D grayscale video (D13) in Figure 3. Processing all point clouds (S6) works in conjunction with the aforementioned processing all point clouds (S4), and is a repetitive process in which brightness conversion (S5) is performed on each detected point in all point clouds.

The processes in Figure 3 from brightness conversion (S5) to 2D grayscale video (D13) correspond to the processing of the generation unit 130. Of these, the time series processing (S11) corresponds to the processing of the editing unit 131, and the 2D grayscale video generation (S12) corresponds to the processing of the video generation unit 132. The 2D grayscale image (D10) and 2D grayscale video (D13) are products (image data, video data) generated by the processing shown in the flowchart in Figure 3.

Point cloud reconstruction (S7) is performed in units of the aforementioned sampling period, and is a process that reconstructs each detected point into a data format that matches the point cloud processing library. In other words, the 3D coordinates and reflection intensity of the point cloud contained in the binary-format measurement data are defined as the 3D coordinates and RGB values of a class object that represents the point cloud in the point cloud processing library.

Viewpoint movement (S8) is a process that adjusts the range of the on-screen display of the point cloud to the viewpoint (field of view) of the vehicle's front camera in a vehicle 2 equipped with the LiDAR measurement device 1 shown in Figure 1, for example. The program executed by the LiDAR analysis device 10 uses Open3D, a 3D point cloud processing library for Python (registered trademark), for point cloud visualization and other purposes. In 2D grayscale image generation (S9), the 3D point cloud is also visualized and the image is converted into a still image. However, the reference viewpoint during visualization is the viewpoint of a bird's-eye view.

Here, to visualize the image from the viewpoint of the front camera attached to the front of the vehicle, a rotation transformation and other processing is performed to move the viewpoint from the bird's-eye view to the driver's viewpoint (field of view). In addition, the height of the point cloud is adjusted so that it is the same as the mounting position of the on-board camera.

The 2D grayscale image generation (S9) corresponds to the processing of the generation unit 130 described above, and is executed in units of the sampling period described above. In this case, functions for point cloud visualization in the Open3D library are used, and the point cloud is drawn from the viewpoint of the front camera. At the same time, a screen capture is performed, and the 2D grayscale image is saved as a still image in one of various formats. For example, the file names of the saved still images are assigned names indicating the respective times (for example, "20xxyyzzaabbcc" for 20xx year, yy month, zz day, aa hour, bb minute, cc). Note that in practice, UNIX time, expressed as the number of seconds elapsed since 00:00:00 on January 1, 1970, is used. As a result, the order of the file names is chronological.

2D grayscale video generation (D13) corresponds to the processing of video generation unit 132 of generation unit 130 described above, and is a process of linking 2D grayscale images (D10) generated by 2D grayscale image generation (S9) to create a video. Specifically, the 2D grayscale images, which are still images, are sorted in file name order and linked in file name order to generate a video file (2D grayscale video (D13)). In this case, the 2D grayscale video is generated at a fixed frame rate. Note that when creating a fixed-frame video file (2D grayscale video (D13)) from the 2D grayscale images (D10) converted from variably sampled data, zero-order interpolation is performed, storing missing values as the most recent sampled data. This makes it possible to compare the data when played back simultaneously with the in-vehicle camera data.

The division unit 140 divides the two-dimensional grayscale image into specified regions to generate multiple divided regions. In particular, after a two-dimensional grayscale video (D13) is generated through two-dimensional grayscale video generation (S12), still image frames are generated from this video, and the still image frames are then divided into multiple divided regions. When generating the multiple divided regions, the target range is defined based on the driver's viewpoint (field of view), with the front direction in which the vehicle 2 equipped with the LiDAR measurement device 1 is traveling as its center.

The determination unit 150 determines whether a defect has occurred in the LiDAR measurement based on the brightness values contained in each of the multiple divided areas. The determination unit 150 includes a video composition unit 151, which composes the divided areas containing the brightness values when a defect has occurred in the LiDAR measurement as a confirmation video of a predetermined length of time. Note that, as another aspect, the determination unit 150 may be configured to determine whether a defect has occurred based on the brightness values contained in each of the divided areas divided into predetermined regions.

When detecting black spots, we want to make a static judgment at a specific point in time, rather than a chronological judgment. Therefore, the 2D grayscale video (D13) is cut out into still images for each scene required for the judgment, and the judgment is made on those still images. Note that the process of first creating a 2D grayscale video (D13) and then converting it back into still images is redundant, but this is because the purpose is to play it back simultaneously with the video from the in-car camera to enable final human confirmation.

The processing in the division unit 140 corresponds to the steps in the flowchart of Figure 4: processing all frames (S14), extracting frames (S15), creating a two-dimensional grayscale image (D16), dividing the search area into grids (S17), processing all grids (S18), and creating a luminance distribution (S19). The processing in the determination unit 150 corresponds to the steps in the flowchart of Figure 4: determining black spots (S20), processing all grids (S21), checking for the presence or absence of black spots (S22), and outputting black spot information (S23), and generates black spot information (D24).

All frame processing (S14) works in conjunction with all frame processing (S27) described below, and is the start of a repetitive process in which the intervening processes are applied to all frames of the 2D grayscale video (D13). Frame extraction (S15) creates still image frames for each frame of the 2D grayscale video (D13) described above. Each frame is then read as an image in the program. A 2D grayscale image (D16) is then generated through the frame extraction (S15) process. The 2D grayscale image (D16) is a still image (frame).

Search area grid division (S17) divides the still image (frame) of the two-dimensional grayscale image (D16) into multiple vertical and horizontal rectangular grids. These vertical and horizontal grids are a collection of multiple "divided areas." In this embodiment, the search for black spots primarily targets the road surface area ahead of the vehicle 2, and this road surface area ahead of the vehicle 2 is set as the divided area to be searched. The subsequent process all grids (S18) functions in conjunction with the process all grids (S21, etc.) described below, and is the start of a repetitive process in which the search area grid division (S17) is performed on all grids in the two-dimensional grayscale image (D16).

The brightness distribution creation process (S19) involves examining the brightness distribution within each grid (i.e., each of multiple divided regions) mentioned above. Specifically, the brightness classes of the individual detection points in the point cloud within each grid are set to 25 levels, and the frequency (occurrence frequency) of each class is counted. Because the reflection intensity of the detection points is converted into grayscale values in 255 levels, the brightness levels are aggregated in stages, for example, as levels 0-9, levels 10-19, ... (intervals omitted), levels 240-249, levels 250-255, etc.

In this embodiment, black spot determination (S20) is a process that calculates the proportion of high-brightness detected points, such as those in tones 240 to 249 and tones 250 to 255, that occupy the grid of the point cloud. The subsequent process all grids (S21) works in conjunction with process all grids (S18) and marks the end of the repeated process in which black spot determination (S20) is performed for all grids in the two-dimensional grayscale still image (D14).

Therefore, if the proportion of high-brightness tone detection points, such as those with tones between 240 and 255, in a grid relative to the total points exceeds a predetermined threshold, it is determined that a black spot exists in that grid. This corresponds to checking for the presence or absence of a black spot (S22). In this case, for the grid containing the black spot, a unique position (grid ID) in the grid, such as the PPth row from the left and the QQth column from the top, is identified. Note that if it is determined that no black spot exists in any grid, processing ends.

The processes of determining black spots (S20), processing all grids (S21), and checking for the presence or absence of black spots (S22) correspond to the process of determining whether a LiDAR measurement has failed based on the ratio of detection points that exceed a predetermined brightness value to detection points that do not exceed the predetermined brightness value for each detection point included in the divided area in the determination unit 150.

Black spot information output (S23) is a process that outputs, as a text file or the like, the unique position (grid ID) in the grid where the black spot occurred and the time of occurrence for each frame of the aforementioned two-dimensional grayscale video (D13) based on the black spot determination (S20). In this way, black spot information (D24) is generated as data that aggregates the unique position (grid ID) in the grid where the black spot occurred and the time of black spot occurrence.

Then, the cut-out video generation (S25) in Figure 5 is a process that cuts out and saves the time period (e.g., 15 seconds before and after) around the time the black spot occurred. As a result, a cut-out video (D26) is generated that shows the progression before and after the black spot occurred. The subsequent process of all frames (S27) works in conjunction with the process of all frames (S14) described above, and marks the end of the repeated process of performing the cut-out video generation (S25) on all frames of the 2D grayscale video (D16).

The extracted video (D26) generated from the series of events through the generation of the extracted video (S25) is a confirmation video of a predetermined time for the divided area that includes the luminance value when the LiDAR measurement failure occurred. Therefore, the processing of the video composition unit 151 corresponds to the processing up to the series of steps for generating the extracted video (S25).

The example image in Figure 7 is a still image Z (frame) captured at a certain point in time from a video captured from the driver's viewpoint (field of view) in a vehicle equipped with a LiDAR measurement device. The vehicle is traveling on a road (on the road surface), and another vehicle is observed traveling ahead. The example image in Figure 8 is a display example in which a grid (divided region) is added to image Zd, which shows a scene from a 2D grayscale video captured from still image Z (frame) in the example image in Figure 7. The largest dashed rectangle shown in the figure is grid G (divided region). The dashed region contained within grid G (divided region) is the search target region f1. Region f2, which includes a thin solid line in grid G (divided region), is a region where no black spots exist. Region f3, which is represented by a thick solid line, is a region where black spots exist. In Figures 6 and 8, the brightness values are reversed, with higher brightness values being displayed lighter.

The presence or absence of black spots in Figure 8 can then be superimposed on the actual driving conditions in Figure 7, enabling analysis. As explained in the extracted video (D26), the display including the grid G in Figure 8 is configured as a 30-second video, from 15 seconds before to 15 seconds after the occurrence of the black spot, as an example. For example, based on Figures 7 and 8, black spots tend to occur in areas where there is a lot of light reflection from the rear window of a vehicle traveling in front of the vehicle, or in areas of low-reflectivity material, such as a black-painted car traveling in front to the right. In this way, the LiDAR analysis device of this embodiment can be used to analyze the factors that cause black spots, such as the surrounding environment during LiDAR measurement.

In the LiDAR analysis device and method of the embodiment, the optical intensity (reflection intensity) obtained from the LiDAR measurement device is used as a brightness value, and the brightness value is judged in stages using a threshold value. This eliminates the cumbersome task of machine-learning brightness values according to the measurement situation to determine pass/fail. As a result, it is possible to reduce and compress the processing time for LiDAR analysis.

Next, the LiDAR analysis method of the embodiment will be described together with the LiDAR analysis program. The LiDAR analysis method of the embodiment is executed by a computing element such as a CPU or GPU within the LiDAR analysis device 10 based on the LiDAR analysis program. The LiDAR analysis method causes the computing element of the LiDAR analysis device 10 to perform an acquisition function, an extraction function, a generation function, a division function, and a determination function, as well as an editing function, a video generation function, and a video composition function. Each function overlaps with the above description, so details will be omitted.

The flowcharts in Figures 9, 10, 11, and 12 show the overall flow of the LiDAR analysis method in the LiDAR analysis device 10. In Figure 9, the acquisition step (S110), extraction step (S120), generation step (S130), division step (S140), determination step (S150), and output step (S160) are executed. In Figure 10, the editing step (S121) is executed. In Figure 11, the editing step (S131) and video generation step (S132) are executed. In Figure 12, the video composition step (S151) is executed.

The acquisition function acquires point cloud data consisting of multiple detection points, each including their position and optical intensity, by LiDAR measurement of the detection target (S110; acquisition step). The extraction function extracts the position and optical intensity of each detection point in the point cloud data (S120; extraction step). The generation function converts the optical intensity of each detection point in the point cloud data to brightness and generates a two-dimensional grayscale image based on the converted brightness (S130; generation step). The division function divides the two-dimensional grayscale image into predetermined regions to generate multiple divided regions (S140; division step). Furthermore, the division function generates the two-dimensional grayscale video as still image frames and divides the still image frames into the multiple divided regions. The judgment function judges whether the LiDAR measurement has failed based on the brightness values contained in each of the multiple divided regions (S150; judgment step). Furthermore, the judgment function judges whether the LiDAR measurement has failed based on the ratio of detection points exceeding a predetermined brightness value to detection points not exceeding a predetermined brightness value for each detection point contained in the divided region. The output function executes a process to output the necessary information (S160; output step).

The editing function, together with the extraction function, edits each detection point of the point cloud data at the same time (S121; editing step). Additionally, the editing function, together with the generation function, edits each detection point of the point cloud data at the same time (S131; editing step). The video generation function animates the two-dimensional grayscale image to generate a two-dimensional grayscale video (S132; video generation step). The video composition function composes a divided area containing the luminance value when a LiDAR measurement error occurs as a confirmation video for a predetermined time (S151; video composition step).

The computer program of the present invention described above may be recorded on a processor-readable recording medium, and the recording medium may be a "non-transitory tangible medium" such as a disk, card, semiconductor memory, or programmable logic circuit.

The above computer program can be implemented using a programming language that supports data processing libraries such as ROS (Robot OS) and Open3D, such as C++ or Python.

REFERENCE SIGNS LIST 1 LiDAR measurement device 2, 5 Vehicle 3 Laser light 10 LiDAR analysis device 11 Irradiation unit 12 Sensor 13 Communication unit 14 Storage unit 15 Control unit 110 Acquisition unit 120 Extraction unit 121 Editing unit 130 Generation unit 131 Editing unit 132 Video generation unit 140 Division unit 150 Determination unit 151 Video composition unit

Claims (7)

an acquisition unit that acquires point cloud data consisting of a plurality of detection points including positions and optical intensities by LiDAR measurement of the detection target;
an extraction unit that extracts the position and optical intensity of each detection point of the point cloud data;
a generating unit that converts the optical intensity of each detection point of the point cloud data into luminance and generates a two-dimensional grayscale image based on the converted luminance;
a dividing unit that divides the two-dimensional grayscale image into predetermined regions to generate a plurality of divided regions;
A LiDAR analysis device comprising: a determination unit that determines whether a measurement error has occurred in the LiDAR based on the ratio of detection points that exceed a predetermined brightness value to detection points that do not exceed the predetermined brightness value for each detection point included in each of the plurality of divided areas . The LiDAR analysis device according to claim 1, wherein the extraction unit and the generation unit are provided with an editing unit that edits each detection point of the point cloud data at the same time. The LiDAR analysis device of claim 2, wherein the generation unit is equipped with a video generation unit that animates the two-dimensional grayscale image to generate a two-dimensional grayscale video. The LiDAR analysis device described in claim 3, wherein the division unit generates the two-dimensional grayscale video as still image frames and divides the still image frames into the plurality of divided regions. The LiDAR analysis device of claim 4, wherein the determination unit includes a video composition unit that composes a divided area containing a luminance value when a measurement error occurs in the LiDAR as a confirmation video of a predetermined duration. The computer
An acquisition step of acquiring point cloud data consisting of a plurality of detection points including positions and optical intensities by LiDAR measurement of the detection target;
an extraction step of extracting the position and optical intensity of each detection point of the point cloud data;
a generating step of converting the optical intensity of each detection point of the point cloud data into brightness and generating a two-dimensional grayscale image based on the converted brightness;
a dividing step of dividing the two-dimensional grayscale image into predetermined regions to generate a plurality of divided regions;
a determination step of determining whether a defect has occurred in the LiDAR measurement based on the ratio of detection points that exceed a predetermined brightness value to detection points that do not exceed the predetermined brightness value for each detection point included in each of the plurality of divided areas. On the computer,
An acquisition function that acquires point cloud data consisting of multiple detection points including positions and optical intensities by LiDAR measurement of the detection target;
an extraction function for extracting the position and optical intensity of each detection point of the point cloud data;
a generation function of converting the optical intensity of each detection point of the point cloud data into brightness and generating a two-dimensional grayscale image based on the converted brightness;
a division function for dividing the two-dimensional grayscale image into predetermined regions to generate a plurality of divided regions;
A LiDAR analysis program characterized by realizing a judgment function that judges whether a defect in the LiDAR measurement has occurred based on the ratio of detection points that exceed a predetermined brightness value to detection points that do not exceed the predetermined brightness value for each detection point included in each of the plurality of divided areas .