JP7636743B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present disclosure relates to an information processing device, an information processing method, and an information processing program.

2. Description of the Related Art Conventionally, image information generated by capturing an image of a subject may contain noise, and therefore, there are cases where it is required to remove noise from the image information.
The device described in Patent Document 1 utilizes deep learning to remove noise from CT images taken of a patient.

JP 2020-064609 A

Incidentally, there is LiDAR measurement, which generates point cloud information by emitting laser light from a measurement point and receiving the light reflected by an object at the measurement point. Point cloud information can also contain noise (such as noise formed by the point cloud), but point cloud noise comes in various patterns, and it has often been impossible to remove noise from the point cloud information.

The present disclosure provides an information processing device, an information processing method, and an information processing program that can identify noise contained in point cloud information.

An information processing device according to one embodiment includes an acquisition unit that acquires point cloud information, and an estimation unit that classifies noise occurring in the point cloud into a plurality of types based on the cause of the noise and the shape of the noise, and estimates the noise contained in the point cloud information based on a trained model trained for each type and the point cloud information acquired by the acquisition unit.

According to one aspect, noise contained in point cloud information can be identified.

FIG. 1 is a diagram illustrating an information processing device according to an embodiment. FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. FIG. 11 is a diagram for explaining an example of moving body noise. FIG. 11 is a diagram for explaining an example of subtraction noise. 1 is a flowchart illustrating an information processing method according to an embodiment.

One embodiment is described below.

[Overview of information processing device 100]
First, an overview of an information processing device 100 according to an embodiment will be described.
FIG. 1 is a diagram illustrating an information processing device 100 according to an embodiment.

The information processing device 100 may be configured as an estimation device that estimates noise included in point cloud information. The information processing device 100 may also be configured as a removal device that removes noise (noise in point clouds) from point cloud information. The information processing device 100 is not limited to the device of the above example, and may be configured as various devices.
The information processing device 100 may be a computer such as a server, a desktop, a laptop, a tablet, or a smartphone.

The information processing device 100 acquires point cloud information obtained by LiDAR measurement. In LiDAR measurement, when light (e.g., laser light) emitted from a light emitting/receiving device (light emitting/receiving device) is reflected by an object, the reflected light (reflected light) is repeatedly received by the light emitting/receiving device to generate information on a point cloud (point cloud information) composed of a plurality of points.
In addition, the information processing device 100 is not limited to the example of acquiring point cloud information by LiDAR measurement, and may acquire point cloud information by using a transmitter/receiver to transmit and receive sound waves, etc. Note that LiDAR measurement and measurement to acquire point cloud information using sound waves may be collectively referred to as "point cloud measurement."

The information processing device 100 estimates various noises (noises generated as point clouds) that occur during point cloud measurement. In this case, the information processing device 100 may estimate the noise by using, for example, machine learning and deep learning. As an example, the information processing device 100 may input point cloud information to a trained model that has learned the noise of the point cloud, and output an estimation result of the noise from the trained model.
During learning, noise is classified into multiple types based on the cause of occurrence and shape, and learning is performed for each type to generate a trained model. There may be multiple types of noise (noise classification) according to the cause of occurrence and shape. As an example, the noise may be moving body noise, pull noise, and other noises other than the above-mentioned two types of noise, or various noises (noises generated as point clouds) other than these three examples. The information processing device 100 may estimate the noise (noise for each type) included in the point cloud information for each of the above-mentioned types of noise.

[Details of information processing device 100]
Next, the information processing device 100 according to an embodiment will be described in detail.
FIG. 2 is a block diagram for explaining the information processing device 100 according to an embodiment.

The information processing device 100 includes, for example, a communication unit 121, a storage unit 122, a display unit 123, and a control unit 110. The communication unit 121, the storage unit 122, and the display unit 123 may be an embodiment of an output unit. The control unit 110 includes, for example, an acquisition unit 111, an estimation unit 112, a removal unit 113, an output control unit 114, and a calculation unit 115. The control unit 110 may be configured, for example, by a calculation processing device of the information processing device 100. The control unit 110 (for example, a calculation processing device, etc.) may realize the functions of each unit (for example, the acquisition unit 111, the estimation unit 112, the removal unit 113, the output control unit 114, and the calculation unit 115, etc.) by, for example, appropriately reading and executing various programs stored in the storage unit 122, etc.

The communication unit 121 is, for example, a communication interface capable of transmitting and receiving various information to and from a device (external device) external to the information processing device 100. The external device may be, for example, a server, a user terminal, etc. The user terminal is, for example, a terminal used by a user of the information processing device 100, and may be a desktop, a laptop, a tablet, a smartphone, etc.

The storage unit 122 may store, for example, various information and programs. Examples of the storage unit 122 may be a memory, a solid state drive, a hard disk drive, etc. The storage unit 122 may be, for example, a storage area and a server on the cloud.

The display unit 123 is, for example, a display capable of displaying various characters, symbols, images, etc.

The acquisition unit 111 acquires point cloud information. The point cloud information may be stored, for example, in an external memory (not shown) and a server (not shown). The acquisition unit 111 may acquire the point cloud information from an external device such as a server via the communication unit 121. Alternatively, for example, when an external memory in which the point cloud information is stored is connected to an interface of the information processing device 100, the acquisition unit 111 may acquire the point cloud information from the external memory.

The estimation unit 112 classifies the noise generated in the point cloud into a plurality of types based on the cause of the noise and the shape of the noise, and estimates the noise contained in the point cloud information based on the learned model trained for each type and the point cloud information acquired by the acquisition unit. There are various causes of noise generated in the point cloud. For example, when classifying the causes of noise, it is often preferable to classify into three types such as moving body noise, pull noise, and other noises excluding these noises. In addition, since the point cloud is acquired by scanning the laser light, the shape of the noise generated in the point cloud is often different from that of the measurement object from which the point cloud is desired to be acquired. Therefore, the estimation unit 112 classifies the types of noise into a plurality of types (for example, into the above-mentioned three types) based on the cause of the noise generated in the point cloud and the shape of the noise, and uses the learned model trained for each type. It has been confirmed that by classifying in this way, each type of noise can be estimated more accurately.
The types of noise are not limited to the above three types, and may be classified into two or less types, or may be classified into four or more types. That is, as an example, when classifying the causes of noise and the shapes of noise, it is preferable to classify into five types such as moving body noise, pull noise, aerial noise, reflection/specular noise, and other noise.

As an example, the estimation unit 112 estimates noise included in the point cloud information according to the learned model based on the point cloud, moving body noise, pull noise occurring around the object, aerial noise occurring in space, reflection/specular noise occurring on the back side of the subject as viewed from the transmitting/receiving device, and at least one of other noises, and the point cloud information acquired by the acquisition unit 111. The other noise may be various noises constituted by a collection of point clouds other than moving body noise, pull noise, aerial noise, and reflection/specular noise. The learned model may be a model that has learned moving body noise, pull noise, aerial noise, reflection/specular noise, other noise, and noise other than the five types of noise described above. The noise other than the five types of noise described above may include noise constituted by a collection of point clouds, or may include a collection of point clouds that are not noise.
The moving object noise, the back noise, the aerial noise, the reflection/mirror noise, and other noises may be, for example, noises constituted by a point cloud, etc. In other words, the noises estimated by the estimation unit 112 may be noises constituted by a point cloud included in the point cloud information, etc.

FIG. 3 is a diagram for explaining an example of moving body noise.
FIG. 4 is a diagram for explaining an example of subtraction noise.

The moving object noise may be, for example, a point cloud detected by light reflected by a moving object moving within a measurement range of the point cloud measurement. As shown in an example in Fig. 3, the moving object noise may be, for example, a point cloud (points 301) when laser light is reflected by an object (moving object 302) other than the measurement target object (object 202), such as a person, bicycle, or vehicle moving between a point cloud measurement transmitting/receiving device 201 and an object 202 (for example, a building).
4, the pull noise may be, for example, a point cloud (points 304) (virtual image) that does not represent the measurement object (object 203) and occurs around the contour (around points 303 (point cloud) of object 203) of the measurement object (object 203). That is, pull noise is likely to occur when the angle of incidence of the laser light and sound waves with respect to the measurement object (object 203) is relatively shallow, and as a specific example, it is particularly likely to occur on poles, thin pillars, fences, etc.
The other noise may be, for example, a point group that is a variety of noises different from the above-mentioned moving object noise and pulling noise.

The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 into the learned model and outputs an estimation result of the point cloud noise (moving object noise, subtraction noise, and other noise) contained in the point cloud information.

The estimation unit 112 may estimate the moving body noise included in the point cloud information based on a trained model that has trained the point cloud and the point cloud noise (moving body noise) generated by a moving body that moves between the measurement point of the point cloud and the subject, and the point cloud information acquired by the acquisition unit 111. The measurement point may be a device for transmitting and receiving the point cloud measurement, etc. The subject may be an object to be measured, etc.
The trained model may be a model that has trained on a moving body noise (point cloud) and a point cloud other than the moving body noise. The above-mentioned non-moving body noise may be a noise (point cloud) other than the moving body noise, or may include a collection of point clouds that are not noise.
That is, the trained model may be a model that estimates moving object noise (moving object noise composed of a point cloud). The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 to the trained model, and outputs an estimation result of the moving object noise (point cloud noise) included in the point cloud information.

The estimation unit 112 may estimate the pull noise occurring around the object, which is included in the point cloud information, based on a trained model that has learned about the point cloud and the point cloud noise (pull noise) occurring around the object, and the point cloud information acquired by the acquisition unit 111.
The trained model may be a model that has trained on a subtraction noise (point cloud) and a non-subtraction noise (point cloud). The non-subtraction noise may be a noise (point cloud) other than the subtraction noise, or may include a collection of non-noise point clouds.
That is, the trained model may be a model that estimates subtraction noise (a subtraction noise composed of a point cloud). The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 to the trained model, and outputs an estimation result of the subtraction noise (a noise of the point cloud) included in the point cloud information.

The estimation unit 112 may estimate aerial noise occurring in space and included in the point cloud information, based on a learned model that has learned a point cloud and point cloud noise (aerial noise) occurring in a space without a subject, as a learned model, and the point cloud information acquired by the acquisition unit 111. The aerial noise is, for example, noise of a collection of point clouds occurring in a space without a measurement object such as a subject.
The trained model may be a model that has been trained on aerial noise (point cloud) and a point cloud other than aerial noise.
In other words, the trained model may be a model that estimates aerial noise (aerial noise composed of a point cloud). The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 to the trained model, and outputs an estimation result of the aerial noise (point cloud noise) included in the point cloud information.

The estimation unit 112 may estimate the reflection/mirror noise that occurs on the far side of the subject as viewed from the transmitting/receiving device, which is included in the point cloud information, based on a learned model that has learned the point cloud and the noise of the point cloud (reflection/mirror noise) that occurs on the far side of the subject reflecting the laser light or sound waves emitted from the transmitting/receiving device as a learned model, and the point cloud information acquired by the acquisition unit 111. The reflection/mirror noise is a noise of a collection of point clouds that occurs on the far side of the subject (object) reflecting the laser light or sound waves emitted from the transmitting/receiving device as viewed from the transmitting/receiving device. As an example, the reflection/mirror noise may occur on a mirror, underwater, etc.
The trained model may be a model that has been trained on reflection/specular noise (point cloud) and a point cloud other than reflection/specular noise.
That is, the trained model may be a model that estimates reflection/specular noise (reflection/specular noise composed of a point cloud). The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 to the trained model, and outputs an estimation result of the reflection/specular noise (noise of the point cloud) included in the point cloud information.

The estimation unit 112 may estimate other noises excluding moving body noise, pull noise, aerial noise, and reflection/mirror noise contained in the point cloud information based on a learned model that has learned the point cloud and other noises excluding moving body noise, pull noise, aerial noise, and reflection/mirror noise, and the point cloud information acquired by the acquisition unit 111.
The trained model may be a model that has trained other noise (point cloud) and non-other noise (point cloud). The non-other noise may be noise other than other noise (for example, moving body noise, back noise, aerial noise, reflection/mirror noise, etc.) (point cloud), or may include a collection of non-noise point clouds.
In other words, the trained model may be a model that estimates other noise (other noise composed of point clouds). The estimation unit 112 inputs the point cloud information acquired by the acquisition unit 111 to the trained model, and outputs an estimation result of other noise (point cloud noise) included in the point cloud information.

In addition, the estimation unit 112 may estimate each noise in sequence, for example, by using a first learned model that estimates moving body noise, a second learned model that estimates subtraction noise, and a third learned model that estimates other noises.
As an example, the estimation unit 112 may connect the output of the result of the estimation using the first learned model (first estimation) to the input of the estimation using the second learned model (second estimation). Similarly, the estimation unit 112 may connect the output of the result of the estimation using the second learned model to the input of the estimation using the third learned model (third estimation). Note that the order of the noise estimation is not limited to the above-mentioned order of the first estimation, the second estimation, and the third estimation, and may be various other orders.

In addition, the estimation unit 112 may estimate each noise by using, for example, the first learned model described above, the second learned model described above, the third learned model described above, a fourth learned model that estimates aerial noise, and a fifth learned model that estimates reflective/specular noise.
As an example, in addition to the first to third estimations described above, the fourth estimation using the fourth trained model and the fifth estimation using the fifth trained model may be performed in the order of the first estimation, second estimation, fourth estimation, fifth estimation, and third estimation, as described above, or may be performed in various other orders.

Alternatively, the estimation unit 112 may input point cloud information in parallel to the first trained model, the second trained model, and the third trained model (for each trained model), and estimate each noise contained in the point cloud based on the point cloud information for each trained model. Similarly, in addition to the first to third trained models, point cloud information may also be input in parallel to the fourth and fifth trained models, and estimate each noise contained in the point cloud based on the point cloud information for each trained model.

The estimation unit 112 may also use, for example, at least one of the first to fifth learned models described above to estimate noise formed by one or more types of point clouds according to the learned model.

The estimation unit 112 is not limited to the first to fifth trained models described above, and may generate trained models according to multiple types based on the cause of noise occurring in the point cloud and the shape of the noise, and estimate noise formed in the point cloud using the trained models. In other words, the number of trained models may be one or more depending on the number of types of noise to be estimated.

The removal unit 113 may remove points that are noise estimated by the estimation unit 112 from the point cloud information acquired by the acquisition unit 111. In other words, the removal unit 113 may remove noise (point cloud noise) estimated as described above from a point cloud based on the point cloud information.

Alternatively, instead of removing noise, the removal unit 113 may assign a color aspect different from that of the point cloud to the noise (noise of the point cloud) estimated as described above among the point clouds based on the point cloud information. As an example, the removal unit 113 may assign white to the point cloud based on the point cloud information, and assign a color different from white to the noise. In this case, the removal unit 113 may change the color aspect according to the type of noise estimated by the estimation unit 112, for example. As an example, the removal unit 113 may assign red to moving body noise, green to subtraction noise, and blue to other noise. In this way, the information processing device 100 can clearly show the type of noise estimated by the estimation unit 112 to the user according to the difference in color aspect.

The output control unit 114 may control the output unit to output the result of noise estimation by the estimation unit 112 and the result of processing by the removal unit 113. The output unit may be, for example, a communication unit 121, a memory unit 122, a display unit 123, etc.

The calculation unit 115 may estimate objects included in the point cloud, for example, based on a trained model (fourth trained model) that has learned about the point cloud and objects included in the point cloud (for example, various objects such as buildings) and information about the point cloud from which noise has been removed by the removal unit 113 described above.

[Information processing method]
Next, an information processing method according to an embodiment will be described.
FIG. 5 is a flowchart illustrating an information processing method according to an embodiment.

In step ST101, the acquisition unit 111 acquires point cloud information.

In step ST102, the estimation unit 112 classifies noise occurring in the point cloud into multiple types based on the cause of the noise and the shape of the noise, and estimates the noise contained in the point cloud information based on the trained model trained for each type and the point cloud information acquired in step ST101.
As an example, the estimation unit 112 estimates point cloud noise included in the point cloud information according to the learned model, based on a learned model that has learned the point cloud and at least one of moving body noise, pull noise, aerial noise, reflection/specular noise, and other noise (moving body noise, pull noise, aerial noise, and other noise other than reflection/specular noise), and the point cloud information acquired in step ST101.
Here, the estimation unit 112 may estimate the moving body noise contained in the point cloud information based on a learned model that has learned the point cloud and the point cloud noise generated by a moving body moving between the measurement point on the point cloud and the subject, and the point cloud information acquired by the acquisition unit 111.
In addition, the estimation unit 112 may estimate the subtraction noise occurring around the object, which is included in the point cloud information, based on a learned model that has learned the point cloud and the point cloud noise occurring around the object, and the point cloud information acquired by the acquisition unit 111.
In addition, the estimation unit 112 may estimate aerial noise occurring in space, which is included in the point cloud information, based on a learned model that has learned about the point cloud and point cloud noise (aerial noise) occurring in a space where there is no subject, and the point cloud information acquired by the acquisition unit 111.
In addition, the estimation unit 112 may estimate the reflection/mirror noise that occurs on the far side of the subject as viewed from the transmitting/receiving device, which is included in the point cloud information, based on a learned model that has learned about the point cloud and the point cloud noise (reflection/mirror noise) that occurs on the far side of the subject reflecting the laser light or sound waves emitted from the transmitting/receiving device, as viewed from the transmitting/receiving device, and the point cloud information acquired by the acquisition unit 111.
In addition, the estimation unit 112 may estimate other noises other than moving body noise, pull noise, aerial noise, and reflective/mirror noise contained in the point cloud information based on a learned model that has learned the point cloud and other noises excluding moving body noise, pull noise, aerial noise, and reflective/mirror noise, and the point cloud information acquired by the acquisition unit 111.

In step ST103, the removal unit 113 removes the noise points estimated in step ST102 from the point cloud information acquired in step ST101.

[Functions and circuits]
Next, the functions and circuits of the information processing device 100 will be described.
Each unit of the information processing device 100 may be realized as a function of a computer's arithmetic processing device, etc. That is, the acquisition unit 111, the estimation unit 112, the removal unit 113, the output control unit 114, and the calculation unit 115 (control unit 110) of the information processing device 100 may be realized as an acquisition function, an estimation function, a removal function, the output control unit 114, and a calculation function (control function) by a computer's arithmetic processing device, etc.
The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded in a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk. The recording medium may be referred to as a non-transitory computer-readable medium, for example.
Also, as described above, each unit of the information processing device 100 may be realized by a computer arithmetic processing device or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each unit of the information processing device 100 may be realized as a circuit constituting the arithmetic processing device or the like. That is, the acquisition unit 111, the estimation unit 112, the removal unit 113, the output control unit 114, and the calculation unit 115 (control unit 110) of the information processing device 100 may be realized as an acquisition circuit, an estimation circuit, a removal circuit, the output control unit 114, and a calculation circuit (control circuit) constituting the arithmetic processing device or the like of a computer.
The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

The information processing device 100 can combine one or any two or more of the above-mentioned units.
In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[Modification]
Next, a modified example will be described.

The information processing device 100 may perform a process (pre-processing) to reduce the number of point clouds (points) included in the point cloud information before performing the above-mentioned noise estimation, etc. By performing the pre-processing, when performing various calculations such as deep learning and machine learning (for example, the above-mentioned noise estimation, etc.), the information processing device 100 can reduce the processing load of the calculation and shorten the calculation time by reducing the number of point clouds (downsampling) compared to when the calculation is performed using all the points that make up the point cloud.

(First Example)
First, a first example of downsampling will be described.
In the first example, an example in which one average value in each voxel (multiple average values for the entire point cloud) is obtained will be described.

As a specific example, the information processing device 100 may perform the following process as preprocessing.
That is, the information processing device 100 may include an acquisition unit 111 that acquires point cloud information obtained by point cloud measurement in which light is output from each of a plurality of instrument points and the light reflected from an object is received at the instrument point; a division unit that divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition unit 111 into voxels; an assignment unit that assigns an instrument point label to each of the plurality of points in each of the voxels divided by the division unit, indicating which of the plurality of instrument points the point in each voxel corresponded to when it was acquired; a calculation unit that identifies the point in the voxel that has the most identical instrument point labels assigned to it and calculates the average value of the features of the points; and a replacement unit that replaces all of the points in each voxel with one point with the average value calculated by the calculation unit, and assigns the one point with the instrument point label identified when calculating the one point, and may perform processing according to each function.
The control unit 110 (e.g., a calculation processing unit, etc.) may realize the functions of each unit (e.g., the acquisition unit 111, the division unit, the assignment unit, the calculation unit, the replacement unit, etc.) by, for example, appropriately reading and executing various programs, etc. stored in the memory unit 122, etc.

The information processing device 100 (acquisition unit 111) acquires point cloud information obtained by point cloud measurement. In the point cloud measurement, when light (e.g., laser light) or sound waves emitted from a transmitting/receiving device are reflected by an object, the reflected light (reflected light) or sound waves (reflected sound waves) are repeatedly received by the transmitting/receiving device to generate information on a point cloud (point cloud information) composed of a plurality of points.
In this modified example, the locations where the transmitting and receiving devices are installed are defined as instrument points, and point cloud information is generated by performing point cloud measurements at each of a plurality of instrument points.
In this case, the point cloud information may be generated by simultaneously performing point cloud measurements at each of the multiple instrument points.Alternatively, the point cloud information may be generated by performing point cloud measurements at each of the multiple instrument points by sequentially performing point cloud measurements at any of the multiple instrument points.
The number of instrument points may be two or more.

The point cloud information may include, for example, information on the position (e.g., coordinates, etc.) of a point measured at each instrument point based on that instrument point, and the distance from that instrument point. Therefore, each of the multiple points included in the point cloud information may include features such as a three-dimensional or two-dimensional positional relationship, reflection intensity (the intensity of laser light or sound waves reflected by an object and received by a transmitting/receiving device), and color information such as RGB (red, green, blue).

The information processing device 100 (division unit) divides a three-dimensional positional relationship (three-dimensional space) including a plurality of points into voxels. As an example, the information processing device 100 (division unit) divides the three-dimensional positional relationship (three-dimensional space) for the plurality of points into a space (voxel) such as a regular cube. Each of the plurality of voxels (e.g., a regular cube) includes a plurality of points obtained by point cloud measurement.

The information processing device 100 (assignment unit) assigns an instrument point label to each of the multiple points in each voxel, indicating at which instrument point each of the multiple points was acquired. As an example of this case, the point cloud information records information indicating the relationship between the instrument point and the point acquired at that instrument point. Therefore, the information processing device 100 (assignment unit) assigns an instrument point label to each of the multiple points based on the point cloud information.

The information processing device 100 (calculation unit) identifies the point that is most frequently assigned with the same instrument point label among the multiple points in a voxel. The information processing device 100 (calculation unit) calculates, for each voxel, the average value of the feature amounts of the points that are most frequently assigned with the same instrument point label. An example of the feature amount may be the coordinates of the point. The information processing device 100 (substitution unit) replaces all of the multiple points in each voxel with one point of the above-mentioned average value. The information processing device 100 (substitution unit) assigns the most frequently assigned instrument point label in the voxel to the one point of the replaced average value.
The information processing device 100 (calculation unit) may calculate the average value of each feature amount for each divided space, for the reflection intensity and RGB (color information) as feature amounts. The average value of each feature amount is found by calculating the average value for each feature amount from the feature amounts of all points existing in the space within the voxel. For example, if the RGB color information is calculated by calculating the average RGB value of a blue point and a red point for each R, G, and B, the average value, purple, may be used as the color of the point after sampling.

The estimation unit 112 of this embodiment may estimate noise contained in multiple points (point cloud) based on one point of the average value replaced by the replacement unit described above (multiple points (point cloud) of the average value because there are multiple voxels) and the trained model described above.

(Second Example)
Next, a second example of downsampling will be described.
In a second example, downsampling may be performed by randomly selecting points until the number of points specified based on random numbers reaches a predetermined number (eg, k).

First, the information processing device 100 (control unit 110) prepares random numbers (for example, random numbers with a uniform distribution).
Next, the information processing device 100 (control unit 110) selects k points from among the multiple points according to the random number described above. In this case, the information processing device 100 (control unit 110) may maintain the feature amounts and feature amounts such as coordinate values of the selected points as they are without performing any processing.

(Third Example)
Next, a third example of downsampling will be described.
In a third example, downsampling may be performed by selecting an arbitrary point based on a random number, and then sequentially selecting the point furthest from the point previously selected.

First, the information processing device 100 (control unit 110) prepares random numbers (for example, random numbers such as those having a normal distribution) and randomly selects any one point from among a plurality of points.
Next, the information processing device 100 (control unit 110) selects the point that is the farthest from the one point selected as described above.
Next, the information processing device 100 (control unit 110) calculates the distance between the points selected so far and the remaining points that have not been selected, and calculates the shortest distance.
Next, the information processing device 100 (control unit 110) selects the point with the greatest shortest distance calculated as described above (the point farthest from the points selected so far).
The information processing device 100 (control unit 110) repeats the process of calculating the shortest distance and the process of selecting the point with the furthest shortest distance until the number of selected points reaches k.

[Aspects and Effects of the Present Embodiment]
Next, one aspect of this embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and this embodiment is not limited to the aspects described below. In other words, this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. In addition, a lower aspect may be able to cite any of the higher aspects.
In addition, the effects described below are merely examples, and the effects of each aspect are not limited to those described below. In addition, each aspect may, for example, exhibit at least one of the effects described below.

(Aspect 1)
An information processing device of one embodiment includes an acquisition unit that acquires point cloud information, and an estimation unit that classifies noise occurring in the point cloud into multiple types based on the cause of the noise and the shape of the noise, and estimates the noise contained in the point cloud information based on a trained model trained for each type, and the point cloud information acquired by the acquisition unit.
In this way, the information processing device can estimate the type of noise contained in the point cloud information and identify the noise. In this case, the information processing device can estimate the type of noise as moving body noise, back noise, and other noises.

In addition, in the past, when point cloud information was obtained by point cloud measurement, the noise was not classified into multiple types based on the cause of the noise occurring in the point cloud and the shape of the noise, and the noise contained in the point cloud information was not classified using a trained model trained for each type.
In the information processing device of this embodiment, for example, noise is classified into three types, namely, moving body noise, pull noise, and other noise, and it has been confirmed through experiments that the noise classification accuracy (accuracy of identifying noise) is improved compared to a case where noise is not classified into three types and estimation is performed.

(Aspect 2)
In one aspect of the information processing device, the estimation unit may estimate moving body noise contained in the point cloud information based on a learned model that has learned a point cloud and point cloud noise generated by a moving body moving between a measurement point on the point cloud and a subject, and the point cloud information acquired by the acquisition unit.
This enables the information processing device to estimate the moving body noise contained in the point cloud information.

(Aspect 3)
In one aspect of the information processing device, the estimation unit may estimate subtraction noise occurring around the object, which is included in the point cloud information, based on a learned model that has learned the point cloud and point cloud noise occurring around the object as a learned model, and the point cloud information acquired by the acquisition unit.
This enables the information processing device to estimate the subtraction noise contained in the point cloud information.

(Aspect 4)
In one aspect of the information processing device, the estimation unit may estimate aerial noise contained in the point cloud information based on a trained model that has learned a point cloud and aerial noise occurring in a space without a subject, and the point cloud information acquired by the acquisition unit.
This enables the information processing device to estimate the aerial noise contained in the point cloud information.

(Aspect 5)
In one embodiment of the information processing device, the estimation unit may estimate the reflection/mirror noise contained in the point cloud information based on a learned model that has learned a point cloud and reflection/mirror noise that occurs on the far side of a subject that reflects laser light or sound waves emitted from the transmitting/receiving device as viewed from the transmitting/receiving device, and the point cloud information acquired by the acquisition unit.
This enables the information processing device to estimate the reflection/specular noise contained in the point cloud information.

(Aspect 6)
In one aspect of the information processing device, the estimation unit may estimate other noises contained in the point cloud information based on a learned model that has learned the point cloud and other noises excluding moving body noise, subtraction noise, aerial noise, and reflection/mirror noise as a learned model, and the point cloud information acquired by the acquisition unit.
This enables the information processing device to estimate other noises (noises other than moving object noise, back noise, aerial noise, and reflective specular noise) contained in the point cloud information.

(Aspect 7)
In one aspect of the information processing device, the estimation unit may estimate at least one noise contained in the point cloud information based on a learned model that has learned a point cloud and at least one noise selected from moving body noise, pull noise, aerial noise, and reflection/specular noise as a learned model, and the point cloud information acquired by the acquisition unit.
This allows the information processing device to estimate at least one learned noise from among moving body noise, pull noise, aerial noise, and reflection/specular noise.

(Aspect 8)
The information processing device according to one aspect may further include a removal unit configured to remove noise points estimated by the estimation unit from the point cloud information acquired by the acquisition unit.
This allows the information processing device to obtain a point cloud in which the effects of the above-mentioned noise have been reduced, and enables various calculations to be performed using the point cloud.

(Aspect 9)
In one aspect of the information processing method, a computer executes an acquisition step of acquiring point cloud information, and an estimation step of classifying noise occurring in the point cloud into multiple types based on the cause of the noise and the shape of the noise, and estimating the noise contained in the point cloud information based on a trained model trained for each type and the point cloud information acquired by the acquisition step.
As a result, the information processing method can achieve the same effects as the information processing device according to the above-described aspect.

(Aspect 10)
An information processing program according to one embodiment provides a computer with an acquisition function for acquiring point cloud information, and an estimation function for classifying noise occurring in the point cloud into a plurality of types based on the cause of the noise and the shape of the noise, and estimating noise contained in the point cloud information based on a trained model trained for each type, and the point cloud information acquired by the acquisition function.
As a result, the information processing program can achieve the same effects as the information processing device according to the above-described aspect.

100 Information processing device 110 Control unit 111 Acquisition unit 112 Estimation unit 113 Removal unit 114 Output control unit 115 Calculation unit 121 Communication unit 122 Storage unit 123 Display unit 301 Mobile noise points (point cloud)
302 Moving object 303 Points (point group) of a measurement target object (object)
304 Noise subtraction points (point cloud)

Claims (10)

An acquisition unit for acquiring point cloud information;
an estimation unit that estimates moving body noise included in the point cloud information based on a trained model that has trained on a point cloud and moving body noise of the point cloud generated by a moving body that moves between a point where a point cloud measurement transmitting/receiving device is installed and a subject, and the point cloud information acquired by the acquisition unit;
An information processing device comprising: An acquisition unit for acquiring point cloud information;
an estimation unit that estimates the subtraction noise occurring around the object, which is included in the point cloud information, based on a trained model that has trained the point cloud and the subtraction noise of the point cloud occurring around the object, and the point cloud information acquired by the acquisition unit ;
An information processing device comprising: An acquisition unit for acquiring point cloud information;
an estimation unit that estimates aerial noise included in the point cloud information based on a trained model that has trained a point cloud and aerial noise occurring in a space without a subject, and the point cloud information acquired by the acquisition unit;
An information processing device comprising: An acquisition unit for acquiring point cloud information;
an estimation unit that estimates the reflection/mirror noise included in the point cloud information based on a trained model that has trained on a point cloud and reflection/mirror noise occurring on the far side of a subject that reflects laser light or sound waves emitted from the transmitting/receiving device as viewed from the transmitting/receiving device, and the point cloud information acquired by the acquisition unit ;
An information processing device comprising: The information processing device according to claim 1 , further comprising a removal unit configured to remove noise points estimated by the estimation unit from the point cloud information acquired by the acquisition unit. The computer
An acquisition step of acquiring point cloud information;
an estimation step of estimating moving body noise included in the point cloud information based on a trained model that has learned the point cloud and moving body noise of the point cloud generated by a moving body that moves between a point where a point cloud measurement transmitting/receiving device is installed and a subject, and the point cloud information acquired by the acquisition step ;
An information processing method for performing the above. On the computer,
A function to acquire point cloud information;
an estimation function that estimates the moving body noise included in the point cloud information based on a trained model that has learned the point cloud and the moving body noise of the point cloud generated by a moving body that moves between a point where a point cloud measurement transmitting/receiving device is installed and a subject, and the point cloud information acquired by the acquisition function ;
An information processing program that realizes this. On the computer,
A function to acquire point cloud information;
An estimation function that estimates the subtraction noise occurring around the object, which is included in the point cloud information, based on a trained model that has learned the point cloud and the subtraction noise of the point cloud occurring around the object and the point cloud information acquired by the acquisition function;
An information processing program that realizes this. On the computer,
A function to acquire point cloud information;
An estimation function that estimates aerial noise included in the point cloud information based on a trained model that has trained a point cloud and aerial noise occurring in a space without a subject and the point cloud information acquired by the acquisition function;
An information processing program that realizes this. On the computer,
A function to acquire point cloud information;
an estimation function that estimates the reflection/mirror noise included in the point cloud information based on a trained model that has learned about the point cloud and the reflection/mirror noise that occurs on the far side of the subject that reflects the laser light or sound waves emitted from the transmitting/receiving device as viewed from the transmitting/receiving device, and the point cloud information acquired by the acquisition function;
An information processing program that realizes this.

Images