CN120599569A - Point cloud processing method, device, equipment, storage medium and program product - Google Patents
Point cloud processing method, device, equipment, storage medium and program productInfo
- Publication number
- CN120599569A CN120599569A CN202510711497.6A CN202510711497A CN120599569A CN 120599569 A CN120599569 A CN 120599569A CN 202510711497 A CN202510711497 A CN 202510711497A CN 120599569 A CN120599569 A CN 120599569A
- Authority
- CN
- China
- Prior art keywords
- columnar
- point cloud
- area
- region
- environmental
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
- G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional [3D] objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application relates to a point cloud processing method, a point cloud processing device, a point cloud processing storage medium and a point cloud processing program product. After the environment point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume, determining the height characteristics of each columnar area according to the environment point cloud contained in each columnar area, determining the three-dimensional characteristic tensor corresponding to each columnar area according to the point cloud characteristics and the number of the environment point clouds contained in each columnar area and the height characteristics of each columnar area, and determining the environment information in the target area according to the three-dimensional characteristic tensor corresponding to each columnar area. By adopting the method, the accuracy of environment information determination can be improved.
Description
Technical Field
The present application relates to the field of radar detection technologies, and in particular, to a point cloud processing method, apparatus, device, storage medium, and program product.
Background
With the rapid development of automatic driving technology, in order to ensure the reliability of driving path planning, a laser radar can be used for collecting environmental point clouds in the vehicle driving process, and according to the environmental point clouds, the environmental information of the environment where the vehicle is located is determined, so as to generate a driving path.
However, in the existing point cloud detection method, the environmental information of the environment where the vehicle is located is generally determined directly according to the point cloud characteristics of the environmental point cloud, but in a complex scene, the laser radar is easily affected by noise and other external influences, so that the accuracy of determining the point cloud characteristics can be reduced, and the accuracy of determining the environmental information is reduced.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a point cloud processing method, apparatus, device, storage medium, and program product capable of improving the accuracy of determining environmental information.
In a first aspect, the present application provides a point cloud processing method, including:
after the environmental point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume;
determining the height characteristics of each columnar area according to the environmental point clouds contained in each columnar area;
Determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the point cloud quantity of the environmental point clouds contained in each columnar region and the height features of each columnar region;
And determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
In one embodiment, determining the height characteristics of each columnar area based on the ambient point cloud contained within each columnar area includes:
Determining the height characteristics corresponding to different types of objects in the columnar areas according to the distribution conditions of the environmental point clouds contained in the columnar areas aiming at each columnar area;
and weighting the height characteristics corresponding to different types of objects to obtain the height characteristics of the columnar area.
In one embodiment, determining the three-dimensional feature tensor corresponding to each columnar region according to the point cloud feature and the number of the environmental point clouds contained in each columnar region and the height feature of each columnar region includes:
for each columnar area, determining the center coordinates of the area center corresponding to the columnar area according to the position information of the columnar area;
Determining the point cloud characteristics of each environmental point cloud contained in the columnar region according to the point cloud coordinates and the reflection intensity of each environmental point cloud contained in the columnar region, the center coordinates of the region center and the offset between each environmental point cloud contained in the columnar region and the region center;
And determining a three-dimensional feature tensor corresponding to the columnar region according to the point cloud features and the number of the environmental point clouds contained in the columnar region and the height features of the columnar region.
In one embodiment, determining the three-dimensional feature tensor corresponding to the columnar region according to the point cloud features and the number of the environmental point clouds contained in the columnar region and the height features of the columnar region includes:
For each environmental point cloud in the columnar area, splicing the height characteristic and the point cloud characteristic of the environmental point cloud to obtain the environmental characteristic of the environmental point cloud, and splicing the environmental characteristic, the number of the point clouds in the columnar area and the number of the areas of the columnar area containing the environmental point cloud in the target area to obtain the point cloud tensor of the environmental point cloud;
and combining the point cloud tensors of the environmental point clouds contained in the columnar region to obtain the three-dimensional characteristic tensors corresponding to the columnar region.
In one embodiment, dividing the target region into at least two columnar regions of equal volume includes:
uniformly dividing the ground size corresponding to the target area to obtain the ground size corresponding to the columnar area;
Uniformly dividing the region heights corresponding to the target regions to obtain the region heights corresponding to the columnar regions;
at least two columnar areas of equal volume are generated based on the ground size and the area height.
In one embodiment, determining the environmental information in the target area according to the three-dimensional feature tensor corresponding to each columnar area includes:
projecting the three-dimensional feature tensor corresponding to each columnar region into a pseudo image;
and carrying out convolution processing on the pseudo image to obtain the environment information in the target area.
In a second aspect, the present application further provides a point cloud processing apparatus, including:
The region dividing module is used for dividing the target region into at least two columnar regions with the same volume after the environmental point cloud in the target region where the vehicle is located is obtained;
the feature determining module is used for determining the height feature of each columnar area according to the environmental point cloud contained in each columnar area;
The tensor determining module is used for determining a three-dimensional characteristic tensor corresponding to each columnar region according to the point cloud characteristics and the point cloud quantity of the environmental point clouds contained in each columnar region and the height characteristics of each columnar region;
And the information determining module is used for determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
after the environmental point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume;
determining the height characteristics of each columnar area according to the environmental point clouds contained in each columnar area;
Determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the point cloud quantity of the environmental point clouds contained in each columnar region and the height features of each columnar region;
And determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
after the environmental point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume;
determining the height characteristics of each columnar area according to the environmental point clouds contained in each columnar area;
Determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the point cloud quantity of the environmental point clouds contained in each columnar region and the height features of each columnar region;
And determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
In a fifth aspect, the application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
after the environmental point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume;
determining the height characteristics of each columnar area according to the environmental point clouds contained in each columnar area;
Determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the point cloud quantity of the environmental point clouds contained in each columnar region and the height features of each columnar region;
And determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
The method, the device, the equipment, the storage medium and the program product for processing the point cloud introduce the height characteristics of each columnar area, divide the target area into at least two columnar areas with the same volume after the environmental point cloud in the target area of the vehicle is obtained, determine the height characteristics of each columnar area according to the environmental point cloud contained in each columnar area, and then determine the three-dimensional characteristic tensor corresponding to each columnar area according to the point cloud characteristics and the number of the point clouds of the environmental point cloud contained in each columnar area and the height characteristics of each columnar area, and further determine the environmental information in the target area according to the three-dimensional characteristic tensor corresponding to each columnar area. Compared with the related art, the method only focuses on the point cloud characteristics of the environmental point cloud, and by dividing the target area into at least two columnar areas with the same volume, determining the environmental information in the target area according to the point cloud characteristics and the number of the point clouds of the environmental point cloud contained in each columnar area and the height characteristics of each columnar area, the accuracy of the environmental information determination can be ensured.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are needed in the description of the embodiments of the present application or the related technologies will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other related drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is a flow chart of a point cloud processing method according to an embodiment;
FIG. 2 is a flow diagram of determining a three-dimensional feature tensor in one embodiment;
FIG. 3 is a flow chart illustrating the determination of a three-dimensional feature tensor in another embodiment;
FIG. 4 is a schematic view of a columnar area in one embodiment;
FIG. 5 is a flow diagram of determining environmental information in one embodiment;
FIG. 6 is a schematic diagram of pseudo-image generation in one embodiment;
FIG. 7 is a schematic diagram of a convolution process in one embodiment;
FIG. 8 is a flowchart of a point cloud processing method according to another embodiment;
FIG. 9 is a block diagram of a point cloud processing device in one embodiment;
fig. 10 is an internal structural view of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
With the rapid development of automatic driving technology, in order to ensure the reliability of driving path planning, a laser radar can be used for collecting environmental point clouds in the vehicle driving process, and according to the environmental point clouds, the environmental information of the environment where the vehicle is located is determined, so as to generate a driving path. The laser radar (LiDAR) is a sensing technology for measuring the distance of an object by utilizing laser beams, and is widely applied to the fields of automatic driving automobiles, unmanned aerial vehicles, robot navigation, intelligent traffic systems and the like. The laser radar generates three-dimensional point cloud data based on the time difference and intensity information of light propagation by emitting a pulse laser beam and receiving the returned reflected light. The point cloud data can be used for detecting objects in the surrounding environment, constructing a high-precision three-dimensional map, and supporting the environment sensing and navigation functions of automatic driving and other intelligent systems.
However, in the existing point cloud detection method, the environmental information of the environment where the vehicle is located is generally determined directly according to the point cloud characteristics of the environmental point cloud, but in a complex scene, the laser radar is easily affected by noise and other external influences, so that the accuracy of determining the point cloud characteristics can be reduced, and the accuracy of determining the environmental information is reduced.
Based on this, in an exemplary embodiment, a point cloud processing method is provided, and an example of application of the method to a terminal deployed in a vehicle is described, as shown in fig. 1, and specifically includes the following steps:
S101, after the environment point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume.
The target area is a preset range where the vehicle is located, and the environment point cloud is a point cloud in the target area.
Optionally, after the point cloud in the target area where the vehicle is located is collected by the laser radar on the vehicle, the collected point cloud may be preprocessed to obtain an environmental point cloud. The step of preprocessing may include, but is not limited to, noise point removal, filtering, downsampling, coordinate conversion, and the like.
Further, in order to analyze the information of the environmental point cloud in more detail, the target area may be divided into at least two columnar areas with the same volume according to the size information of the target area.
S102, determining the height characteristics of each columnar area according to the environmental point clouds contained in each columnar area.
The height feature is a feature describing height information of an environmental point cloud included in the columnar region.
Alternatively, for each columnar area, the height feature of the columnar area may be determined according to the height information of the ambient point cloud contained in the columnar area and/or the height information of the columnar area in the target area. For example, the environmental point clouds and the position information of the columnar area contained in the columnar area may be input into a trained first feature determination model, and the first feature determination model outputs the height feature of the columnar area according to the point cloud coordinates, the position information and the model parameters of each environmental point cloud.
In another alternative, for each columnar area, the height information of the environmental point cloud contained in the columnar area may be encoded according to the distribution condition of the environmental point cloud, and the maximum pooling process is performed, so as to extract the height feature of the columnar area.
And S103, determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the number of the environmental point clouds contained in each columnar region and the height features of each columnar region.
The point cloud feature is a feature describing position information of an environmental point cloud contained in a columnar area, the tensor is a multidimensional array used for representing a data structure, and the three-dimensional feature tensor is a three-dimensional tensor used for describing the feature of the environmental point cloud.
Alternatively, for each columnar area, the point cloud characteristics of the environmental point cloud contained in the columnar area may be determined according to the position information of the environmental point cloud contained in the columnar area. For example, the position information of the environmental point clouds contained in the columnar region may be input into a trained second feature determination model, and the second feature determination model outputs the point cloud features of the environmental point clouds contained in the columnar region according to the position information and the model parameters of each environmental point cloud.
Further, the point cloud characteristics, the number of the point clouds and the height characteristics of the columnar area of the environmental point clouds contained in the columnar area can be combined to obtain a three-dimensional characteristic tensor corresponding to the columnar area. For example, the point cloud features and the number of the point clouds of the environmental point clouds contained in the columnar area and the height features of the columnar area can be directly spliced to obtain the three-dimensional feature tensor corresponding to the columnar area.
S104, determining the environment information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
The environmental information is information about the environment in the columnar area, and is used to generate a driving route.
Optionally, after determining the three-dimensional feature tensor corresponding to each columnar region, a depth algorithm may be used to process each three-dimensional feature tensor, so as to obtain the environmental information in the target region. For example, the three-dimensional feature tensor corresponding to each columnar region may be encoded, and the three-dimensional feature tensor after the encoding process may be input into a deep learning network and a single polygon detection (Single Shot MultiBox Detector, SSD) head, and the three-dimensional feature tensor after the encoding process may be shared and learned by the deep learning network and the SSD detection head, so as to obtain the environmental information in the target region.
Further, after the environmental information in the target area is determined, operations such as obstacle early warning, path generation, environmental display and the like can be performed based on the environmental information.
According to the point cloud processing method, the height characteristics of each columnar area are introduced, after the environmental point cloud in the target area of the vehicle is obtained, the target area is divided into at least two columnar areas with the same volume, the height characteristics of each columnar area are determined according to the environmental point cloud contained in each columnar area, then the three-dimensional characteristic tensor corresponding to each columnar area is determined according to the point cloud characteristics and the number of the environmental point clouds contained in each columnar area and the height characteristics of each columnar area, and the environmental information in the target area is determined according to the three-dimensional characteristic tensor corresponding to each columnar area. Compared with the related art, the method only focuses on the point cloud characteristics of the environmental point cloud, and by dividing the target area into at least two columnar areas with the same volume, determining the environmental information in the target area according to the point cloud characteristics and the number of the point clouds of the environmental point cloud contained in each columnar area and the height characteristics of each columnar area, the accuracy of the environmental information determination can be ensured.
In order to more reasonably embody the height characteristics of the columnar areas, the embodiment of the application provides an optional mode for determining the height characteristics, specifically, for each columnar area, the height characteristics corresponding to different types of objects in the columnar area are determined according to the distribution condition of the environmental point clouds contained in the columnar area, and the height characteristics corresponding to different types of objects are weighted to obtain the height characteristics of the columnar area.
Among these, the so-called different types of objects may include, but are not limited to, people, vehicles, obstacles, etc.
Optionally, for each columnar area, according to the distribution condition of the environmental point cloud contained in the columnar area, different types of object information and height features corresponding to different types of objects in the columnar area can be determined.
Further, the weighting parameters corresponding to the various types of objects can be determined according to the influence degree of the various types of objects on the driving path generation, and then the weighting parameters corresponding to the various types of objects are adopted to carry out weighting processing on the height characteristics corresponding to the various types of objects, so that the height characteristics of the columnar area are obtained.
In the embodiment of the application, the height characteristics of the columnar area are obtained by weighting the height characteristics corresponding to different types of objects, so that the rationality of the determination of the height characteristics can be ensured.
In order to ensure the accuracy of the three-dimensional feature tensor, in the embodiment of the present application, an alternative way of determining the three-dimensional feature tensor is provided, as shown in fig. 2, specifically including the following steps:
S201, for each columnar area, determining the center coordinates of the area center corresponding to the columnar area according to the position information of the columnar area.
The point cloud center is the center point of the columnar areas, and further, the center point of the columnar areas is used for providing global position references, so that spatial consistency among the columnar areas is ensured, and the method is helpful for more accurately positioning the target object especially in a large-scale scene. The center coordinates are coordinates of the center point of the columnar region.
Alternatively, for each columnar region, the center coordinate of the center of the region corresponding to the columnar region may be determined according to the coordinate information of the vertex of each region in the position information of the columnar region. Or the position information of the columnar area can be input into a trained center determination model, and the center determination model outputs the center coordinates of the area center corresponding to the columnar area according to the position information of the columnar area and the model parameters.
S202, determining the point cloud characteristics of each environmental point cloud contained in the columnar area according to the point cloud coordinates and the reflection intensity of each environmental point cloud contained in the columnar area, the center coordinates of the area center and the offset between each environmental point cloud contained in the columnar area and the area center.
The point cloud coordinates are coordinate information of the environmental point cloud, and are three-dimensional coordinates, and further, the three-dimensional coordinates of the environmental point cloud represent absolute positions of the environmental point cloud in a global space, and can be used for identifying positions and shapes of objects.
The reflected intensity is the intensity of reflected light when the laser radar collects the environmental point cloud, and further, the reflected intensity is used for distinguishing the material or the surface characteristics of objects, so that the objects of different types can be distinguished in the same scene.
The offset is used for representing the relative coordinates between the environmental point cloud and the center of the point cloud, and further, the relative coordinates can capture the relative relation between the local geometric structure and the points and other points in the columnar units, so that the expression capability of the local spatial characteristics is improved.
It can be understood that, in order to ensure the rationality of feature determination between the columnar areas, a standard point cloud number can be preset, for each columnar area, if the point cloud number of the environmental point clouds in the columnar area is greater than the standard point cloud number, the random sampling processing is performed on the environmental point clouds, that is, the environmental point clouds with the standard point cloud number are selected, and the point cloud features of the environmental point clouds are calculated, and if the point cloud number of the environmental point clouds in the columnar area is less than the standard point cloud number, the filling is performed by using the point cloud feature 0 until the environmental point cloud number reaches the standard point cloud number. At this time, the number of ambient point clouds in each columnar area is the same.
Optionally, for each environmental point cloud in each columnar area, the point cloud characteristics of the environmental point cloud may be determined according to the point cloud coordinates and the reflection intensity of the environmental point cloud, the center coordinates of the area center of the columnar area where the environmental point cloud is located, and the offset between the environmental point cloud and the area center. For example, the point cloud coordinates and the reflection intensity of the environmental point cloud, the center coordinates of the area center of the columnar area where the environmental point cloud is located, and the offset between the environmental point cloud and the area center may be spliced to obtain the point cloud characteristics of the environmental point cloud.
And S203, determining a three-dimensional feature tensor corresponding to the columnar region according to the point cloud features and the number of the environmental point clouds contained in the columnar region and the height features of the columnar region.
The number of point clouds is the number of ambient point clouds contained in the columnar area.
Optionally, for each columnar area, the point cloud characteristics and the number of the point clouds of the environmental point clouds and the height characteristics of the columnar area contained in the columnar area may be input into a trained tensor determination model, and the tensor determination model outputs a three-dimensional characteristic tensor corresponding to the columnar area according to the point cloud characteristics, the number of the point clouds, the height characteristics and the model parameters of each environmental point cloud.
Or combining the point cloud characteristics of the environmental point cloud, the number of the point clouds of the columnar area where the environmental point cloud is located and the height characteristics of each environmental point cloud in each columnar area to obtain the point cloud tensor of the environmental point cloud, and then combining the point cloud tensors of the environmental point clouds in any columnar area to obtain the three-dimensional characteristic tensor corresponding to the columnar area.
It can be understood that, in the case where the number of ambient point clouds in each columnar area is uniformly processed, the number of point clouds in each columnar area in step S203 is the standard number of point clouds.
In the embodiment of the application, the three-dimensional feature tensor corresponding to the columnar region is determined according to the point cloud features and the point cloud quantity of the environmental point clouds contained in the columnar region and the height features of the columnar region, so that the rationality of the determination of the three-dimensional feature tensor can be ensured.
In order to further ensure the accuracy of the three-dimensional feature tensor, in the embodiment of the present application, another alternative way of determining the three-dimensional feature tensor is provided, as shown in fig. 3, specifically including the following steps:
S301, aiming at each environment point cloud in the columnar area, splicing the height characteristic and the point cloud characteristic of the environment point cloud to obtain the environment characteristic of the environment point cloud, and splicing the environment characteristic, the number of the point clouds in the columnar area and the number of the areas of the columnar area containing the environment point cloud in the target area to obtain the point cloud tensor of the environment point cloud.
The environmental feature is a feature of the environmental point cloud in a plurality of dimensions such as a height dimension, a coordinate dimension, a reflection dimension, and an offset dimension.
Optionally, for each environmental point cloud in the columnar area, the height feature of the columnar area where the environmental point cloud is located and the point cloud feature of the environmental point cloud may be spliced to obtain the environmental feature of the environmental point cloud. For example, when the coordinates of the environmental point cloud a are (x, y, z), the reflection intensity is r, the height characteristic of the columnar area where the environmental point cloud is located is s, the center coordinates are (x c,yc,zc), and the offset is (x p,yp), the point cloud of the environmental point cloud a is (x, y, z, x c,yc,zc,xp,yp, r), the environmental characteristic is (x, y, z, x c,yc,zc,xp,yp, s, r), that is, the environmental characteristic is ten-dimensional data.
Furthermore, the number of the areas of the columnar areas containing the environmental point clouds before unifying the number of the point clouds in each columnar area can be counted, and then the environmental characteristics of the environmental point clouds, the number of the point clouds in the columnar areas and the number of the areas are spliced, so that the point cloud tensor of the environmental point clouds can be obtained. For example, when the environmental feature of the environmental point cloud a is D A, the number of point clouds is N, and the number of areas is S, the point cloud tensor of the environmental point cloud a is (P, D A, N).
S302, combining point cloud tensors of all the environmental point clouds contained in the columnar area to obtain a three-dimensional characteristic tensor corresponding to the columnar area.
Optionally, after determining the point cloud tensor of each environmental point cloud, for each columnar area, the point cloud tensors of each environmental point cloud in the columnar area may be combined, so as to obtain the three-dimensional feature tensor corresponding to the columnar area. For example, the three-dimensional feature tensor of the columnar region where the ambient point cloud a is located is (P, D, N), where D is a feature combination of the environmental features corresponding to each ambient point cloud in the columnar region where the ambient point cloud a is located.
In the embodiment of the application, the environmental characteristics are obtained by splicing the height characteristics and the point cloud characteristics corresponding to the environmental point clouds, the environmental characteristics, the number of the point clouds in the columnar area and the number of the areas of the columnar area containing the environmental point clouds in the target area are spliced to obtain the point cloud tensor of the environmental point clouds, and then the point cloud tensors of the environmental point clouds contained in the columnar area are combined to obtain the three-dimensional characteristic tensor corresponding to the columnar area, so that the rationality of the three-dimensional characteristic tensor can be ensured.
In order to ensure the rationality of the division of the columnar areas, in the embodiment of the application, an alternative way for dividing the columnar areas is provided, specifically, the ground size corresponding to the target area is uniformly divided to obtain the ground size corresponding to the columnar areas, the area height corresponding to the target area is uniformly divided to obtain the area height corresponding to the columnar areas, and at least two columnar areas with the same volume are generated according to the ground size and the area height.
The ground size is the size information of the two-dimensional space where the ground is located, and the area height is the height information of each space.
Optionally, the ground size corresponding to the target area may be uniformly divided based on a preset plane step length, so as to obtain the ground size corresponding to the columnar area. For example, the ground dimensions corresponding to the target area are X and Y, and the preset plane step size is Δx and Δy, and then the ground division of each columnar area may refer to the formula p= { X/Δx, Y/Δy }, where P is the ground division set of each columnar area.
Further, a preset height threshold may be adopted to uniformly divide the region heights corresponding to the target regions, so as to obtain the region heights corresponding to the columnar regions.
After the ground division and the height division of each columnar area are determined, at least two columnar areas with the same volume can be generated. For example, refer to the schematic diagram of the columnar area shown in fig. 4.
In the embodiment of the application, the accuracy of the division of the columnar areas can be ensured by generating at least two columnar areas with the same volume according to the ground size and the area height.
In order to ensure the accuracy of determining the environmental information, in the embodiment of the present application, an alternative manner of determining the environmental information is provided, as shown in fig. 5, which specifically includes the following steps:
S501, a three-dimensional feature tensor corresponding to each columnar region is projected as a pseudo image.
Alternatively, the three-dimensional feature tensors corresponding to the columnar areas may be arranged according to the positions of the columnar areas in the target area, and each arranged three-dimensional feature tensor is projected as a pseudo image.
For example, the pseudo image hxw may be determined from the environmental characteristics of each environmental point cloud in each columnar area with reference to the following formula (1). Where H and W are the number of rows and columns of the original grid, corresponding to the X-Y plane grid division, and each pixel in the pseudo-image contains a multi-dimensional feature vector (i.e., an environmental feature that contains a height feature).
(1)
Wherein, the Is the pixel value of the pseudo image in the pixel (X, Y), and F i is the environmental characteristic of the environmental point cloud i in the columnar area.
For example, referring to the pseudo-image generation schematic diagram shown in fig. 6, after determining the three-dimensional feature tensor (P, D, N) of each columnar region, for each columnar region, local features may be extracted from the environmental features of each environmental point cloud in the columnar region by using a fully connected layer (for example, dimensions from 9 to 64), and the local features of all environmental point clouds in the same columnar region are maximally pooled (Max Pooling) to obtain a columnar region-level feature (dimension 64), and then the pooled columnar region-level feature and the local features of each environmental point cloud are spliced (64+64=128 dimensions), and the spliced features are compressed to a fixed length (for example, c=64 dimensions) by using the fully connected layer. And converting the three-dimensional feature tensor corresponding to the columnar region into a C-dimensional feature vector.
Further, the C-dimensional feature vectors of all columnar areas are arranged according to the original grid positions to form a pseudo image. Pseudo image size H X W X C (H and W are the number of rows and columns of the original grid, corresponding to the X-Y plane grid division, C is the feature dimension).
S502, performing convolution processing on the pseudo image to obtain environment information in the target area.
Alternatively, a Backbone network (Backbone) may be employed, and a two-dimensional convolutional neural network may be used to convolve the pseudo-image to extract multi-scale features, thereby obtaining environmental information within the target region.
Illustratively, the encoded pseudo-image is input into a convolutional neural network, learning spatial features. In this process, feature extraction is often performed using a convolution operation shown in the following formula (2). Through the convolution layer, the network can capture the spatial characteristics in the pseudo image, including the fine difference of the height information, and the detection effect on the target is enhanced.
(2)
Where O (x, y) is the convolved output, i (x, y) is the input pseudo-image, K (x, y) is the convolution kernel, and K is the convolution kernel size.
For example, referring to the convolution processing schematic diagram shown in fig. 7, the pseudo image is input into the convolution layer Block 1, the Block 1 adopts 3×3 convolution, batch normalization processing (BatchNorm) and activation processing (RECTIFIED LINEAR Unit, reLU) are performed on the pseudo image, local details (such as object edges) of the pseudo image can be captured to obtain first output data, the first output data is input into the convolution layer Block 2, the Block 2 adopts 3×3 convolution, batch normalization processing and activation processing are performed on the pseudo image, mesoscale features (such as object components) of the pseudo image can be extracted to obtain second output data, the second output data is input into the convolution layer Block 3, the Block 3 adopts 3×3 convolution, batch normalization processing and activation processing are performed on the pseudo image, and global semantic features (such as an object) of the pseudo image can be extracted.
Further, the resolution may be recovered by transpose convolution (deconvolution) and fused with shallow features to generate a multi-scale feature map. Specifically, the depth features are input into an up-sampling Block 1, the up-sampling Block 1 carries out deconvolution on the depth features to obtain first deconvolution data, the first deconvolution data and second output data output by the Block 2 are subjected to element-by-element addition/channel splicing to obtain first processing data, deep semantic and middle layer details can be fused, the first processing data are input into the up-sampling Block 2, the up-sampling Block 2 carries out deconvolution on the first processing data to obtain second deconvolution data, and the second deconvolution data and the first output data output by the Block 1 are subjected to feature fusion to obtain final output data (environment information) used for fusing the middle layer details and shallow high-resolution features.
In the embodiment of the application, the three-dimensional feature tensor corresponding to each columnar area is projected into the pseudo image, and the pseudo image is subjected to convolution processing to obtain the environment information in the target area, so that the accuracy of environment information determination can be ensured.
Fig. 8 is a schematic flow chart of a point cloud processing method in another embodiment, and on the basis of the foregoing embodiment, this embodiment provides an alternative example of the point cloud processing method. In connection with fig. 8, the specific implementation procedure is as follows:
s801, after the environmental point cloud in the target area of the vehicle is obtained, dividing the target area into at least two columnar areas with the same volume.
Optionally, uniformly dividing the ground size corresponding to the target area to obtain the ground size corresponding to the columnar area, uniformly dividing the area height corresponding to the target area to obtain the area height corresponding to the columnar area, and generating at least two columnar areas with the same volume according to the ground size and the area height.
S802, determining the height characteristics corresponding to different types of objects in the columnar areas according to the distribution condition of the environmental point clouds contained in the columnar areas, and carrying out weighting processing on the height characteristics corresponding to the different types of objects to obtain the height characteristics of the columnar areas.
S803, the center coordinates of the region center corresponding to each columnar region are determined from the position information of each columnar region.
S804, determining the point cloud characteristics of the environmental point clouds contained in each columnar area according to the point cloud coordinates and the reflection intensity of the environmental point clouds contained in each columnar area, the center coordinates of the area center and the offset between the environmental point clouds contained in each columnar area and the area center.
S805, determining a three-dimensional feature tensor corresponding to each columnar region according to the point cloud features and the number of the environmental point clouds contained in each columnar region and the height features of the columnar regions.
Optionally, for each environmental point cloud in the columnar area, the height feature and the point cloud feature of the environmental point cloud are spliced to obtain the environmental feature of the environmental point cloud, the environmental feature, the number of the point clouds in the columnar area and the number of the areas of the columnar area containing the environmental point cloud in the target area are spliced to obtain the point cloud tensor of the environmental point cloud, and the point cloud tensors of the environmental point clouds contained in the columnar area are combined to obtain the three-dimensional feature tensor corresponding to the columnar area.
S806, the three-dimensional feature tensor corresponding to each columnar region is projected as a pseudo image.
S807, convolution processing is performed on the pseudo image to obtain environment information in the target area.
The specific process of S801 to S807 may be referred to the description of the method embodiment, and its implementation principle and technical effect are similar, and will not be described herein.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a point cloud processing device for realizing the above-mentioned point cloud processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the point cloud processing device or devices provided below may refer to the limitation of the point cloud processing method hereinabove, and will not be described herein.
In an exemplary embodiment, as shown in fig. 9, there is provided a point cloud processing apparatus 1, including a region dividing module 10, a feature determining module 20, a tensor determining module 30, and an information determining module 40, wherein:
the region dividing module 10 is configured to divide the target region into at least two columnar regions with the same volume after acquiring an environmental point cloud in the target region where the vehicle is located;
the feature determining module 20 is configured to determine a height feature of each columnar area according to an environmental point cloud included in each columnar area;
A tensor determining module 30, configured to determine a three-dimensional feature tensor corresponding to each columnar region according to the point cloud feature and the number of the environmental point clouds included in each columnar region, and the height feature of each columnar region;
The information determining module 40 is configured to determine environmental information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.
In one exemplary embodiment, the feature determination module 20 is specifically configured to:
And aiming at each columnar area, determining the height characteristics corresponding to different types of objects in the columnar area according to the distribution condition of the environmental point clouds contained in the columnar area, and carrying out weighting treatment on the height characteristics corresponding to different types of objects to obtain the height characteristics of the columnar area.
In one exemplary embodiment, the tensor determination module 30 includes:
A first determining unit, configured to determine, for each columnar area, a center coordinate of an area center corresponding to the columnar area according to position information of the columnar area;
The second determining unit is used for determining the point cloud characteristics of each environmental point cloud contained in the columnar area according to the point cloud coordinates and the reflection intensity of each environmental point cloud contained in the columnar area, the center coordinates of the area center and the offset between each environmental point cloud contained in the columnar area and the area center;
And the third determining unit is used for determining a three-dimensional feature tensor corresponding to the columnar region according to the point cloud features and the number of the point clouds of the environmental point clouds contained in the columnar region and the height features of the columnar region.
In an exemplary embodiment, the third determining unit is specifically configured to:
For each environmental point cloud in the columnar area, splicing the height characteristic and the point cloud characteristic of the environmental point cloud to obtain the environmental characteristic of the environmental point cloud, and splicing the environmental characteristic, the number of the point clouds in the columnar area and the number of the areas of the columnar area containing the environmental point cloud in the target area to obtain the point cloud tensor of the environmental point cloud; and combining the point cloud tensors of the environmental point clouds contained in the columnar region to obtain the three-dimensional characteristic tensors corresponding to the columnar region.
In one exemplary embodiment, the region dividing module 10 is specifically configured to:
The method comprises the steps of uniformly dividing the ground size corresponding to a target area to obtain the ground size corresponding to a columnar area, uniformly dividing the area height corresponding to the target area to obtain the area height corresponding to the columnar area, and generating at least two columnar areas with the same volume according to the ground size and the area height.
In an exemplary embodiment, the information determination module 40 is specifically configured to:
and carrying out convolution processing on the pseudo images to obtain the environment information in the target area.
The respective modules in the point cloud processing device may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In an exemplary embodiment, a computer device, which may be a terminal, is provided, and an internal structure thereof may be as shown in fig. 10. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The Communication interface of the computer device is used for conducting wired or wireless Communication with an external terminal, and the wireless Communication can be realized through WIFI, a mobile cellular network, near field Communication (NEAR FIELD Communication) or other technologies. The computer program is executed by a processor to implement a point cloud processing method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the data related to the present application (including but not limited to the environmental point cloud data and the like) are all data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile memory and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (RESISTIVE RANDOM ACCESS MEMORY, reRAM), magneto-resistive Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computation, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the present application.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510711497.6A CN120599569A (en) | 2025-05-29 | 2025-05-29 | Point cloud processing method, device, equipment, storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510711497.6A CN120599569A (en) | 2025-05-29 | 2025-05-29 | Point cloud processing method, device, equipment, storage medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120599569A true CN120599569A (en) | 2025-09-05 |
Family
ID=96892544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510711497.6A Pending CN120599569A (en) | 2025-05-29 | 2025-05-29 | Point cloud processing method, device, equipment, storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120599569A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN121685940A (en) * | 2026-02-12 | 2026-03-17 | 国汽(北京)智能网联汽车研究院有限公司 | Target detection method, device, equipment, medium and product based on vehicle-mounted chip |
-
2025
- 2025-05-29 CN CN202510711497.6A patent/CN120599569A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN121685940A (en) * | 2026-02-12 | 2026-03-17 | 国汽(北京)智能网联汽车研究院有限公司 | Target detection method, device, equipment, medium and product based on vehicle-mounted chip |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113819890B (en) | Distance measuring method, distance measuring device, electronic equipment and storage medium | |
| CN114663598B (en) | Three-dimensional modeling method, device and storage medium | |
| CN114859938B (en) | Robot, dynamic obstacle state estimation method, device and computer equipment | |
| CN112200129A (en) | Three-dimensional target detection method and device based on deep learning and terminal equipment | |
| CN116091998B (en) | Image processing method, device, computer equipment and storage medium | |
| CN115546760B (en) | Point cloud sequence data processing method and device, computer equipment and storage medium | |
| CN117132649A (en) | Artificial intelligence integrated Beidou satellite navigation ship video positioning method and device | |
| CN115223146A (en) | Obstacle detection method, obstacle detection device, computer device, and storage medium | |
| KR20250131818A (en) | Parking space detection method and system | |
| CN117727001A (en) | A BEV perception prediction method for targets around vehicles based on omnidirectional vision | |
| CN115830073A (en) | Map element reconstruction method, map element reconstruction device, computer equipment and storage medium | |
| CN117576653A (en) | Target tracking methods, devices, computer equipment and storage media | |
| CN120599569A (en) | Point cloud processing method, device, equipment, storage medium and program product | |
| Lin et al. | A-SATMVSNet: An attention-aware multi-view stereo matching network based on satellite imagery | |
| CN117576199A (en) | A driving scene visual reconstruction method, device, equipment and medium | |
| CN120032182B (en) | Training method and detection method for power transmission channel tree obstacle hidden danger detection model | |
| US20250156997A1 (en) | Stochastic dynamic field of view for multi-camera bird’s eye view perception in autonomous driving | |
| CN117830991B (en) | Multimode fusion-based four-foot robot complex scene sensing method and system | |
| CN118314543A (en) | Target detection method and system for automatic driving perception | |
| Roman-Rivera et al. | 3D Point Cloud Outliers and Noise Reduction Using Neural Networks | |
| CN116664809A (en) | Three-dimensional information acquisition method, device, computer equipment, storage medium and product | |
| CN115239841A (en) | Lane line generation method, device, computer equipment and storage medium | |
| CN116664996A (en) | Feature fusion method, device, computer equipment and storage medium thereof | |
| CN115222815A (en) | Obstacle distance detection method, device, computer equipment and storage medium | |
| Faulkner et al. | Semantic segmentation on 3D point clouds with high density variations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |