JP7851868B2 - Point cloud decoding device, point cloud decoding method, and program - Google Patents

Description

This invention relates to a point cloud decoding device, a point cloud decoding method, and a program.

Non-patent document 1 discloses a technique for performing predictive coding.

Furthermore, Non-Patent Document 2 discloses a technique for performing inter-frame prediction using an inter-frame predictor selected from a reference frame in predictive coding.

G-PCC codec description, ISO/IEC JTC1/SC29/WG7 N00271 G-PCC 2nd Edition codec description, ISO/IEC JTC1/SC29/WG7 N00314

However, the method described in Non-Patent Document 1 only performs intra-prediction, which has the problem that in non-uniform scenes, the prediction residual becomes large, and the compression performance can be impaired.

Furthermore, the method described in Non-Patent Document 2 performs interpretation in addition to intraprediction. However, because it selects the interpreter from a point in the reference frame based on the laser ID and azimuth angle of the point decoded immediately beforehand, there is a problem that if the positional shift between the decoded frame and the reference frame is large, an appropriate predictor cannot be selected, and the compression performance may be impaired.

Therefore, the present invention has been made in view of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve the compression performance of encoding.

The first feature of the present invention is a point cloud decoding device that, in predictive coding, includes a tree synthesis unit that selects an interpreter based on the laser ID of the parent node of the node to be processed and the angle obtained by subtracting an offset value from the azimuth angle of the parent node.

The second feature of the present invention is a point cloud decoding method that includes a step in predictive coding in which an interpreter is selected based on the laser ID of the parent node of the node to be processed and the angle obtained by subtracting an offset value from the azimuth angle of the parent node.

The third feature of the present invention is a program that causes a computer to function as a point cloud decoder, wherein the point cloud decoder includes a tree synthesis unit that, in predictive coding, selects an interpreter based on the laser ID of the parent node of the node to be processed and the angle obtained by subtracting an offset value from the azimuth angle of the parent node.

The fourth feature of the present invention is a point cloud decoding device that, in predictive coding, includes a tree synthesis unit that decodes the laser ID and azimuth angle of the target node and selects a node having coordinate values equal to the laser ID and azimuth angle of the target node from a reference frame as an inter-predictor.

The fifth feature of the present invention is a point cloud decoding device that, in predictive coding, includes a tree synthesis unit that selects a node corresponding to the target node from a reference frame and uses the intra-predicted residual of the node corresponding to the target node as the predicted value of the intra-predicted residual of the target node.

According to the present invention, a point cloud decoding device, a point cloud decoding method, and a program capable of improving the encoding compression performance can be provided.

Figure 1 shows an example of the configuration of a point cloud processing system 10 according to one embodiment. Figure 2 shows an example of a functional block of a point cloud decoding device 200 according to one embodiment. Figure 3 shows an example of the configuration of encoded data (bitstream) received by the geometric information decoding unit 2010 of the point cloud decoding device 200 according to one embodiment. Figure 4 shows an example of the syntax configuration of GPS2011. Figure 5 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point cloud decoding device 200 according to one embodiment. Figure 6 is a flowchart showing an example of the slice data decoding process in step S505. Figure 7 is a flowchart showing an example of the coordinate prediction process in step S604. Figure 8 shows an example of the process of selecting a predictor from a reference frame in step S704. Figure 9 shows an example of the process of selecting a predictor from a reference frame in step S704. Figure 10 shows an example of the process of selecting a predictor from a reference frame in step S704. Figure 11 shows an example of the process of selecting a predictor from a reference frame in step S704. Figure 12 is a diagram showing an example of the functional blocks of the point cloud coding device 100 according to this embodiment.

The embodiments of the present invention will be described below with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components as appropriate, and various variations are possible, including combinations with other existing components. Therefore, the description of the following embodiments does not limit the content of the invention as described in the claims.

(First Embodiment)
The point cloud processing system 10 according to the first embodiment of the present invention will be described below with reference to Figures 1 to 12. Figure 1 is a diagram showing the point cloud processing system 10 according to the present embodiment.

As shown in Figure 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

The point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding the input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.

The input and output point cloud signals consist of positional and attribute information for each point within the point cloud. Attribute information includes, for example, the color and reflectivity of each point.

Here, such a bitstream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission line. Alternatively, the bitstream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200.

(Point cloud decoder 200)
The point cloud decoding device 200 according to this embodiment will be described below with reference to Figure 2. Figure 2 is a diagram showing an example of the functional blocks of the point cloud decoding device 200 according to this embodiment.

As shown in Figure 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, an inverse color transformation unit 2110, and a frame buffer 2120.

The geometric information decoding unit 2010 is configured to receive the bitstream related to geometric information (geometric information bitstream) from the bitstream output from the point cloud encoding device 100 as input and decode the syntax.

The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of the location information.

The tree synthesis unit 2020 is configured to generate tree information indicating which regions within the decoding target space contain points, using the control data decoded by the geometric information decoding unit 2010 and the occupancy code (described later) indicating which nodes in the tree contain the points as input.

Furthermore, the occupancy code decoding process may be configured to be performed within the tree synthesis unit 2020.

This process recursively divides the data space to be decoded into rectangular prisms, determines whether a point exists within each prism by referring to the occupancy code, divides the prism containing a point into multiple prisms, and then refers to the occupancy code again. This process is repeated to generate tree information.

Here, when decoding such an occupancy code, the inter-prediction method described later may be used.

In this embodiment, two methods can be used: one called "Octree," which recursively performs octree partitioning by always treating the aforementioned rectangular prism as a cube; and another called "QtBt," which performs quadtree partitioning and binary partitioning in addition to octree partitioning. Whether or not to use "QtBt" is transmitted as control data from the point cloud encoding device 100.

Alternatively, if the control data specifies the use of predictive geometry coding, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree configuration determined by the point cloud encoding device 100.

The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on this approximate surface information.

Approximate surface information is used, for example, when decoding 3D point cloud data of an object. In cases where the point cloud is densely distributed on the object's surface, instead of decoding each individual point cloud, the region where the point cloud exists is approximated and represented by a small plane.

Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode point clouds using a method called "Trisoup," for example. A specific example of the "Trisoup" process will be described later. Furthermore, this process can be omitted when decoding sparse point clouds acquired by Lidar or similar systems.

The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point in the point cloud data to be decoded, based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

The inverse coordinate transformation unit 2050 is configured to take the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform it from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

The frame buffer 2120 is configured to receive geometric information reconstructed by the geometric information reconstruction unit 2040 as input and store it as a reference frame. The stored reference frame is read from the frame buffer 2130 and used as a reference frame when the tree synthesis unit 2020 performs inter-frame prediction for frames that are different in time.

Here, the choice of which timestamp reference frame to use for each frame may be determined, for example, based on the control data transmitted as a bitstream from the point cloud encoding device 100.

The attribute information decoding unit 2060 is configured to receive the bitstream related to attribute information (attribute information bitstream) from the bitstream output from the point cloud encoding device 100 as input and decode the syntax.

The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of attribute information.

Furthermore, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, in order to generate inverse quantized residual information.

The inversely quantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090, depending on the characteristics of the point cloud to be decoded. Which unit it is output to is specified by the control data decoded by the attribute information decoding unit 2060.

The RAHT unit 2080 takes the inversely quantized residual information generated by the inverse quantization unit 2070 and the geometric information reconstruction unit 2040 as input, and is configured to decode the attribute information of each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform) (inverse Haar transform in the decoding process). For example, the method described in Non-Patent Document 1 can be used as a specific example of the RAHT process.

The LoD calculation unit 2090 is configured to receive geometric information generated by the geometric information reconstruction unit 2040 as input and generate the LoD (Level of Detail).

The Line of Data (LoD) is information used to define reference relationships (referring points and referenced points) for implementing predictive coding, which involves predicting the attribute information of one point from the attribute information of another point, and then encoding or decoding the predicted residual.

In other words, LoD (Line of Data) is information that defines a hierarchical structure in which each point included in geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

As a specific method for determining the Line of Disorder (LoD), for example, the method described in Non-Patent Document 1 above may be used.

The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inversely quantized residual information generated by the inverse quantization unit 2070. For example, the method described in Non-Patent Document 1 above can be used as a specific processing method for inverse lifting.

The reverse color conversion unit 2110 is configured to perform reverse color conversion on the attribute information output from the RAHT unit 2080 or the reverse lifting unit 2100, if the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether or not to perform such reverse color conversion is determined by the control data decoded by the attribute information decoding unit 2060.

The point cloud decoding device 200 is configured to decode and output attribute information for each point within the point cloud through the above processing.

(Geometric information decoding unit 2010)
The control data decoded by the geometric information decoding unit 2010 will be explained below using Figures 3 and 4.

Figure 3 shows an example of the configuration of encoded data (bitstream) received by the geometric information decoding unit 2010.

Firstly, the bitstream may include GPS2011. GPS2011, also known as the geometry parameter set, is a set of control data related to the decoding of geometric information. Specific examples will be described later. Each GPS2011 includes at least GPS ID information to identify each individual GPS2011 when multiple GPS2011s exist.

Secondly, the bitstream may include GSH2012A/2012B. GSH2012A/2012B, also known as geometry slice headers or geometry data unit headers, are sets of control data corresponding to slices, which will be described later. Hereafter, the term "slice" will be used, but "slice" can also be read as "data unit." Specific examples will be described later. Each GSH2012A/2012B includes at least GPS ID information for specifying the GPS2011 corresponding to each GSH2012A/2012B.

Thirdly, the bitstream may include slice data 2013A/2013B after GSH2012A/2012B. Slice data 2013A/2013B contains data encoded with geometric information. Examples of slice data 2013A/2013B include encoded data using occupancy code or predictive coding, as described later.

As described above, the bitstream is configured so that each slice data 2013A/2013B corresponds to one GSH2012A/2012B and one GPS2011.

As described above, GSH2012A/2012B allows specifying which GPS2011 to refer to using GPS ID information, thus enabling the use of a common GPS2011 for multiple slice data 2013A/2013B.

In other words, GPS2011 does not necessarily need to be transmitted for each slice. For example, as shown in Figure 3, the bitstream can be configured so that GPS2011 is not encoded immediately before GSH2012B and slice data 2013B.

Note that the configuration shown in Figure 3 is merely an example. As long as the GSH2012A/2012B and GPS2011 correspond to each slice data 2013A/2013B, other elements besides those mentioned above may be added to the bitstream components.

For example, as shown in Figure 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, it may be formatted to a different configuration than that shown in Figure 3 during transmission. Furthermore, it may be combined with the bitstream decoded by the attribute information decoding unit 2060 (described later) and transmitted as a single bitstream.

Figure 4 shows an example of the GPS2011 syntax configuration.

Please note that the syntax names described below are merely examples. If the functionality of the syntax described below is the same, the syntax names may differ.

Each GPS2011 may include GPS ID information (gps_geom_parameter_set_id) to identify each GPS2011.

Note that the Descriptor column in Figure 4 indicates how each syntax is encoded. ue(v) means it is an unsigned zero-order exponential Golomb code, and u(1) means it is a 1-bit flag.

GPS2011 may include a flag (interpretation_enabled_flag) that controls whether or not interpretation prediction is performed in the tree synthesis unit 2020.

For example, you could define that interpretation prediction is disabled when the value of `interpretation_enabled_flag` is "0", and enabled when the value of `interpretation_enabled_flag` is "1".

Note that `interpretation_enabled_flag` may be included in SPS2001 instead of GPS2011.

The GPS2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020. For example, a value of "1" for geom_tree_type may define the use of Predicative coding, while a value of "0" for geom_tree_type may define the disabling of Predicative coding.

Note that geom_tree_type may be included in SPS2001 instead of GPS2011.

The GPS 2011 may include a flag (geom_angular_enabled) to control whether the tree synthesis unit 2020 processes the data in Angular mode.

For example, if the value of geom_angular_enabled is "1", it may be defined that predictive coding is performed in Angular mode, and if the value of geom_angular_enabled is "0", it may be defined that predictive coding is not performed in Angular mode.

Note that geom_angular_enabled may be included in SPS2001 instead of GPS2011.

GPS2011 may include a flag (global_motion_enabled_flag) that controls whether or not to perform global motion compensation for interpretation in the tree synthesis unit 2020.

For example, you could define that global motion compensation is disabled when the value of global_motion_enabled_flag is "0", and disabled when the value of global_motion_enabled_flag is "1".

When performing global motion compensation, each slice data may include a global motion vector.

Note that `global_motion_enabled_flag` may be included in SPS2001 instead of GPS2011.

(Tree composition section 2020)
The processing of the tree synthesis unit 2020 will be explained below using Figures 5 to 11. Figure 5 is a flowchart showing an example of the processing in the tree synthesis unit 2020. In the following, an example of synthesizing a tree using "Predictive geometry coding" will be explained.

Note that terms such as "Predictive geometry," "Predictive geometry coding," and "Predictive tree" may also be used instead of "Predictive coding."

As shown in Figure 5, in step S501, the tree synthesis unit 2020 determines whether to use interprediction based on the value of interprediction_enabled_flag.

If the tree synthesis unit 2020 determines that interpretation prediction should be used, it proceeds to step S502; if it determines that interpretation prediction should not be used, it proceeds to step S505.

In step S502, the tree synthesis unit 2020 obtains a reference frame from the frame buffer 2120. The frame buffer 2120 may already contain one previously decoded frame, and the addition of decoded frames to the frame buffer 2020 may occur one frame at a time, or a predetermined number of frames at a time, after the decoding of each frame is completed. After obtaining the reference frame, the tree synthesis unit 2020 proceeds to step S503.

In step S503, the tree synthesis unit 2020 determines whether to perform global motion compensation based on the global_motion_enabled_flag.

If the tree synthesis unit 2020 determines that global motion compensation should be performed, it proceeds to step S504. If it determines that global motion compensation should not be performed, it proceeds to step S505.

In step S504, the tree synthesis unit 2020 performs global motion compensation on the reference frame acquired in step S502.

Here, global motion compensation is a process that corrects the global positional shift for each frame. It applies rotation and translation based on the global motion vector decoded by the geometric information decoding unit 2010 to all or a specified range of points in the reference frame.

The tree synthesis unit 2020 performs global motion compensation and then proceeds to step S505.

In step S505, the tree synthesis unit 2020 decodes the slice data. The specific processing in step S505 will be described later. After decoding the slice data, the tree synthesis unit 2020 proceeds to step S506.

In step S506, the tree synthesis unit 2020 terminates its processing. Note that the processing in steps S503 and S504, i.e., the determination and execution of global motion compensation, may be performed during the slice data decoding process in step S505.

Figure 6 is a flowchart showing an example of the slice data decoding process in step S505.

As shown in Figure 6, in step S601, the tree synthesis unit 2020 constructs a prediction tree corresponding to the slice data.

The slice data may contain a list of the number of child nodes of each node in the prediction tree, sorted in depth-first order. The prediction tree can be constructed by starting from the root node and adding the number of child nodes specified in the above list to each node, in depth-first order.

After completing the construction of the prediction tree, the tree synthesis unit 2020 proceeds to step S602.

In step S602, the tree synthesis unit 2020 determines whether processing of all nodes in the prediction tree has been completed.

If the tree synthesis unit 2020 determines that processing of all nodes in the prediction tree is complete, it proceeds to step S607. If it determines that processing of all nodes in the prediction tree is not complete, it proceeds to step S603.

In step S603, the tree synthesis unit 2020 selects the node to be processed from the prediction tree.

The tree synthesis unit 2020 may select the node following the previously processed node as the node to be processed, in depth-first order.

After the tree synthesis unit 2020 has finished selecting the nodes to be processed, it proceeds to step S604.

In step S604, the tree synthesis unit 2020 predicts the coordinates of the points corresponding to the nodes to be processed. The specific method for predicting these coordinates will be described later.

After completing the prediction, the tree synthesis unit 2020 proceeds to step S605.

In step S605, the tree synthesis unit 2020 decodes the predicted residuals of the coordinates of the points corresponding to the nodes to be processed. The slice data may include the predicted residuals of the coordinates of the points corresponding to each node.

The tree synthesis unit 2020 predicts the "predicted residuals" using a method described later, and the slice data may include the "predicted residual of the predicted residual" for the coordinates of the points corresponding to each node. In this case, the tree synthesis unit 2020 decodes the sum of the predicted value of the "predicted residual" and the decoded "predicted residual of the predicted residual" as the "predicted residual" for the point coordinates. The slice data may also include a flag indicating whether residual prediction is performed for each node.

After the tree synthesis unit 2020 completes the decoding of the predicted residuals of the target node, it proceeds to step S606.

In step S606, the tree synthesis unit 2020 reconstructs the coordinates of the points corresponding to the nodes to be processed. The tree synthesis unit 2020 may determine the point coordinates by summing the coordinates predicted in step S604 and the residuals decoded in step S605.

When Angular mode is used, the tree synthesis unit 2020 may reconstruct the coordinates using the methods described in Non-Patent Documents 1 and 2, taking into consideration that the predicted residuals and predicted coordinates are based on a spherical coordinate system.

When Angular mode is used, the tree synthesis unit 2020 may convert the reconstructed coordinates from a spherical coordinate system to a Cartesian coordinate system using the methods described in Non-Patent Documents 1 and 2.

After completing the coordinate reconstruction, the tree synthesis unit 2020 returns to step 602.

In step S607, the tree synthesis unit 2020 terminates the process in step S505.

Here, the order of steps S604 and S605 may be reversed.

Figure 7 is a flowchart showing an example of the coordinate prediction process in step S604.

As shown in Figure 7, in step S701, the tree synthesis unit 2020 decodes the predictor flag.

Here, the slice data may include flags indicating the predictor used for each node. For example, the slice data may include flags similar to those described in Non-Patent Documents 1 and 2, such as a flag indicating whether it is an inter-predictor or an intra-predictor, or an index for the inter-predictor. Furthermore, the slice data may include other flags described later.

After decoding the predictor flag, the tree synthesis unit 2020 proceeds to step S702.

In step S702, the tree synthesis unit 2020 determines whether to use the interpreter based on the predictor flag decoded in step S701.

If the tree synthesis unit 2020 determines that the interpreter should be used, it proceeds to step S704. If it determines that the interpreter should not be used, it proceeds to step S703.

In step S703, the tree synthesis unit 2020 performs intra-prediction of the coordinates of the target node.

When performing intra-prediction, the tree synthesis unit 2020 configures a predictor based on the coordinates of the parent node or ancestor nodes (e.g., the parent node's parent node, etc.) of the node to be processed, and predicts the coordinates of the node to be processed.

The tree synthesis unit 2020 may use the methods described in Non-Patent Documents 1 and 2 for configuring the intra-predictor, and may use the predictor indicated by the predictor flag decoded in step S701 from among multiple intra-predictors.

After completing the intra-prediction, the tree synthesis unit 2020 proceeds to step S705.

In step S704, the tree synthesis unit 2020 performs inter-coordinate prediction for the nodes to be processed.

When performing inter-prediction, the tree synthesis unit 2020 selects a node corresponding to the target node from the reference frame as a predictor, and uses the coordinates of the selected predictor as the predicted value of the target node's coordinates. The method for selecting predictors from the reference frame will be described later.

After completing the inter-prediction, the tree synthesis unit 2020 proceeds to step S705.

In step S705, the tree synthesis unit 2020 terminates the process in step S605.

Figures 8 to 10 show an example of the process of selecting a predictor from a reference frame in step S704 of the tree synthesis unit 2020.

However, it is assumed that Angular mode is being used. In Angular mode, the points of the parent node of the node being processed can be assumed to have been decoded immediately before or earlier.

In the example shown in Figure 8, the tree synthesis unit 2020 searches the reference frame for nodes with the same laser ID (L) as the parent node of the node to be processed and with a large azimuth angle (φ). Of these, the two nodes with the smallest azimuth angle (φ) are designated as Predictor 1 and Predictor 2, respectively.

In the example shown in Figure 9, the tree synthesis unit 2020 searches the reference frame for nodes whose laser ID (L) is equal to the parent node of the node to be processed and whose azimuth angle (φ) is greater than the angle obtained by subtracting the offset value (φ _offset ). Of these, the two nodes with the smallest azimuth angle (φ) are designated as predictor 1 and predictor 2, respectively.

The tree synthesis unit 2020 may take any value from negative to positive as the offset value.

Furthermore, such offset values may be included in the slice data as node-specific values. Additionally, such offset values may be included in the header (GSH, SPS, GPS, or other header) as values for each laser ID, slice, frame, or sequence.

For example, the tree synthesis unit 2020 may calculate and set this offset value each time as the difference between the azimuth angle of the parent node of the node to be processed and the azimuth angle of the parent node's parent node (or the ancestor node of the node to be processed).

The tree synthesis unit 2020 may set such offset values to fixed values that are integer multiples of the quantization unit of the azimuth angle.

With this configuration, by selecting a predictor based on the laser ID, azimuth angle, and azimuth angle offset value of the parent node of the processing node, a robust and appropriate predictor can be selected even if there are discrepancies or differences between the processing frame and the reference frame.

In the example shown in Figure 10, the tree synthesis unit 2020 searches the reference frame for a node that has a laser ID (L) equal to the laser ID (L) of the node to be processed and an azimuth angle (φ) equal to the azimuth angle (φ _current ) of the node to be processed, and uses it as a predictor.

The tree synthesis unit 2020, if no corresponding node exists in the reference frame, searches for a node (first node) that has the smallest azimuth angle among nodes with a laser ID (L) equal to the laser ID (L) of the node to be processed and an azimuth angle greater than the azimuth angle of the node to be processed, and a node (second node) that has the largest azimuth angle among nodes with a laser ID (L) equal to the laser ID (L) of the node to be processed and an azimuth angle smaller than the azimuth angle of the node to be processed. It then generates and interpolates a node whose coordinate values are the weighted average of the coordinate values of these two nodes (first and second nodes).

The tree synthesis unit 2020 sets the weights so that the weighted average of the azimuth angles equals the azimuth angles of the nodes being processed.

However, in the example shown in Figure 10, the tree synthesis unit 2020, after selecting the target node in step S603, must decode only the laser ID and azimuth angle by performing steps S604 to S606, and then decode the complete coordinate values by performing steps S604 to S606 again for the radius.

The tree synthesis unit 2020 may determine which predictor to use from among the predictors 1 and 2 shown in Figure 8, predictors 1 and 2 shown in Figure 9, and the predictors shown in Figure 10, based on the predictor flag.

The slice data may include a predictor flag for each node, and such predictor flags may be decoded in step S701.

Figure 11 shows an example of the residual prediction process in step S605 of the tree synthesis unit 2020.

The tree synthesis unit 2020 selects the node corresponding to the node to be processed from the reference frame, using the coordinate values of the parent node of the node to be processed, in a manner similar to that shown in Figure 8 or Figure 9.

The tree synthesis unit 2020 uses the intra-predicted residual of the node corresponding to the node being processed as the predicted value of the intra-predicted residual of the node being processed.

The tree synthesis unit 2020 may also record the intra-predicted residual of the reference frame in the frame buffer 2120 along with the reference frame.

The tree synthesis unit 2020 may apply such residual prediction only when the coordinate prediction in step S604 was performed by intra-prediction.

(Point cloud encoding device 100)
The point cloud coding device 100 according to this embodiment will be described below with reference to Figure 12. Figure 12 is a diagram showing an example of the functional blocks of the point cloud coding device 100 according to this embodiment.

As shown in Figure 14, the point cloud coding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information coding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, an LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, an attribute information coding unit 1130, and a frame buffer 1140.

The coordinate transformation unit 1010 is configured to perform a transformation process from the three-dimensional coordinate system of the input point cloud to an arbitrary different coordinate system. The coordinate transformation may, for example, be performed by rotating the input point cloud to transform the x, y, and z coordinates of the input point cloud into arbitrary s, t, and u coordinates. Alternatively, as one variation of the transformation, the coordinate system of the input point cloud may be used as is.

The geometric information quantization unit 1020 is configured to quantize the positional information of the input point cloud after coordinate transformation and to remove points with overlapping coordinates. When the quantization step size is 1, the positional information of the input point cloud and the positional information after quantization match. In other words, when the quantization step size is 1, it is equivalent to not performing quantization.

The tree analysis unit 1030 is configured to take the positional information of the quantized point cloud as input and generate an occupancy code indicating which node in the encoding target space the points reside at, based on the tree structure described later.

The tree analysis unit 1030 is configured to generate a tree structure in this process by recursively dividing the encoding target space into rectangular parallelepipeds.

Here, if a point exists within a given rectangular prism, a tree structure can be generated by recursively dividing that prism into multiple smaller prisms until the prism reaches a predetermined size. Each of these prisms is called a node. Furthermore, each prism generated by dividing a node is called a child node, and the occupancy code represents whether or not a child node contains a point, expressed as 0 or 1.

As described above, the tree analysis unit 1030 is configured to generate an occupancy code by recursively dividing the nodes until they reach a predetermined size.

In this embodiment, a method called "Octree," which recursively performs octree partitioning by always treating the aforementioned rectangular prism as a cube, and a method called "QtBt," which performs quadtree partitioning and binary tree partitioning in addition to octree partitioning, can be used.

The decision of whether or not to use "QtBt" is transmitted to the point cloud decoder 200 as control data.

Alternatively, the system may be specified to use predictive coding with an arbitrary tree configuration. In this case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoder 200 as control data.

For example, the control data in a tree structure may be configured to be decoded using the procedure described in Figures 5 to 13.

The approximate surface analysis unit 1040 is configured to generate approximate surface information using the tree information generated by the tree analysis unit 1030.

Approximate surface information is used, for example, when decoding 3D point cloud data of an object. In cases where the point cloud is densely distributed on the object's surface, instead of decoding each individual point cloud, the region where the point cloud exists is approximated and represented by a small plane.

Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using a method such as "Trisoup." Furthermore, this process can be omitted when decoding sparse point clouds acquired by a Lidar or the like.

The geometric information encoding unit 1050 is configured to encode the syntax of the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040, thereby generating a bitstream (geometric information bitstream). Here, the bitstream may include, for example, the syntax described in Figure 4.

The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of the location information.

The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information of each point in the point cloud data to be encoded (the coordinate system assumed by the encoding process, i.e., the position information after coordinate transformation in the coordinate transformation unit 1010) based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

The frame buffer 1140 is configured to receive geometric information reconstructed by the geometric information reconstruction unit 1060 as input and store it as a reference frame.

The saved reference frames are read from the frame buffer 1140 and used as reference frames when the tree analysis unit 1030 performs interpretation.

The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. Color conversion is not always necessary; whether or not the color conversion process is performed is encoded as part of the control data and transmitted to the point cloud decoder 200.

The attribute transfer unit 1080 is configured to correct attribute values to minimize distortion of attribute information based on the position information of the input point cloud, the position information of the point cloud after reconstruction by the geometric information reconstruction unit 1060, and the attribute information after color conversion by the color conversion unit 1070. A specific correction method, for example, can be applied as described in Non-Patent Document 2.

The RAHT unit 1090 takes the attribute information after the attribute transfer by the attribute transfer unit 1080 and the geometric information reconstruction unit 1060 as input, and is configured to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). For example, the method described in Non-Patent Document 2 above can be used for the specific processing of RAHT.

The LoD calculation unit 1100 is configured to receive geometric information generated by the geometric information reconstruction unit 1060 as input and generate the LoD (Level of Detail).

The Line of Data (LoD) is information used to define reference relationships (referring points and referenced points) for implementing predictive coding, which involves predicting the attribute information of one point from the attribute information of another point, and then encoding or decoding the prediction residual.

In other words, LoD (Line of Data) is information that defines a hierarchical structure in which each point included in geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

As a specific method for determining the Line of Disorder (LoD), for example, the method described in Non-Patent Document 2 above may be used.

The lifting unit 1110 is configured to generate residual information through a lifting process using the LoD calculated by the LoD calculation unit 1100 and the attribute information after attribute transfer in the attribute transfer unit 1080.

As for the specific lifting process, for example, the method described in Non-Patent Document 2 above may be used.

The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, a quantization step size of 1 is equivalent to not performing quantization.

The attribute information encoding unit 1130 is configured to encode the quantized residual information output from the attribute information quantization unit 1120 using the syntax, thereby generating a bitstream related to attribute information (attribute information bitstream).

The encoding process is, for example, context-adaptive binary arithmetic encoding. Here, for example, the syntax includes control data (flags and parameters) to control the decoding process of attribute information.

The point cloud encoding device 100 is configured to perform encoding processing on the positional and attribute information of each point within the point cloud, and to output a geometric information bitstream and an attribute information bitstream, based on the above processing.

Furthermore, the point cloud encoding device 100 and point cloud decoding device 200 described above may be implemented as programs that cause a computer to execute each function (each process).

In the above embodiments, the present invention was described using the application to a point cloud coding device 100 and a point cloud decoding device 200 as examples. However, the present invention is not limited to such examples and can also be similarly applied to a point cloud coding/decoding system equipped with the functions of the point cloud coding device 100 and the point cloud decoding device 200.

Furthermore, according to this embodiment, for example, it is possible to achieve an overall improvement in service quality in video communication, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs): "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

10...Point cloud processing system 100...Point cloud coding device 1010...Coordinate transformation unit 1020...Geometric information quantization unit 1030...Tree analysis unit 1040...Approximate surface analysis unit 1050...Geometric information coding unit 1060...Geometric information reconstruction unit 1070...Color conversion unit 1080...Attribute transfer unit 1090...RAHT unit 1100...LoD calculation unit 1110...Lifting unit 1120...Attribute information quantization unit 1130...Attribute Information encoding unit 1140... Frame buffer 200... Point cloud decoding device 2010... Geometric information decoding unit 2020... Tree synthesis unit 2030... Approximate surface synthesis unit 2040... Geometric information reconstruction unit 2050... Inverse coordinate transformation unit 2060... Attribute information decoding unit 2070... Inverse quantization unit 2080... RAHT unit 2090... LoD calculation unit 2100... Inverse lifting unit 2110... Inverse color transformation unit 2120... Frame buffer

Claims (9)

A point cloud decoding device that decodes a point cloud using predictive coding ,
A point cloud decoding device characterized by comprising a tree synthesis unit in which, in predictive coding, the device searches for nodes from the reference frame that have the same laser ID and a large azimuth angle as the parent node of the node to be processed, and uses the two nodes with the smallest azimuth angles among the found nodes as predictor 1 and predictor 2, respectively . A point cloud decoding device that decodes a point cloud using predictive coding,
A point cloud decoding device characterized by comprising a tree synthesis unit in which, in predictive coding, the laser ID is equal to the parent node of the node to be processed and the azimuth angle is greater than the angle obtained by subtracting the offset value from the reference frame, and the two nodes with the smallest azimuth angles among the found nodes are designated as predictor 1 and predictor 2, respectively. The point cloud decoding apparatus according to claim 2 , characterized in that the tree synthesis unit sets the offset value according to the value contained in the slice data or header. The point cloud decoding device according to claim 2, characterized in that the tree synthesis unit calculates and sets the offset value for each node to be processed based on the difference between the azimuth angle of the parent node of the node to be processed and the azimuth angle of the parent node's parent node, or the difference between the azimuth angle of the parent node of the node to be processed and the azimuth angle of the ancestor node of the node to be processed. The point cloud decoding apparatus according to claim 2 , characterized in that the tree synthesis unit sets the offset value in advance as a fixed value that is an integer multiple of the quantization unit of the azimuth angle. A point cloud decoding device that decodes a point cloud using predictive coding ,
The predictive coding includes a tree synthesis unit that decodes the laser ID and azimuth angle of the target node and selects a node having coordinate values equal to the laser ID and azimuth angle of the target node from the reference frame as an inter predictor .
The aforementioned tree synthesis unit,
If no node with coordinate values equal to the laser ID and azimuth angle of the node to be processed exists in the reference frame, the system searches for a first node with the smallest azimuth angle among nodes that have a laser ID equal to the laser ID of the node to be processed and an azimuth angle greater than the azimuth angle of the node to be processed, and a second node with the largest azimuth angle among nodes that have a laser ID equal to the laser ID of the node to be processed and an azimuth angle smaller than the azimuth angle of the node to be processed.
A new node is created whose coordinate values are the weighted average of the coordinate values of the first node and the coordinate values of the second node.
A point cloud decoding device characterized by selecting the newly generated node as the interpreter . A point cloud decoding device that decodes a point cloud using predictive coding ,
A point cloud decoding device characterized by comprising a tree synthesis unit in which, in the predictive coding, a node corresponding to the node to be processed is selected from a reference frame, and the intra-predicted residual of the node corresponding to the node to be processed is set to the predicted value of the intra-predicted residual of the node to be processed. A point cloud decoding method that decodes a point cloud using predictive coding ,
A point cloud decoding method characterized by having the step of searching the reference frame for nodes with the same laser ID and a larger azimuth angle as the parent node of the node to be processed in the predictive coding, and setting the two nodes with the smallest azimuth angle among the found nodes as predictor 1 and predictor 2, respectively . A program that makes a computer function as a point cloud decoder that decodes point clouds using predictive coding ,
The point cloud decoding device is,
The program is characterized by comprising a tree synthesis unit in which, in the predictive coding, it searches for nodes from the reference frame that have the same laser ID and a larger azimuth angle as the parent node of the node to be processed, and sets the two nodes with the smallest azimuth angles among the found nodes as predictor 1 and predictor 2, respectively .