JP7599400B2 - Computer generated hologram generating device, method and program - Google Patents

Description

The present invention relates to a computer-generated hologram (CGH) generating device, method, and program, and in particular to a computer-generated hologram generating device, method, and program that reduces the number of object points used in CGH calculations by merging nearby object points.

Non-Patent Document 1 discloses a CGH calculation method that enables highly realistic rendering based on CG ray tracing. In Non-Patent Document 1, the hologram surface that records the propagating light waves from an object is divided into multiple small areas called element holograms as shown in Figure 15, and the rendering result of the 3D scene is obtained for each element hologram based on the ray tracing method to obtain motion parallax.

However, when a hologram surface is divided into multiple element holograms, the points on the 3D model visible from the center position of each element hologram must be acquired and stored using the ray tracing method, and the light wave distribution must be calculated. This poses the problem that the number of object points to be stored is at most the number of element holograms E x the number of light rays N, which means that the memory size required for CGH calculations becomes enormous.

To solve these technical problems, the inventors of the present invention have filed a patent application (Patent Document 1) for an invention that reduces the number of object points by integrating multiple object points that are close to each other in the point clouds acquired for each element hologram into one, thereby reducing the memory size required to hold the point clouds that are the subject of calculations of object light wave propagation.

Patent Application No. 2021-068626

Ichikawa, T. Yoneyama and Y. Sakamoto, "CGH calculation with the ray tracing method for the Fourier transform optical system," Opt. Express 21, 32019-32031 (2013). William E. Lorensen, Harvey E. Cline: Marching Cubes: A high resolution 3D surface construction algorithm. In: Computer Graphics, Vol. 21, Nr. 4, July 1987. J. -D. Boissonnat, Geometric structures for three-dimensional shape representation. ACM Trans. Graph. 3, 4 (1984) 266. Ryosuke Watanabe, Takamasa Nakamura, Masaya Mitobe, Yuji Sakamoto and Sei Naito, "Fast calculation method for viewpoint movements in computer-generated holograms using a Fourier transform optical system," Appl. Opt. 58, G71-G83 (2019)

According to Patent Document 1, the number of object points to be calculated for object light wave propagation can be reduced, and the memory size required to store them can be reduced. Specifically, the memory size required to secure a memory area for a point cloud that registers the position coordinates and color information for each object point, and a memory area for an occlusion table that registers occlusion information can be reduced.

On the other hand, if such point cloud integration processing could be performed in parallel between object points, it would be possible to reduce memory size and generate CGHs quickly. In order to perform parallel processing efficiently, it is desirable to reserve the necessary memory size in advance for the point cloud and occlusion table so that there is no excess or shortage during processing.
However, if the number of object points reduced by the integration process and used to generate the CGH (number of object points after integration) is unknown, the required memory size cannot be grasped and must be overestimated, which prevents effective memory use.
Moreover, if the required memory size is underestimated, it becomes necessary to expand the memory size during processing. Moreover, if the expansion amount is kept to a minimum with an eye on efficient memory usage, memory shortages will occur successively, and each time this occurs, it will be necessary to expand the memory area, preventing the full benefit of parallel processing, such as the high-speed generation of CGHs.

The object of the present invention is to provide a computer generated hologram generating device, method, and program that solves the above technical problems, and that quickly generates a CGH by estimating in advance the number of object points to be included in the point cloud after integration, thereby reserving in advance a memory area of the size required to register the object points to be calculated for object light wave propagation, and enabling object point integration processing to be performed in parallel between object points without causing a memory size shortage.

To achieve the above object, the present invention is characterized in that it is an apparatus for generating a computer-generated hologram for each frame based on interference calculations between an object beam and a reference beam on a hologram surface, and has the following configuration.

(1) The device is provided with a means for arranging a 3D model used in hologram calculation in a virtual space, a means for acquiring object points of the 3D model for each element hologram on the hologram surface, a means for registering the acquired object points in a memory area, and in this case, a means for integrating object points that are in the same position or close to each other into a single object point in advance, a means for estimating the number of object points after integration for each frame, a means for preserving the memory area with a size according to the number of object points after integration, and a means for calculating light wave propagation to each element hologram for each object point registered in the memory area, and the acquisition, integration, and registration of object points in the memory area are performed in parallel between object points.

(2) The means for estimating the number of integrated object points is such that the number of integrated object points in the current frame is determined by multiplying the number of integrated object points in the previous frame by a predetermined ratio.

(3) The means for acquiring object points includes a means for acquiring, as object points, intersections between light rays and the 3D model by ray tracing from each element hologram, and a means for calculating the density of light rays passing through an area in the virtual space in which the 3D model is placed, and the means for estimating the number of object points after integration is configured to set the ratio to a smaller value for frames in which the light ray density in the area in which the 3D model is placed is higher.

(4) A means for calculating the amount of movement of the 3D model between frames is provided, and the means for estimating the number of object points after integration is configured to set the ratio to a larger value as the amount of movement of the 3D model increases.

(5) If the allocated memory area becomes insufficient, some of the registered object points will be randomly discarded.

The present invention achieves the following effects:

(1) When integrating identical or nearby object points to reduce the number of object points in a point cloud, the number of object points after integration is calculated in advance, and a memory area of the size required to register the object points used in the light wave propagation calculation is reserved in advance, so that the occurrence of processing to expand the memory area during the process of performing the processes related to object point registration in parallel can be suppressed. As a result, it is possible to maximize the effect of performing the processes related to object point registration in parallel, namely, high-speed CGH generation.

(2) The number of integrated object points in the current frame is estimated based on the number of integrated object points in the previous frame, enabling highly accurate estimation with a small amount of calculation.

(3) The ratio by which the number of integrated object points in the previous frame is multiplied to estimate the number of integrated object points in the current frame is optimized based on the light density distribution in the area where the 3D model is located, making it possible to estimate the number of integrated object points in the current frame more accurately.

(4) The ratio by which the number of integrated object points in the previous frame is multiplied to estimate the number of integrated object points in the current frame is optimized based on the amount of movement of the 3D model, making it possible to estimate the number of integrated object points in the current frame more accurately.

(5) If the allocated memory space becomes insufficient, registered object points are randomly discarded, so that the processing related to object point registration can be performed in parallel within the allocated memory space without significantly degrading the quality of the CGH.

1 is a functional block diagram showing a configuration of a computer-generated hologram generating device according to a first embodiment of the present invention. FIG. 13 is a diagram showing a method of integrating multiple object point light sources at the same or nearby positions and each registered in a different elementary hologram into a single common object point light source. FIG. 13 is a diagram showing an example of defining an integrated range of an object point light source. FIG. 13 is a diagram showing an example of integrating multiple object point light sources in the same voxel into one. FIG. 13 is a diagram showing an example of a method for calculating occlusion. FIG. 13 is a diagram showing an example of an occlusion table. FIG. 11 is a functional block diagram showing the configuration of a computer-generated hologram generating device according to a second embodiment of the present invention. This is a diagram showing an example of the distribution of rays in the ray tracing method used to obtain object points in a virtual space where a 3D model is placed. FIG. 13 is a diagram showing an example of calculating a one-dimensional light ray density distribution by obtaining the number of passing light rays for each grid. FIG. 13 is a diagram showing an example of the arrangement distribution of 3D models. FIG. 13 is a diagram showing an example in which the light density is high near the central axis. FIG. 13 is a diagram showing an example of defining a central region of a virtual space. FIG. 13 is a diagram showing an example of a filter in which the probability distribution becomes large near the center. FIG. 11 is a functional block diagram showing a configuration of a computer-generated hologram generating device according to a third embodiment of the present invention. FIG. 13 is a diagram showing an example of obtaining motion parallax by obtaining rendering results of a 3D scene for each element hologram based on a ray tracing method.

The following describes in detail the embodiments of the present invention with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of a computer-generated hologram generating device 1 according to a first embodiment of the present invention, and includes a 3D model input unit 10, an object point acquisition unit 20, a post-integration object point number estimation unit 30, a memory area allocation unit 40, an object point integration unit 50, a light wave propagation calculation unit 60, an interference fringe calculation unit 70, a CGH output unit 80, and a global memory 90 as main components, and the object point integration unit 50 includes an occlusion determination unit 501.

A hologram is constructed by virtually arranging multiple element holograms in a two-dimensional space. A 3D model is input to the 3D model input unit 10 on a frame-by-frame basis, and a holographic video is output from the CGH output unit 80.

Such a computer-generated hologram generating device 1 can be configured by implementing applications (programs) that realize each function on at least one general-purpose computer or server equipped with a CPU, ROM, RAM, storage, bus, interface, etc. Alternatively, it can be configured as a dedicated machine or a single-function machine in which part of the application is implemented as hardware or software.

The 3D model input unit 10 acquires the 3D model used for hologram calculation on a frame-by-frame basis and places it in the virtual space as mesh data. The 3D model may be directly input as mesh data such as an OBJ file, or voxel data may be converted into mesh data using a meshing algorithm such as the Martin Cube method in Non-Patent Document 2 and then input. Alternatively, point cloud data may be converted into mesh data using the Boissonnat method in Non-Patent Document 3 or the like and then input.

The object point acquisition unit 20 acquires points on the 3D model that can be observed from the center of each element hologram as object points to be used in light wave propagation calculations using the ray tracing method disclosed in Non-Patent Document 1. The ray tracing method is a method that enables realistic rendering that takes into account hidden surface removal, shading, etc. by sending multiple light rays from a single point and registering position information and color information of polygons in the 3D model that intersect with the light rays.

In this embodiment, an object point is placed at the intersection of a ray from the center of each element hologram and the 3D model using ray tracing, and its three-dimensional coordinates (x, y, z) and color information (R, G, B) are acquired and stored. Because the position from which the ray is projected differs for each element hologram, a different point cloud can be acquired for each element hologram, making it possible to reproduce holograms with smooth continuous parallax.

The object point integration unit 50 targets the object points acquired by the object point acquisition unit 20 for each element hologram, and as shown in FIG. 2, integrates object points detected from different element holograms that have the same three-dimensional coordinates or are close to each other into a single common point, thereby reducing the number of object points used in the light wave propagation calculation.

In this embodiment, the three-dimensional space in which object points exist is divided into a cubic voxel grid as shown in Figure 3, and multiple object points in the same voxel are integrated into one object point as shown in Figure 4. For example, if a total of eight object points registered in eight selected element holograms are in the same voxel, the eight pieces of coordinate information and color information are represented by one piece of coordinate information and color information, so the amount of data related to registering the coordinate information and color information is reduced to 1/8.

For the position information of one integrated point, the average value of the 3D coordinates of the integrated multiple points can be used as the integrated 3D coordinate. Alternatively, to reduce the amount of calculations, the barycentric coordinates of the voxel can be used as the integrated 3D coordinate. For the color information of one integrated point, the average value of the color information of the integrated multiple points can be used, or the color information of any of the integrated multiple points can be used.

In this embodiment, the object points to be integrated are assumed to be close enough to be considered the same point when observed by the human eye, and the voxel size is automatically set using the following formula (1) based on the shortest distance l0 between the hologram surface and the surface of the 3D model, so that the length of one side L is observed to be equal to or less than the spatial resolution of human vision (1/60°), based on the distance l between the 3D model and the hologram surface. As a result, the voxel size is set based on the spatial resolution of human vision, making it possible to reduce memory size without compromising subjective image quality even when multiple point groups are integrated into one.

Note that, although a rectangular parallelepiped voxel grid is used in this embodiment, the present invention is not limited to this. For example, when considering the detection of object points using the ray tracing method in this embodiment, the distance between points tends to increase the farther the object point is located from the hologram surface. Therefore, a grid may be formed such that the size of the grid increases the farther the distance from the hologram surface.

One way to achieve this is to consider the arrangement of object point light sources in a 3D polar coordinate system expressed by the distance l from the center of the hologram surface, the elevation angle θ, and the azimuth angle Φ, and form a grid by dividing each axis into fixed lengths or angles.

The grid may also be represented as a viewing cone with a base parallel to the hologram surface. In this case, the space may be divided by creating a grid of smaller viewing cones with a constant length in the z-axis direction (the z-axis is perpendicular to the hologram surface) and a constant angle of view within the viewing cone that contains the entire 3D region in which the points reside.

As described in detail below, in order to reserve in advance the memory size required to register object points used in light wave propagation calculations and to enable parallel processing of object point registration, the post-integration object point number estimation unit 30 estimates the post-integration object point number of the current frame based on the post-integration object point number of the previous frame for each frame before the object point integration unit 50 integrates identical or adjacent object points to reduce the number of object points in the point cloud.

In this embodiment, the number of integrated object points N ^to _f in the current frame (fth frame) is estimated as a number αN'f _-1 obtained by multiplying the number of integrated object points N'f _-1 in the previous frame (f-1th frame) by a constant rate α.

In a CGH video, the 3D model does not move significantly between frames, and it is empirically recognized that there is no significant change in the number of object points integrated by object point integration unit 50. Therefore, for example, the post-integration object points N ^to _f in the current frame can be estimated with high accuracy simply by multiplying the post-integration object point number N' _f-1 in the previous frame by a constant rate α.

The ratio α may be a predetermined fixed value, or may be changed for each frame depending on changes in the model placement or the amount of movement of the model between frames. Alternatively, both may be combined, with the ratio α being set to a fixed value for frame sections where the amount of movement of the 3D model is small, and the ratio α being adaptively changed depending on the amount of movement for frame sections where the amount of movement is large.

The ratio α should be set to α>1 so that a larger memory area than the previous frame can be secured, taking into account errors in the estimated value. Also, since the object position does not change significantly between frames, a value of α of at most 2 is sufficient. For example, when displaying a person's upper body in 3D using holography during a remote conference, the position of the 3D captured person in the virtual space does not change significantly, so it is effective to set an appropriate fixed ratio α based on past experience.

Before the object point integration unit 50 starts the integration process, the memory area allocation unit 40 allocates a memory area of a size required for the object point integration process in the global memory 90 based on the result of estimating the number of object points after integration. In the allocated memory area, a point cloud area M1 (hereinafter, sometimes referred to as a point cloud integration memory area), an occlusion table M2, and a grid number storage array M3 are allocated.

In the point cloud area M1, the position coordinates and color information for each integrated object point are registered. In the occlusion table M2, occlusion information is registered for each integrated object point indicating from which element hologram the object point is visible and from which element hologram it is invisible. In the grid number storage array M3, the number of the grid in which the object point exists is registered.

In this embodiment, by reserving memory areas M1, M2, and M3 for the integrated point cloud in advance, as described in detail below, the object point acquisition unit 20 and the object point integration unit 50 cooperate to acquire object points for each element hologram, and furthermore, object points located within the same grid are integrated and stored. Furthermore, the process of generating an occlusion table can be performed in parallel for each element hologram and between element holograms.

In this embodiment, when multiple object points are stored in the same grid, they are merged. A consecutive grid number is assigned to each grid in advance as an identifier, and an occlusion table M2 is also created at the time of the merger to manage from which element hologram each object point after the merger can be observed.

When calculating the CGH for the first frame, it is not possible to estimate the number of object points after integration from previous frames, so similar to conventional technology, the process of acquiring object points, integrating nearby object points, and registering them in the point cloud region M1 and occlusion table M2 is repeated serially for each object point.

From the second frame onwards, a memory area of a size according to the number of integrated object points in the current frame, estimated based on the number of integrated object points in the previous frame, is secured in global memory 90, and the acquisition of object points, integration of nearby object points, and registration in point cloud area M1 and occlusion table M2 are processed in parallel up to the number of element holograms E x number of rays N.

The memory area allocation unit 40 basically reuses the memory area used in the previous frame. If there is a memory area shortage, it may allocate additional memory area to make up for the shortage, or it may repeatedly release and allocate memory area for each frame.

Once the memory area is secured, the object point acquisition unit 20 uses the ray tracing method to virtually irradiate light rays from the center point of each element hologram simultaneously, and places object points at the intersections of each light ray and the 3D model using parallel processing.

The object point integration unit 50 uses parallel processing to calculate the storage grid number for each placed object point based on its position information. If the storage grid number does not exist in the grid number storage array M3, it is added, and the position information and color information of the object point are newly registered and recorded in the point cloud area M1. Furthermore, an occlusion determination unit 501 determines from which element hologram the object point is visible and from which element hologram it is invisible, as shown in FIG. 5, and the determination result is registered in an occlusion table M2.

In occlusion table M2, as shown diagrammatically in FIG. 6, occlusion information is managed in table format for each combination of object point and element hologram. In this embodiment, the object point is used as an index, and a "1" is set in the corresponding cell of the element hologram where the object point can be observed, and a "0" is set in the corresponding cell of the element hologram where the object point cannot be observed.

On the other hand, if the storage destination grid number exists in the grid number storage array M3, the current object point is integrated with the object point already registered in the point cloud area M1, and the point cloud area M1 and occlusion table M2 are updated.

In this embodiment, the same grid number is calculated at the same time as the storage destination of the object point, and it is possible that the same grid number and object point are stored in duplicate, though this is extremely rare. To prevent this, a function may be provided that checks for duplicate grid numbers after the integration process and deletes duplicate data. Specifically, the point cloud region M1, occlusion table M2, and grid number storage array M3 are sorted by grid number, and if adjacent data have the same grid number, one of them is deleted.

Furthermore, if the actual number of integrated object points exceeds the estimated value and the memory allocated in advance is insufficient, the number of integrated object points can be reduced to below the estimated value by randomly deleting some of the integrated object points. Since the optically reconstructed image of a CGH can be considered as a collection of many point light sources, the shape of the reconstructed image does not change even if some of them are randomly deleted. This makes it possible to perform processes related to object point registration in parallel within the range of the allocated memory size without significantly degrading the image quality.

Instead of randomly deleting some of the integrated object points, the insufficient memory size may be sequentially secured, or when memory is insufficient, the integrated point cloud may be discarded and a larger memory area may be re-secured, and the object point acquisition unit 20 and object point integration unit 50 may then perform the above process again from the beginning.

In this embodiment, the number of integrated object points in the current frame is estimated from one past frame, but the present invention is not limited to this, and estimation may be performed by referring to the number of integrated object points in multiple past frames. For example, by adding the standard deviation to the maximum number of integrated object points in multiple past frames, it becomes possible to obtain a highly accurate estimation result from the tendency of change in the number of integrated object points in past frames.

The light wave propagation calculation unit 60 performs light wave propagation calculations from each object point to an element hologram that can observe the object point, based on the information of the point cloud region M1 and the occlusion table M2. Note that the point source method disclosed in Non-Patent Document 2 can be used for the light wave propagation calculation, and the calculation formula is expressed by the following formulas (2) and (3).

Here, (x, y) indicates the pixel position on the hologram surface through which the light wave propagates, and si(x, y) is the light wave distribution on the hologram surface propagated from each object point pi. N' is the number of object points after integration, Ai is the brightness of each object point pi, ri is the distance between each object point pi and pixel (x, y) on the hologram surface, and O(i, e) is the value of the occlusion table corresponding to each object point pi and element hologram e, indicating whether or not the object point pi is observable in the element hologram e. k represents the wave number calculated from the wavelength of light, and u(x, y) is the calculated object light wave distribution.

The interference fringe calculation unit 70 has the function of performing interference calculations by inserting a reference light wave as a computer simulation into the object light wave u(x, y) on the hologram surface, and outputting the result as an image. The reference light in this embodiment uses a convergent spherical reference light wave. The method disclosed in Non-Patent Document 4 can be applied to interference calculations using a convergent spherical reference light wave.

The complex amplitude distribution R(x, y) of the light wave when this convergent spherical reference light wave is propagated onto the hologram surface is expressed by the following equation (4). Here, Ro is the intensity of the reference light, and r is the distance from the position of the reference light to the position (x, y) on the hologram surface.

The reference light is not limited to the above formula (4), but may be a simple spherical wave reference light as expressed by the following formula (5), or a parallel wave reference light as expressed by the following formula (6). Note that β in formula (6) is the angle of incidence of the reference light on the hologram surface.

The interference between the reference light wave and the object light wave is expressed by the following equation (7), where I(x, y) is the luminance distribution of the CGH.

Finally, this brightness distribution is output as an image. For example, the interference fringes are normalized to the range of 0-255 and output as an image.

According to this embodiment, when integrating identical or nearby object points to reduce the number of object points in a point cloud, the number of object points after integration is calculated in advance, and a memory area of the size required for registering object points to be used in light wave propagation calculations is reserved in advance, so that it is possible to prevent the occurrence of processing to expand the memory area during the process of performing the processes related to object point registration in parallel. As a result, it is possible to maximize the effect of performing the processes related to object point registration in parallel, namely, speeding up CGH generation.

Figure 7 is a functional block diagram showing the configuration of the main parts of a computer-generated hologram generating device 1 according to a second embodiment of the present invention. The same reference numerals as those described above represent the same or equivalent parts, and therefore their explanation will be omitted.

This embodiment is characterized in that the 3D model input unit 10 includes a ray density distribution calculation unit 101, and the coefficient α used by the integrated object point number estimation unit 30 when estimating the integrated object point number of the current frame based on the integrated object point number of the past frame is adaptively determined for each frame according to the relationship between the density distribution of the rays used in the ray tracing method and the arrangement of the 3D model.

Figure 8 shows the distribution of light rays in the ray tracing method used to acquire object points in a virtual space in which a 3D model is placed, and it can be seen that the light rays are concentrated in an area close to the central axis of the hologram surface. Since the probability of object points being integrated is higher near the central axis where the light ray density distribution is high than in other areas, it is desirable to make the ratio α relatively small for frames in which the 3D model is placed in an area of high light ray density and relatively large for frames in which the 3D model is placed in an area of low light ray density.

In this embodiment, therefore, the 3D model input unit 10 is provided with a light ray density distribution calculation unit 101, and the coefficient α used by the post-integration object point number estimation unit 30 when estimating the post-integration object point number of the current frame based on the post-integration object point number of the past frame is adaptively set based on the calculation result of the light ray density distribution calculation unit 101.

Since the ray density distribution is an index of how many rays of the ray tracing method pass through each part of the 3D model, it can be calculated in advance by evenly dividing the virtual space in which the 3D model is placed into a voxel grid and calculating the number of rays passing through each grid (number of ray passes).

In this embodiment, the number of passing rays is obtained for each grid in which a 3D model of the virtual space exists, and a one-dimensional ray density distribution as shown in FIG. 9 is calculated. Note that this voxel grid is different from the voxel grid used by the object point integration unit 50 to identify the object points to be integrated, and is used only for calculating the ray density distribution, so the voxel size does not need to be as small as the voxel size used to identify the object points to be integrated. Therefore, the ray density distribution can be calculated without requiring an additional large memory area.

In this embodiment, when the light ray distribution density is high, α can be set to 1.2, and when the light ray distribution density is low, α can be set to 2.0. On the other hand, if it is known in some way that the number of points after reduction will be smaller than in the previous frame, the ratio α can be set outside the above range. When comparing the magnitude of light ray density between frames, it is possible to compare representative values such as the median or average value of the light ray density distribution, or to compare the integral value of the distribution.

Note that the light ray density distribution may be replaced by the model layout distribution without precise calculation. For example, as shown in Figure 10, the model layout distribution can be found by labeling each 3D model that intersects with the light rays of the ray tracing method from the central coordinates of the hologram and calculating the two-dimensional model layout distribution. This is equivalent to roughly estimating the light ray density distribution based on how much the model layout distribution depends on the central axis, since the light rays of the ray tracing method used to acquire the point cloud are concentrated in an area close to the central axis of the hologram surface in virtual space, as shown in Figure 11.

When replacing the light density distribution with the model arrangement distribution, the central region of the virtual space is defined by the following equation (8), as shown in Figure 12, and the probability of the existence of a 3D model in the central region is represented by the maximum value, average value, or integral value.

By decreasing the ratio α when the representative value is higher than in the previous frame, and increasing the ratio α when the probability of the surrounding presence increases, it becomes possible to determine the optimal ratio α in the same way as when the light density distribution is used.

As shown in an example in Figure 13, it is also possible to calculate the similarity of the probability distribution with a filter designed to have larger values near the center, and obtain the model placement distribution based on the results of this calculation.

According to this embodiment, the ratio α for calculating the number of integrated object points in the current frame based on the number of integrated object points in the previous frame can be optimized based on the light density of the area in which the 3D model is placed, making it possible to estimate the number of integrated object points more accurately.

Figure 14 is a functional block diagram showing the configuration of a computer-generated hologram generating device 1 according to a third embodiment of the present invention. The same reference numerals as those described above represent the same or equivalent parts, and therefore their explanation will be omitted.

This embodiment is characterized in that the 3D model input unit 10 includes a movement amount calculation unit 102, and the coefficient α used by the integrated object point number estimation unit 30 when estimating the integrated object point number of the current frame based on the integrated object point number of the past frame is adaptively set for each frame based on the movement amount of the 3D model.

When placing a 3D model in a virtual space, the movement amount calculation unit 102 calculates the change in the center of gravity position of the 3D model from the previous frame as the movement amount. Since the amount of change in the number of integrated object points between frames differs depending on the movement amount between frames, the estimation accuracy can be improved by using the movement amount of the model to estimate the number of integrated object points.

Generally, the greater the movement of the 3D model between frames, the greater the change in the 3D model arrangement between frames, and therefore the greater the change in the number of object points after integration. Therefore, in this embodiment, the ratio α is optimized by increasing the ratio α the greater the movement of the 3D model between frames.

According to this embodiment, the ratio α for calculating the number of integrated object points in the current frame based on the number of integrated object points in the previous frame is calculated based on the amount of movement of the model, so that the number of integrated object points can be estimated more accurately.

Furthermore, according to each of the above embodiments, high-quality CGH can be generated in a short time and can be provided in real time even via communication infrastructure, making it possible to provide a variety of entertainment to many people regardless of geographic or economic disparities. As a result, it will be possible to contribute to Goal 9 "Build resilient infrastructure and promote inclusive and sustainable industrialization" and Goal 11 "Make cities inclusive, safe, resilient and sustainable" of the Sustainable Development Goals (SDGs) led by the United Nations.

10...3D model input section, 20...object point acquisition section, 30...post-integration object point number estimation section, 40...memory area allocation section, 50...object point integration section, 60...light wave propagation calculation section, 70...interference fringe calculation section, 80...CGH output section, 90...global memory, 101...light ray density distribution calculation section, 102...movement amount calculation section, 501...occlusion table determination section

Claims (15)

1. An apparatus for generating a computer-generated hologram for each frame based on interference calculation between an object beam and a reference beam on a hologram plane, comprising:
A means for placing a 3D model used for hologram calculation in a virtual space; and
A means for obtaining object points of a 3D model for each element hologram of the hologram surface;
a means for registering the acquired object points in a memory area, and at that time, integrating object points having the same position or close positions into a single object point in advance;
A means for estimating the number of object points after integration for each frame;
a means for reserving in advance the memory area having a size corresponding to the number of object points after integration;
a means for calculating light wave propagation to each element hologram for each object point registered in the memory area,
A computer generated hologram generating device, characterized in that the acquisition, integration and registration of the object points in the memory area are performed in parallel between object points. The memory area is
a memory area for integrating point clouds in which position information and color information of each object point are registered;
2. The computer-generated hologram generating device according to claim 1, further comprising an occlusion table in which occlusion information is registered for each combination of an object point and an elementary hologram. The computer generated hologram generating device according to claim 1 or 2, characterized in that the means for estimating the number of integrated object points estimates the number of integrated object points of the current frame based on the number of integrated object points of a past frame. The computer generated hologram generating device according to claim 3, characterized in that the means for estimating the number of integrated object points estimates the number of integrated object points in the current frame based on the number of integrated object points in the previous frame or a statistical value of the number of integrated object points in the past few frames. The computer generated hologram generating device according to claim 3 or 4, characterized in that the means for estimating the number of integrated object points is a value obtained by multiplying the number of integrated object points of the past frame by a predetermined ratio, and setting the number of integrated object points of the current frame to this value. the means for acquiring the object point acquires an intersection point between a light ray and a 3D model as an object point by ray tracing from each element hologram;
A means for calculating a density of a light ray passing through an area in which the 3D model of the virtual space is placed,
6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a smaller value for a frame having a higher light ray density in an area in which a 3D model is placed. the means for calculating the light ray density divides the virtual space into a number of small regions and calculates the number of light rays passing through each of the small regions;
7. The computer-generated hologram generating device according to claim 6, wherein the ratio is determined based on the number of light rays in a small region including a 3D model. means for calculating frame-to-frame displacement of a 3D model;
6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a larger value as the amount of movement of the 3D model increases. a means for calculating a similarity between a placement probability distribution of the 3D model in the virtual space and a filter designed in advance so that the probability distribution near the center of the virtual space is large;
6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a smaller value as the degree of similarity increases. The computer-generated hologram generating device according to any one of claims 2 to 9, characterized in that, when the secured memory area for point cloud integration becomes insufficient, some of the registered object points are randomly discarded. The computer generated hologram generating device according to any one of claims 1 to 10, characterized in that the integrating means divides the virtual space into a number of small regions and integrates object points located within the same small region. A method for generating a computer-generated hologram for each frame by a computer based on interference calculation between an object beam and a reference beam on a hologram plane, comprising:
The 3D model used for hologram calculation is placed in the virtual space,
Obtain the object points of the 3D model for each element hologram on the hologram surface,
The acquired object points are registered in a memory area, and object points that are in the same or close positions are integrated into a single object point in advance.
Estimate the number of object points after integration for each frame.
The memory area is allocated in advance with a size corresponding to the number of object points after integration;
Calculating light wave propagation to each element hologram for each object point registered in the memory area;
A computer-generated hologram generating method, comprising the steps of: acquiring and integrating the object points and registering them in the memory area in parallel between the object points. The computer-generated hologram generating method according to claim 12, characterized in that the number of integrated object points in the current frame is estimated based on the number of integrated object points in the past frame. A program for generating a computer-generated hologram for each frame based on interference calculation between an object beam and a reference beam on a hologram plane,
The procedure for placing the 3D model used for hologram calculation in the virtual space,
A procedure for obtaining object points of a 3D model for each element hologram on the hologram surface;
A step of registering the acquired object points in a memory area, and at that time, integrating object points having the same position or close positions into a single object point in advance;
A step of estimating the number of object points after integration for each frame;
a step of allocating in advance the memory area having a size according to the number of object points after integration;
a step of calculating light wave propagation to each element hologram for each object point registered in the memory area;
A computer-generated hologram generating program, characterized in that the acquisition, integration and registration of each object point in the memory area are performed in parallel between object points. The computer-generated hologram generation program according to claim 14, characterized in that in the step of estimating the number of object points after integration, the number of object points after integration in the current frame is estimated based on the number of object points after integration in a previous frame.

Images