JP7634861B2 - Image generating device and program - Google Patents

Description

The present invention relates to an image generating device and a program.

Traditionally, an exterior drawing of a proposed building has been created to show an image of what the building will look like after completion. Showing an exterior drawing in this way is useful for sales purposes such as home sales.

Nowadays, 3D rendering images (hereinafter simply referred to as rendering images) are often used as exterior views to show the completed image. Using rendering images, the exterior can be confirmed from various viewpoints. Furthermore, by superimposing a rendering image on an actual image of the proposed construction site, it is possible to confirm that the building will blend in with the surrounding scenery. For example, a construction image display strategy has been proposed in which a three-dimensional image of the building (exterior image 9) is superimposed on an image taken of the proposed location of the building (see, for example, Patent Document 1).

JP 2014-89697 A

By converting a real-life image of the proposed construction site into a spherical image, it is possible to check 360 degrees in all directions from a specified viewpoint. On the other hand, the rendering image also needs to be corrected to match the spherical image. Therefore, it would be ideal if the rendering image could be easily corrected to match the spherical image.

The present invention aims to provide an image generating device and program that can easily correct a rendering image to match a spherical image.

The present invention relates to an image generating device that generates a superimposed image by superimposing a rendering image that imitates a building on a captured image of a planned construction site, and that includes an image acquiring unit that acquires the captured image, which is a celestial sphere image, and the rendering image, an imaging position acquiring unit that acquires an imaging position including a height at which the captured image is captured, a superimposition position determining unit that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position, an image converting unit that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position.

The image generating device further includes a reference point acquisition unit that acquires a predetermined reference point in the captured image, a feature point acquisition unit that acquires a feature point that indicates a position in the rendering image that corresponds to the acquired reference point, and an estimation unit that estimates the imaging position based on the acquired reference point and the acquired feature point, and it is preferable that the imaging position acquisition unit acquires the estimated imaging position.

The image generating device further includes a reference point acquisition unit that acquires a predetermined reference point in the captured image, a location information acquisition unit that acquires the position of the acquired reference point on the map as location information, and an estimation unit that estimates the imaging position based on the acquired reference point and the location information, and it is preferable that the imaging position acquisition unit acquires the estimated imaging position.

It is also preferable that the image acquisition unit acquires a captured moving image including the captured image, and the image generation unit superimposes a rendering moving image including the rendering image, the temporal length of which is adjusted to match the temporal length of the captured moving image that is preset, on the captured moving image to generate a superimposed moving image.

The present invention also relates to a program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site, and causes the computer to function as an image acquisition unit that acquires the captured image, which is a 360° image, and the rendering image, an imaging position acquisition unit that acquires an imaging position including the height at which the captured image was captured, a superimposition position determination unit that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position, an image conversion unit that converts the rendering image into a 360° image based on the determined superimposition position, and an image generation unit that generates a superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position.

The present invention provides an image generating device and program that can easily correct a rendering image to match a spherical image.

1 is a schematic diagram showing a relationship between an imaging position of a captured image synthesized by an image generating device according to a first embodiment of the present invention and a planned construction site included in an imaging target. 2 is a schematic diagram showing a superimposed image superimposed by the image generating device of the first embodiment. FIG. 1 is a schematic diagram showing a captured image synthesized by an image generating device according to a first embodiment; 2 is a schematic diagram showing a rendering image synthesized by the image generating device according to the first embodiment. FIG. 1 is a block diagram showing a configuration of an image generating apparatus according to a first embodiment; FIG. 11 is a block diagram showing a configuration of an image generating device according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a configuration of an image generating device according to a third embodiment of the present invention.

Hereinafter, an image generating device 1 and a program according to each embodiment of the present invention will be described with reference to FIGS.
First, in describing the image generating device 1 and the program according to each embodiment, a generated image (hereinafter, also referred to as a superimposed image) will be described with reference to FIGS. 1 to 4. FIG.

As shown in Figures 1 and 2, for example, a superimposed image is an image in which a rendering image of a building is superimposed on an image of the proposed construction site A captured from an imaging position P. Specifically, a superimposed image is an image in which a rendering image of the building to be built is superimposed on a real-life image of the proposed construction site A. By superimposing the rendering image on the real-life image, it is possible to confirm the harmony between the building to be built and neighboring buildings, etc.

In the following embodiments, a spherical image is used as the captured image, as shown in Figure 3. This allows the entire periphery to be confirmed from a specified viewpoint. Therefore, it is possible to confirm the harmony with the surrounding environment, not just neighboring buildings, etc.

On the other hand, the rendering image is, for example, a 3D model as shown in FIG. 4. The rendering image is created using, for example, a mesh and a texture. Therefore, the rendering image needs to be converted so that it can be superimposed on a spherical image from a specific viewpoint. The image generating device 1 according to each of the following embodiments converts the rendering image based on the imaging position P including the height. This makes it possible to easily correct the rendering image to match the spherical image.

[First embodiment]
Next, an image generating device 1 and a program according to a first embodiment of the present invention will be described with reference to FIGS.
The image generating device 1 is, for example, a device that acquires a captured image and a rendered image and superimposes them. As shown in Fig. 5, the image generating device 1 includes an image acquiring unit 11, an imaging position acquiring unit 12, a superimposition position determining unit 13, an image converting unit 14, an image generating unit 15, and an output unit 16.

The image acquisition unit 11 is realized, for example, by the operation of a CPU. The image acquisition unit 11 acquires a captured image, which is a panoramic image, and a rendering image. The image acquisition unit 11 acquires, as the captured image, for example, a panoramic image captured by a user from an imaging position P from which the planned construction site A can be viewed. In addition, the image acquisition unit 11 acquires, as the rendering image, for example, an image that has been created in advance as a 3D model of a building.

The imaging position acquisition unit 12 is realized, for example, by the operation of a CPU. The imaging position acquisition unit 12 acquires an imaging position P including the height at which the captured image was captured. The imaging position acquisition unit 12 acquires, for example, latitude, longitude, and height (altitude) as the imaging position P. In this embodiment, the imaging position acquisition unit 12 acquires the imaging position P input using an input device (not shown) such as a keyboard.

The superimposition position determination unit 13 is realized, for example, by the operation of a CPU. The superimposition position determination unit 13 determines the superimposition position of the captured image and the rendering image based on the acquired imaging position P. For example, the superimposition position determination unit 13 determines the position of the planned construction site A included in the captured image as the superimposition position of the rendering image with the imaging position P as the viewpoint. The superimposition position determination unit 13 also determines a feature point of the rendering image to be superimposed on a reference point of the planned construction site A. For example, the superimposition position determination unit 13 determines a feature point of the rendering image corresponding to a reference point using multiple points of the boundary position between the planned construction site A and a road as the reference points. In other words, the superimposition position determination unit 13 determines a feature point corresponding to the reference point from multiple points of the boundary position between the road and the planned construction site A included in the rendering image.

The image conversion unit 14 is realized, for example, by the operation of a CPU. The image conversion unit 14 converts the rendering image into a celestial sphere image based on the determined superimposition position. The image conversion unit 14 converts the rendering image, whose viewpoint is the imaging position P, into a celestial sphere image, so that, for example, the reference point and the feature point are positioned at overlapping positions.

The image generating unit 15 is realized, for example, by the operation of a CPU. The image generating unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image at a determined superimposition position. The image generating unit 15 generates the superimposed image by superimposing the captured image and the converted rendering image, for example, so that the position of the feature point is superimposed on the position of the reference point.

The output unit 16 is realized, for example, by the operation of the CPU. The output unit 16 outputs the generated superimposed image to the outside. The output unit 16 displays the superimposed image on a display device (not shown), for example. The output unit 16 also outputs (transmits) the superimposed image to, for example, another terminal, etc.

Next, the operation of the image generating device 1 according to this embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the imaging position acquisition unit 12 acquires an imaging position P of the captured image. Next, the superimposition position determination unit 13 determines a superimposition position of the rendering image relative to the captured image.

Next, the image conversion unit 14 converts the rendering image into a spherical image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.

Next, the program according to this embodiment will be described.
Each component included in the image generating device 1 can be realized by hardware, software, or a combination of these. Here, being realized by software means being realized by a computer reading and executing a program.

The program can be stored and supplied to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)). The display program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can supply the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

The image generating device 1 and the program according to the present embodiment described above provide the following advantages.
(1) An image generating device 1 that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site A, and includes an image acquiring unit 11 that acquires a captured image that is a celestial sphere image and a rendering image, an imaging position acquiring unit 12 that acquires an imaging position P including a height at which the captured image was captured, a superimposition position determining unit 13 that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position P, an image converting unit 14 that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit 15 that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position.
The program also causes a computer to function as an image generating device 1 that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site A, and causes the computer to function as an image acquiring unit 11 that acquires a captured image that is a celestial sphere image and a rendering image, an imaging position acquiring unit 12 that acquires an imaging position P including the height at which the captured image was captured, a superimposition position determining unit 13 that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position P, an image converting unit 14 that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit 15 that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position.
This allows the rendering image to be easily corrected to match the spherical image. That is, the rendering image can be easily corrected to match the captured image. Therefore, a superimposed image in which the captured image and the rendering image are superimposed can be easily obtained. In addition, since the captured image and the rendering image are superimposed based on the imaging position P including the height, a superimposed image with less awkwardness can be obtained. In addition, since the reference point and the feature point are associated using information on the height of the imaging position, the accuracy of superimposition can be improved compared to a case where there is no information on the height of only the latitude and longitude.

[Second embodiment]
Next, an image generating device 1 and a program according to a second embodiment of the present invention will be described with reference to FIG. 6. In describing the second embodiment, the same components as those in the previous embodiment are given the same reference symbols, and their description will be omitted or simplified. The image generating device 1 and a program according to the second embodiment acquire a reference point and a feature point, and estimate an imaging position P. The image generating device 1 according to the second embodiment differs from the first embodiment in that the imaging position acquisition unit 12 acquires an estimated imaging device. As shown in FIG. 6, the image generating device 1 according to the second embodiment further includes a reference point acquisition unit 17, a feature point acquisition unit 18, and an estimation unit 19.

The reference point acquisition unit 17 is realized, for example, by the operation of a CPU. The reference point acquisition unit 17 acquires a predetermined reference point in the captured image. The reference point acquisition unit 17 acquires, for example, a position in the captured image that serves as a reference point. The reference point acquisition unit 17 acquires a reference point by accepting a specification of a position in the captured image. In this embodiment, the reference point acquisition unit 17 acquires multiple points on the boundary with the road and the boundary with the neighbor's land as reference points.

The feature point acquisition unit 18 is realized, for example, by the operation of a CPU. The feature point acquisition unit 18 acquires feature points that indicate positions in the rendering image that correspond to the acquired reference points. The feature point acquisition unit 18 acquires feature points, for example, by accepting a specification of a position in the rendering image.

The estimation unit 19 is realized, for example, by the operation of a CPU. The estimation unit 19 estimates the imaging position P based on the acquired reference point and feature point. The estimation unit 19 estimates the imaging position P including the height, for example, from the position of the reference point and the position of the corresponding feature point. The estimation unit 19 estimates the imaging position P, for example, from the degree of deviation between the position of the reference point and the position of the corresponding feature point.

Next, the operation of the image generating device 1 according to this embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires a reference point. Furthermore, the feature point acquisition unit 18 acquires a feature point. Next, the estimation unit 19 estimates an imaging position P including a height from the acquired reference point and feature point. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

Next, the superimposition position determination unit 13 determines the superimposition position of the rendering image on the captured image. Next, the image conversion unit 14 converts the rendering image into a spherical image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.

The image generating device 1 and the program according to the present embodiment described above provide the following advantages.
(2) The image generating device 1 further includes a reference point acquisition unit 17 that acquires a predetermined reference point in the captured image, a feature point acquisition unit 18 that acquires a feature point that indicates a position in the rendering image that corresponds to the acquired reference point, and an estimation unit 19 that estimates the imaging position P based on the acquired reference point and feature point, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the imaging position P cannot be acquired, the imaging position P can be estimated as long as the reference point and the feature point can be obtained. Therefore, it is possible to suppress a sense of incongruity caused by the superimposition of the captured image and the rendering image.

[Third embodiment]
Next, an image generating device 1 and a program according to a third embodiment of the present invention will be described with reference to FIG. 7. In describing the third embodiment, the same components as those in the above-mentioned embodiment are denoted by the same reference symbols, and their description will be omitted or simplified. The image generating device 1 and the program according to the third embodiment estimate an imaging position P including a height by acquiring a reference point and accepting a designation of an imaging point on a map. The image generating device 1 according to the third embodiment differs from the first embodiment in that the imaging position acquisition unit 12 acquires an estimated imaging device. The image generating device 1 according to the third embodiment differs from the second embodiment in that it further includes a point information acquisition unit 20 as shown in FIG. 7. The image generating device 1 according to the third embodiment differs from the second embodiment in that it does not include a feature point acquisition unit 18. The image generating device 1 according to the third embodiment differs from the second embodiment in that the estimation unit 19 estimates an imaging position P based on the acquired reference point and point information.

The location information acquisition unit 20 is realized, for example, by the operation of a CPU. The location information acquisition unit 20 acquires the position of the acquired reference point on the map as location information. The location information acquisition unit 20 acquires the location information, for example, by using EXIF (Exchangeable image file format) included in the captured image.

Next, the operation of the image generating device 1 according to this embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires a reference point. Furthermore, the location information acquisition unit 20 acquires location information. Next, the estimation unit 19 estimates an imaging position P including a height from the acquired reference point and location information. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

Next, the superimposition position determination unit 13 determines the superimposition position of the rendering image on the captured image. Next, the image conversion unit 14 converts the rendering image into a spherical image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.

The image generating device 1 and the program according to the present embodiment described above provide the following advantages.
(3) The image generating device 1 further includes a reference point acquisition unit 17 that acquires a predetermined reference point in the captured image, a point information acquisition unit 20 that acquires the position of the acquired reference point on the map as point information, and an estimation unit 19 that estimates the imaging position P based on the acquired reference point and point information, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the height of the imaging position P cannot be acquired, the height of the imaging position P can be estimated as long as the reference point and point information can be obtained. Therefore, it is possible to suppress the awkwardness of superimposing the captured image and the rendering image.

Although preferred embodiments of the image generating device and program of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be modified as appropriate.
For example, in the above embodiment, the imaging position acquisition unit 12 acquires the imaging position P by input from an input device, but is not limited to this. For example, the imaging position acquisition unit 12 may acquire the imaging position P from an imaging position storage unit (not shown) that stores the imaging position P in advance. Also, the imaging position acquisition unit 12 may acquire the imaging position P from an external terminal or the like.

In the second or third embodiment, the reference point acquisition unit 17, the feature point acquisition unit 18, and the location information acquisition unit 20 may acquire the reference point, feature point, or location information input by an input device. The reference point acquisition unit 17 and the feature point acquisition unit 18 may acquire the reference point or feature point from the edge position (position with high spatial frequency).

In the above embodiment, the captured image and the rendering image are described as examples, but the captured video including the captured image and the rendered captured image including the rendering image may be superimposed. The image acquisition unit 11 acquires the captured video including the captured image. The image generation unit 15 superimposes the rendering video including the rendering image, the time length of which is adjusted to match the time length of the captured video set in advance, on the captured video to generate a superimposed video. Here, the image generation device 1 may further include a time acquisition unit (not shown) that acquires the time (e.g., start time and end time) to be used for superimposition from the acquired captured video. The image generation device 1 may further include a rendering video generation unit (not shown) that adjusts or generates the rendering video according to the acquired time.

REFERENCE SIGNS LIST 1 Image generating device 11 Image acquisition section 12 Imaging position acquisition section 13 Superimposition position determination section 14 Image conversion section 15 Image generating section 17 Reference point acquisition section 18 Feature point acquisition section 19 Estimation section 20 Point information acquisition section A Planned construction site P Imaging position

Claims (5)

An image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model of a building onto a captured image of a planned construction site,
an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model ;
an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured;
a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position;
an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image ;
an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position;
a reference point acquisition unit for acquiring a predetermined reference point in the captured image;
a feature point acquisition unit that acquires feature points indicating positions in the three-dimensional model corresponding to the acquired reference points;
an estimation unit that estimates the imaging position based on the acquired reference point and the acquired feature point;
Equipped with
The imaging position acquisition unit is an image generating device that acquires the estimated imaging position . An image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model of a building onto a captured image of a planned construction site,
an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model;
an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured;
a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position;
an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image;
an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position;
a reference point acquisition unit for acquiring a predetermined reference point in the captured image;
a point information acquisition unit that acquires the positions of the acquired reference points on a map as point information;
an estimation unit that estimates the imaging position based on the acquired reference point and the spot information;
Equipped with
The imaging position acquisition unit is an image generating device that acquires the estimated imaging position. the image acquisition unit acquires a captured moving image including the captured image,
The image generating device according to claim 1 or 2, wherein the image generating unit generates a superimposed moving image by superimposing a moving image including the three-dimensional model , the moving image having a temporal length adjusted to match a predetermined temporal length of the captured moving image , on the captured moving image. A program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model of a building onto a captured image of a planned construction site,
The computer,
an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model ;
an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured;
a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position;
an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image ;
an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position;
a reference point acquisition unit for acquiring a predetermined reference point in the captured image;
a feature point acquisition unit that acquires feature points indicating positions in the three-dimensional model corresponding to the acquired reference points;
an estimation unit that estimates the imaging position based on the acquired reference point and the acquired feature point;
Functioning as a
The imaging position acquisition unit is a program for acquiring the estimated imaging position . A program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model of a building onto a captured image of a planned construction site,
The computer,
an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model;
an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured;
a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position;
an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image;
an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position;
a reference point acquisition unit for acquiring a predetermined reference point in the captured image;
a spot information acquisition unit that acquires the positions of the acquired reference points on a map as spot information;
an estimation unit that estimates the imaging position based on the acquired reference point and the spot information;
Functioning as a
The imaging position acquisition unit is a program for acquiring the estimated imaging position.

Images