JP7598994B2 - 3D image generating method, 3D image generating device, guidance device, and program - Google Patents

Description

The present invention relates to a method for generating a three-dimensional image, a three-dimensional image generating device, a guidance device, and a program.

Patent Document 1 describes a technology that uses a 3D scanner to measure the 3D data of the surface of the completed form at a construction site in three dimensions as point cloud data in order to grasp the shape of the completed form at the construction stage, and then modifies the construction plan according to the actual shape of the completed form before the construction stage.

JP 2007-277813 A

The 3D images used at construction sites as described in the above document are sufficient if they allow the shape to be grasped, but when generating 3D images to preserve cultural assets or other buildings as historical legacy for future generations, it is desirable to represent not only the shape but also the texture, color, and scenery as closely as possible to the real thing.

The present invention was made in consideration of the above circumstances, and its purpose is to provide a technology for generating three-dimensional images that can express the texture and color of structures.

In order to solve the above-mentioned problems, each aspect of the present invention adopts the following configuration.

The first aspect relates to a method for generating a three-dimensional image.
A method for generating a three-dimensional image according to a first aspect of the present invention comprises the steps of:
a point cloud data generating step of measuring the structure a plurality of times while varying the installation position of a measuring device in at least one of the horizontal and vertical directions to generate a plurality of point cloud data representing shapes and colors of different portions of the structure;
a three-dimensional image generating step of generating a three-dimensional image of the structure by processing the plurality of point cloud data;
Including,
In the point cloud data generating step, an interval between the measuring devices arranged at a position higher than the structure is set to be wider than an interval between the measuring devices arranged at a position lower than the structure;
The measuring device is a 3D laser scanner,
In the point cloud data generation process, the 3D laser scanner positioned at a position higher than the structure performs scanning with a coarser laser irradiation interval than the 3D laser scanner positioned at a position lower than the structure, and generates the point cloud data.

The second aspect relates to a three-dimensional image generating apparatus.
A three-dimensional image generating device according to a second aspect includes a point cloud data acquisition means for acquiring a plurality of point cloud data representing shapes and colors of different portions of a structure by measuring the structure a plurality of times while varying at least one of the horizontal and vertical directions of an installation position of a measuring device, the point cloud data being generated by the three-dimensional image generating method according to the first aspect;
a three-dimensional image generating means for generating a three-dimensional image of the structure by processing the plurality of point cloud data,
The distance between the measuring devices arranged at a position higher than the structure is set to be greater than the distance between the measuring devices arranged at a position lower than the structure;
The measuring device is a 3D laser scanner;
The 3D laser scanner arranged at a position higher than the structure performs scanning by making the laser irradiation interval coarser than the 3D laser scanner arranged at a position lower than the structure;
The point cloud data acquisition means acquires the point cloud data generated by the 3D laser scanner.

A third aspect relates to a guidance device.
A guidance device according to a third aspect includes a presenting means for presenting guidance on a method of generating a three-dimensional image to an operator in order to execute the method of generating a three-dimensional image according to the first aspect.

Another aspect of the present invention may be a program executed by at least one computer to realize the apparatus according to the second and third aspects, or a computer-readable recording medium having such a program recorded thereon. This recording medium includes a non-transitory tangible medium.
The computer program includes computer program code which, when executed by a computer, causes the computer to carry out the processing method on each device.

In addition, any combination of the above components, and any transformation of the present invention into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present invention.

Furthermore, the various components of the present invention do not necessarily have to be independent entities, but may be formed as a single member by multiple components, one component may be formed from multiple components, one component may be part of another component, or part of one component may overlap with part of another component, etc.

In addition, although the method and computer program of the present invention describe multiple steps in a sequential order, the order does not limit the order in which the multiple steps are executed. Therefore, when implementing the method and computer program of the present invention, the order of the multiple steps can be changed to the extent that does not interfere with the content.

Furthermore, the multiple steps of the method and computer program of the present invention are not limited to being executed at different times. Therefore, a step may occur while another step is being executed, or the execution timing of a step may overlap in part or in whole with the execution timing of another step, etc.

The above aspects provide a technology for generating 3D images that can express the texture and color of structures.

1 is a diagram conceptually illustrating a system configuration of an image processing system according to an embodiment of the present invention. FIG. 1 illustrates an example of the configuration of a computer that realizes a three-dimensional image generating device. FIG. 2 is a functional block diagram showing a logical configuration of the three-dimensional image generating device. 1 is a flow chart illustrating steps of a method for generating a three-dimensional image. FIG. 1 is a diagram showing an example of an arrangement of a plurality of measuring devices. FIG. 2 is a diagram for explaining an overlapping portion of a plurality of point cloud data; 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 11A and 11B are diagrams for explaining a method of deleting overlapping portions of point cloud data. 1A and 1B are diagrams showing an example in which a plurality of measuring devices are arranged in a staggered manner at different heights, and a comparative example in which the measuring devices are arranged at the same height. FIG. 13 is a diagram showing an example in which measuring devices are placed at positions higher and lower than a structure. FIG. 2 is a functional block diagram showing a logical configuration of the guidance device. FIG. 4 is a diagram illustrating a laser irradiation direction of the measuring device. FIG. 13 is a diagram for explaining a measurement range depending on an installation position of a measurement device. FIG. 13 is a diagram for explaining a measurement range depending on an installation position of a measurement device. FIG. 13 is a diagram for explaining a measurement range depending on an installation position of a measurement device. FIG. 13 is a diagram for explaining a measurement range depending on an installation position of a measurement device. FIG. 13 is a diagram for explaining a measurement range depending on an installation position of a measurement device. FIG. 13 is a diagram for explaining an example of installation of a measuring device. FIG. 13 is a diagram for explaining an example of installation of a measuring device. FIG. 13 is a diagram for explaining an example of installation of a measuring device.

The following describes an embodiment of the present invention with reference to the drawings. Note that in all drawings, similar components are given similar reference numerals and descriptions are omitted where appropriate.

(First embodiment)
1 is a diagram conceptually showing the system configuration of an image processing system 1 according to the present embodiment. Note that in the following drawings, configurations of parts that are not related to the essence of the present invention are omitted and are not shown.
The image processing system 1 includes a three-dimensional image generating device 100 that generates a three-dimensional image of a structure using measurement data obtained by using a plurality of measuring devices 5. The measuring devices 5 are, for example, 3D laser scanners that perform 3D digital measurement and output point cloud data. The point cloud data is a collection of data of a plurality of points that indicate the three-dimensional shape of the structure, and includes measurement data (relative or absolute coordinate information) that indicates the position of each point, and color information of each point.

The point cloud data measured by the measuring device 5 may be stored in the storage device 110 via the communication network 3, or may be stored in a recording medium of the measuring device 5. The three-dimensional image generating device 100 can read the point cloud data from the storage device 110 or the recording medium.

The functions of the 3D image generating device 100 of this embodiment, which will be described later, are realized by a computer. Figure 2 is a diagram showing an example of the configuration of a computer 60 that realizes the 3D image generating device 100.

The computer 60 comprises a CPU (Central Processing Unit) 62, a memory 64, a program 80 that realizes the components of the 3D image generating device 100 loaded into the memory 64, a storage 66 that stores the program 80, an I/O (Input/Output) 68, and an interface (communication I/F) 70 for connecting to a communication network.

The CPU 62, memory 64, storage 66, I/O 68, and communication I/F 70 are connected to each other via a bus 69, and the entire 3D image generating device 100 is controlled by the CPU 62. However, the method of connecting the CPU 62 and other components to each other is not limited to bus connection.

Memory 64 is memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Storage 66 is a storage device such as a hard disk, SSD (Solid State Drive), or memory card.

Storage 66 may be a memory such as a RAM or a ROM. Storage 66 may be provided inside computer 60, or may be provided outside computer 60 and connected to computer 60 by wire or wirelessly as long as computer 60 can access it. Alternatively, storage 66 may be provided detachably on computer 60.

The CPU 62 reads the program 80 stored in the storage 66 into the memory 64 and executes it to realize the functions of each unit of the 3D image generating device 100 in FIG. 3, which will be described below.

The I/O 68 controls the input and output of data and control signals between the computer 60 and other input and output devices. The other input and output devices include, for example, input devices 72 such as a keyboard, touch panel, mouse, and microphone connected to the computer 60, output devices 74 such as a display, printer, and speaker, and interfaces between these input and output devices and the computer 60. Furthermore, the I/O 68 may control the input and output of data with other recording medium reading or writing devices (not shown).

The communication I/F 70 is a network connection interface for communication between the computer 60 and an external device. The communication I/F 70 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, the computer 60 that realizes the three-dimensional image generating device 100 may be connected to each measuring device 5 via the communication I/F 70 through the communication network 3.

Each component of the 3D image generating device 100 of this embodiment shown in FIG. 3, which will be described later, is realized by any combination of hardware and software of the computer 60 shown in FIG. 2. Those skilled in the art will understand that there are various variations in the realization method and device. The functional block diagram showing the 3D image generating device 100 of each embodiment described below shows logical functional unit blocks, rather than a hardware unit configuration.

The three-dimensional image generating device 100 may be configured with multiple computers 60, or may be realized by a virtual server. Alternatively, it may be in a form in which SaaS (Software as a Service) provides the user's terminal with access to an application running on a server computer.

The program 80 of this embodiment is written to cause the computer 60 to execute the following procedures to realize the three-dimensional image generating device 100. The procedures executed by the computer include a procedure for acquiring multiple point cloud data indicating the shapes and colors of different parts of the structure by measuring the structure multiple times with the installation position of the measuring device 5 being changed in at least one of the horizontal and vertical directions, and a procedure for generating a three-dimensional image of the structure by processing the multiple point cloud data.

The computer program 80 of this embodiment may be recorded on a recording medium readable by the computer 60. The recording medium is not particularly limited, and various forms are possible. The program 80 may be loaded from the recording medium into the memory 64 of the computer 60, or may be downloaded to the computer 60 via the communication network 3 and loaded into the memory 64.

The recording medium on which the computer program 80 is recorded includes a non-transitory, tangible medium usable by the computer 60, in which program code readable by the computer 60 is embedded. When the computer program 80 is executed on the computer 60, it causes the computer 60 to realize the three-dimensional image generating device 100.

3 is a functional block diagram showing the logical configuration of the 3D image generating device 100 of this embodiment. The 3D image generating device 100 includes a point cloud data acquisition unit 102 and an image generating unit 104.
The point cloud data acquisition unit 102 acquires multiple point cloud data indicating the shapes and colors of different parts of the structure, obtained as a result of measuring the structure multiple times while changing the installation position of the measuring device 5 in at least one of the horizontal and vertical directions.
The image generator 104 processes a plurality of point cloud data to generate a three-dimensional image of the structure.
However, the different parts of the structure acquired by the point cloud data acquisition unit 102 may include not only different parts of the structure but also the same parts.

In the embodiment, "acquisition" includes at least one of the following: the device itself goes to retrieve data or information stored in another device or recording medium (active acquisition), and the device itself inputs data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or recording medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

In this embodiment, the structure preferably includes a cultural property, a temple, a shrine, a world heritage site (recognized by the World Heritage Committee), an industrial heritage site (recognized by the International Committee for the Conservation of Industrial Heritage), a modern industrial heritage site (recognized by the Ministry of Economy, Trade and Industry), a civil engineering heritage site (recommended by the Japan Society of Civil Engineers), a scenic spot, a topography, or a historical building and ruins including at least one of a building and a scenic spot that need to be preserved for future generations. In this case, the three-dimensional image generated by the three-dimensional image generating device 100 of this embodiment is intended to convey the texture and color of the structure to future generations. At a typical construction site, the purpose is to survey the structure, so it is sufficient if the shape can be grasped from the three-dimensional image. In contrast, in this embodiment, the purpose is to convey the shape and texture of the structure to future generations, so it is preferable that not only the shape but also the texture, color, and scene are expressed in the three-dimensional image.

In addition, in historical wooden buildings such as cultural properties, multiple beams and small beams are arranged in a lattice pattern under the roof, eaves, ceiling, etc., and the laser does not directly hit the backs of these beams and small beams. Furthermore, Japanese-style rooms often have complicated shapes such as a tokonoma alcove, a top drawer, a transom, a drop-down hanging, and a bookshelf, which creates many blind spots for the laser. This embodiment provides a method for eliminating such blind spots.

FIG. 4 is a flow chart illustrating the steps of the method for generating a three-dimensional image according to the present embodiment.
The three-dimensional image generating method of this embodiment includes a point cloud data generating step (step S101) and a three-dimensional image generating step (step S103).
In the point cloud data generation process of step S101, the structure is measured multiple times while varying the installation position of the measuring device 5 in at least one of the horizontal and vertical directions, thereby generating multiple point cloud data representing the shapes and colors of different parts of the structure.
In the three-dimensional image generating process of step S103, a three-dimensional image of the structure is generated by processing a plurality of point cloud data.

Figure 16 is a diagram explaining the laser irradiation direction of the measuring device 5. The measuring device 5 changes the laser irradiation angle in a range of 360 degrees in the horizontal direction (Figure 16(a)), and in the vertical direction, it scans the measurement object while changing the laser irradiation angle in a range of 90 degrees toward the zenith and -60 degrees downward, with the horizontal position being 0 degrees (Figure 16(b)). Figure 16(c) shows the entire measurement range of the measuring device 5.

As mentioned above, in historical wooden buildings such as cultural properties, multiple beams and small beams are arranged under the roof and eaves, resulting in blind spots where the laser does not directly hit the backs of these beams and small beams. Below, we will use the drawings to explain an example of how to efficiently reduce these blind spots using a method for installing the measuring device 5 of the present invention.

Figures 17 to 21 are vertical cross-sectional views of a roof. In an example where there are small beams 10 under the roof as shown in Figure 17, when the measuring device 5a is installed at position La under the roof, the bold line indicates the range where the laser can be irradiated, i.e., the measurable range. The rest of the area is a blind spot.

The example of Figure 18 shows a case where measuring device 5b is installed at a position Lb that is the same horizontal position as position La of measuring device 5a in Figure 17, but lower vertically. The measurement range of measuring device 5b (shown by the bold line in the figure) covers a wider range than the measurement range of measuring device 5a in Figure 17. However, the point cloud density of measuring device 5b is lower than the point cloud density of measuring device 5a.

The example of Figure 19 shows an example in which measuring device 5c is installed at position Lc, which is different in both the horizontal and vertical directions from the position of measuring device 5a in Figure 17. The measurement range of measuring device 5c is indicated by a thick line in the figure. In the example of Figure 19, the point cloud density is approximately the same as the example of Figure 17.

The example in FIG. 20 shows a case where measuring device 5d is installed at a position Ld that is the same horizontal position as position Lc of measuring device 5c in FIG. 19, lower in the vertical direction, and higher than measuring device 5b in FIG. 18. The measurement range of measuring device 5d is shown by a thick line in the figure. The point cloud density of measuring device 5d in FIG. 20 is an intermediate value between measuring devices 5a, 5b, and 5c.

Consider these combinations. When measuring device 5b at position Lb is combined with measuring device 5d at position Ld, it is possible to cover the joists 10 and the underside of the roof between the joists 10. In other combinations, blind spots remain. However, the combination of measuring device 5b and measuring device 5d has a lower point cloud density than other combinations. Therefore, using measuring device 5a at position La and measuring device 5c at position Lc, a scan can be performed to complement the low point cloud density areas of the above combinations.

On the other hand, in a comparative example in which measuring device 5a is positioned at position Le, slightly lower in the vertical direction, as shown in FIG. 21, measuring device 5a and measuring device 5d can cover the entire measurement target. However, in this example, the overlapping part of the point cloud is mainly only on the underside of the beam 10. In other words, the point cloud density is high only on the underside of the beam 10, and the difference in point cloud density with other parts becomes clear. For this reason, it is not preferable for historical structures such as cultural properties, which are the measurement targets of the present invention, where it is important to express the texture.

FIG. 5 is a diagram showing an example of an arrangement of a plurality of measuring devices 5. As shown in FIG.
The multiple measuring devices 5 are arranged at different installation positions in the horizontal and vertical directions. Here, the number of measuring devices 5 does not necessarily need to be multiple, and at least one measuring device 5 may be moved to multiple installation locations to perform measurements. In other words, multiple measuring devices 5 means measuring devices 5 installed at multiple installation locations, and does not mean the number of measuring devices 5 used for measurement. Scanning may be performed on the multiple measuring devices 5 one by one in sequence, or a predetermined number of measuring devices may be simultaneously performed.

The method of arranging the measuring device 5 is exemplified below.
(1) Multiple measuring devices 5 are arranged at intervals so that the scanning areas (point cloud data) of adjacent measuring devices 5 overlap, at positions that allow measurement from a relatively low viewpoint on one side of the target (hereinafter also referred to as the measurement target), and at a predetermined distance from the one side.
(2) In order to eliminate blind spots of (1), place multiple sensors at a specified distance from one side as in (1) at a location where measurements can be taken from a higher viewpoint than (1).
(3) Furthermore, in order to eliminate the blind spot of (1), it is placed at a position farther away from the first surface than (1) and at a position where it can be measured from an oblique viewpoint. The position may be on both sides of the object, or in a position where the object's accessories can be measured. The height may be between (1) and (2). The distance from the first surface should be such that the scanning area of the measuring device 5 of (1) and the scanning area of the measuring device 5 of (3) at least partially overlap.
(4) In order to measure objects around the target object, a plurality of measuring devices 5 are placed at positions further away from the surface than (3). The heights of the plurality of measuring devices 5 may be different. The distance from the surface is preferably such that the scanning area of the measuring device 5 in (3) and the scanning area of the measuring device 5 in (4) overlap at least partially.
(5) In order to eliminate the blind spots of (4), a plurality of measuring devices 5 may be placed at positions further away from the one surface than (4). The heights of the plurality of measuring devices 5 may be different. The distance from the one surface is preferably such that the scanning area of the measuring device 5 of (4) and the scanning area of the measuring device 5 of (5) at least partially overlap. (5) is repeated until the measurement target and its associated objects are measured. Note that the blind spots include not only the areas caused by the shape of the structure to be measured, but also the blind spots of the measuring device 5 itself (its own feet, tripods, and the range from -60 degrees to -90 degrees (to the ground) below in FIG. 16(b)).

A specific example will be given using the example of Figure 5. For example, the four measuring devices 5a are arranged in positions that allow measurement from a relatively low viewpoint, and measure the shape of the gate, the inside of the pillars, and the underside of the eaves. It is desirable to arrange the number of measuring devices 5a so as to eliminate the blind spots of the small beams under the eaves and the empty space of the small beams. The two measuring devices 5b are arranged in positions that allow measurement from a higher viewpoint than the measuring device 5a in order to compensate for the blind spot of the measuring device 5a. The two measuring devices 5c are also arranged in positions that allow measurement from a lower viewpoint diagonally behind the measuring device 5a in order to compensate for the blind spot of the measuring device 5a, and measure the outside of the gate, the outside of the pillars, and under the tiles of the earthen wall. The four measuring devices 5d are also arranged in positions that allow measurement from a lower viewpoint diagonally behind the measuring devices 5a, 5b, and 5c in order to compensate for the blind spots of the measuring devices 5a, 5b, and 5c, and measure the steps (treads, landings) and risers of the stairs in front of the gate. The four measuring devices 5e are positioned so that they can measure from a lower viewpoint diagonally behind the measuring devices 5a, 5b, 5c, and 5d in order to compensate for the blind spots of the measuring devices 5a, 5b, 5c, and 5d.

In this way, by changing the position and height of the multiple measuring devices 5, blind spots due to the back side of the beams of a wooden building or complex shapes can be effectively eliminated. In addition, the measurement direction of the measuring device 5 may be, for example, 240 degrees vertically as shown in FIG. 16(b), or the measurement direction (in other words, the laser irradiation angle) may be limited to a predetermined range. For example, in the case of outdoor measurement, if it is clear that there are no structures or trees in the zenith direction of the measuring device 5, the zenith direction may be excluded from the measurement range. By limiting the measurement range, the volume of point cloud data can be reduced. However, if the measurement range is partially excluded, it may be difficult to combine point cloud data. Therefore, it is more preferable to limit the measurement range to a range that does not affect the combination of point cloud data.

In the point cloud data generation process of step S101, the measurement object is divided into multiple regions and measured to generate multiple point cloud data. In two pieces of point cloud data that measure adjacent regions, the adjacent regions partially overlap each other. In other words, the measurement device 5 is positioned so that the adjacent regions in the two generated point cloud data partially overlap each other. As shown in FIG. 6(a), the regions (hereinafter sometimes referred to as ranges) R1 and R2 indicated by the two generated point cloud data respectively include a portion where the same location on the measurement object was measured (the portion indicated by the star mark S). In other words, the regions R1 and R2 of the two point cloud data overlap in the portion indicated by the star mark S.

In the three-dimensional image generation process in step S103, the two point cloud data can be combined using this overlapping portion as a reference. As shown in FIG. 6(b), the ranges R1 and R2 of the two point cloud data in FIG. 6(a) can be aligned and combined using the star mark S. The overlapping portion is shown by hatching.

As a specific example, in the case of a building with a gate that cannot be opened or closed, and where visibility is poor from the inside to the outside of the gate, scanning is performed from both the inside and outside of the gate. When scanning, the position of the measuring device 5 is selected so that the scanning range includes a common part. The two point cloud data obtained as a result of scanning from the inside and outside of the gate are aligned and combined by overlapping the common part. The common part may be selected as a landmark when overlapping the point cloud data.

It is preferable to select the position of the measuring device 5 so that there are multiple overlapping parts. In addition, when combining the overlapping parts, it is preferable to select from among the multiple overlapping parts the parts with the highest combining accuracy.

In this way, blind spots can be eliminated by using multiple measuring devices 5, but the result is that a large number of point cloud data are overlapped to generate a three-dimensional image, which generates a lot of noise and further reduces the texture and quality of the object being measured.

Therefore, in this embodiment, the point cloud data obtained by the measuring device 5 is used partially, not all of it. In the three-dimensional image generating process of step S103, when combining point cloud data obtained by multiple measuring devices 5, at least a part of the overlapping parts is deleted. For example, overlapping parts and unnecessary parts of the point cloud data are deleted by image processing and used partially. This reduces noise and makes it possible to ensure the texture. As explained with reference to FIG. 6(b), for the hatched part where the ranges R1 and R2 of two point cloud data overlap, one of the point cloud data may be deleted.
Alternatively, statistical processing such as finding the average value or median value may be performed on the overlapping portion of the two point cloud data.

The method for removing overlapping parts will be explained using Figures 7 to 12. Here, we assume that there are three structures to be measured: target A, target B, and target C. Target A is a cultural asset, and targets B and C are associated with it, such as earthen walls.

As shown in FIG. 7, six measuring devices 5a are placed to measure object A, and the ranges of regions R11 and R12 are measured. Next, as shown in FIG. 8, three measuring devices 5b are placed to measure object B, and the ranges of regions R21 and R22 are measured. Next, as shown in FIG. 9, two measuring devices 5c are placed to measure object C, and regions R31 to R34 are measured.

When all the point cloud data obtained by such measurements is overlaid, there are many overlapping areas, as shown in Figure 10. In this case, there is a high possibility that the texture will be lost. Therefore, as shown in Figure 11, the overlapping partial point cloud data is deleted. For example, regions R21, R33, and R34 are deleted.

Furthermore, as shown in FIG. 12, the point cloud data within region R12 measured by measuring device 5a of object B may be deleted. In this way, the texture can be maintained.

In cultural properties and other structures, transoms, nageshi, small beams, and other elements that are located high up cast shadows themselves, creating blind spots for the laser. Also, in long corridors, passageways, streets, and the like, if the scanning position is low, many blind spots will also occur on the tops of chests of drawers, the tops of eaves, and so on.

For these measurements, the scanning position can be increased in height (e.g., 7 meters), allowing scanning from a high viewpoint and eliminating blind spots.

In addition, in long corridors, passages, streets, etc., it is preferable to change the scanning heights of multiple measuring devices 5 arranged along the longitudinal direction of the passage. A first height, a second height higher than the first height, a third height higher than the second height, etc. may be combined. For example, it is preferable to change the scanning heights of multiple measuring devices 5 to a third height, a first height, a third height, a first height, a second height, a first height, a third height, etc., and arrange the measuring devices 5 at a predetermined interval. As described above, the predetermined interval between the measuring devices 5 is set so that the scanning areas of each measuring device 5 at least partially overlap.

As shown in FIG. 13(a), the heights of the multiple measuring devices 5 in this embodiment are set in a staggered pattern, for example, at a first height h1 and a second height h2 that is higher than the first height h1.

In the point cloud data generation process of step S101, the spacing between multiple measuring devices 5 arranged at a position higher than the structure is set to be wider than the spacing between measuring devices 5 arranged at a position lower than the structure. Furthermore, in the point cloud data generation process of step S101, the density of the point cloud data acquired by the measuring devices 5 arranged at a position higher than the structure is set to be lower than the density of the point cloud data acquired by the measuring devices 5 arranged at a position lower than the structure.

When scanning is performed from the second height h2, the angle of the laser irradiated to the ground is closer to vertical than when scanning is performed from the lower position of the first height h1, resulting in a higher laser irradiation density. The reflection intensity of the laser is also higher. Therefore, when scanning is performed from the second height h2, the laser irradiation interval can be made coarser than when scanning from the first height h1, and as a result, the scanning time to obtain point cloud data of the same accuracy and range can be shortened. By combining scans from a lower viewpoint with scans from a higher viewpoint in a staggered arrangement, blind spots can be effectively eliminated.

For example, as in FIG. 13(b), when the heights of multiple measuring devices 5 are set to the first height h1, the installation interval of the measuring devices 5 becomes half that of FIG. 13(a), and the number of measuring devices 5 required also increases to five. This is because scanning from a high viewpoint widens the scanning coverage area and makes it easier to combine point clouds, allowing the scanning interval to be increased. As a result, the on-site scanning work can be shortened.

Even when obtaining point cloud data of similar accuracy, the arrangement in FIG. 13(a) allows for a lower scanning density and shorter scan time than the arrangement in FIG. 13(b). Therefore, the data volume of the obtained point cloud data can be reduced with the arrangement in FIG. 13(a). Furthermore, the arrangement in FIG. 13(a) also allows for a shorter work process time, which can reduce labor costs. Furthermore, as will be described later, scanning of structures is often performed outdoors, and in such cases, if the work takes a long time, the direction of the sunlight will change, which will affect the way shadows are formed and the intensity of the light, which is likely to affect the texture and color of the three-dimensional image, so the shorter the work time, the better.

In addition, in the point cloud data generation process of step S101, when generating at least a portion of the point cloud data, the measuring device 5 is positioned at a height higher than the structure. Furthermore, as shown in FIG. 14, in the point cloud data generation process of step S101, the installation position of the measuring device 5 is alternately repeated between a position higher than the structure (height h2) and a position lower than the structure (height h1).

Figures 22 to 24 are diagrams for explaining installation examples of the measuring device 5. Figures 22(a), 23(a), and 24(a) are plan views seen from above, and Figures 22(b), 23(b), and 24(b) are cross-sectional views seen from line II in Figures 22(a), 23(a), and 24(a), respectively.
22 shows an example in which the measuring devices 5a are lined up in a row at the same height in the center of the passage, which is an example of an arrangement commonly used in surveying. In this arrangement, the earthen walls on both sides and under the eaves of the buildings are blind spots.

Figure 23 shows an example in which a measuring device 5b of the same height is added to eliminate blind spots under the eaves. The total number of measuring devices 5 increases to 12, and the number of scans becomes 12, which is twice as many as the 6 scans in the example of Figure 22. Therefore, in the example of Figure 23, the measurement range is wider than in the example of Figure 22, but noise is generated and the measurement time is longer.

Figure 24 shows an example where measuring devices 5 are placed at different heights. In this example, measuring device 5c is placed at a higher height in the center of the passageway, and measuring device 5d is placed to scan under the eaves. Measuring device 5c in a higher position widens the scanning area, so the entire area can be measured, including the roof. Measuring device 5d in a lower position can measure under the eaves. In this example, measuring device 5c in a higher position covers a wider range, so the spacing between measuring devices 5d in lower positions can be made wider than the spacing between measuring devices 5b in the example of Figure 23. Therefore, in the example of Figure 24, scanning is possible with fewer scans compared to the example of Figure 23, and without blind spots such as on the roof or under the eaves.

As described above, according to this embodiment, a three-dimensional image of a structure is generated using point cloud data obtained by placing multiple measuring devices 5 at different heights and taking measurements, thereby eliminating blind spots of the measuring devices 5. In addition, partially overlapping parts of the point cloud data obtained by multiple measuring devices 5 are deleted, so a three-dimensional image closer to the real thing can be generated without compromising the texture, color, and scenery of the structure.

By increasing the height of the measuring device 5, the scanning density can be reduced, and the scanning time can be shortened. This allows the data volume of the point cloud data to be reduced. When transferring point cloud data, the transfer time is also shortened. Work efficiency is improved, and work costs can be reduced.

Second Embodiment
The three-dimensional image generating method of this embodiment is similar to the above embodiment except that measurement conditions are specified in the point cloud data generating step.

The texture and color of a 3D image of a structure are greatly affected by the scanning environment, such as the weather, time of day, season, the position and angle of the sun, and the presence or absence of sunlight. Scanning for one location takes about 3 to 15 minutes, depending on the density of the point cloud being scanned. Therefore, especially outdoors, if the sun repeatedly disappears behind and emerges from the clouds during scanning, the sunlight will vary, and the way shadows are formed on the object to be measured, the way the light hits it, and the intensity of the light will differ by the second, making it difficult to reproduce the texture and color. In addition, when there are many scans, the way shadows are formed will differ depending on the time of day, and even if one building is scanned, the shadows will be unnatural between morning and afternoon scans, making it similarly difficult to reproduce the texture and color.

Because of these issues, in this embodiment, the point cloud data generation process in step S101 is preferably performed when the weather is cloudy, and may also be performed on rainy days if indoors. Furthermore, it is preferable to align the time period during which the point cloud data generation process in step S101 is performed for each part of the structure. For example, if the target is a cultural property, the cultural property architectural parts of the structure are distinguished from associated parts such as earthen walls, and the measurement time periods are aligned for each part.

In addition, in the point cloud data generation process of step S101, if the sunlight is oblique, the measuring device 5 is installed in a position where the shadow of the measuring device 5 is not reflected. In addition, it is preferable to perform scanning of outdoor structures when the wind strength is equal to or lower than a predetermined wind speed. The predetermined wind speed is preferably, for example, equal to or lower than a value (e.g., 6 m/s) at which the measuring device 5 is not swayed by the wind.

Depending on how shadows appear, it may be a good idea to limit scanning to certain times of the day. For example, in an example in which each part of a structure consists of object A and object B, object A can be limited to "mornings only" and object B to "evenings only." This will standardize the way shadows are cast for each object, and also the intensity of the light, bringing out the texture and color. Also, when the sun is low in the sky (in the case of so-called oblique light), there is a possibility that the shadows of the body of the measuring device 5 or the tripod will be reflected in the image, so the position of the measuring device 5 should be selected so that these will not be reflected.

Indoors, light coming in through windows or openings can affect the texture and color. If there is a possibility of this affecting the appearance, take the same measures as outdoors. Indoors, it is preferable to keep the lighting constant.

Furthermore, in the point cloud data generation process of step S101, if the structure includes a reflective surface, the measuring device 5 is placed in a position where the reflective surface is not reflected.

For example, if a structure includes a reflective surface such as window glass or a mirror, the image of the object reflected in the reflective surface will be included in the point cloud data. Therefore, when installing the measuring device 5, select a location where there is no reflection on the reflective surface.

As described above, this embodiment allows for efficient scanning while taking into account the weather and time of day. It also prevents the texture and color from being damaged by light intensity and shadows.

Third Embodiment
This embodiment is the same as at least one of the above embodiments except that it is provided with a guidance device 200 that outputs guidance to an operator on the method of generating a three-dimensional image in order to execute the method of generating a three-dimensional image of the above embodiment.

FIG. 15 is a functional block diagram showing the logical configuration of a guidance device 200 of the present embodiment.
The guidance device 200 of this embodiment is preferably realized by, for example, a portable terminal device so that a worker can use it on-site. The guidance device 200 is, for example, a smartphone, a tablet terminal, a notebook personal computer, or a PDA (Personal Digital Assistant). These terminal devices are realized by the computer 60 in FIG. 2 described above. Alternatively, in the case of guidance for three-dimensional image processing described later, the guidance device 200 may be included in the three-dimensional image generating device 100.

When the program 80 for realizing the guidance device 200 is executed on the computer 60, it causes the computer 60 to realize the guidance device 200. The program 80 causes the computer 60 to execute a procedure for presenting guidance to the operator on how to generate a three-dimensional image.

The guidance device 200 of this embodiment may be realized by installing and executing an application program that realizes the guidance device on the computer 60. Alternatively, the guidance device 200 may be realized by starting a program such as a browser on the computer 60, accessing a specific website, and displaying content including the contents of the guidance. It is preferable that the specific website can be accessed by logging in by entering authentication information (such as a user ID and password) that has been registered in advance by the user, for example.

The guidance device 200 includes a presentation unit 202 that presents guidance to the worker to execute the three-dimensional image generation method described in the above embodiment.

The guidance is presented by content including at least one of audio, images, video, and text. The presentation unit 202 outputs the guidance to an output device 74 including at least one of a display for displaying a screen and a speaker for outputting audio, and also accepts operations from an operation unit that accepts operations from an operator, such as an input device 72 including at least one of a touch panel, an operation switch, a keyboard, and the like. The operation unit also includes a graphical user interface (GUI (Graphical User Interface)) that is displayed on the display.

The guidance includes the necessary procedures and precautions for workers to take when performing measurement work on-site, as well as the necessary procedures and precautions for 3D image processing.

<Guidance for eliminating blind spots>
The installation positions of the measuring devices 5 for each cultural property and its associated objects are announced.
Specifically, first, an announcement is made that the measuring devices 5 are to be placed around the cultural property at a predetermined interval and at different heights. Recommended interval and height settings may also be announced. Recommended scanning density and time may also be announced.

Then, announce that measuring devices 5 will be placed around the structure at different heights with a set distance between them. Recommended spacing and height settings may be announced. Recommended scanning density and time may be announced. Announce that it is preferable to place measuring devices 5 both higher and lower than the structure.

It is also announced that it is preferable to alternate between high and low positions. Furthermore, it is announced that the distance between measuring devices 5 placed at a position higher than the structure should be wider than the distance between measuring devices 5 placed at a position lower than the structure. Furthermore, it is announced that the density of the point cloud data acquired by measuring devices 5 placed at a position higher than the structure should be set lower than the density of the point cloud data acquired by measuring devices 5 placed at a position lower than the structure.

A screen for selecting the measurement target (e.g., selecting either a cultural property or ancillary objects) may be presented, the selection may be prompted, and after the selection is accepted, guidance on the installation location of the corresponding measurement device 5 may be selected and presented. As described in the above embodiment, the placement method of the measurement device 5 and the recommended values for scanning density and time are changed and presented depending on whether the target is a cultural property or ancillary objects.

In another example, the system may accept a selection of the type of structure (gate, staircase, approach to a temple, torii gate, main hall, etc.) and provide guidance suggesting a location to install the measuring device 5 according to the type of structure accepted. Also, the system may accept whether or not the structure has beams, and if so, position the measuring device 5 so that the back side of the beam can be measured and provide guidance.

For example, when measuring a gate of a structure as shown in Figure 5, it is first announced that measuring devices 5 will be placed in a row below and in front of the gate. Next, it is announced that measuring devices 5 will be placed at a higher position than the first one placed in order to eliminate blind spots behind the beams on the underside of the gate's roof. It may also be announced that measuring devices 5 will be placed in positions that allow the appearance of the gate to be measured from a position away from the gate. In the case of stairs, it may also be announced that measuring devices 5 will be placed at different heights in positions that allow the measurement of each step and the rise of the stairs.

<Guidance for ensuring texture and color>
The weather forecast for the measurement date and time is presented, announcing that cloudy weather is most preferable, followed by rainy weather. Also, the cloudy hours on the measurement date and the fact that it is preferable to measure outdoors during those hours are announced. Alternatively, the rainy hours on the measurement date and the fact that it is preferable to measure indoors during those hours are announced.

Specifically, the presentation unit 202 displays a screen for receiving input of the measurement date and time and measurement location to the operator, outputs a message prompting input, and receives input through the operator's screen operation. Then, the presentation unit 202 accesses a weather forecast website based on the input measurement date and time and measurement location to obtain weather forecast information. Then, the presentation unit 202 presents the obtained weather forecast information and also provides guidance.

The recommended scanning density and time of the measuring device 5 may also be changed depending on the weather, such as sunny or cloudy. Furthermore, wind speed information may also be obtained from a weather forecast. If the wind speed at the measurement date and time is equal to or greater than a predetermined value, an announcement may be made that outdoor measurement is not suitable. Alternatively, if the wind speed varies depending on the time of day, a message may be displayed recommending that outdoor measurement be performed during a time period when the wind speed is less than the predetermined value.

If the measurement date and time is during a time period with oblique light, an announcement is made to select a location for installing the measuring device 5 in a position where the shadow of the measuring device 5 itself or the tripod will not be reflected in the image. If the weather is fine or the measurement is indoors, the user is asked to select whether or not the structure has a reflective surface (window glass or mirror), and if there is a reflective surface, an announcement is made to select a location for installing the measuring device 5 in a position where the shadow will not be reflected in the reflective surface.

<Guidance on image processing for 3D image generation>
7 to 12 in the above embodiment, the guidance device 200 may provide guidance on a method for deleting overlapping portions of point cloud data acquired from a plurality of measuring devices 5. The guidance device 200 may further include a detection unit (not shown) that detects overlapping portions of point cloud data.

The guidance for deleting overlapping portions of the point cloud data may be configured as follows. For example, when generating point cloud data from measurement values acquired on-site from the measuring device 5, the presentation unit 202 may be configured to find overlapping portions from the measurement positions of the measuring device 5, output a message inquiring whether or not to delete the point cloud data of the overlapping portions, accept an operation as to whether or not to delete, and perform a process of deleting the overlapping portions if an operation to delete is accepted.

With this configuration, overlapping portions can be deleted when generating point cloud data, so the data volume of the point cloud data acquired by the point cloud data acquisition unit 102 of the 3D image generation device 100 can be reduced. The data transfer volume from the measuring device 5 to the 3D image generation device 100 can be reduced, so the transfer time can be shortened. Alternatively, the data volume of the recording medium to which the point cloud data is written from the measuring device 5 can be reduced, so the time required to write or read data to or from the recording medium can be reduced.

Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.
For example, the method for generating a three-dimensional image according to the above embodiment may further include a scanning planning step of planning the measurement conditions, the installation position of the measuring device 5, the scanning density, time, and the like in advance before the method is performed on-site. The presentation unit 202 of the guidance device 200 according to the above embodiment may further present guidance for causing the operator to perform the scanning planning step. In other words, the presentation unit 202 may present guidance including the contents required for making a scanning plan in advance.

In this scanning planning process, the measurement conditions of this embodiment and the installation position of the measuring device 5 of the above embodiment are appropriately combined. It is preferable to consider in advance the time period during which scanning is performed for each target. For example, the constraints on the time period during which scanning is performed for each target are considered. It is preferable to predict the weather using a weather forecast in advance and create a scanning plan. For example, when scanning target A, which is the measurement target of the cultural property building itself, target B such as an earthen wall that is an accessory to the cultural property, and target C indoors outdoors, if the weather forecast is cloudy in the morning and sunny in the afternoon, a plan is made to scan targets A and B in the morning and target C in the afternoon.

In addition, the scanning plans for targets A and B are made by distinguishing between the installation position of the measuring device 5 and the scanning accuracy and time for target A, which is the measurement target itself, and target B, which is an accessory.

The present invention has been described above with reference to embodiments and examples, but the present invention is not limited to the above embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Below, examples of reference forms are given.
1. A point cloud data generation process for generating a plurality of point cloud data representing shapes and colors of different portions of the structure by measuring the structure a plurality of times while varying at least one of the horizontal and vertical installation positions of a measuring device;
a three-dimensional image generating step of generating a three-dimensional image of the structure by processing the plurality of point cloud data;
A method for generating a three-dimensional image, comprising:
2. In the method for generating a three-dimensional image according to 1.
A method for generating a three-dimensional image, wherein in the point cloud data generation step, in two pieces of point cloud data obtained by measuring adjacent portions, the two adjacent portions partially overlap each other.
3. In the method for generating a three-dimensional image according to 2.,
A three-dimensional image generating method, comprising the steps of: deleting at least a part of the overlapping portion when combining the point cloud data acquired by the plurality of measuring devices in the three-dimensional image generating step.
4. In the method for generating a three-dimensional image according to any one of 1 to 3,
A three-dimensional image generating method, comprising: disposing the measuring device at a position higher than the structure when generating at least a portion of the point cloud data in the point cloud data generating step.
5. In the method for generating a three-dimensional image according to 4.,
A three-dimensional image generating method, wherein in the point cloud data generating step, the installation position of the measuring device is alternately repeated between a position higher than the structure and a position lower than the structure.
6. In the method for generating a three-dimensional image according to any one of 1 to 5,
A method for generating a three-dimensional image, wherein, in the point cloud data generation process, the distance between the measuring devices arranged at a position higher than the structure is wider than the distance between the measuring devices arranged at a position lower than the structure.
7. In the method for generating a three-dimensional image according to 6.,
A method for generating a three-dimensional image, wherein, in the point cloud data generation process, the density of the point cloud data acquired by the measuring device positioned at a position higher than the structure is set to be lower than the density of the point cloud data acquired by the measuring device positioned at a position lower than the structure.
8. In the method for generating a three-dimensional image according to any one of 1 to 7,
The point cloud data generating step is performed when the weather is cloudy.
9. In the method for generating a three-dimensional image according to any one of 1 to 8,
A method for generating a three-dimensional image, comprising: aligning a time period during which the point cloud data generation process is performed for each part of the structure.
10. In the method for generating a three-dimensional image according to any one of 1. to 9.,
In the point cloud data generating step,
A method for generating a three-dimensional image, comprising the steps of: when the sunlight is oblique, placing the measuring device in a position where the shadow of the measuring device is not captured.
11. In the method for generating a three-dimensional image according to any one of 1. to 10.,
In the point cloud data generating step,
A method for generating a three-dimensional image, in which, when the structure includes a reflective surface, the measuring device is positioned at a position where the reflective surface is not reflected.
12. In the method for generating a three-dimensional image according to any one of 1. to 11.,
The structure includes at least one of cultural properties, temples and shrines, world heritage sites (certified by the World Heritage Committee), industrial heritage sites (certified by the International Committee for the Conservation of Industrial Heritage), modern industrial heritage sites (certified by the Ministry of Economy, Trade and Industry), civil engineering heritage sites (recommended by the Japan Society of Civil Engineers), scenic spots, topography, or historical buildings and ruins that need to be preserved for future generations,
The three-dimensional image generating method is intended to convey the texture and color of the structure to future generations.

13. A point cloud data acquisition means for acquiring a plurality of point cloud data representing shapes and colors of different portions of a structure by measuring the structure a plurality of times while varying at least one of the horizontal and vertical installation positions of a measuring device, the point cloud data being generated by the three-dimensional image generating method according to any one of 1. to 12.;
and a three-dimensional image generating means for generating a three-dimensional image of the structure by processing the plurality of point cloud data.
14. A program for causing a computer to realize the three-dimensional image generating device according to 13.
15. A guidance device comprising a presentation means for presenting guidance on a method of generating a three-dimensional image to an operator in order to carry out the method of generating a three-dimensional image according to any one of 1. to 12.
16. A program for causing a computer to realize the guidance device according to 15.

1 Image processing system 3 Communication network 5, 5a, 5b, 5c, 5d, 5e Measuring device 10 Beam 60 Computer 62 CPU
64 Memory 66 Storage 68 I/O
69 Bus 70 Communication I/F
72 Input device 74 Output device 80 Program 100 Three-dimensional image generating device 102 Point cloud data acquisition unit 104 Image generating unit 110 Storage device 200 Guidance device 202 Presentation unit A, B, C Object h1, h2 Height La, Lb, Lc, Ld, Le Position R1, R2, R11, R12, R21, R22, R31, R32, R33, R34 Area S Star mark S101 Point cloud data generating process S103 Three-dimensional image generating process

Claims (11)

a point cloud data generating step of measuring the structure a plurality of times while varying the installation position of a measuring device in at least one of the horizontal and vertical directions to generate a plurality of point cloud data representing shapes and colors of different portions of the structure;
a three-dimensional image generating step of generating a three-dimensional image of the structure by processing the plurality of point cloud data;
Including,
In the point cloud data generating step, an interval between the measuring devices arranged at a position higher than the structure is set to be wider than an interval between the measuring devices arranged at a position lower than the structure;
The measuring device is a 3D laser scanner,
A method for generating three-dimensional images, wherein in the point cloud data generation process, the 3D laser scanner positioned at a position higher than the structure performs scanning with a coarser laser irradiation interval than the 3D laser scanner positioned at a position lower than the structure, and generates the point cloud data. 2. The method for generating a three-dimensional image according to claim 1,
A method for generating a three-dimensional image, wherein in the point cloud data generation step, in two pieces of point cloud data obtained by measuring adjacent portions, the two adjacent portions partially overlap each other. The method for generating a three-dimensional image according to claim 2,
A three-dimensional image generating method, comprising the steps of: deleting at least a part of the overlapping portion when combining the point cloud data acquired by the plurality of measuring devices in the three-dimensional image generating step. The method for generating a three-dimensional image according to any one of claims 1 to 3,
A three-dimensional image generating method, comprising: disposing the measuring device at a position higher than the structure when generating at least a portion of the point cloud data in the point cloud data generating step. The method for generating a three-dimensional image according to any one of claims 1 to 4 ,
The point cloud data generating step is performed when the weather is cloudy. The method for generating a three-dimensional image according to any one of claims 1 to 5 ,
A method for generating a three-dimensional image, comprising: aligning a time period during which the point cloud data generation process is performed for each part of the structure. The method for generating a three-dimensional image according to any one of claims 1 to 6 ,
The structure includes at least one of cultural properties, temples and shrines, world heritage sites, industrial heritage sites, modern industrial heritage sites, civil engineering heritage sites, scenic spots, topography, or historical buildings and ruins that need to be preserved for future generations,
The three-dimensional image generating method is intended to convey the texture and color of the structure to future generations. a point cloud data acquisition means for acquiring a plurality of point cloud data representing shapes and colors of different portions of a structure by measuring the structure a plurality of times while varying at least one of an installation position of a measuring device in a horizontal direction and a vertical direction, the point cloud data being generated by the three-dimensional image generating method according to any one of claims 1 to 7;
a three-dimensional image generating means for generating a three-dimensional image of the structure by processing the plurality of point cloud data,
The distance between the measuring devices arranged at a position higher than the structure is set to be greater than the distance between the measuring devices arranged at a position lower than the structure;
The measuring device is a 3D laser scanner,
The 3D laser scanner arranged at a position higher than the structure performs scanning by making the laser irradiation interval coarser than the 3D laser scanner arranged at a position lower than the structure;
The point cloud data acquisition means of the three-dimensional image generating device acquires the point cloud data generated by the 3D laser scanner. A program for causing a computer to realize the three-dimensional image generating device according to claim 8 . A guidance device comprising: a presentation unit that presents guidance of a three-dimensional image generating method to an operator in order to execute the three-dimensional image generating method according to any one of claims 1 to 7 . A program for causing a computer to realize the guidance device according to claim 10 .

Images