JP7638110B2 - Information processing system, information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing system, an information processing device, an information processing method, and a program.

Generally, when a worker uses heavy machinery or the like to carry out a specified construction project on an object, the worker carries out the construction while looking at a construction drawing. In addition, a technology has been devised in which guidance according to the construction status is displayed on a monitor device installed near the driver's seat where the worker operates the machine, and the worker carries out the construction project by following the guidance displayed on the monitor device (see, for example, Patent Document 1).

JP 2012-136985 A

When a worker performs construction while looking at construction drawings, the worker must judge for himself whether the actual construction is proceeding in accordance with the construction drawings. Therefore, even if the technology described in Patent Document 1 is used, the worker must temporarily stop construction in order to look at the guidance displayed on the monitor device. This poses the problem that it is difficult to perform construction smoothly.

The object of the present invention is to provide an information processing system, information processing device, information processing method, and program that enable smooth construction work.

The information processing system of the present invention comprises:
A transmissive display;
a position information acquisition unit that acquires position information indicating a position of the heavy equipment operated by the worker;
An input unit for inputting three-dimensional drawing information of a construction target from an external device;
a calculation unit that calculates a position at which the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit is to be displayed on the transmissive display based on the position information acquired by the position information acquisition unit;
a display control unit that displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position calculated by the calculation unit of the transmissive display,
The transmissive display is disposed at a position where the worker can visually recognize an object to be worked on through the transmissive display.

The information processing device of the present invention comprises:
a position information acquisition unit that acquires position information indicating a position of the heavy equipment operated by the worker;
An input unit for inputting three-dimensional drawing information of a construction target from an external device;
A calculation unit that calculates a position for displaying the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit on a transparent display mounted on the heavy equipment, the transparent display being arranged at a position where the worker can view the target to be constructed through the transparent display mounted on the heavy equipment, based on the position information acquired by the position information acquisition unit;
and a display control unit that displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position calculated by the calculation unit on the transmissive display.

The information processing method of the present invention further comprises:
A process of acquiring location information indicating a location of a heavy machine operated by a worker;
A process of inputting three-dimensional drawing information of the construction target from an external source;
A process of calculating a position for displaying the three-dimensional drawing indicated by the input three-dimensional drawing information on a transparent display mounted on the heavy equipment, the transparent display being arranged at a position where the worker can visually recognize the target to be constructed through the transparent display mounted on the heavy equipment, based on the acquired position information;
and displaying the three-dimensional drawing indicated by the three-dimensional drawing information at the calculated position on the transmissive display.

In addition, the program of the present invention is
A program executed by a computer,
On the computer,
A step of acquiring location information indicating a location of a heavy machine operated by a worker;
A procedure for inputting 3D drawing information of the construction target from outside;
A step of calculating a position where the three-dimensional drawing indicated by the input three-dimensional drawing information is displayed on a transparent display mounted on the heavy equipment, the transparent display being arranged at a position where the worker can visually recognize the target to be constructed through the transparent display mounted on the heavy equipment, based on the acquired position information;
and a procedure of displaying the three-dimensional drawing indicated by the three-dimensional drawing information at the calculated position on the transmissive display.

In the present invention, construction can be carried out smoothly.

1 is a diagram showing a first embodiment of an information processing system according to the present invention; 2 is a diagram illustrating an example of an internal configuration of the information processing device illustrated in FIG. 1 . 2 is a flowchart illustrating an example of an information processing method in the information processing system illustrated in FIG. 1 . FIG. 11 is a diagram showing a second embodiment of an information processing system according to the present invention. 5 is a diagram showing an example of an internal configuration of the information processing device shown in FIG. 4 and a connection with components other than the information processing device. 6 is a diagram showing an example of a state in which the camera shown in FIG. 5 captures an image of a marker; 11 is a diagram for explaining an example of a method for correcting a measured position of a heavy machine. FIG. 13 is a diagram showing an example of measuring the direction from a moving heavy equipment to a marker and the distance between the heavy equipment and the marker. FIG. FIG. 13 is a diagram showing an example of a state in which there is an angle between a horizontal plane and an actual construction surface. 1A to 1C are diagrams illustrating examples of lines of sight when a construction target area is viewed from viewpoints at mutually different positions. 5 is a diagram showing an example of an AR display displayed by the transmissive display shown in FIG. 4 . 5 is a flowchart illustrating an example of an information processing method in the information processing system illustrated in FIG. 4 .

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
First Embodiment

Fig. 1 is a diagram showing a first embodiment of an information processing system of the present invention. As shown in Fig. 1, the information processing system in this embodiment has a transmissive display 100 and an information processing device 200. The transmissive display 100 and the information processing device 200 can exchange information with each other. In other words, the transmissive display 100 and the information processing device 200 are connected by wire or wirelessly.

The transparent display 100 is a display capable of displaying AR (Augmented Reality) with a structure that allows information (screen) to be displayed on the display while allowing the other side of the display to be seen through. The transparent display 100 is disposed at a position where a worker performing work using heavy machinery can visually confirm the target to be constructed through the transparent display 100. For example, when a worker performing work using heavy machinery sits in the driver's seat of the heavy machinery to perform work, the transparent display 100 may be disposed along the windshield of the heavy machinery on the worker's side of the windshield. If the transparent member for displaying AR is a goggle-type member worn by the worker, there is a concern that the safety of the worker wearing the goggle-type member may be reduced. For this reason, in the present invention, the transparent display for displaying AR is disposed on the front of the heavy machinery, particularly along the windshield. This makes it easier for the worker to check the surrounding situation without only viewing the AR display, and thus prevents a decrease in safety.

The information processing device 200 displays a predetermined screen on a transmissive display. FIG. 2 is a diagram showing an example of the internal configuration of the information processing device 200 shown in FIG. 1. As shown in FIG. 2, the information processing device 200 shown in FIG. 1 has a position information acquisition unit 210, an input unit 220, a calculation unit 230, and a display control unit 240. Note that FIG. 2 shows only the main components related to this embodiment among the components of the information processing device 200 shown in FIG. 1.

The position information acquisition unit 210 acquires position information indicating the position of the heavy machinery operated by the worker. The position information acquisition unit 210 acquires position information using GNSS (Global Navigation Satellite System). The position information acquisition unit 210 acquires position information using an antenna mounted on the heavy machinery. The position information acquired by the position information acquisition unit 210 may be information indicating a position in three-dimensional space using three-dimensional coordinates.

The input unit 220 inputs three-dimensional drawing information of the construction object from the outside. The input unit 220 inputs the three-dimensional drawing information based on an operation received from the outside. The input unit 220 may use software for reading a specified drawing, or may use a general mechanism for inputting information from the outside, such as a keyboard, mouse, or touch panel. The three-dimensional drawing information input by the input unit 220 is information showing the construction drawing, and includes information showing the dimensions and positions of the construction object.

The calculation unit 230 calculates the position at which the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit 220 is to be displayed on the transparent display 100, based on the position information acquired by the position information acquisition unit 210. In other words, the calculation unit 230 calculates the position on the transparent display 100 at which the three-dimensional drawing is to be displayed, so that the object seen by the worker through the transparent display 100 overlaps with the object on the three-dimensional drawing displayed on the transparent display 100. The position calculated by the calculation unit 230 in this way is the position at which, when the transparent display 100 displays the three-dimensional drawing, the object in the three-dimensional drawing displayed by the transparent display 100 appears to overlap with the object seen by the worker through the transparent display 100.

The display control unit 240 displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position calculated by the calculation unit 230 on the transmissive display 100. By the display control unit 240 controlling the display of the three-dimensional drawing in this manner, the worker can see the object in the three-dimensional drawing and the object seen through the transmissive display 100 superimposed on each other.

The following describes an information processing method in the information processing system shown in FIG. 1. FIG. 3 is a flowchart for explaining an example of an information processing method in the information processing system shown in FIG. 1.

First, the position information acquisition unit 210 acquires position information indicating the position of the heavy equipment operated by the worker (step S1). The input unit 220 also inputs three-dimensional drawing information from the outside (step S2). Either the process of step S1 or the process of step S2 may be performed first. Then, the calculation unit 230 calculates the position at which the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit 220 is to be displayed on the transmissive display 100 based on the position information acquired by the position information acquisition unit 210 (step S3). Then, the display control unit 240 displays the three-dimensional drawing indicated by the three-dimensional drawing information on the transmissive display 100 at the position calculated by the calculation unit 230 (step S4).

In this way, the construction drawing is displayed on the transparent display 100, and the worker can check the actual construction object and the object on the construction drawing by overlapping them. This allows the worker to check the construction drawing on the transparent display 100 while performing the actual work, and construction can be carried out smoothly.
Second Embodiment

Fig. 4 is a diagram showing a second embodiment of the information processing system of the present invention. As shown in Fig. 4, the information processing system in this embodiment has a transmissive display 100, an information processing device 201, a camera 301, a direction calculation unit 311, a distance measurement unit 321, a marker 401, an acceleration sensor 501, a viewpoint position calculation unit 601, and a detection unit 701. The transmissive display 100 is the same as that in the first embodiment. Each component will be described with reference to Fig. 5.

Figure 5 is a diagram showing an example of the internal configuration of the information processing device 201 shown in Figure 4 and the connection with components other than the information processing device 201. As shown in Figure 5, the information processing device 201 shown in Figure 4 has a location information acquisition unit 210, an input unit 220, a calculation unit 231, and a display control unit 241. The location information acquisition unit 210 and the input unit 220 are the same as those in the first embodiment. Also, Figure 5 shows only the main components related to this embodiment among the components of the information processing device 201 shown in Figure 4.

Marker 401 is a component placed at a predetermined position. Marker 401 may be any component that can be imaged by camera 301 and whose placement (installation) position can be recognized, and may be, for example, an LED marker that uses an LED (Light Emitting Diode).

Camera 301 captures the surroundings around the heavy equipment. Camera 301 captures still images and videos at a predetermined timing. In particular, camera 301 captures images of markers 401. Camera 301 is mounted, for example, on heavy equipment used by a worker for work. It is preferable that camera 301 be capable of capturing images in a 360-degree range.

Figure 6 is a diagram showing an example of how the camera 301 shown in Figure 5 captures the image of the marker 401. The example shown in Figure 6 shows a case where there are two markers 401, marker 401-1 and marker 401-2. As shown in Figure 6, the camera 301 is installed in a position where it can capture images of the markers 401-1 and 401-2. The reason why the camera 301 captures images of the markers 401-1 and 401-2 will be described below.

Figure 7 is a diagram for explaining an example of a method for correcting the measured position of heavy equipment. The GNSS antenna shown in Figure 7 is one of the components constituting the position information acquisition unit 210, and is mounted on the heavy equipment. As shown in Figure 7, there is a slight error between the position of the heavy equipment measured using the GNSS and the actual position of the heavy equipment in the system. This is because a certain range of error is allowed in the GNSS. Therefore, if the position indicated by the position information acquired by the position information acquisition unit 210 is used as the position of the heavy equipment and processing is performed as it is, a deviation occurs from the actual position of the heavy equipment, and more accurate processing (AR display) cannot be performed. Therefore, a camera 301 mounted on the heavy equipment captures an image of a marker 401 installed at a predetermined position, the relative position of the marker 401 with respect to the camera 301 is calculated, and the position information acquired by the position information acquisition unit 210 is corrected using the calculated value. Specifically, for this correction, a direction calculation unit 311 and a distance measurement unit 321 are used.

The direction calculation unit 311 calculates the direction of the marker captured by the camera 301 relative to the camera 301. The direction calculation unit 311 calculates the imaging direction of the camera 301 relative to the direction in which the heavy equipment faces. At this time, the direction calculation unit 311 calculates the imaging direction of the camera 301 relative to the direction in which the heavy equipment faces, assuming that the direction in which the heavy equipment is moving is the direction in which the heavy equipment faces.

The distance measurement unit 321 measures the distance from the camera 301 to the marker 401 captured by the camera 301. In this case, the distance measurement unit 321 may measure the distance using a laser radar, for example. Note that this system may not include the distance measurement unit 321.

FIG. 8 is a diagram showing an example of how the direction from a moving heavy machine to a marker and the distance between the heavy machine and the marker are measured. As shown in FIG. 8, when the heavy machine moves from position A to position B, the calculation unit 231 calculates the direction (facing direction) of the heavy machine based on the movement. For example, if the three-dimensional coordinates of position A are (x ₀ , y ₀ , z ₀ ) and the three-dimensional coordinates of position B are (x ₁ , y ₁ , z ₁ ), the components of the vector of the facing direction when moving from position A to position B are (x ₁ -x ₀ , y ₁ -y ₀ , z ₁ -z ₀ ). The calculation unit 231 regards this direction as the facing direction of the heavy machine at that time. The direction calculation unit 311 calculates the directions of the markers 401-1 and 401-2 captured by the camera 301 with respect to the facing direction of the heavy machine calculated by the calculation unit 231. Further, distance measurement unit 321 measures the distance between the moving heavy equipment and each of markers 401-1 and 401-2.

The acceleration sensor 501 measures the inclination of the heavy equipment relative to a horizontal plane. Specifically, the acceleration sensor 501 is mounted on the heavy equipment and measures the gravitational acceleration, calculates the vertical inclination based on the measured value, and measures the horizontal inclination (inclination of the construction surface), i.e., the inclination of the heavy equipment, based on the calculated vertical inclination.

Figure 9 is a diagram showing an example of a situation when there is an angle between the horizontal plane and the actual construction surface. As shown in Figure 9, the position that can be acquired by the position information acquisition unit 210 is a position on the horizontal plane. Therefore, in a conventional system that detects a position using only the position information acquisition unit 210, when there is an angle between the actual construction surface and the horizontal plane, a difference occurs between the coordinate position of the target on the system and the position where the target is actually seen by the worker. As a result, the accurate position of the target cannot be recognized due to the inclination of the construction surface. Therefore, the inclination of the heavy equipment is measured using the acceleration sensor 501, and the measurement result is used.

The viewpoint position calculation unit 601 calculates the relative position of the operator's viewpoint with respect to the transmissive display 100. The viewpoint position calculation unit 601 may be attached to the transmissive display 100.

Figure 10 is a diagram showing an example of the line of sight when the construction target area is viewed from viewpoints at different positions. As shown in Figure 10, the line of sight positions on the transparent display 100 are different when the construction target area is viewed from viewpoint A through the transparent display 100 and when the construction target area is viewed from viewpoint B through the transparent display 100. In other words, if the display position of the AR image displayed on the transparent display 100 is not determined according to the viewpoint, the overlap between the image displayed on the transparent display 100 and the construction target area visible on the other side will not be as desired.

The detection unit 701 detects the status of construction. The detection unit 701 detects whether construction has been completed for the area to be constructed. For example, the detection unit 701 may detect whether heavy machinery has performed an operation for performing a specified construction. More specifically, in the case where the construction involves compaction using a vibrating roller, the detection unit 701 may detect whether heavy machinery has performed an operation for compaction.

The calculation unit 231 has the following functions in addition to the functions of the calculation unit 230 in the first embodiment. The calculation unit 231 calculates the orientation of the heavy equipment based on the position information acquired by the position information acquisition unit 210 as the heavy equipment moves. The specific method of calculating the orientation of the heavy equipment is as described with reference to FIG. 8. Then, the calculation unit 231 calculates the position at which the three-dimensional drawing is to be displayed on the transmissive display 100 based on the calculated orientation of the heavy equipment and the position information acquired by the position information acquisition unit 210. Furthermore, the calculation unit 231 calculates the positions at which the markers 401-1 and 401-2 are installed based on the calculated orientation of the heavy equipment, the directions of the markers 401-1 and 401-2 relative to the camera 301 calculated by the direction calculation unit 311, and the distances between the heavy equipment and the markers 401-1 and 401-2 measured by the distance measurement unit 321. The calculation unit 231 then corrects the position of the heavy equipment acquired by the position information acquisition unit 210 based on the difference between the calculated positions of the markers 401-1 and 401-2 and the positions of the markers 401-1 and 401-2 that have been set (stored) in advance. Furthermore, the calculation unit 231 corrects the position information using the inclination measured by the acceleration sensor 501. Furthermore, the calculation unit 231 corrects the position information using the viewpoint position calculated by the viewpoint position calculation unit 601. Then, the calculation unit 231 calculates the position at which the three-dimensional drawing is to be displayed on the transmissive display 100 based on the corrected position information. Note that if the present system does not include the distance measurement unit 321, the calculation unit 231 calculates the positions at which the markers 401-1 and 401-2 are installed based on the calculated orientation of the heavy equipment and the directions of the markers 401-1 and 401-2 relative to the camera 301 calculated by the direction calculation unit 311.

The display control unit 241 has the following functions in addition to the functions of the display control unit 240 in the first embodiment. The display control unit 241 changes the display mode of the three-dimensional drawing displayed on the transmissive display 100 based on the construction status detected by the detection unit 701. For example, the display control unit 241 may color in a predetermined color the area where the detection unit 701 detects that construction has been completed.

Figure 11 is a diagram showing an example of an AR display displayed by the transparent display 100 shown in Figure 4. As shown in Figure 11, the display control unit 241 causes the transparent display 100 to display the area to be worked on from the three-dimensional drawing surrounded by a dashed line, and causes the transparent display 100 to display the area of the work that the detection unit 701 detects as having been worked on in color. Note that the display control unit 241 may use a method other than coloring as long as it can distinguish the area that the detection unit 701 detects as having been worked on from the other areas. By the transparent display 100 performing such a display, the worker can recognize the areas where work has been completed and the areas where work has not been completed while looking at the transparent display 100 through which the work area is seen.

The heavy machinery may also be equipped with a function to turn off the AR display of the transmissive display 100 in an emergency, or a function to prevent the mounted mechanism from malfunctioning due to the vibration of the heavy machinery. An abnormality detection mechanism may be provided to detect an abnormality in the operation of the heavy machinery, and when the abnormality detection mechanism detects an abnormality in the heavy machinery, the AR display of the transmissive display 100 may be turned off. The timing for turning off the AR display of the transmissive display 100 may be based on the level of the abnormality detected by the abnormality detection mechanism. For example, if the level of the abnormality is high, the AR display may be turned off immediately when the abnormality is detected, and if the level of the abnormality is low, the AR display may be turned off depending on the number of times the abnormality is detected. In addition, a vibration-proof rubber may be used as a function to prevent the mounted mechanism from malfunctioning due to the vibration of the heavy machinery. For example, the vibration-proof rubber may be installed on the transmissive display 100 or the mounting portion of the transmissive display 100 to reduce the vibration that the heavy machinery applies to the transmissive display 100.

The following describes an information processing method in the information processing system shown in FIG. 4. FIG. 12 is a flowchart for explaining an example of an information processing method in the information processing system shown in FIG. 4.

First, the input unit 220 inputs three-dimensional drawing information from the outside (step S11). The position information acquisition unit 210 also acquires position information indicating the position of the heavy equipment operated by the worker (step S12). Here, either the process of step S11 or the process of step S12 may be performed first. Next, the acceleration sensor 501 measures the inclination of the heavy equipment with respect to the horizontal plane (step S13). Also, the viewpoint position calculation unit 601 calculates the relative position of the viewpoint of the worker with respect to the transparent display 100 (step S14). Here, either the process of step S13 or the process of step S14 may be performed first. Then, the calculation unit 231 calculates the position at which the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit 220 is to be displayed on the transparent display 100 based on the position information acquired by the position information acquisition unit 210. Furthermore, the calculation unit 231 corrects the calculated position based on the inclination of the heavy equipment measured by the acceleration sensor 501 and the viewpoint position calculated by the viewpoint position calculation unit 601 to create an AR model (step S15). The display control unit 241 displays the AR model calculated by the calculation unit 231 on the transmissive display 100 (step S16). This AR model displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position corrected by the calculation unit 230 on the transmissive display 100.

After that, when the heavy equipment moves (step S17), the location information acquisition unit 210 further acquires location information indicating the location of the heavy equipment (step S18). Next, the calculation unit 231 calculates the orientation of the heavy equipment based on the location information acquired by the location information acquisition unit 210 as the heavy equipment moves. Furthermore, the calculation unit 231 corrects the location information using the direction calculated by the direction calculation unit 311 based on the calculated orientation of the heavy equipment and the distance measured by the distance measurement unit 321 (step S19). Then, the processing of step S13 is performed. Note that if the present system does not include the distance measurement unit 321, in step S19, the calculation unit 231 corrects the location information using the direction calculated by the direction calculation unit 311 based on the calculated orientation of the heavy equipment.

In this way, the construction drawing is displayed in AR on the transparent display 100, so that the worker can check the actual construction object and the object of the construction drawing by overlapping them. Therefore, the worker who operates the heavy equipment can check the construction status while facing forward in the driver's seat, and there is no need to temporarily stop work due to checking the construction location on the system management screen in a typical rolling compaction system. This enables continuous construction and improves the construction speed. In addition, by displaying the completed area so that it can be distinguished from the area where construction has not yet been performed, it is possible to prevent construction omissions in the unlikely event of an accident. Furthermore, the display position of the construction drawing is calculated using not only the position acquired by the position information acquisition unit 210, but also the relative position and direction with respect to the marker 401 captured by the camera 301 mounted on the heavy equipment, the inclination of the construction surface, and the position of the worker's viewpoint relative to the transparent display 100, and the AR display is performed. Therefore, a more accurate AR display can be performed. This allows construction to be performed smoothly.

Although the above description has been given with each component assigned a different function (process), this allocation is not limited to the above. Furthermore, the configuration of the components is not limited to the above-mentioned form, and the above-mentioned form is merely an example.

The processing performed by the above-mentioned information processing devices 200, 201 and other components (hereinafter referred to as information processing devices) may be performed by logic circuits created according to the purpose. Also, a computer program (hereinafter referred to as a program) describing the processing contents as procedures may be recorded on a recording medium readable by the information processing device, and the program recorded on the recording medium may be read into the information processing device and executed. Recording media that can be read by information processing devices include removable recording media such as floppy (registered trademark) disks, optical magnetic disks, DVDs (Digital Versatile Discs), CDs (Compact Discs), Blu-ray (registered trademark) Discs, and USB (Universal Serial Bus) memories, as well as memories such as ROMs (Read Only Memory) and RAMs (Random Access Memory) built into the information processing devices, and HDDs (Hard Disc Drives). The programs recorded on these recording media are read by a CPU provided in the information processing device, and the same processing as described above is performed under the control of the CPU. Here, the CPU operates as a computer that executes the programs read from the recording media on which the programs are recorded.

REFERENCE SIGNS LIST 10 Heavy machinery 100 Transmissive display 200, 201 Information processing device 210 Position information acquisition unit 220 Input unit 230, 231 Calculation unit 240, 241 Display control unit 301 Camera 311 Direction calculation unit 321 Distance measurement unit 401, 401-1, 401-2 Marker 501 Acceleration sensor 601 Viewpoint position calculation unit 701 Detection unit

Claims (6)

A transmissive display;
a position information acquisition unit that acquires position information indicating a position of the heavy equipment operated by the worker;
An input unit for inputting three-dimensional drawing information of a construction target from an external device;
a calculation unit that calculates an orientation of the heavy equipment based on position information acquired by the position information acquisition unit in association with the movement of the heavy equipment, and calculates a position for displaying the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit on the transmissive display based on the calculated orientation and the position information acquired by the position information acquisition unit;
a display control unit that displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position of the transmissive display calculated by the calculation unit;
A camera that captures an image of the surroundings centered on the heavy machinery;
A marker installed at a predetermined position;
a direction calculation unit that calculates a direction of the marker captured by the camera relative to the camera based on the direction of the heavy equipment calculated by the calculation unit ;
A detection unit that detects the status of construction ,
The transmissive display is disposed at a position where the worker can visually recognize an object to be constructed through the transmissive display,
The calculation unit corrects the position information acquired by the position information acquisition unit based on the direction calculated by the direction calculation unit and a predetermined position where the marker is installed, and calculates a position for displaying the three-dimensional drawing on the transmissive display based on the corrected position information ;
The display control unit is an information processing system that displays in color the portion where the detection unit detects that construction has been completed . An acceleration sensor is provided to measure the inclination of the heavy machinery relative to a horizontal plane,
2. The information processing system according to claim 1, wherein the calculation unit corrects the position information acquired by the position information acquisition unit using the tilt measured by the acceleration sensor, and calculates a position at which the three-dimensional drawing is to be displayed on the transmissive display based on the corrected position information. a viewpoint position calculation unit that calculates a position of a viewpoint of the operator with respect to the transmissive display,
3. The information processing system according to claim 1, wherein the calculation unit corrects the position information acquired by the position information acquisition unit using the viewpoint position calculated by the viewpoint position calculation unit, and calculates a position at which the three-dimensional drawing is to be displayed on the transmissive display based on the corrected position information. a position information acquisition unit that acquires position information indicating a position of the heavy equipment operated by the worker;
An input unit for inputting three-dimensional drawing information of a construction target from an external device;
A calculation unit that calculates the orientation of the heavy equipment based on the position information acquired by the position information acquisition unit as the heavy equipment moves, and calculates a position for displaying the three-dimensional drawing indicated by the three-dimensional drawing information input by the input unit on a transparent display mounted on the heavy equipment, the transparent display being disposed at a position where the worker can view the target of construction through the transparent display mounted on the heavy equipment, based on the calculated orientation and the position information acquired by the position information acquisition unit;
a display control unit that displays the three-dimensional drawing indicated by the three-dimensional drawing information at the position calculated by the calculation unit of the transmissive display,
The calculation unit corrects the position information acquired by the position information acquisition unit based on a direction of a marker installed at a predetermined position captured by a camera that captures an image of the surroundings centered on the heavy machine, the direction being calculated based on the calculated orientation of the heavy machine, and a predetermined position at which the marker is installed, and calculates a position for displaying the three-dimensional drawing on the transmissive display based on the corrected position information ;
The display control unit is an information processing device that displays in color a portion where a detection unit that detects the status of the construction has detected that construction has been completed . A process of acquiring location information indicating a location of a heavy machine operated by a worker;
A process of inputting three-dimensional drawing information of the construction target from an external source;
A process of calculating a direction of the heavy equipment based on position information acquired in association with the movement of the heavy equipment;
A process of calculating a position for displaying the three-dimensional drawing indicated by the input three-dimensional drawing information on a transparent display mounted on the heavy equipment, the transparent display being arranged at a position where the worker can visually recognize the target to be constructed through the transparent display mounted on the heavy equipment, based on the calculated orientation and the acquired position information;
A process of displaying the three-dimensional drawing indicated by the three-dimensional drawing information at the calculated position on the transmissive display;
a process of correcting the acquired position information based on a direction of a marker installed at a predetermined position captured by a camera capturing an image of the surroundings centered on the heavy equipment, the direction being calculated based on the calculated orientation of the heavy equipment, and the predetermined position at which the marker is installed, and calculating a position for displaying the three-dimensional drawing on the transmissive display based on the corrected position information ;
and coloring the parts of the three-dimensional drawing that a detection unit that detects the status of construction detects as having been completed and displaying them on the transmissive display . On the computer,
A step of acquiring location information indicating a location of a heavy machine operated by a worker;
A procedure for inputting 3D drawing information of the construction target from outside;
A step of calculating a direction of the heavy equipment based on position information acquired in association with the movement of the heavy equipment;
A step of calculating a position where the three-dimensional drawing indicated by the input three-dimensional drawing information is displayed on a transparent display mounted on the heavy equipment, the transparent display being arranged at a position where the worker can view the target to be constructed through the transparent display mounted on the heavy equipment, based on the calculated orientation and the acquired position information;
a step of displaying the three-dimensional drawing indicated by the three-dimensional drawing information at the calculated position on the transmissive display;
a step of correcting the acquired position information based on a direction of a marker installed at a predetermined position captured by a camera capturing an image of the surroundings centered on the heavy equipment, the direction being calculated based on the calculated orientation of the heavy equipment, and the predetermined position at which the marker is installed, and calculating a position for displaying the three-dimensional drawing on the transmissive display based on the corrected position information ;
and a procedure for displaying on the transmissive display a portion of the three-dimensional drawing that has been detected as completed by a detection unit that detects the status of construction .

Images