JP7689879B2 - Joining operation support device, joining operation support system, joining operation support method, and program - Google Patents

Description

The present disclosure relates to a joining operation support device, a joining operation support system, a joining operation support method, and a program for supporting joining operations such as brazing or welding.

Conventionally, technology has been widespread that supports joining work by presenting joining work support information to a worker performing joining work such as brazing or welding. For example, Patent Literature 1 discloses a technology that supports joining work by presenting abnormalities in a joint to a worker performing joining work.

JP 2019-56622 A

In recent years, there has been a shortage of workers with extensive work experience in the field of joining techniques to support joining work such as brazing or welding. Workers with extensive work experience are required to instruct workers with less work experience. When instructing workers on joining work, it is believed that if the instructor can instruct the worker during the joining work, the worker's joining technique will improve quickly. In contrast, the technique disclosed in Patent Document 1 does not allow the instructor to instruct the worker during the joining work.

This disclosure has been made to solve the above problems, and aims to provide a joining work support device that allows an instructor to instruct a worker during joining work.

The joining operation support device according to the present disclosure includes an image acquisition unit that acquires each visible image taken by a visible image acquisition unit that is provided on each of the two display devices and that captures a joining operation area including the materials to be joined and a marker, and each thermal image taken by a thermal imaging unit that is provided on each of the two display devices and that captures a temperature distribution in the joining operation area, a temperature detection unit that detects the temperature of the joining portion of the materials to be joined by superimposing each of the visible image and each of the thermal image acquired by the image acquisition unit and connecting the superimposed images together in three dimensions, and a temperature determination unit that determines whether or not the temperature of the joining portion detected by the temperature detection unit is acceptable. and a three-dimensional data acquisition unit that acquires from the memory unit three-dimensional data indicating the three-dimensional shapes and dimensions of the components to be joined, the base material to which the components to be joined are joined, and the jig for fixing the components to be joined and the base material, and a display control unit that displays joining work support information for supporting the joining work on the display units of each of the two display devices based on the judgment result determined by the temperature judgment unit, and displays the joining work seen through one of the display devices in three dimensions on the display unit of the other display device based on the position and orientation of a marker included in one of the visible photographed images acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit.

According to the present disclosure, an instructor can instruct a worker during a joining operation.
The worker and the trainer who trains the worker can quantitatively view the actual work of the worker, which they could only learn by watching and imitating the movements and timing of skilled workers based on intuition. This allows the trainer to accurately train the worker on the joining work in a short time. In addition, by quantitatively displaying the joining work support information for the joining work based on the worker's intuition, it becomes possible to stabilize the joining quality and manage that joining quality.

1 is a block diagram showing a configuration of a joining operation assistance system to which a joining operation assistance device according to a first embodiment is applied; 1A and 1B are diagrams showing the configuration of a jig used when joining piping members. 11 is a diagram showing the correspondence between the configuration of the display device and a joining operation. FIG. 4A to 4D are diagrams showing the display states of the display device worn by the instructor. 5A to 5D are diagrams sequentially showing the brazing process. 6A and 6B are diagrams showing a state in which brazing has not been performed correctly. FIG. 2 is a diagram showing a configuration of a pipe having a valve portion. FIG. 4 is a diagram showing a temperature change of a joint. 4 is a flowchart showing a joining operation supporting method according to the first embodiment. Fig. 10A is a block diagram showing a hardware configuration for implementing the functions of the joining operation supporting device according to the embodiment 1. Fig. 10B is a block diagram showing a hardware configuration for executing software for implementing the functions of the joining operation supporting device according to the embodiment 1.

The following describes in detail the embodiments of the present disclosure with reference to the drawings.

Embodiment 1.
A joining operation supporting device 10 according to a first embodiment will be described with reference to Figs. 1 to 10 .

Figure 1 is a block diagram showing the configuration of a joining operation support system to which a joining operation support device 10 according to the first embodiment is applied. Figure 2 is a diagram showing the configuration of a jig 61 used when joining piping members 71, 72, and 73. Figure 3 is a diagram showing the correspondence between the configurations of the display devices 20 and 30 and the joining operation.

The joining work assistance system shown in FIG. 1 assists a worker performing joining work such as brazing or welding by presenting joining work assistance information to assist the worker in the joining work. In this embodiment, a case will be described in which the joining work assistance system assists with brazing work, but the present invention is not limited to brazing work.

As shown in FIG. 2, the worker uses a jig 61 to braze the piping member 71, which is the member to be joined, to the piping members 72 and 73, which are the base materials, to assemble the piping 70. At this time, as shown in the upper diagram in FIG. 3, the worker supplies brazing material 63 to the joint of the piping member 71 while heating it with a flame sprayed from a gas burner 62.

Specifically, first, the worker fixes the piping members 71 and 72 in a vertical position at one end of the jig 61, and brazes one end of the piping member 71 to one end of the piping member 72 (part I in FIG. 2). Next, the worker fixes the piping member 71 brazed to the piping member 72 and the piping member 73 in a horizontal position at the other end of the jig 61, and brazes the other end of the piping member 71 to one end of the piping member 73 (part II in FIG. 2).

The jig 61 also has multiple AR markers 51, 52, 53 provided at different positions. Figure 2 shows an example in which three AR markers 51, 52, 53 are provided on the jig 61. The AR markers 51, 52, 53 are, for example, composed of a square black frame and a rectangular portion provided inside the frame. Each rectangular portion has a different shape, letter, color, etc. drawn on it. For this reason, the AR markers 51, 52, 53 are distinguished from one another.

As shown in FIG. 1, the joining work support system according to the first embodiment includes a joining work support device 10, display devices 20 and 30, and a CAD device 40. The joining work support device 10 is a computer connected to the display devices 20 and 30 by wire or wirelessly. The joining work support device 10 is not electrically connected to the CAD device 40. The CAD data created by the CAD device 40 is transferred to the joining work support device 10, for example, when necessary, using a storage medium or the like.

As shown in FIG. 3, the display devices 20 and 30 are, for example, head-mounted display type transmissive smart glasses that display digital information superimposed on a real image seen through the glass. Such display devices 20 and 30 may project the digital information onto a display surface (reflective surface) or directly onto a person's retina. One of the display devices 20 is worn by a worker performing the joining work. The other display device 30 is worn by an instructor who instructs the worker. The display devices 20 and 30 shown in FIG. 3 are configured to face the brazing work. The instructor who instructs the worker is, for example, a skilled worker with ample work experience.

As shown in Figures 1 and 3, the display device 20 has a visible light imaging section 21, a thermal imaging section 22, and a display section 23.

The visible photography unit 21 has a photography range that overlaps with most of the worker's field of vision. This visible photography unit 21 is, for example, a visible image camera. Therefore, when the worker is performing joining work, the visible photography unit 21 can photograph at least the joining work area including the piping member 71 fixed to the jig 61 and the AR markers 51, 52, and 53 provided on the jig 61 from the worker's side. In addition, the visible photography unit 21 outputs the captured visible photography image to the joining work support device 10.

The thermal imaging unit 22 has an imaging range that overlaps with most of the worker's field of vision. The thermal imaging unit 22 is, for example, a thermo camera. Therefore, the thermal imaging unit 22 can image the temperature distribution of the joining work area from the worker's side. The thermal imaging unit 22 also outputs the captured thermal image to the joining work support device 10.

The display unit 23 displays joining work support information in the field of view of the worker. The joining work support information is information for supporting the worker performing the joining work. This joining work support information is, for example, the temperature of the joint in the piping member 71, the timing of inserting and removing the brazing material 63 (timing of end of insertion), an alarm due to a rise in temperature of the piping member 71, etc. The display unit 23 is controlled by the display control unit 15, which will be described later, to display or not display the joining work support information.

As shown in Figures 1 and 3, the display device 30 has a visible light imaging section 31, a thermal imaging section 32, and a display section 33.

The visible photography unit 31 has a photography range that overlaps with most of the instructor's field of vision. This visible photography unit 31 is, for example, a visible image camera. Therefore, when the worker is performing the joining work, the visible photography unit 31 can photograph, from the instructor's side, at least the joining work area including the piping member 71 fixed to the jig 61 and the AR markers 51, 52, and 53 provided on the jig 61. In addition, the visible photography unit 31 outputs the captured visible photography image to the joining work support device 10.

The thermal imaging unit 32 has an imaging range that overlaps with most of the instructor's field of vision. The thermal imaging unit 32 is, for example, a thermo camera. Therefore, the thermal imaging unit 32 is capable of imaging the joining work area from the instructor's side. The thermal imaging unit 32 also outputs the captured thermal image to the joining work support device 10.

The display unit 33 displays the joining work support information in the field of vision of the instructor. The same joining work support information displayed on the display unit 23 is simultaneously displayed on this display unit 33. In addition, although details will be described later, the display unit 33 is capable of displaying the joining work of the worker as seen through the display device 20 in three dimensions. The display control unit 15, which will be described later, controls whether the display unit 33 displays the information.

The CAD device 40 designs the jig 61, the pipe 70, and the pipe members 71, 72, and 73. This CAD device 40 creates CAD data relating to them individually and stores it in the memory unit 41. The CAD data relating to the jig 61, the pipe 70, and the pipe members 71, 72, and 73 is three-dimensional data indicating the three-dimensional shape and dimensions of the jig 61, the three-dimensional shape and dimensions of the pipe 70, and the three-dimensional shape and dimensions of the pipe members 71, 72, and 73. The memory unit 41 may be provided in the joining work support device 10.

In contrast, the joining work support device 10 has an image acquisition unit 11, a temperature detection unit 12, a temperature determination unit 13, a three-dimensional data acquisition unit 14, and a display control unit 15.

The image acquisition unit 11 acquires a visible image captured by the visible light capture unit 21 and a thermal image captured by the thermal capture unit 22. The image acquisition unit 11 also acquires a visible image captured by the visible light capture unit 31 and a thermal image captured by the thermal capture unit 32. The image acquisition unit 11 outputs each visible image and each thermal image to the temperature detection unit 12.

The temperature detection unit 12 detects the temperature of the joint in the piping member 71 based on each visible image and each thermal image acquired by the image acquisition unit 11. The temperature detection unit 12 also outputs the detected temperature of the joint to the temperature determination unit 13.

Specifically, the temperature detection unit 12 superimposes the visible light image taken by the visible light imaging unit 21 and the thermal image taken by the thermal imaging unit 22. The temperature detection unit 12 also superimposes the visible light image taken by the visible light imaging unit 31 and the thermal image taken by the thermal imaging unit 32. The temperature detection unit 12 then three-dimensionally stitches together the superimposed images, and detects the temperature of the joint in the piping member 71 from this stitched three-dimensional image.

The temperature determination unit 13 determines whether the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 is acceptable. The temperature determination unit 13 also outputs the determination result to the display control unit 15. Specifically, the temperature determination unit 13 has a brazing material temperature determination unit 13a and a base material temperature determination unit 13b.

The brazing material temperature determination unit 13a determines whether the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63. The base material temperature determination unit 13b determines whether the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 exceeds the melting temperature of the piping members 72 and 73 that serve as the base material.

The three-dimensional data acquisition unit 14 acquires, from the memory unit 41 of the CAD device 40, three-dimensional data indicating the three-dimensional shape and dimensions of the piping member 71, the three-dimensional shape and dimensions of the piping members 72 and 73 to which the piping member 71 is joined, and the three-dimensional shape and dimensions of the jig 61 that fixes the piping members 71, 72, and 73. The three-dimensional data acquisition unit 14 also outputs the acquired three-dimensional data to the display control unit 15.

Based on the result of the determination made by the temperature determination unit 13, the display control unit 15 displays joining work support information to support the joining work on the display units 23 and 33 of the two display devices 20 and 30, respectively.

Specifically, when the brazing material temperature determination unit 13a determines that the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63 and reaches the preset optimal melting temperature, the display control unit 15 displays the joining work support information on each display unit 23, 33. That is, since the brazing material 63 has been properly melted and good brazing has been performed, the display control unit 15 presents the joining work support information to support the worker in the brazing work.

In addition, when the base material temperature determination unit 13b determines that the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 is a temperature just before the melting temperature of the base material piping members 72, 73, the display control unit 15 displays an alarm on each display unit 23, 33. In other words, the display control unit 15 alerts the worker to the brazing work before the base material melts, because if the base material melts, good brazing has not been performed.

Furthermore, the display control unit 15 displays the joining work seen through one display device 20 in three dimensions on the display unit 33 of the other display device 30 based on the positions and orientations of the AR markers 51, 52, 53 contained in one visible image acquired by the image acquisition unit 11 and the three-dimensional data acquired by the three-dimensional data acquisition unit 14.

Specifically, the display control unit 15 calculates the three-dimensional coordinates and orientation of the display device 20 based on the positions and orientations of the AR markers 51, 52, 53 included in the visible image captured by the visible image capture unit 21, and the three-dimensional data. Next, the display control unit 15 displays the joining work as seen by the worker through the display device 20 in three dimensions on the display unit 33 of the display device 30 worn by the instructor, based on the calculation result and the visible image captured by the visible image capture unit 21.

Therefore, the instructor can virtually view the joining work of the worker from the same direction as the worker views the joining work area. As a result, the instructor can provide detailed guidance to the worker on the joining work. The joining work support device 10 may display the joining work that the worker sees through the display device 20 in real time on a display device other than the display devices 20 and 30, so that the instructor can provide guidance to the worker. Furthermore, the joining work support device 10 may record the joining work that the worker sees through the display device 20, and display the recorded joining work on a display device other than the display devices 20 and 30, so that the instructor can provide guidance to the worker.

In contrast, workers can now quantitatively confirm the actual joining work, whereas previously they could only learn by imitating the movements of experienced workers, relying on intuition. This allows workers to acquire high-level joining techniques equivalent to those of experienced workers, with accurate and short instruction.

Figure 4 is a diagram showing the display state of the display device 30 worn by the instructor. Figures 4A to 4D show a state in which a three-dimensional image showing the joining work that the worker can see through the display device 20 and joining work support information or a warning are displayed on the display unit 33 of the display device 30. As described above, the joining work support information or warning displayed on the display unit 33 is the same as that displayed on the display unit 23. Also, the visible light imaging unit 31 and the thermal imaging unit 32 are omitted in Figures 4A to 4D.

The joining work support information shown in FIG. 4A is the temperature of the joint in the piping member 71 and a corresponding instruction to heat the joint. The temperature is displayed as a graph and numbers. The joining work support information shown in FIG. 4B is the temperature of the joint in the piping member 71 and a corresponding instruction to start inserting the brazing material. The joining work support information shown in FIG. 4C is the temperature of the joint in the piping member 71 and a corresponding instruction to end inserting the brazing material. The joining work support information shown in FIG. 4D is the temperature of the joint in the piping member 71. In response to this, a warning is displayed on the display unit 33 to the effect that the temperature of the joint is rising.

Figure 5 shows the brazing process in sequence. Figure 5D is a vertical cross-sectional view of Figure 5C.

The worker brazes the piping member 71 to the piping member 72 while viewing the joining operation support information displayed on the display unit 23 of the display device 20. As shown in Fig. 5A to Fig. 5D, for example, the brazing operation performed by the worker is performed by melting and supplying the brazing material 63 into a gap formed between one end of the piping member 71 and one end of the piping member 72, with the end of the piping member 71 being inserted therein. Therefore, the brazing material 63 protruding from the gap forms a fillet. This fillet is formed over the entire circumferential area of the piping members 71 and 72. Such brazing operation by the worker is displayed as a three-dimensional image on the display unit 33 of the display device 30 worn by the instructor.

At this time, if the display control unit 15 determines that the fillet of the brazing material 63 is properly formed based on the two visible images acquired by the image acquisition unit 11, it may display on the display units 23 and 33 that the brazing work is complete.

Figure 6 shows a state where brazing has not been performed correctly. Figure 6A shows a state where the brazing material 63 has not been sufficiently supplied to the gap between the piping members 71 and 72, and the fillet is not connected. Figure 6B shows a state where the brazing material 63 has been excessively supplied to the gap between the piping members 71 and 72, and the fillet is not properly formed in an annular shape.

At this time, if the display control unit 15 determines that a brazing defect has occurred, as shown in Figures 6A and 6B, using the two visible images acquired by the image acquisition unit 11, it may display on the display units 23 and 33 that a brazing defect has occurred.

Figure 7 is a diagram showing the configuration of a pipe 70 having a valve portion 74. As shown in this Figure 7, the pipe 70 may have a valve portion 74. This valve portion 74 has a built-in pressure sensor, solenoid valve, etc., and is a part that may be damaged if kept at high temperatures. In particular, when the pipe members 71, 72, and 73 are made of aluminum or copper, etc., these materials have good thermal conductivity, so there is a risk that the heat generated when the pipe member 71 is brazed will be transmitted to the valve portion 74.

As shown in FIG. 7, the joining operation support device 10 is equipped with a cooling liquid supply pipe 81. This cooling liquid supply pipe 81 supplies cooling liquid to the outside of the valve portion 74. The cooling liquid is, for example, water or oil. When the joining operation is started by the operator, the joining operation support device 10 starts to supply cooling liquid to the outside of the valve portion 74 using the cooling liquid supply pipe 81. This supply of cooling liquid continues until the joining operation is completed. At this time, the joining operation support device 10 determines the start and completion of the joining operation by the operator based on the visible photographic image and the thermal photographic image acquired by the image acquisition unit 11.

When the joining operation supporting device 10 determines that the temperature of the valve portion 74 has exceeded a preset temperature based on the visible image and the thermal image acquired by the image acquisition unit 11, it displays a message that the temperature of the valve portion 74 is rising on the display units 23, 33. Thus, the joining operation supporting device 10 can prevent the valve portion 74 from being damaged by heat generated when the piping members 71 are brazed.

Figure 8 is a diagram showing the temperature change of the junction. The vertical axis of Figure 8 indicates the temperature of the junction. The horizontal axis of Figure 8 indicates the passage of time. The solid line shown in Figure 8 is the ideal temperature curve of the temperature change of the junction. The two-dot chain line shown in Figure 8 is the upper temperature limit curve of the temperature change of the junction. The one-dot chain line shown in Figure 8 is the lower temperature limit curve of the temperature change of the junction.

The joining operation support device 10 calculates in advance an ideal temperature curve, an upper temperature limit curve, and a lower temperature limit curve corresponding to the joint of the piping member 71 based on the diameter and thickness of the piping members 71, 72, and 73. The joining operation support device 10 then detects the temperature change of the joint based on the two visible images and two thermal images acquired by the image acquisition unit 11, and displays on the display units 23 and 33 that the temperature of the joint is rising or falling before the detected temperature exceeds the upper temperature limit curve or the lower temperature limit curve.

Next, the joining operation support method according to the first embodiment will be described with reference to FIG. 9. FIG. 9 is a flowchart showing the joining operation support method according to the first embodiment.

In step ST11, the image acquisition unit 11 acquires each visible image captured by the visible image capture units 21 and 31, and each thermal image captured by the thermal image capture units 22 and 32.

In step ST12, the temperature detection unit 12 detects the temperature of the joint in the piping member 71 based on each visible image and each thermal image acquired by the image acquisition unit 11.

In step ST13, the temperature determination unit 13 determines whether or not the temperature of the joint detected by the temperature detection unit 12 is acceptable.

In step ST14, the three-dimensional data acquisition unit 14 acquires, from the memory unit 41 of the CAD device 40, three-dimensional data indicating the three-dimensional shape and dimensions of the piping member 71, the three-dimensional shape and dimensions of the piping members 72 and 73 to which the piping member 71 is joined, and the three-dimensional shape and dimensions of the jig 61 that fixes the piping members 71, 72, and 72.

In step ST15, the display control unit 15 displays joining work support information on the display units 23 and 33 of the display devices 20 and 30 based on the judgment result judged by the temperature judgment unit 13. The display control unit 15 also displays the joining work visible from the display device 20 in three dimensions on the display unit 33 of the display device 30. Then, the joining work support method ends.

Next, FIG. 10A is a block diagram showing a hardware configuration for implementing the functions of the joining operation support device 10 according to embodiment 1. FIG. 10B is a block diagram showing a hardware configuration for executing software for implementing the functions of the joining operation support device 10 according to embodiment 1.

In Figures 10A and 10B, the input interface 104 is an interface that relays the visible images and thermal images output from the display devices 20 and 30 to the joining work support device 10, and the three-dimensional data output from the CAD device 40 to the joining work support device 10. The output interface 105 is an interface that relays the display control signals output from the joining work support device 10 to the display devices 20 and 30.

When the processing circuit is the dedicated hardware processing circuit 101 shown in FIG. 10A, the processing circuit 101 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. The functions of the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, three-dimensional data acquisition unit 14, and display control unit 15 provided in the joining work support device 10 may be realized by separate processing circuits, or these functions may be realized together by a single processing circuit.

When the processing circuit is the processor 102 shown in FIG. 10B, the functions of the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, three-dimensional data acquisition unit 14, and display control unit 15 provided in the joining operation support device 10 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 103.

The processor 102 reads out and executes the programs stored in the memory 103 to realize the functions of the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15 of the joining operation support device 10. For example, the joining operation support device 10 includes a memory 103 for storing a program that, when executed by the processor 102, results in the processing of steps ST11 to ST15 shown in FIG. 9 being executed. These programs cause the computer to execute the procedures or methods of the processing performed by the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15. The memory 103 may be a computer-readable storage medium in which programs for causing the computer to function as the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15 are stored.

The memory 103 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically-EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.

Some of the functions of the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, three-dimensional data acquisition unit 14, and display control unit 15 provided in the joining work support device 10 may be realized by dedicated hardware, and some may be realized by software or firmware. For example, the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, and three-dimensional data acquisition unit 14 realize their functions by a processing circuit 101, which is dedicated hardware, and the display control unit 15 realizes its functions by the processor 102 reading and executing a program stored in the memory 103. In this way, the processing circuit can realize the above functions by hardware, software, firmware, or a combination of these.

As described above, the joining work support device 10 according to the first embodiment includes an image acquisition unit 11 that is provided on each of the two display devices 20, 30 and acquires each visible photographed image taken by the visible photographing units 21, 21 that photograph the joining work area including the piping member 71 and the AR markers 51, 52, 53, and each thermal photographed image taken by the thermal photographing units 22, 32 that are provided on each of the two display devices 20, 30 and photograph the temperature distribution of the joining work area, a temperature detection unit 12 that detects the temperature of the joining part in the piping member 71 based on each visible photographed image and each thermal photographed image acquired by the image acquisition unit 11, a temperature determination unit 13 that determines whether or not the temperature of the joining part detected by the temperature detection unit 12 is acceptable, and a temperature determination unit 13 that determines whether or not the temperature of the joining part detected by the temperature detection unit 12 is acceptable from a memory unit 41. The apparatus includes a three-dimensional data acquisition unit 14 that acquires three-dimensional data indicating the three-dimensional shapes and dimensions of the pipe member 71, the pipe members 72 and 73 to which the pipe member 71 is joined, and the jig 61 that fixes the pipe members 71, 72, and 73, and a display control unit 15 that displays joining work support information for supporting the joining work on the display units 23 and 33 of the two display devices 20 and 30 based on the judgment result judged by the temperature judgment unit 13, and displays the joining work seen through one display device 20 in three dimensions on the display unit 33 of the other display device 30 based on the position and orientation of the AR markers 51, 52, and 53 included in one visible photographed image acquired by the image acquisition unit 11 and the three-dimensional data acquired by the three-dimensional data acquisition unit 14. Therefore, the joining work support device 10 allows an instructor to instruct a worker during the joining work.

In the joining work support device 10, the temperature determination unit 13 has a brazing material temperature determination unit 13a that determines whether the temperature of the joining part detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63, and the display control unit 15 displays joining work support information on each display unit 23, 33 when the brazing material temperature determination unit 13a determines that the temperature of the joining part detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63. Therefore, the joining work support device 10 can immediately present the joining work support information to the worker when the joining work is being performed appropriately.

In the joining operation support device 10, the temperature determination unit 13 has a base material temperature determination unit 13b that determines whether the temperature of the joining portion detected by the temperature detection unit 12 exceeds the melting temperature of the piping members 72, 73 that serve as the base material, and the display control unit 15 displays an alarm on each display unit 23, 33 when the base material temperature determination unit 13b determines that the temperature of the joining portion detected by the temperature detection unit 12 is a temperature just before the melting temperature of the piping members 72, 72 that serve as the base material. Therefore, the joining operation support device 10 can immediately present an alarm to the worker if the joining operation is not being performed properly.

In addition, within the scope of this disclosure, any of the components of the embodiments may be modified or omitted.

10 Joining work support device, 11 Image acquisition unit, 12 Temperature detection unit, 13 Temperature determination unit, 13a Brazing material temperature determination unit, 13b Base material temperature determination unit, 14 3D data acquisition unit, 15 Display control unit, 20, 30 Display device, 21, 31 Visible light photography unit, 22, 32 Thermal photography unit, 23, 33 Display unit, 40 CAD device 41 Memory unit, 51 to 53 AR marker, 61 Jig, 62 Gas burner, 63 Brazing material, 70 Pipe, 71, 72, 73 Pipe member, 74 Valve unit, 81 Cooling liquid supply pipe, 101 Processing circuit, 102 Processor, 103 Memory, 104 Input interface, 105 Output interface.

Claims (6)

an image acquisition unit that acquires each visible image captured by a visible image capturing unit that is provided on each of the two display devices and captures an image of a joining work area including the joining target material and a marker, and each thermal image captured by a thermal image capturing unit that is provided on each of the two display devices and captures a temperature distribution of the joining work area;
a temperature detection unit that detects the temperature of the joint of the joining target members by superimposing each of the visible light photographed images and each of the thermal photographed images acquired by the image acquisition unit and three-dimensionally connecting the superimposed images;
a temperature determination unit that determines whether or not the temperature of the joint detected by the temperature detection unit is acceptable;
a three-dimensional data acquisition unit that acquires, from a storage unit, three-dimensional data indicating three-dimensional shapes and dimensions of the members to be joined, a base material to which the members to be joined are joined, and a jig for fixing the members to be joined and the base material;
a display control unit that displays joining work support information for supporting the joining work on a display unit possessed by each of the two display devices based on a determination result determined by the temperature determination unit, and that displays in three dimensions the joining work seen through one of the display devices on the display unit of the other display device based on the position and orientation of the marker included in one of the visible photographed images acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit. The temperature determination unit is
a brazing material temperature determination unit that determines whether or not the temperature of the joint detected by the temperature detection unit exceeds a melting temperature of the brazing material,
The display control unit is
The joining work assistance device according to claim 1, characterized in that, when the brazing material temperature determination unit determines that the temperature of the joining portion detected by the temperature detection unit exceeds the melting temperature of the brazing material, the joining work assistance information is displayed on each display unit. The temperature determination unit is
a base material temperature determination unit that determines whether or not the temperature of the joint detected by the temperature detection unit exceeds a melting temperature of the base material;
The display control unit is
The joining work support device according to claim 1 or 2, characterized in that when the base material temperature determination unit determines that the temperature of the joining portion detected by the temperature detection unit is a temperature just before the melting temperature of the base material, an alarm is displayed on each display unit. A joining operation support device according to any one of claims 1 to 3,
One display device held by a worker performing a joining operation;
Another display device is provided for an instructor who is instructing the worker; and
and a memory unit that stores the three-dimensional data. An image acquisition unit is provided on each of the two display devices, and acquires each visible image captured by a visible image capture unit that captures an image of a joining work area including the joining target material and a marker, and each thermal image captured by a thermal image capture unit that is provided on each of the two display devices and captures a temperature distribution in the joining work area,
The temperature detection unit superimposes each of the visible light photographed images and each of the thermal photographed images acquired by the image acquisition unit, and three-dimensionally connects the superimposed images to each other to detect the temperature of the joint in the joining target members;
a temperature determination unit that determines whether or not the temperature of the joint detected by the temperature detection unit is acceptable;
A three-dimensional data acquisition unit acquires, from a storage unit, three-dimensional data indicating three-dimensional shapes and dimensions of the joining target members, a base material to which the joining target members are joined, and a jig for fixing the joining target members and the base material;
a display control unit displays joining work support information for supporting the joining work on a display unit of each of the two display devices based on a determination result determined by the temperature determination unit, and displays the joining work seen through one of the display devices in three dimensions on the display unit of the other display device based on the position and orientation of the marker included in one of the visible photographed images acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit. Computer,
an image acquisition unit that acquires each visible image captured by a visible image capturing unit that is provided on each of the two display devices and captures an image of a joining work area including the joining target material and a marker, and each thermal image captured by a thermal image capturing unit that is provided on each of the two display devices and captures a temperature distribution of the joining work area;
a temperature detection unit that detects the temperature of the joint of the joining target members by superimposing each of the visible light photographed images and each of the thermal photographed images acquired by the image acquisition unit and three-dimensionally connecting the superimposed images;
a temperature determination unit that determines whether or not the temperature of the joint detected by the temperature detection unit is acceptable;
a three-dimensional data acquisition unit that acquires, from a storage unit, three-dimensional data indicating three-dimensional shapes and dimensions of the members to be joined, a base material to which the members to be joined are joined, and a jig that fixes the members to be joined and the base material;
A program for functioning as a display control unit that displays joining work support information for supporting the joining work on the display units of the two display devices based on the judgment result judged by the temperature judgment unit, and that displays the joining work seen through one display device in three dimensions on the display unit of the other display device based on the position and orientation of the marker included in one of the visible photographed images acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit.

Images