JP6652446B2 - Work support device - Google Patents

Description

  The present invention relates to a work support device, and is suitably applied to, for example, a device that supports a work such as a maintenance check at a place where a large number of devices are installed.

  In the maintenance and inspection work of a server in a data center and a meter in a plant, maintenance and inspection work is performed on a large number of devices having the same or similar shape. In such work, the operator checks the model number and serial number attached to each device one by one for each device, performs unique settings for each device, and records the unique status. Or

  In Patent Literature 1, a center information processing server performs matching processing of a photographed image of a plant device received from a portable information terminal with a device mask image by an image recognition processing unit, and acquires inspection area information from the photographed image. Then, the inspection area information is superimposed on the device display image for displaying the maintenance inspection result, and the inspection character information is further superimposed to synthesize a projection image, which is transmitted to the portable information terminal, and is projected and displayed from the projector unit. Is disclosed.

JP 2010-003190 A

  However, the characters for identifying individual devices attached to the devices are arranged in gap spaces so as not to hinder the operation and functions of the devices. It is difficult to recognize an identification character attached to a device that performs such operations. Further, the device may not be specified due to an error or erroneous recognition.

  Further, even when the device to be operated can be specified from the image of the entire device, for example, in a case where the same or similar devices are lined up, when the worker removes his / her gaze from the device to be operated. In this case, there was a risk that the equipment to be operated was unclear, and it was necessary to repeat the work of checking the equipment to be subjected to maintenance and inspection again.

  The present invention has been made in consideration of the above points, and accurately identifies a device to be worked on, even if a worker's eyes are away from the device to be worked during work such as maintenance and inspection. Another object of the present invention is to provide a work support device and a work support method that can accurately recognize a device to be worked.

  In order to solve such a problem, a work support device according to the present disclosure includes a first camera that captures a first image including a character string attached to a device, and a second camera that includes the first camera directed to the character string. A second camera that captures an image, recognizes the character string included in the first image, determines whether the device is a target of the work based on the character string, and determines whether the device is a target of the work. And a display unit that displays a device recognition image based on the second image in accordance with an instruction from the control unit.

  Further, in order to solve such a problem, a work support method of the present disclosure captures a first image including a character string attached to a device, recognizes the character string included in the first image, and converts the character string into a character string. Determine whether the device is the target of the work based on the, if the target of the work, instruct the second camera to shoot, based on the instruction, directed to the character string A second image including the first camera, and a device recognition image based on the second image is displayed.

  According to the present invention, it is possible to accurately specify a device to be worked, and to accurately identify a device to be worked even if a worker's eyes are away from the device to be worked during work such as maintenance and inspection. Can be recognized.

It is a figure showing roughly the work support device concerning one embodiment of the present invention. FIG. 2 is a diagram illustrating functional blocks of a control unit in FIG. 1. It is a figure showing about an example of an appearance composition of a work support device. It is a figure explaining the case where an inspection of equipment is performed using a work support device. FIG. 9 is a sequence diagram illustrating a character recognition device determination process performed using the work support device. FIG. 7 is a diagram illustrating an example of an image displayed when it is determined that the device is a work target device. FIG. 7 is a diagram illustrating an example of a device recognition image displayed on a display unit. FIG. 9 is a sequence diagram illustrating a process of a control unit when performing a work on a device to be worked. 5 is a work content table showing the work procedure and work contents of a work target device stored in a work database. It is a figure showing about an example of a display which shows a work procedure about a certain work object device. FIG. 11 is a diagram illustrating an example in which a device recognition image is displayed on a glasses-type wearable device. FIG. 7 is a diagram illustrating a device recognition image indicating a work target device in a pointing image. It is a figure showing about an example of the 2nd picture. It is a figure which shows the image image | photographed when a worker's arm is lowered in the visual field of a 2nd image. FIG. 13 is a diagram for describing another example of the image processing for creating the device recognition image in FIG. 12. FIG. 13 is a diagram for describing another example of the image processing for creating the device recognition image in FIG. 12. FIG. 9 is a diagram illustrating another example of the external configuration of the work support device. FIG. 9 is a diagram illustrating an image displayed on a display unit using a second image as a device recognition image. FIG. 11 is a diagram illustrating an example of a state in which a device recognition image is displayed on a tablet terminal as a second image and a work target device is checked. FIG. 8 is a sequence diagram of a process of displaying a device recognition image on a display unit by an operation of an operator. FIG. 9 is a sequence diagram of a process of displaying a device recognition image on a display unit by voice recognition. FIG. 8 is a sequence diagram of a process of displaying a device recognition image on a display unit by image recognition. FIG. 8 is a sequence diagram of a process of displaying a device recognition image on a display unit 305 based on an output of a motion sensor. FIG. 23 is a sequence diagram of a process for displaying a device recognition image on a display unit based on an output of a posture sensor such as an acceleration sensor according to another embodiment different from that of FIG. 22. FIG. 24 is a sequence diagram of a device recognition image display start determination process according to the sequence diagram of FIG. 23. FIG. 24 is a sequence diagram of a device recognition image display end determination process according to the sequence diagram of FIG. 23. FIG. 22 is a sequence diagram of a process for displaying a device recognition image on a display unit by image recognition according to a mode different from that of FIG. 21. FIG. 26 is a sequence diagram of a device recognition image display start determination process according to the sequence diagram of FIG. 25. FIG. 26 is a sequence diagram of a device recognition image display end determination process according to the sequence diagram of FIG. 25.

  Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following description, similar elements are denoted by the same reference numerals, and duplicate description will be omitted.

(1) Configuration of Work Support Device According to Present Embodiment FIG. 1 is a diagram schematically showing a work support device 100 according to one embodiment of the present invention. As shown in this figure, the work support device 100 has a first camera 301, a second camera 302, a display unit 305, an input unit 310, and a control unit 200. The control unit 200 receives a central processing unit (CPU) 201, a volatile storage unit 202 such as a random access memory (RAM), a nonvolatile storage unit 203 such as a hard disk or a flash memory, and an image captured by the first camera 301. A first camera interface 204, a second camera interface 205 for receiving an image captured by the second camera 302, a display output interface 206 for outputting an image to be displayed on the display unit 305, and a signal output from the input unit 310. It has an input interface 207 for receiving. These are connected by a bus 209 and the like, and can transmit and receive signals to and from each other. Here, the input unit 310 can include a mouse, a keyboard, a touch panel, a touch sensor, and other input devices, a voice input device, an accelerometer, a device that outputs signals from a sensor such as a GPS, and the like. However, a configuration without the input unit 310 may be employed. In FIG. 1, the first camera 301, the second camera 302, the display unit 305, and the input unit 310 are connected to the control unit 200 by solid lines or broken lines, respectively. Shall be included.

  FIG. 2 is a diagram showing functional blocks of the control unit 200 of FIG. The control unit 200 implements a plurality of functions by operating hardware in cooperation with software. As shown in this figure, the control unit 200 includes an identification character string for identifying the device attached to the device, identification of whether or not the device is to be operated, a work procedure for the device to be operated, By extracting a character string from an image captured by the first camera 301 and comparing the character string stored in the operation database 258 with the character string stored in the work database 258 in which the contents, the work result, and the like are stored, the work is performed. When the character recognition device determination unit 251 and the character recognition device determination unit 251 determine whether the device is the device to be worked, the second camera 302 shoots the image. , An image processing unit 253 that processes an image captured by the second camera 302 to create a device recognition image 540 (see, for example, FIG. 7), and an input. A device recognition image display determination unit 254 that determines whether to display the device recognition image 540 based on an input signal from the input device 310, an image recognized by the second camera 302, a timing in work, and the like. When it is determined that the device recognition image 540 is to be displayed according to 254, the device recognition image display instructing unit 255 instructs the display unit 305 to display the device recognition image 540, and when it is determined that the device is a work target. A work data display instructing unit 256 for acquiring work procedures and work contents from the work database 258 and instructing the display on the display unit 305, and inputting a work result during or at the end of work and saving the work results in the work database 258. And a result input storage unit 257.

  FIG. 3 is a diagram illustrating an example of an external configuration of the work support device 100. In this example, a part of the work support device 100 is configured as a glasses-type wearable device 400 that can be mounted on the head 499 of a person. The glasses-type wearable device 400 has a head attachment 401, and the control unit 200, the display unit 305, and the second camera 302 are attached to the head attachment 401, and can be one of the input units 310. It has an acceleration sensor 313, a voice receiving unit 312, and a touch sensor 311. In addition, the image processing apparatus includes a wireless communication unit 402 that receives image information from the pen-type first camera 301 by wireless communication. The wireless communication unit 402 can receive not only information from the first camera 301 but also information from other input devices, sensor devices, and the like. The display unit 305 can use a projection type display device using a light source and LCOS (Liquid Crystal on Silicon), and recognizes an image displayed on the projection surface 404 in space without impairing the field of view of the wearer. can do.

(2) Character Recognition Device Judgment Process FIG. 4 is a diagram for explaining a case where a device such as a server device is inspected in a data center, for example, using the work support device 100. As shown in this figure, for example, it is assumed that five devices 501 to 505 are arranged in a work place. Further, the worker 480 attaches the glasses-type wearable device 400 of FIG. 3 to the head and holds the pen-shaped first camera 301 in his hand. The worker 480 checks the device to be worked on, and performs work according to the device. FIG. 5 is a sequence diagram illustrating a character recognition device determination process S100 performed using the work support device 100. As shown in the sequence diagram, first, the worker 480 operates the input unit 310 to instruct the control unit 200 to start a program of the character recognition device determination unit 251 for determining a device to be worked. (S101, S102). The control unit 200 activates the program of the character recognition device determination unit 251 (S103). Here, the input unit 310 can use the touch sensor 311, the voice receiving unit 312, and other input units 310. Further, the information may be input via the wireless communication unit 402. As shown in FIG. 4, the worker 480 sets the first camera 301 to a shooting area including a label on which the model number and the serial number of the device 503 are described, in order to confirm whether the device 503 is a work target. In accordance with 531, an instruction for shooting is issued to the first camera 301 (S 104). The first camera 301 captures a first image (S105), and the captured first image is sent to the control unit 200 (S106).

  The character recognition device determination unit 251 of the control unit 200 that has received the first image recognizes the character string from the first image (S107), and determines whether the device is the work target device 503 based on the information stored in the work database 258. It is determined whether it is (S108). The display unit 305 is instructed to display the determination result (S109), and the display unit 305 displays the determination result (S110). In the case of the work target device 503, for example, as shown in FIG. 6, a screen showing a character string indicating that the work target device 503 is recognized and the recognized work target device 503 is displayed. It may be. If the device is not the work target device 503, the character recognition device determination processing S100 ends here. If the device is the work target device 503, the control unit 200 issues a shooting instruction to the second camera 302 (S111), and the second camera 302 shoots a second image (S112). Here, as shown in FIG. 4, the second image can be a shooting area 532 including the first camera 301, and further includes not only the work target device 503 but also other devices 501 to 502 and 504 to 505. It may be that. By photographing with the second camera 302 having such a photographing area 532, it is possible to use the first camera 301 as an image as a pointer pointing to the work target device 503. In order for the operator to recognize the photographing timing, the photographing timing may be counted down by voice or screen display.

  The photographed second image is transmitted to the control unit 200 (S113), and if necessary, is processed into an image in which the work target device 503 can be more easily identified by using image processing. For example, as shown in FIG. The device recognition image 540 is generated (S114). However, the second image may be used as it is as the device recognition image 540 without processing the second image. In this case, the first camera 301 included in the second image can be identified as a pointer indicating the work target device 503. Subsequently, the control unit 200 instructs the display unit 305 to display the device recognition image 540 (S115), and the display unit 305 displays the device recognition image 540 (S116).

  As shown in the example of the device recognition image 540 in FIG. 7, the direction pointed by the first camera 301 is the work target device 503, and the worker 480 reads this image when starting work or resuming work. Is displayed, it is possible to confirm which device the current work target device 503 is. As shown in the display image of FIG. 7, the device recognition image 540 may be displayed together with the character string of the model number or serial number recognized in the first image. By adopting such a configuration, the worker 480 can accurately specify the device to be worked on, and can keep the worker away from the device to be worked during the work such as maintenance and inspection. The device to be worked can be accurately recognized.

(3) Processing during Work FIG. 8 is a sequence diagram illustrating processing of the control unit when performing work on a device to be worked. As shown in this sequence diagram, the processing during the work is mainly a work procedure display processing S210, a work content display processing S220, and a work result input storage processing S230. In the work procedure display process S210, first, the worker 480 gives an instruction for confirming the work procedure to the input unit 310 (S211), and the input unit 310 gives an instruction to display the work procedure to the control unit (S212). ). The control unit 200 acquires the work procedure of the work target device 503 from the work database 258 (S213), and instructs the display unit 305 to display the acquired work procedure (S214). The display unit 305 displays the work procedure of the received content (S215).

  FIG. 9 is a work content table 580 showing the work procedure and work contents of the work target device 503 stored in the work database 258. As shown in this figure, the work content table 580 includes, for example, a column indicating the location of the device, a device number for each location, a model number and a serial number (S / N), a work item number as a work procedure, and a work ( Inspection) items and work contents are included. The work content table 580 is not limited to this, and can be changed as appropriate. Further, a work result entry column and the like can be further included. FIG. 10 is a diagram illustrating an example of a display indicating a work procedure for a certain work target device 503, which is displayed in step S235. As shown in this figure, for the device specified as the work target device 503, a check of a MAC (Media Access Control) address, a connector, and a cable is listed as a work procedure.

  Returning to FIG. 8, in the work content display processing S220, when the worker 480 issues a work start instruction to the input unit 310 (S221), the input unit gives a work start instruction to the control unit 200 (S222). ). The control unit 200 acquires the work content of the work target device 503 (S223), and instructs the display unit 305 to display the work content (S224). The display unit 305 displays the work content (S225). Here, the work content may be one by one of the items displayed in the work procedure, or may be displayed one by one according to the progress of the work. When the work content is displayed, the worker 480 performs a work corresponding to the displayed work content.

  Subsequently, the work result input storage processing S230 will be described. After finishing the work corresponding to the displayed work content, the worker 480 inputs the work result using the input unit 310 (S231), and the input unit 310 sends the input content to the control unit 200. The work is transmitted (S232), and the work result is recorded and stored, for example, in a corresponding column of the work content table 580 (S233). Subsequently, an instruction to display the work result is issued to the display unit 305 to confirm the input content (S234), and the display unit 305 displays the work result (S235). As described above, an example of the processing at the time of work has been described. However, the processing of the work support apparatus 100 is not limited to the work procedure display processing S210, the work content display processing S220, and the work result input storage processing S230. The contents of the work content display processing S220 and the work result input storage processing S230 are not limited to these.

(4) Example of Display of Device Recognition Image FIG. 11 is a diagram illustrating an example in which a device recognition image 540 is partially displayed when the glasses-type wearable device is displayed over the entire field of view. As shown in this figure, by providing an area for displaying the device recognition image 540 in a part of the visual field of the screen for displaying the work content, in addition to the area for directly viewing the work device in the field of view 542, the work target device 503 is provided. The operator 480 can immediately recognize the work target device 503 even when the user returns his / her eyes.

  FIG. 12 is a diagram illustrating a device recognition image 540 in which the hand of the worker 480 and the portion of the first camera 301 are deleted from the second image, and the pointing image 552 indicates the work target device 503. By using such an image, the worker 480 can easily recognize the work target device 503. Hereinafter, image processing for processing the second image into the device recognition image 540 will be described.

  FIG. 13A is a second image captured by the second camera 302 when it is determined that the device is the work target device 503 based on the first image captured by the first camera 301. FIG. 13B is an image when the hand holding the first camera 301 is lowered after capturing the second image, and the first camera 301 and the hand cannot enter the field of view of the second camera 302. The image processing unit 253 performs processing of attaching the pointing image 552 to the position of the second image in FIG. 13A indicated by the first camera 301 in the image in FIG. 13B, thereby obtaining the device recognition image 540 in FIG. Can be performed.

  14 and 15 are diagrams illustrating another example of the image processing for creating the device recognition image 540 of FIG. In this example, as shown in FIG. 14, an AR (Augmented Reality) code is attached to the first camera 301 at a position where it can be easily read by the second camera 302, and character recognition device determination processing S100 is performed. FIG. 15 is a diagram when the second image captured at this time is used as it is as the device recognition image 540. The image processing unit 253 recognizes the AR code in the processing of the image in FIG. 15 and replaces the AR code with the pointing image 552 to obtain an image as illustrated in FIG.

(5) Another Example of the External Configuration of the Work Supporting Apparatus FIG. 16 is a diagram illustrating another example of the external configuration of the work supporting apparatus 100. In this example, a part of the work support device 100 is configured by a tablet terminal 600. The tablet terminal 600 includes a control unit 200, a display unit 305, and a second camera 302, and further includes an acceleration sensor 313, which can be one of the input units 310, a voice receiving unit 312, and a touch panel 611. In addition, a wireless communication unit 402 that receives image information from the second camera 302 attached to the head of the worker 480 by wireless communication is provided. The wireless communication unit 402 can receive not only information from the second camera 302 but also information from other input devices, sensor devices, and the like. FIG. 17 is an image in which the second image is displayed as the device recognition image 540 on the display unit 305. FIG. 18 is a diagram in which the second image is displayed as the device recognition image 540 on the display unit 305 of the tablet terminal 600. FIG. 9 is a diagram when a device 503 is checked. Even with such a configuration, the worker 480 can accurately specify the device to be worked on, and even if the user gets away from the device to be worked during work such as maintenance and inspection, the worker 480 can be used. The device to be operated can be accurately recognized.

(6) Judgment for Displaying Device Recognition Image FIGS. 19 to 26B show the device recognition image 540 in order to start the operation again when the operator releases his / her gaze from the target device 503 during the operation. FIG. 9 is a diagram for describing processing to be displayed on a display 305.

  FIG. 19 is a sequence diagram of the process S310 of displaying the device recognition image 540 on the display unit 305 by the operation of the worker 480. As shown in this figure, the worker 480 operates the input unit 310 such as a button, a string-shaped input device or a touch sensor of the glasses-type wearable device 400, or a touch panel of the tablet terminal 600 to display the device recognition image 540. Is instructed (S311). The input unit 310 instructs the control unit 200 to display the device recognition image 540 (S312), and the device recognition image display instruction unit 255 of the control unit 200 instructs the display unit 305 to display the device recognition image 540 (S312). S313). The display unit 305 displays the device recognition image 540 (S314). Thus, even if the user moves his / her eyes away from the device to be worked during the work such as maintenance and inspection, the worker 480 can accurately recognize the device to be worked.

  FIG. 20 is a sequence diagram of the process S320 of displaying the device recognition image 540 on the display unit 305 by voice recognition. As shown in this figure, the worker 480 performs predetermined device recognition by voice via a voice receiving unit 312 such as the voice receiving unit 312 of the glasses-type wearable device 400 and the voice receiving unit 312 of the tablet terminal 600. A voice instruction for displaying the image 540 is issued (S321). The voice receiving unit 312 having received the voice transmits a voice signal to the control unit 200 (S322), and the device recognition image display determination unit 254 of the control unit 200 determines that the display of the device recognition image 540 is instructed by voice recognition. When it is determined (S323), the device recognition image display instructing unit 255 of the control unit 200 instructs the display unit 305 to display the device recognition image 540 (S324). The display unit 305 displays the device recognition image 540 (S325). Therefore, by receiving the instruction by voice, the worker 480 can accurately recognize the device to be worked even when the user moves away from the device to be worked during the work such as maintenance and inspection.

  FIG. 21 is a sequence diagram of the process S330 of displaying the device recognition image 540 on the display unit 305 by image recognition. As shown in this figure, the worker 480 gives an instruction to display a predetermined device recognition image 540 by hand movement such as a gesture so as to be captured by the second camera 302 (S331). The second camera that has captured the hand movement transmits the moving image to the control unit 200 (S332), and the device recognition image display determination unit 254 of the control unit 200 analyzes the moving image and instructs the display of the device recognition image 540 by an instruction. If it is determined that there is (S333), the device recognition image display instructing unit 255 of the control unit 200 instructs the display unit 305 to display the device recognition image 540 (S334). The display unit 305 displays the device recognition image 540 (S335). Here, the analysis of the moving image can be performed by analyzing a difference from the previous frame, or by using a moving image analysis process such as feature point matching or optical flow. Thus, even if the user moves his / her eyes away from the device to be worked during the work such as maintenance and inspection, the worker 480 can accurately recognize the device to be worked.

  FIG. 22 is a sequence diagram of the process S340 of displaying the device recognition image 540 on the display unit 305 based on the output of the motion sensor such as the acceleration sensor 313. As shown in this figure, the worker 480, for example, in the case of the glasses-type wearable device 400, because the acceleration sensor 313 is on the head, moves the head in a predetermined motion, In the case of 600, since the acceleration sensor 313 is inside the tablet terminal 600, an instruction to display a predetermined device recognition image 540 is given by setting the tablet terminal 600 in a predetermined posture, for example, standing the tablet terminal 600. (S341). The acceleration sensor 313 that has detected the acceleration transmits the detected signal to the control unit 200 (S342), and the device recognition image display determination unit 254 of the control unit 200 analyzes the signal and instructs display of the device recognition image 540. Is determined (S343), the device recognition image display instructing unit 255 of the control unit 200 instructs the display unit 305 to display the device recognition image 540 (S344). The display unit 305 displays the device recognition image 540 (S345). In addition, other than the acceleration sensor 313, a position sensor such as a GPS, a motion sensor such as an angular velocity sensor, or the like may be used, or a combination of these may be used. Therefore, even if the user moves his / her eyes away from the work target device during the work such as maintenance and inspection by receiving the signal from the acceleration sensor, the worker 480 can accurately recognize the work target device.

  FIG. 23 is a sequence diagram of a process S400 according to a mode different from that of FIG. 22, in which the device recognition image 540 is displayed on the display unit 305 based on an output of a posture sensor such as an acceleration sensor. In this embodiment, an acceleration sensor 313 on the head, such as a glasses-type wearable device 400, is used, and the display is switched depending on whether or not the camera is facing the same direction when the second image is captured. As shown in this sequence diagram, first, a signal detected by a posture sensor such as the acceleration sensor 313 is notified to the control unit 200 (S401), and the device recognition image display determination unit 254 of the control unit 200 displays the device recognition image display. Start determination processing S410 is performed.

  FIG. 24A is a sequence diagram of the device recognition image display start determination processing S410. As shown in the figure, in the device recognition image display start determination processing S410, first, it is determined whether or not the posture is in a stationary state based on a signal notified by a posture sensor such as the acceleration sensor 313 (S411). ). Here, the stationary state includes not only a case where the signal of the posture sensor is completely constant, but also a case where the change of the signal of the posture sensor is a change within a predetermined range. If it is determined in step S411 that the apparatus is in the stationary state, the apparatus recognition image display start determination processing S410 ends. If it is not determined that the camera is in the stationary state, it is determined whether or not the orientation of the posture sensor is the same as when the second image was captured (S412). In this case, information from a direction sensor or the like can also be used. Here, “same as when capturing the second image” includes not only the completely same direction but also a predetermined range including the same direction, such as a range of an angle difference within 20 degrees from the same direction. . If it is determined that the direction is the same as that when the second image was captured, the device recognition image display start determination processing S410 ends. If it is not determined that the direction is the same as the direction at the time of capturing the second image, it is determined that the device recognition image display is to be started (S413), and the device recognition image display start determination process S410 ends.

  Returning to FIG. 23, in the device recognition image display start determination processing S410, when the device recognition image display start is determined, the device recognition image display instruction unit 255 of the control unit 200 causes the display unit 305 to display the device recognition image 540. Is displayed (S403). The display unit 305 displays the device recognition image 540 (S404).

  Next, when a signal detected by the attitude sensor such as the acceleration sensor 313 is further notified to the control unit 200 (S405), the device recognition image display determination unit 254 of the control unit 200 determines the device recognition image display end determination process S420. I do.

  FIG. 24B is a sequence diagram of the device recognition image display end determination processing S420. As shown in this figure, in the device recognition image display end determination processing S420, first, based on the signal notified by the attitude sensor such as the acceleration sensor 313, the orientation of the attitude sensor is the same as when the second image was captured. It is determined whether or not there is (S421). In this case, information from an orientation sensor or the like can be used as in FIG. 24A. If it is not determined that the direction is the same as when the second image was captured, the device recognition image display end determination processing S420 ends. If it is determined that the direction is the same as that at the time of capturing the second image, it is determined whether or not the posture is at rest (S422). If it is not determined that the apparatus is in the stationary state, the apparatus recognition image display end determination processing S420 ends. If it is determined that the apparatus is in the stationary state, it is determined that the device recognition image display has ended (S423), and the device recognition image display end determination processing S420 ends.

  Returning to FIG. 23, in the device recognition image display end determination processing S420, when the device recognition image display termination is determined, the device recognition image display instructing unit 255 of the control unit 200 sends the device recognition image 540 to the display unit 305. Is displayed (S407). The display unit 305 ends the display of the device recognition image 540 (S408). Note that, in addition to the acceleration sensor 313, a position sensor such as a GPS, an attitude sensor such as an angular velocity sensor, and a direction sensor may be used. Therefore, by receiving the signal from the acceleration sensor, the worker 480 can accurately recognize the device to be worked even if the user moves away from the device to be worked during the work such as maintenance and inspection. When the signal from the attitude sensor is within a predetermined range from the second image capturing signal, the display of the device recognition image 540 is started. When the signal is out of the predetermined range of the second image capturing signal, the device is displayed. Since the display of the recognition image 540 ends, the device recognition image 540 can be displayed only when the worker 480 needs to recognize the device.

  FIG. 25 is a sequence diagram of a process S500 for displaying the device recognition image 540 on the display unit 305 by image recognition according to another mode different from that of FIG. In this embodiment, an image captured by the second camera 302, which is a line-of-sight camera attached to the head, is used to indicate whether the captured image is the device recognition image 540 or an image similar to the second image. It is decided to switch. As shown in the sequence diagram, first, the image data captured by the second camera 302 is transmitted to the control unit 200 (S501), and the device recognition image display determination unit 254 of the control unit 200 determines the device recognition image display start determination. Step S510 is performed.

  FIG. 26A is a sequence diagram of the device recognition image display start determination processing S510. As shown in this figure, in the device recognition image display start determination processing S510, first, the image data received by the control unit 200 is acquired (S511), and an optical flow is detected based on the acquired image data (S512). ). It is determined whether the image is in a stationary state (S513). If the image is determined to be stationary, the process returns to step S511 to acquire the next image data. If it is not determined that the image is in the stationary state, the acquired image is compared with the second image or the device recognition image 540 (S514), and it is determined whether or not the similarity is equal to or larger than a threshold (S515). If the similarity is equal to or larger than the threshold, the process returns to step S511 to acquire the next image data. If the similarity is less than the threshold value, it is determined that the device recognition image display has started (S516), and the device recognition image display start determination process S510 ends. Here, the stationary state is determined to be the stationary state not only when the images obtained continuously are completely identical, but also when, for example, half or two-thirds or more of the images obtained continuously have almost the same field of view. Including cases.

  Returning to FIG. 25, in the device recognition image display start determination processing S510, when the device recognition image display start is determined, the device recognition image display instruction unit 255 of the control unit 200 sends the device recognition image 540 to the display unit 305. Is displayed (S503). The display unit 305 displays the device recognition image 540 (S504).

  Next, the image data captured by the second camera 302 is transmitted to the control unit 200 (S505), and the device recognition image display determination unit 254 of the control unit 200 performs a device recognition image display end determination process S520.

  FIG. 26B is a sequence diagram of the device recognition image display end determination processing S520. As shown in this figure, in the device recognition image display end determination processing S520, first, the image data received by the control unit 200 is acquired (S521), and an optical flow is detected based on the acquired image data (S522). ). It is determined whether the image is in a stationary state (S523). Here, if the image is not determined to be in the stationary state, the process returns to step S521 to acquire the next image data. If it is determined that the image is in the stationary state, the acquired image is compared with the second image or the device recognition image 540 (S524), and it is determined whether or not the similarity is equal to or larger than a threshold (S525). If the similarity is less than the threshold, the process returns to step S521 to acquire the next image data. If the similarity is equal to or larger than the threshold value, it is determined that the device recognition image display has ended (S526), and the device recognition image display end determination processing S520 ends. Here, the stationary state is determined to be the stationary state not only when the images obtained continuously are completely identical, but also when, for example, half or two-thirds or more of the images obtained continuously have almost the same field of view. Including cases.

  Returning to FIG. 25, in the device recognition image display end determination processing S520, when the device recognition image display termination is determined, the device recognition image display instructing unit 255 of the control unit 200 sends the device recognition image 540 to the display unit 305. Is displayed (S507). The display unit 305 ends the display of the device recognition image 540 (S508). Therefore, even if the user moves his / her eyes away from the work target device during the work such as maintenance and inspection due to the reception of the signal based on the processing result of the image captured by the second camera 302, the worker 480 is notified of the work target device. Accurate recognition is possible. Further, when the image data as the signal from the second camera 302 is within a predetermined range (similarity is equal to or more than a threshold value) with the image data of the second image or the device recognition image 540 as the second image photographing signal, When the display of the device recognition image 540 is started, and the display of the device recognition image 540 ends when the second image or the predetermined range of the device recognition image 540 is deviated, the worker 480 needs to recognize the device. Only in this case, the device recognition image 540 can be displayed.

(7) Effects of the present embodiment In the present embodiment, the work support device 100 is directed to the first camera 301 that captures the first image including the character string attached to the device 503 and faces the character string. A second camera 302 that captures a second image including the first camera 301 and a character string included in the first image, and determines whether or not the device 503 is a work target based on the character string; In order to include a control unit 200 that instructs the second camera 302 to shoot when the object is a work target, and a display unit 305 that displays a device recognition image 540 based on the second image according to the instruction of the control unit 200. In addition to accurately specifying a device to be worked on, even if the worker's eyes are away from the device to be worked during work such as maintenance and inspection, the device to be worked can be accurately recognized. .

  The present invention relates to a work support device, and can be applied to, for example, a device that supports work such as maintenance and inspection at a place where many devices are installed.

Reference Signs List 100 work support device, 200 control unit, 202 volatile storage unit, 203 nonvolatile storage unit, 204 first camera interface, 205 second camera interface, 206 display output interface, 207 input interface, 209 bus, 251 character recognition device determination Unit, 252 shooting instruction unit, 253 image processing unit, 254 device recognition image display determination unit, 255 device recognition image display instruction unit, 256 work data display instruction unit, 257 work result input storage unit, 258 work database, 301 first camera , 302 second camera, 305 display unit, 310 input unit, 311 touch sensor, 312 voice receiving unit, 313 acceleration sensor, 400 glasses-type wearable device, 401 head mount, 402 wireless communication unit, 404 projection surface, 480 work Person, 499 head, 503 work target device, 540 device recognition image, 552 pointing image, 580 work content table, 600 tablet terminal, 611 touch panel

Claims (12)

A first camera that captures a first image including a character string attached to the device,
A second camera that captures a second image including the first camera directed at the character string;
Recognize the character string included in the first image, determine whether the device is the target of the work based on the character string, if the device is the target of the work, the second camera A control unit for instructing shooting,
A display unit for displaying a device recognition image based on the second image according to an instruction from the control unit. The work support device according to claim 1,
The device recognition image is the processed second image,
A work support device, characterized in that: The work support device according to claim 2,
The work support device, wherein the device recognition image is an image obtained by replacing the first camera in the second image with a pointing image. The work support device according to any one of claims 1 to 3,
The control unit issues an instruction to display the device recognition image based on the reception of the input signal,
A work support device, characterized in that: The work support device according to claim 4,
The input signal is based on reception of a voice instruction. The work support device according to claim 4,
The input signal is based on reception of a signal from an acceleration sensor. The work support device according to claim 4,
The work support apparatus, wherein the input signal is based on reception of a signal based on a result of processing of an image captured by the second camera. The work support device according to claim 4,
Displaying the device recognition image when the input signal is within a predetermined range from the second image photographing signal as compared with the second image photographing signal which is a signal at the time of the second image photographing; Start,
The input signal is compared with a second image capturing signal that is a signal at the time of the second image capturing, and when the signal is out of the predetermined range, the display of the device recognition image is terminated.
A work support device, characterized in that: The work support device according to claim 4,
The work support device, wherein the display unit and the second camera are fixed to a head attachment. The work support device according to claim 4,
The operation support device, wherein the display unit has a touch panel input unit and is integrated with the first camera. The work support device according to claim 4,
An input unit for inputting a result for the work item to the control unit,
The work support device, wherein the control unit displays the work item on the display unit when it is determined that the work item is the target of the work. Taking a first image including a character string attached to the device,
Recognizing the character string included in the first image, determining whether the device is the target of the work based on the character string, and instructing the photographing of the second image when the device is the target of the work And
Based on the instruction, photographing a second image including the first camera that photographed the first image directed to the character string,
A work support method for displaying a device recognition image based on the second image.

Images