JP7340792B2 - Work display method and work display device - Google Patents

Description

The present invention relates to a work display method and a work display device.

2. Description of the Related Art In the past, studies have been conducted to detect assembly errors made by workers performing assembly work in factories and the like. Patent Document 1 discloses that by detecting the position of the worker's hand based on an image taken with a camera, it is possible to determine whether the worker has picked up a component from the correct component placement section among multiple component placement sections. A production system (work display system) for determining whether

JP 2018-45669 Publication

Incidentally, in such a production system, it is desired that the processing load in the determination process be reduced and that erroneous detections be suppressed.

Therefore, the present invention provides a work display method and a work display device that can reduce the processing load in the determination process and suppress false detections.

A work display method according to an aspect of the present invention includes a first work in which a first part is installed in a first area, and the first part is not installed in a second area different from the first area. and a second work in which a second part is mounted in the second region and the second part is not mounted in the first region. A work display method for displaying, the method comprising: a first display step for displaying a first work content in the first work; a second display step for displaying a second work content in the second work; In the first display step, the detection of the first part in the first area is enabled, the detection of the second part in the second area is disabled, and the sensing result obtained from the sensor unit is displayed. If the first part is detected in the first region based on In the step, the detection of the second part in the second region is enabled, the detection of the first part in the first region is disabled, and the second part is detected based on the sensing result obtained from the sensor unit. When the second part is detected in the area, information indicating that the second work has been performed normally is output to the display device as the second work content.

Further, in the work display device according to one aspect of the present invention, a first part is installed in a first area, and the first part is not installed in a second area different from the first area. and a second work in which a second part is installed in the second area and the second part is not installed in the first area. A work display device for displaying contents, wherein when the first work is performed, the detection of the first part in the first area is enabled, and the detection of the second part in the second area is enabled. Control that disables detection and, when the second work is performed, enables detection of the second part in the second area and disables detection of the first part in the first area and when the first work is performed and the first part is detected in the first region based on the sensing result obtained from the sensor part, the first work is successfully performed. When the second work is performed, the second part is displayed in the second area based on the sensing result obtained from the sensor section. and an output unit that, when detected, outputs information indicating that the second work has been performed normally to the display device as the second work content.

Note that these comprehensive or specific aspects may be realized in a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, and the system, integrated circuit, computer program, and recording It may be implemented with any combination of media.

According to the work display method and the like according to one aspect of the present invention, it is possible to reduce the processing load in the determination process and suppress false detections.

FIG. 1 is a block diagram showing the functional configuration of a work instruction system according to an embodiment. FIG. 2A is a diagram showing an example of a work table. FIG. 2B is a diagram showing an example of a detection table. FIG. 3 is a schematic diagram showing a detection area according to the embodiment. FIG. 4A is a first diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4B is a second diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4C is a third diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4D is a fourth diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 5 is a sequence diagram showing the operation of the work instruction system according to the embodiment. FIG. 6 is a flowchart showing the operation of the work instruction device according to the embodiment. FIG. 7A is a diagram showing how detection of the first component in the first region is enabled. FIG. 7B is a diagram showing how detection of the second component in the second region is enabled. FIG. 8 is a flowchart showing the operation of the work instruction device according to the first modification of the embodiment. FIG. 9 is a diagram showing how the prohibited area corresponding to work 1 is enabled. FIG. 10 is a flowchart showing the operation of the work instruction device according to the second modification of the embodiment.

Embodiments will be described below with reference to the drawings. The embodiments described below are all inclusive or specific examples. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of constituent elements, processes (steps), order of processes (steps), etc. shown in the following embodiments are merely examples, and are intended to limit the present invention. isn't it. Further, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims will be described as arbitrary constituent elements.

Furthermore, each figure is a schematic diagram and is not necessarily strictly illustrated. Furthermore, in each figure, substantially the same configurations are denoted by the same reference numerals, and overlapping explanations may be omitted or simplified.

(Embodiment)
[1. Work instruction system configuration]
First, the configuration of a work instruction system according to an embodiment will be explained using FIGS. 1 to 3. FIG. 1 is a block diagram showing the functional configuration of a work instruction system 1 according to an embodiment.

As shown in FIG. 1, the work instruction system 1 includes a work instruction device 10, a first driver 20, a second driver 30, an imaging device 40, and a display device 70. Although an example in which the work instruction system 1 includes two tools (the first driver 20 and the second driver 30) will be described below, the work instruction system 1 is not limited thereto. Note that hereinafter, when the first driver 20 and the second driver 30 are not distinguished, they will also be simply referred to as drivers. Note that the work instruction system 1 is an example of a work display system.

The work instruction device 10 is a server device that enables prevention of work mistakes, support for learning, observance of work time, work improvement, notification of work mistakes, and the like. The basic operation of the work instruction device 10 will be described later. The work instruction device 10 according to the present embodiment provides work instructions, detects work errors, etc. to a worker who performs a one-person assembly work in which one person is in charge of multiple tasks, for example, in an assembly work or the like. Note that the work performed by the worker includes, for example, work that uses a tool (eg, a screwdriver) and work that does not use a tool. A work without tools is a work that does not use any of the tools used in any of the plurality of works. Operations without tools include manual operations, such as inserting washers, tightening clips, and inspection operations. Note that the work instruction device 10 is an example of a work display device for displaying on the display device 70 the contents of the work performed by the worker.

The work instruction device 10 includes a communication section 11, a control section 12, and a storage section 13.

The communication unit 11 is a communication interface that is communicably connected to various devices included in the work instruction system 1 via wireless communication or wired communication. The communication unit 11 outputs, for example, control information for controlling the imaging unit 60 generated by the control unit 12 to the imaging device 40. The communication unit 11 also acquires information based on image data captured by the imaging unit 60 from the imaging device 40 . In this way, the communication unit 11 functions as an output unit that outputs control information and an acquisition unit that acquires information based on image data. Note that the image data is an example of sensing results (sensing data).

Further, the communication unit 11 may acquire a start signal indicating the start and an end signal indicating the end of each of the plurality of tasks. For example, when the work is a work using a driver, the communication unit 11 may acquire at least one of a start signal and an end signal from the first driver 20 or the second driver 30. Further, the communication unit 11 may acquire at least one of the start signal and the end signal, for example, through an operator's operation (for example, an operation on a button or a foot switch (not shown)).

The control unit 12 generates and outputs information related to processing executed by the device to various devices connected to the work instruction device 10, and information acquired by the device from the various devices (for example, acquired by measurement). This is a processing unit that acquires information and performs predetermined processing based on the information.

Specifically, the control unit 12 sets the driver control information for controlling the first driver 20 and the second driver 30 based on the work table T1 (see FIG. 2A) stored in the storage unit 13, for example. The signal is output to the first driver 20 and the second driver 30. The control unit 12 also performs control for controlling each camera (for example, cameras a to camera d) included in the imaging device 40 based on the detection table T2 (see FIG. 2B) stored in the storage unit 13. The information is output to the imaging device 40.

Further, the control unit 12 determines whether the work has been performed normally based on information based on the image data of each camera acquired from the imaging device 40. Then, the control unit 12 outputs information for displaying the determination result to the display device 70 via the communication unit 11.

The storage unit 13 is a storage device that stores a control program executed by the control unit 12 included in the work instruction device 10. The storage unit 13 also stores a work table T1 and a detection table T2. Furthermore, the storage unit 13 may store information acquired via the communication unit 11 (for example, information based on image data, etc.). The storage unit 13 is realized by, for example, a semiconductor memory.

Here, the working table T1 and the detection table T2 stored in the storage unit 13 will be explained with reference to FIGS. 2A and 2B. FIG. 2A is a diagram showing an example of the work table T1. Note that FIG. 2A shows a case where a screw is tightened as an example of a screw. Note that hereinafter, a screw will also be simply referred to as a screw.

As shown in FIG. 2A, the storage unit 13 stores a work table T1 in which work procedures, work names, driver names, numbers of screws, and upper and lower limit pulses are associated with each other. The work in work procedures 1, 2, and 4 is an example of work that uses a driver, and the work in work procedure 3 is an example of work that does not use a driver. In other words, the work table T1 is a table for performing a plurality of tasks including tasks that use a driver and tasks that do not use a driver. Note that the number and order of tasks using a driver and tasks not using a driver included in the work table T1 are not particularly limited.

The work procedure indicates the order of multiple tasks performed by a worker. The work procedure is predetermined. The example in FIG. 2A shows that work steps 1 to 4 are performed in that order.

The work name indicates the content of the work, and is set for each of a plurality of works. The worker can know the content of the work just by looking at the work name.

The driver name indicates a name for the operator to identify the driver, and is set for each driver. The driver name may be the driver's model number, etc., as long as the operator can identify the driver.

The number of screws indicates the number of screws to be tightened in the relevant work.

The upper limit pulse and the lower limit pulse are threshold values (number of pulses) used by the control unit 12 to determine whether the work using the screwdriver is completed based on the screw tightening information. The upper limit pulse and the lower limit pulse are determined, for example, based on the number of rotations (or number of pulses) when the screw tightening work (screw tightening work) is properly completed by each of the plurality of workers. The upper limit pulse and the lower limit pulse are, for example, based on the rotation speed of each of the plurality of workers, the rotation speed above a predetermined number (for example, the maximum rotation speed) and the rotation speed below a predetermined value (for example, the minimum rotation speed). may be determined. This allows the upper limit pulse and the lower limit pulse to be appropriately set even if the number of rotations when the screw tightening operation is appropriately completed by a plurality of workers is different.

Note that the work table T1 only needs to include at least one of the upper limit pulse and the lower limit pulse. Furthermore, when the screw tightening information includes torque information, the work table T1 may include at least one of an upper limit and a lower limit of the torque. Further, the work table T1 does not need to include information on the upper limit pulse and the lower limit pulse when determining whether the work has been completed in the driver.

Work in work procedure 1 (hereinafter also referred to as work 1) indicates that a screw A is tightened using a screwdriver X. Driver X is, for example, the first driver 20. In addition, the upper limit pulse and the lower limit pulse are standards for determining whether screw tightening of screw A is completed when screw A is tightened using driver X, and corresponds to the number of rotations of screw A. This is the numerical value.

The work in work procedure 2 (hereinafter also referred to as work 2) is a work in which one screw B is tightened using a driver Y. In work 2, a different driver is used than in work 1. Driver Y is, for example, the second driver 30.

The work in work procedure 3 (hereinafter also referred to as work 3) shows an example in which one plate C is attached.

Work in work procedure 4 (hereinafter also referred to as work 4) shows an example in which one screw D, which is different from screw A, is tightened using the same driver as in work procedure 1.

As described above, the work instruction device 10 provides information for a worker to perform a plurality of tasks, including tasks using a driver (for example, screw tightening) and tasks not using a driver, according to predetermined work procedures. (For example, a work table T1) is held.

Next, the detection table T2 stored in the storage unit 13 will be explained. FIG. 2B is a diagram showing an example of the detection table T2.

As shown in FIG. 2B, the storage unit 13 stores a detection table T2 in which work procedures, work names, and information regarding the imaging device 40 are associated with each other. Information regarding the imaging device 40 includes a detection camera, a prohibited camera, a stop camera, and a detection color. Note that the information regarding the imaging device 40 only needs to include at least information regarding the detection camera.

The detection camera indicates a camera among a plurality of cameras that is operated to determine whether or not the current work has been performed normally. For example, in work 1, an example is shown in which camera a among cameras a to camera d is operated as a detection camera.

The prohibited camera indicates a camera among a plurality of cameras that is operated in order to determine whether or not a part in the current work has been mounted in a prohibited area where parts are not allowed to be mounted in the current work. For example, in work 1, an example is shown in which cameras b and d among cameras a to camera d are operated as prohibited cameras. Further, for example, in tasks 3 and 4, an example is shown in which none of the cameras is operated as a prohibited camera.

The stopped camera indicates a camera among a plurality of cameras that stops operating during the current operation. In other words, cameras listed as stopped cameras do not take pictures during the operation. For example, in work 1, an example is shown in which the operation of camera c among cameras a to camera d is stopped. Note that at least one camera is set as the stopped camera.

The detection color indicates the color of the part for determining whether the work has been performed normally. For example, in work 1, it is shown that the color of the component needs to be "blue" in order for the work 1 to be performed normally.

Referring again to FIG. 1, the first driver 20 and the second driver 30 are examples of tools that can be operated by power used by an operator in work using the tools. Note that the tool here means, for example, a power tool. In addition, the motive power is electricity, air, or the like. The first driver 20 is, for example, a fastening tool used for tightening screws A and D, and is an example of a first tool. The second driver 30 is a fastening tool different from the first driver 20. The second driver 30 is, for example, a fastening tool used for tightening the screw B, and is an example of a second tool. Note that the number and types of drivers connected to the work instruction device 10 are not particularly limited.

Although not shown, each of the first driver 20 and the second driver 30 includes a communication section for communicating with the work instruction device 10, a pulse measurement section for measuring the number of rotations of the driver, and a torque-up, rotation start, and and a detection section that detects the end of rotation. The pulse measuring section has a circuit that is built into a motor (not shown) of the driver and can measure a pulse signal generated by an encoder (not shown), for example, and detects the rotation speed from the measurement result of the pulse signal. When the motor rotates, the encoder emits a pulse signal as a corresponding rotation signal. The detection unit detects torque increase, start of rotation, and end of rotation, for example, by measuring current consumption of the driver. The communication unit processes at least one of the measurement results measured by the pulse measurement unit and the detection results detected by the detection unit (for example, rotation start and rotation end detection results, which are examples of start signals and end signals). Output to the instruction device 10.

Note that each of the first driver 20 and the second driver 30 may have a processing unit that determines whether screw tightening is completed based on the number of rotations measured by the pulse measurement unit.

The imaging device 40 photographs the detection area based on the control information from the work instruction device 10, and outputs image data representing the photographed image to the work instruction device 10. The imaging device 40 includes a camera control section 50 and an imaging section 60. The detection area is an area in the main body 100 (see FIG. 3) that includes a position where a component is attached. The main body portion 100 may be a board on which components are mounted.

The camera control unit 50 is a processing unit that controls the operation of the imaging unit 60 based on control information from the work instruction device 10. The camera control unit 50 also acquires image data from the imaging unit 60 and outputs information based on the acquired image data to the work instruction device 10.

The information based on the image data may include the image data itself, or may include the results of analyzing the image data (for example, the results of image analysis).

The camera control unit 50 is realized by, for example, a microcomputer. The camera control unit 50 reads a control program from a built-in storage unit and executes the read program.

The imaging unit 60 includes one or more cameras that take images to detect whether a component is attached to a predetermined position. In this embodiment, the imaging unit 60 is composed of cameras a to d, but the number of cameras is not particularly limited. Each of the cameras a to camera d photographs a detection area set for each camera, and outputs image data obtained by photographing to the camera control unit 50. Note that each of cameras a to camera d may have the same performance (eg, resolution), or may have different performance from each other. Cameras a to d may be determined as appropriate, for example, depending on the work content, detection accuracy, and the like. The imaging section 60 is an example of a sensor section.

The work instruction system 1 may include sensor devices other than the imaging device 40 as long as they can detect parts. The work instruction system 1 may include a sensor device other than the imaging device 40, for example, instead of the imaging device 40 or together with the imaging device 40. The sensor included in the sensor device is not particularly limited, but may include, for example, an optical sensor that detects the presence or absence of a component by blocking light.

Note that the imaging device 40 does not need to include the camera control section 50. In this case, the work instruction device 10 may have the functions of the camera control section 50, for example. The work instruction device 10 may output control information to each camera of the imaging section 60 via the communication section 11, and may acquire image data from each camera of the imaging section 60.

As shown in FIG. 2B, in this embodiment, camera a is a camera for detecting whether or not there is a screw A corresponding to work 1 in the detection area for work 1. Camera b is a camera for detecting whether or not there is a screw B corresponding to work 2 in the detection area for work 2. The camera c is a camera for detecting whether or not the plate C corresponding to the work 3 is present in the detection area for the work 3. The camera d is a camera for detecting whether or not there is a screw D corresponding to the work 4 in the detection area for the work 4.

Work 1 is a work (an example of the first work) in which a first part (for example, screw A) is installed in a first area, and the first part is not installed in a second area different from the first area. Work 2 is a work (an example of a second work) in which a second part (for example, screw B) is installed in the second area, and the second part is not installed in the first area. Furthermore, in the first work, the first part is prohibited from being installed in the second and fourth areas, and in the second work, the second part is prohibited from being installed in the fourth area. ing. Therefore, in the first work, cameras b and d are set as prohibited cameras, and in the second work, camera d is set as prohibited cameras.

Here, the detection area of the imaging section 60 will be explained with reference to FIG. 3. FIG. 3 is a schematic diagram showing detection regions Ra to Rd according to the embodiment. FIG. 3 shows the main body portion 100 to which screws A, B, plate C, and screw D are attached (attached), and detection areas corresponding to each of the components. Note that the plate C is arranged to cover the screw A. Further, the area surrounded by the dashed line shown in FIG. 3 indicates the detection area.

As shown in FIG. 3, the detection area Ra is an area of the main body 100 where the screw A is attached. The detection area Ra is photographed by camera a. The detection region Rb is a region of the main body portion 100 where the screw B is attached. The detection region Rb is photographed by camera b. The detection area Rc is an area of the main body 100 to which the plate C is attached. The detection region Rc is photographed by a camera c. The detection region Rd is a region of the main body portion 100 where the screw D is attached. The detection area Rd is photographed by a camera d. At least a portion of the detection areas Ra to Rd may overlap in plan view. In this embodiment, the detection area Rc is an area including the detection area Ra in plan view. Note that each of the detection regions Ra to Rd may be, for example, regions that do not overlap with each other.

Further, detection regions Rb and Rd are prohibited regions corresponding to work 1, and are photographed by cameras b and d, respectively, in work 1. The detection area Rd is a prohibited area corresponding to work 2, and is photographed by camera d in work 2.

The prohibited area is, for example, an area in which parts used in a certain work must not be attached. The prohibited area may be, for example, an area where a component similar in appearance to a component used in a certain work is mounted. In this way, the prohibited area is an area for detecting that the operator attaches a component to the wrong area during work.

Note that when the sensor device is an imaging device that captures images at the angle of view shown in FIG. It may be composed of one camera.

Referring again to FIG. 1, the display device 70 displays the information acquired from the work instruction device 10 as an image. Images include photographs, videos, illustrations, characters, etc. The display device 70 is realized by a liquid crystal display or the like. The image output by the display device 70 is visually recognized by the worker and used for checking the work content and determination results. The display device 70 displays information indicating that the current work was performed normally, information indicating that the work was not performed normally, and information indicating that the work was not performed as the determination results. Display information, etc.

Furthermore, the display device 70 displays information regarding the work currently being performed. The display device 70 displays, for example, at least one of the work procedure, information identifying the driver to be used (for example, driver name), the number of screws to be tightened, the determination result of screw tightening, and the elapsed time of the work. The display device 70 is placed in a position where it does not get in the way of the worker's work.

The display device 70 is an example of an output device that outputs information acquired from the work instruction device 10. Note that the work instruction system 1 includes, as an output device, a device that displays information on an object (for example, a screen) such as a sound output device (for example, a speaker), a projector, etc., in place of or in addition to the display device 70; It may also include a device that outputs information using light (for example, the color of light) such as a light emitting device.

[2. Basic operation of work instruction system]
Next, basic operations in the work instruction system 1 as described above will be explained with reference to FIGS. 4A to 4C. Specifically, the operation of the work instruction device 10 instructing a worker to perform a work will be described. This is realized by each processing unit of the work instruction device 10 executing a predetermined application program (hereinafter also referred to as a dedicated application).

First, basic operations performed using the work instruction image etc. stored in the storage unit 13 will be explained. FIG. 4A is a first diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4A, the worker first starts the dedicated application and inputs predetermined information into the items displayed on the menu. The predetermined information includes, for example, information for specifying a worker and the work performed by the worker. Items displayed on the menu include, for example, "Worker input" for entering information to identify the worker, "Terminal number input" for identifying the work terminal to be used, and "Input for the product number of the product to be worked on". "Product number input". Furthermore, the items displayed on the menu further include handling of performance data obtained during past work, adjustment of screen layout, etc.

When predetermined information is input, an image corresponding to the work is displayed on the display device 70. FIG. 4B is a second diagram for explaining the basic operation of the work instruction device 10 according to the embodiment. Specifically, a work instruction image P displayed on the display device 70 by the work instruction device 10 is shown.

As shown in FIG. 4B, when the product type is set (in FIG. 4B, "DEMO-2017" is set), a work instruction image P is displayed on the display device 70. When the work instruction device 10 obtains an operation from the worker, the work instruction device 10 causes the display device 70 to display a work instruction image P corresponding to the operation. The work instruction image P includes the product type, work name p1 (in FIG. 4B, "take part A"), precautions during work p2 (in FIG. 4B, "take it securely"), and an explanation explaining the work. An image p3, a part name p4 ("part A" in FIG. 4B), etc. are displayed. Further, although not shown, the work instruction system 1 includes a sound output device, and the work instruction corresponding to the explanation image p3 is output as a sound from the sound output device. Note that the explanatory image p3 is, for example, a moving image. Further, a work name p1 (work content), a precaution p2, an explanatory image p3, a part name p4, etc. are created for each of the plurality of works.

Thereby, the worker receives a work instruction using the work instruction image P and voice. The work instruction device 10 can support the prevention of work errors by the worker by providing work instructions using the explanatory image p3 and voice, text work instructions, and displaying the precautions p2. Furthermore, for example, if the worker is an expert, he or she can perform the work using voice instructions from the sound output device without looking at the explanatory image p3 on the display device 70. Further, for example, if the worker is not accustomed to the work concerned, the work can proceed smoothly by viewing the explanatory image p3 on the display device 70 only when necessary. Therefore, according to the work instruction device 10, it is possible to prevent work errors and support learning.

For example, if the work is a work using a screwdriver, the work name p1 includes the driver used, the type of screw, and the number of screws, and is displayed as, for example, "tighten screw 1 in two places with screwdriver X". Ru.

Further, the work instruction image P may further display a selection image for selecting a worker classification according to the skill level of the worker. FIG. 4C is a third diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4C, when the worker clicks on a drop-down menu or the like before starting work or during work, a selection image p5 for selecting the worker classification of the worker is displayed. The example in FIG. 4C shows an example in which three categories are displayed: "Newcomer," "Normal," and "Expert." However, the number of worker categories is not particularly limited, and may be 2 or more. . The output content on the display device 70 may be changed by the worker selecting his/her own worker category. For example, when "new employee" is selected, the work instruction device 10 outputs all the work instructions stored in the storage unit 13. Furthermore, when "normal" or "skilled" is selected, for example, the work instruction device 10 does not need to output a work instruction for a preset work among the plurality of works. Thereby, the work instruction system 1 can output a work instruction image P suitable for the skill level of the worker.

Referring again to FIG. 4B, the work instruction image P displays an elapsed time display bar p6. The elapsed time display bar p6 displays bars indicating progress rates of a plurality of tasks and progress times. The progress rate bar shows the elapsed time of the current task ("work unit progress" shown in the upper part of the elapsed time display bar p6), and the cumulative total of standard working time for each of multiple tasks (the elapsed time display bar p6 shows This includes the "cumulative work base" shown in the middle row) and the elapsed time during one cycle of operation (the "total work progress" shown in the lower row of the elapsed time display bar p6). Standard working hours are set in advance. Note that multiple tasks performed by a worker are also referred to as elemental tasks. 1 cycle operation in progress means that the worker is in the process of performing multiple tasks in accordance with the work procedure, where one cycle is defined as the worker performing multiple tasks in accordance with the work procedure. means.

Thereby, the worker can confirm whether or not the standard time is being kept by checking the elapsed time display bar p6, which is useful for maintaining the work pace. Therefore, according to the work instruction device 10, it is possible to support the worker in observing the work time. Note that the work instruction device 10 may alert the worker when the elapsed time during the work exceeds a predetermined percentage of the standard time for the work. The work instruction device 10 may, for example, call attention by changing the color of the work unit progress bar and the progress time, or may call attention by causing a sound output device to output a sound. For example, the work instruction device 10 changes the color of the work unit progress bar and progress time to orange when the elapsed time during the work exceeds 70% of the standard time of the work, and the elapsed time during the work changes to the standard time of the work. When the time period has elapsed, the color of the work unit progress bar and progress time may be changed to red to draw further attention.

In addition, as shown in explanatory image p3, if there are multiple types of washers and each of the multiple types of washers is housed in a different case, in explanatory image p3, the case containing the washer that the worker takes during the work is highlighted. It may be displayed as

The work instruction device 10 displays a work instruction image P corresponding to the next task on the display device 70 every time the current task is performed normally.

The work instruction device 10 may display the work result on the display device 70 when one cycle of work (a plurality of tasks assigned to a worker) is completed. FIG. 4D is a fourth diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4D, the work instruction device 10 may cause the display device 70 to display the working time upon completion of one cycle. Further, the work instruction device 10 may further display the standard time of one cycle on the display device 70.

Referring again to FIG. 4B, performance information p7 indicating the performance of the work is displayed on the work instruction image P. FIG. 4B shows an example in which the work is a work using a driver, and the work instruction image P displays performance information p7 indicating the performance of the work using the driver. That is, the work instruction image P including the performance information p7 shown in FIG. 4B is an image during driver cooperation that is displayed when performing work using a driver. As the performance information p7, for example, the work result ("OK" shown in FIG. 4B), the name of the driver used for the work ("Driver X" shown in FIG. 4B), the pulse, the count, and the number of NGs are displayed.

The work result indicates the determination result as to whether or not the screw tightening was successful. In the work result, for example, each time the control unit 12 determines that the screw is properly tightened or not, the determination result regarding the screw tightening is displayed. The work results are displayed for each screw, for example.

The driver name indicates information that identifies the driver used in the work using the driver. Only the drivers displayed in the driver name are operational.

The pulse is the actual number of pulses displayed for each screw tightening operation.

The count indicates the number of times the control unit 12 has determined that the screw has been tightened (for example, has been determined to be OK). Specifically, the count includes the actual number ("0" shown in Figure 4B), which is the number of screws that were determined to be "OK", and the planned number ("0", shown in Figure 4B), which is the total number of screws to be installed in the relevant work. 3") is displayed.

The number of NGs is the number of times screw tightening has failed. The number of NGs is counted up when the control unit 12 determines that screw tightening has failed.

By displaying the track record information p7 in this way, the worker can confirm the track record in work using the driver. Furthermore, since the work results, driver name, etc. are displayed on the display device 70, the worker can easily know the work results and information on the driver to be used.

Note that the performance information p7 for work that does not use a driver includes information indicating that the current work was performed normally, information indicating that the work was not performed normally, and information indicating that the work was not performed normally. Contains information indicating that there was no such thing.

Further, in the work instruction image P, an operation image p8 for performing operations such as switching the explanation image p3 is displayed. For example, when a predetermined button of the operation image p8 is selected with a mouse or the like, the explanation image p3 etc. are switched according to the button. The buttons of the operation image p8 include, for example, a button to stop the currently displayed explanation image p3 or restart the stopped explanation image p3, a button to proceed to the next task, a button to return to the previous task, and a button to read out voice instructions. It includes buttons to start or stop, etc.

Note that instead of the above buttons, shortcut keys may be assigned to the keyboard for easy operation. Further, the work instruction system 1 includes a sound collection device (not shown, for example, a microphone), and the work instruction device 10 displays the work instruction image P based on the worker's voice acquired by the sound collection device. You may change the contents.

Thereby, the worker can change the display contents of the work instruction image P according to the progress of the work and the like. For example, by redisplaying the explanation image p3 of the part that was difficult to understand, work errors can be prevented. Further, for example, if the work is completed before the explanation of the explanation image p3 is finished, the work can be smoothly proceeded by proceeding to the next work.

Further, the work instruction image P may display information such as a work list p9 showing a list of elemental works of one cycle.

This allows the worker to know information about the work performed before the current work and the work to be performed after the current work.

For example, when the work instruction device 10 detects that the current work is completed (e.g., detected based on the analysis result from the imaging device 40), the work instruction device 10 outputs information to the display device 70 as the next work to be performed. automatically switch to the information. For example, the work instruction device 10 automatically switches the work instruction image P displayed on the display device 70 to the work instruction image P of the next task to be performed. Note that the work instruction device 10 is not limited to automatically switching the work instruction images P, etc., and may switch the work instruction images P etc. by an operation from the worker (for example, operation of a foot switch).

[3. Operation of work instruction system]
Next, the operation of the work instruction system 1 as described above will be explained with reference to FIG. 5. FIG. 5 is a sequence diagram showing the operation of the work instruction system 1 according to the embodiment. Note that an example in which no prohibited area is set will be described below. Further, it is assumed that the work procedure includes work 1 to work n. The Nth work is also written as work N. Further, in FIG. 5, an example will be described in which image analysis is performed based on image data in the imaging device 40 (for example, the camera control unit 50), but the imaging device 40 is not limited to performing image analysis.

As shown in FIG. 5, the work instruction device 10 reads work procedures (1 to n) by activating a dedicated application and inputting predetermined information by an operation of a worker (S11). The work instruction device 10 can determine whether the worker is the worker who performs the work with the production number by inputting the "worker" and the "product number." The work instruction device 10 extracts a work instruction image P, a work table T1, and a detection table T2 according to predetermined information from among the plurality of work instruction images and information tables stored in the storage unit 13. Note that reading includes reading the work instruction image P, the work table T1, and the detection table T2 from a recording medium (CD-ROM, USB memory, etc.) external to the work instruction device 10. Further, reading the work procedure also includes the communication unit 11 acquiring the work instruction image P, the work table T1, and the detection table T2 through communication from an external device.

Next, the work instruction device 10 reads the information on the work N and outputs it to the display device 70 (S12). Work N is the work to be performed from now on. In step S12, the control unit 12 acquires the work instruction image P corresponding to the work N and information regarding the camera to be operated in the work N (for example, the detection camera) from the detection table T2. Then, the control unit 12 outputs the work instruction image P corresponding to the work N to the display device 70.

The display device 70 acquires information on work N from the work instruction device 10 and displays the information (S13). Thereby, the worker can work after or while checking the display device 70. Note that, for example, when the work instruction system 1 includes a sound output device as an output device, a sound based on the acquired information may be output in step S13.

Next, the work instruction device 10 outputs control information for work N to the camera control unit 50 (S14). Specifically, the work instruction device 10 generates control information including information for specifying the camera to be operated in the work N (that is, the camera that photographs the detection area) based on the detection table T2, and It is output to the control section 50. Note that the image obtained by operating the camera may be a still image or a moving image.

In this embodiment, to explain work 1 as an example, the work instruction device 10 operates the camera a to photograph the detection area Ra, and stops photographing the detection areas Rb to Rd in other tasks. In order to do so, control information indicating that the operations of cameras b to d are to be stopped is generated and output to the camera control unit 50. This enables the detection of the first part (for example, screw A) in the first area (for example, detection area Ra), and the detection of the second part (for example, screw B) in the second area (for example, detection area Rb). ) is an example of disabling detection.

The camera control unit 50 acquires control information and controls the imaging unit 60 based on the acquired control information (S15). In this embodiment, the camera control unit 50 controls the imaging unit 60 to operate only the camera corresponding to work N based on the control information. Note that the control of the imaging section 60 by the camera control section 50 is not limited to this. The camera control unit 50 may operate cameras other than the camera corresponding to work N.

The imaging unit 60 operates only the camera corresponding to work N (S16), and outputs image data (an example of a sensing result) indicating an image taken of the detection area corresponding to work N to the camera control unit 50 (S17). . Further, if the sensor corresponding to work N is an optical sensor, the sensing result includes, for example, at least one of whether or not a part of the detection area is shielded from light, and the number of times that light is shielded. Note that the sensing results may be outputted sequentially or at predetermined intervals.

The camera control unit 50 acquires image data from the imaging unit 60 (S18), and analyzes the acquired image data (S19). The camera control unit 50 detects the presence or absence of parts in the work by image analysis of the acquired image data. Then, the camera control unit 50 outputs the image data and the analysis result to the work instruction device 10 (S20). The image data and the analysis results may be outputted sequentially or at predetermined intervals.

When cameras other than the camera corresponding to work N are also operated in step S15 (for example, when all cameras a to d are operated), in step S20, the camera control unit 50 controls each of cameras a to d. Image analysis may be performed only on the image data of the camera corresponding to work N from the image data of . In this way, the camera control unit 50 extracts the image data of the camera corresponding to the work N from among the image data of the plurality of cameras acquired from the imaging unit 60, and performs image analysis of the extracted image data. good. Then, the camera control unit 50 may output only the image data and analysis results corresponding to the work N to the work instruction device 10. The camera control unit 50 outputting only the image data and analysis results of the camera corresponding to the work N to the work instruction device 10 means that the first part (for example, Detection of a second part (for example, a part corresponding to a work other than work N) is enabled in a second area (for example, a region corresponding to a work other than work N). This is an example of invalidating.

Note that the camera control unit 50 does not need to analyze the image data. The camera control unit 50 may function as a relay device that acquires image data from the imaging unit 60 and outputs the acquired image data to the work instruction device 10, for example. If cameras other than the camera corresponding to work N are operated in step S15 (for example, all cameras a to d are operated), the camera control unit 50 controls all cameras a to d in step S20. From the image data of d, only the camera image data corresponding to work N may be output to the work instruction device 10.

In this way, the camera control unit 50 may extract the image data of the camera corresponding to the work N from among the image data of the plurality of cameras acquired from the imaging unit 60, and output it to the work instruction device 10. The camera control unit 50 outputting only the image data of the camera corresponding to the work N to the work instruction device 10 enables the detection of the first part in the first area and the detection of the second part in the second area. This is an example of invalidating.

In addition, when the imaging unit 60 has a camera that captures images with the angle of view shown in FIG. By performing image analysis only in Ra), it may be detected whether a component corresponding to work N is attached to the detection area, and the detection result may be output to the work instruction device 10. In other words, in the work N, the detection result as to whether or not the part corresponding to the work N is attached to a detection area different from the detection area corresponding to the work N is not output to the work instruction device 10. The fact that the camera control unit 50 outputs only the detection result of the detection area corresponding to the work N from the image data to the work instruction device 10 enables the detection of the first part in the first area and the detection result of the detection area corresponding to the work N in the second area. This is an example of disabling the detection of two parts. In addition, when the imaging unit 60 has a camera that captures an image with the angle of view shown in FIG. It may be output to 10.

Next, the work instruction device 10 acquires image data and analysis results corresponding to work N from the camera control unit 50 (S21). The work instruction device 10 determines whether work N was performed normally based on the acquired image data and the analysis result (S22). The determination will be described later. Then, the work instruction device 10 outputs the determined result to the display device 70 (S23). The determination result (for example, information indicating that work N was performed normally) is included in the work content. The work content according to this embodiment includes at least a determination result.

The display device 70 acquires the determination result and displays the acquired determination result (S24). Thereby, the work instruction device 10 can notify the worker whether or not work N has been performed normally.

The processes of steps S12 to S24 are repeatedly executed until all of the work procedures (for example, work 1 to work n) included in the work table T1 are completed. Note that when the work instruction device 10 determines in step S22 that the work N is not being performed normally, it outputs a determination result indicating that the work N is not being performed normally to the display device 70. Then, the work instruction device 10 does not execute the processing after step S12 in the next work. In other words, no further work will be performed.

In addition, although the example in which the work instruction device 10 directly outputs the determination result to the display device 70 has been described above, the work instruction device 10 outputs the determination result directly to the display device 70. You can also output it. The work instruction device 10 outputting the determination result to the relay device is included in outputting the determination result to the display device 70.

Note that the work instruction device 10 may cause the display device 70 to display the determination result superimposed on the work instruction image P for the task.

Next, the operation of the work instruction device 10 will be explained with reference to FIG. 6. FIG. 6 is a flowchart showing the operation of the work instruction device 10 according to the embodiment. Note that each step shown in FIG. 6 is an example of a display step for displaying the work details to the worker.

As shown in FIG. 6, the control unit 12 reads the work procedure (S31). Step S31 corresponds to step S11 shown in FIG. Then, the control unit 12 sets work N to be performed from now on to work 1 in the read work procedure. That is, the control unit 12 sets N=1 (S32). From step S33 onwards, the case where N=1 will be mainly explained.

Next, the control unit 12 reads the information on the work N, and outputs the read information on the work N to the display device 70 (S33). When the work N is the work 1, the control unit 12 outputs the work instruction image P corresponding to the work 1 to the display device 70.

Next, the control unit 12 sets the detection area (an example of the first area) for work N to be valid (S34). Enabling the detection area of work N means that the work instruction device 10 makes the determination in step S36 using information based on image data of a camera that photographs the detection area. In other words, it means that the determination in step S36, which will be described later, is performed without using image data of a camera that photographs a detection area other than the detection area of work N (an example of the second area). Step S34 is an example of validating the detection of the first part in the first area and invalidating the detection of the second part in the second area.

Specifically, the control unit 12 generates control information based on the detection table T2 and outputs it to the imaging device 40. When work N is work 1, the control information here includes, for example, information for specifying camera a, which is the camera that acquires image data in work 1. The control information may include, for example, an instruction to operate only camera a among cameras a to camera d. Step S34 corresponds to step S14 shown in FIG.

Next, the control unit 12 acquires the detection result for work N (S35). The control unit 12 obtains, for example, image data and analysis results as detection results. When the work N is the work 1, the control unit 12 acquires, for example, image data indicating an image taken by the camera a and an analysis result of the image data. Note that the control unit 12 may acquire at least one of image data and analysis results from the imaging device 40. Hereinafter, an example will be described in which the control unit 12 acquires only image data from the imaging device 40 and acquires an analysis result in its own device based on the acquired image data. It's okay. Step S35 corresponds to step S21 shown in FIG. Note that step S35 may be performed multiple times for each task, for example.

Next, the control unit 12 determines whether a component has been mounted in the detection area based on the image data (S36). When the work N is the work 1, the control unit 12 determines whether the screw A is installed in the detection area Ra. The control unit 12 determines whether the screw A is attached to the detection area Ra by, for example, analyzing the image data of the camera a. Note that the image analysis method does not need to be particularly limited, and conventional techniques may be used. Step S36 corresponds to step S22 shown in FIG.

For example, when the image data indicates that the screw A has been detected in the detection area Ra corresponding to work 1, the control unit 12 determines that the screw A has been attached to the detection area Ra. When the control unit 12 determines that the screw A is attached to the detection area Ra (Yes in S36), it outputs information indicating that work 1 has been performed normally (S37). The control unit 12 outputs the information to the display device 70, for example. Step S37 corresponds to step S23 shown in FIG. Further, the information indicating that work 1 was performed normally is an example of the first work content.

Then, the control unit 12 determines whether all the work included in the read work procedure has been completed. Specifically, the control unit 12 determines whether N=n (S38). If N=n (Yes in S38), the control unit 12 ends the process. Further, if N=n is not true (No in S38), the control unit 12 proceeds to the next operation (S39). Specifically, the control unit 12 sets N=n+1 and continues the processing from step S33 onwards. For example, after work 1, work 2 is performed. For example, the information indicating that work 2 in step S37 was performed normally is an example of the second work content.

As a result, when a part (an example of the first part) corresponding to the work N (an example of the first part) is detected in the detection area (an example of the first area) in the work N (an example of the first work), the next work (an example of the first work) is detected. 2) Information for performing an example of work 2) is output to the display device 70. In other words, if a component corresponding to the work N is not detected in the detection area for the work N, information for performing the next work is not output to the display device 70.

Further, when the control unit 12 determines that the detection area has not been passed (No in S36), the control unit 12 further determines whether a predetermined time has elapsed (S40). The predetermined period is the elapsed time since the start of the work N, and may be, for example, the elapsed time since the detection area of the work N was effectively set in step S34.

If the predetermined time has elapsed (Yes in S40), the control unit 12 outputs information indicating that work N has not been performed (S41). The control unit 12 outputs the information to the display device 70, for example. Then, the control unit 12 ends the process. Step S41 corresponds to step S23 shown in FIG. Further, if the predetermined time has not elapsed (No in S40), the control unit 12 returns to step S35 and continues the process. Note that the information indicating that work N was not performed is an example of the first work content.

FIG. 7A is a diagram showing how detection of the first component (eg, screw A) in the first area (eg, detection area Ra) is enabled. Specifically, FIG. 7A shows a state in which the camera a is photographing the detection area Ra in work 1 in order to determine whether the screw A is attached to the detection area Ra. Note that diagonal hatching in FIG. 7A indicates a detection area where detection is enabled.

As shown in FIG. 7A, only the screw A, which is a part corresponding to work 1 among work 1 to work 4, is attached to the main body portion 100. At this time, the camera control section 50 controls the imaging section 60 so that the detection of the screw A in the detection area Ra becomes effective. For example, the camera control unit 50 operates only the camera a among the cameras a to d. Thereby, it is possible to determine whether work 1 has been performed normally based on whether or not the screw A is attached to the detection area Ra.

For example, in work 1, when each of the cameras a to d is operating and it is determined whether a component has been mounted in each of the detection areas Ra to Rd, in the area other than the detection area Ra, In this case, it may be determined that the part is not installed, and an erroneous determination may be made that work 1 is not being performed normally. According to the work instruction device 10 according to the present embodiment, since detection is not performed in areas other than the detection area Ra, it is possible to suppress such erroneous determination from being performed.

Next, a case will be described in which work 4 is performed after works 1 to 3 are performed. FIG. 7B is a diagram showing how the detection of the screw D in the second region (for example, detection region Rd) is enabled.

As shown in FIG. 7B, screws A, B, plate C, and screw D, which are parts corresponding to work 1 to work 4, are attached to the main body 100, respectively. At this time, the camera control section 50 controls the imaging section 60 so that the detection of the screw D in the detection area Rd is enabled. For example, the camera control unit 50 operates only the camera d among the cameras a to d. Thereby, it is possible to determine whether work 4 has been performed normally based on whether or not the screw D is attached to the detection region Rd.

For example, in work 4, when each of the cameras a to d is operating and it is determined whether a component is mounted in each of the detection areas Ra to Rd, the part covered by the plate C is detected. Since the screw A that is present cannot be detected by the camera a, it may be determined that the screw A is not installed, and an erroneous determination may be made that the work 1 is not being performed normally. Therefore, as described above, in the work 4, it is preferable that only the detection of the screw D in the detection area Rd corresponding to the work 4 is performed.

Note that in work 4, each of the cameras a to d may be in operation, and it may be determined whether a component has been mounted in each of the detection areas Ra to Rd. In this case, only the determination result of the screw D corresponding to work 4 is adopted, and it is determined whether work 4 was performed normally. This also enables the detection of the screw D (an example of the first part) in the detection area Rd (an example of the first area), and also enables the detection of the screw A (an example of the second part) in the detection area Ra (an example of the second area). ) is an example of disabling detection.

As described above, the work instruction device 10 enables the detection of the parts corresponding to the work N in the detection area corresponding to the work N, and determines whether the work N is performed normally based on the image data of the detection area. Determine whether it has been performed. Further, the work instruction device 10 disables detection of parts corresponding to the work N in detection areas other than the detection area corresponding to the work N. That is, the work instruction device 10 makes the above determination without using image data of detection areas other than the detection area corresponding to the work N.

[4. Effects, etc.]
As explained above, in the work instruction method (an example of the work display method) according to the present embodiment, the first part is attached to the first area, and the first part is attached to the second area different from the first area. A task in which a plurality of tasks are performed, including a first task in which the second part is not attached to the second area, and a second task in which the second part is not attached to the first area. This is a work display method for displaying the work details to the person. The work display method includes a first display step for displaying a first work content in a first work, and a second display step for displaying a second work content in a second work. In the first display step, the detection of the first part in the first area is enabled and the detection of the second part in the second area is disabled (S34), and the If the first part is detected in the first area based on the image data (an example of sensing results) (Yes in S36), information indicating that the first work was performed normally is displayed as the first work content. It is output to the device 70 (S37). In the second display step, the detection of the second part in the second area is enabled and the detection of the first part in the first area is disabled (S34), and the If the second part is detected in the second area based on the sensing result (Yes in S36), information indicating that the second work was performed normally is output to the display device 70 as the second work content ( S37).

For example, when performing the first task, if detection in the second region is effective, it is determined whether the first task was performed normally based on the image data of the first region and the second region. be done. In this case, even if the first work is performed normally, the part corresponding to the second work is not detected in the second area, or the part corresponding to the first work is detected in the second area. In this case, it may be determined that the first task was not performed normally. In other words, false detection may occur.

On the other hand, in the work instruction method (an example of a work display method) according to the present embodiment, since the detection in the first area is invalid in the second work, it is difficult to determine whether the first work has been performed normally. erroneous detection can be suppressed. For example, when the second work is performed after the first work, at least a portion of the first part of the first work may be hidden in the work performed between the first work and the second work. If a part of the first work is detected in the second work in this state, the part may not be detected accurately and an erroneous detection may occur. For example, it is possible to suppress the occurrence of such false detections.

Further, in the work instruction method according to the present embodiment, since the detection in the second area is invalid in the first work, the process of determining whether the first work in the first work has been performed normally. The amount of processing can be reduced. For example, it is possible to reduce the load of image processing for determining whether the first task has been performed normally based on the image taken by the camera.

Therefore, according to the work instruction method according to the present embodiment, it is possible to reduce the processing load in the determination process and to suppress false detections.

The sensor unit also includes an imaging unit 60 that captures an image including the first area and the second area as a sensing result.

Thereby, it is possible to determine whether or not the work has been performed normally based on image data (an example of a sensing result) indicating an image photographed by the imaging unit 60. For example, when image analysis results of image data (for example, the position, shape, color, etc. of parts) are used, it is possible to more accurately determine whether the work has been performed normally. In other words, according to the work display method according to the present embodiment, false detection can be further suppressed.

Further, the second work is a work to be performed continuously after the first work, and in the first display step, an image for performing the first work is further displayed on the display device 70, and in the second display step, Furthermore, in the first display step, when the first part is detected in the first area, an image for performing the second work is displayed on the display device 70.

As a result, when the first task is successfully performed, an image for performing the next task is automatically displayed, improving work efficiency.

Further, as described above, in the work instruction device 10 (an example of a work display device) according to the present embodiment, the first part is attached to the first area, and the second area different from the first area is A plurality of tasks including a first task in which the first component is not mounted, and a second task in which the second component is mounted in the second region and the second component is not mounted in the first region. This is a work display device for displaying work details to the worker performing the work. When the first work is to be performed, the work instruction device 10 enables the detection of the first part in the first region, disables the detection of the second part in the second region, and disables the detection of the second part in the second region. When the first work is performed, the control unit 12 enables detection of the second part in the second area and disables the detection of the first part in the first area, and the imaging unit 60 ( When the first part is detected in the first area based on the image data (an example of sensing results) obtained from the sensor unit (an example of a sensor unit), information indicating that the first work was performed normally is sent to the first When outputting the work content to the display device 70 and performing the second work, when the second part is detected in the second area based on the image data acquired from the imaging unit 60, the second work is performed. The communication unit 11 (an example of an output unit) outputs information indicating that the process has been successfully performed to the display device 70 as the second work content.

This produces the same effect as the work instruction method described above.

(Modification 1 of the embodiment)
Next, a work instruction method according to modification 1 of the embodiment will be described with reference to FIGS. 8 and 9. FIG. 8 is a flowchart showing the operation of the work instruction device 10 according to the first modification of the embodiment. Note that each step shown in FIG. 8 is an example of a display step for displaying the work details to the worker.

Note that the configuration of the work instruction device according to this modification is the same as that of the work instruction device 10 according to the embodiment, and a description thereof will be omitted. Further, in the flowchart in FIG. 8 as well, processes similar to those in the flowchart in FIG. 6 are denoted by the same reference numerals, and the description thereof will be omitted or simplified.

As shown in FIG. 8, after reading the work N in step S33, the control unit 12 sets the detection area and prohibited area of the work N to be valid (S134). Enabling the detection area and prohibited area of work N means to perform the determinations in steps S136 and S142, which will be described later, using image data of a camera that photographs the detection area and prohibited area of work N. Step S134 enables detection of the first part (for example, screw A) in the first region (for example, detection region Ra), and enables detection of the second component (for example, plate) in the second region (for example, detection region Rc). This is an example of disabling the detection of C) and further enabling the detection of the first component in the third region (for example, detection regions Rb and Rd) which is a prohibited region.

Specifically, the control unit 12 generates control information based on the detection table T2 and outputs it to the imaging device 40. When work N is work 1, the control information here may include an instruction to operate only cameras a, b, and d among cameras a to camera d. The control information may include an instruction to output image data of camera a, which is the detection camera corresponding to work 1, and cameras b and d, which are prohibited cameras corresponding to work 1, to the work instruction device 10. Step S134 corresponds to step S14 shown in FIG.

Next, the control unit 12 acquires the detection result for work N from the imaging device 40 (S135). The control unit 12 obtains, for example, image data and analysis results. When the work N is work 1, the control unit 12 acquires image data of cameras a, b, and d, and an analysis result of the image data. Note that the control unit 12 may acquire at least the image data from the image capturing device 40 as the detection result among the image data and the analysis result. Hereinafter, an example will be described in which the control unit 12 acquires only image data from the imaging device 40 in step S135. Step S135 corresponds to step S21 shown in FIG. Note that step S135 may be performed multiple times for each task, for example.

Next, the control unit 12 determines whether a component has been mounted in the detection area based on the image data (S36). If the control unit 12 determines that the component is attached to the detection area (Yes in S36), it further determines whether the component is attached to the prohibited area based on the image data (S142). Specifically, the control unit 12 determines whether the part corresponding to work N is mounted in the prohibited area based on the image data of the prohibited camera. When the work N is work 1 and the image data of cameras b and d indicate that the screw A is not detected in the detection areas Rb and Rd, which are the prohibited areas corresponding to work 1, the control unit 12 prohibits the screw A from being detected. It is determined that screw A is not attached to the area. The image data used for the determinations in steps S36 and S142 may be, for example, image data representing images taken at the same time. Steps S36 and S142 correspond to step S22 shown in FIG. 5.

Here, the detection area and prohibited area in Work 1 will be explained with reference to FIG. 9. FIG. 9 is a diagram showing how the prohibited area corresponding to work 1 is enabled. The diagonal line downward to the left in FIG. 9 indicates the detection area in work 1, and the diagonal line downward to the right indicates the prohibited area in work 1.

As shown in FIG. 9, in work 1, detection of the screw A in the detection area Ra is set to be effective. Thereby, in work 1, it is possible to determine whether or not the screw A is attached to the detection area Ra (determination shown in S36) based on the image data of the camera a. In the example of FIG. 9, since the screw A is attached to the detection area Ra of the main body 100, the work instruction device 10 determines Yes in step S36.

Further, in work 1, detection of the screw A in the detection regions Rb and Rd is set to be effective. As a result, in work 1, it is determined based on the image data of cameras b and d whether or not the screw A is erroneously attached to the detection areas Rb and Rd, which are prohibited areas in work 1 (determination shown in S142). It can be carried out. In the example of FIG. 9, since the screw A is not installed in the prohibited area (detection areas Rb and Rd) in the work 1 of the main body 100, the work instruction device 10 determines No in step S142.

Referring again to FIG. 8, if the control unit 12 determines that no component is mounted in the prohibited area (No in S142), it outputs information indicating that work N was performed normally (S37). . The control unit 12 outputs the information to the display device 70, for example.

Further, when the control unit 12 determines that the component has been mounted in the prohibited area (Yes in S142), it outputs information indicating that work N was not performed normally (S143). Then, the control unit 12 ends the process. Note that steps S37, S41, and S143 correspond to step S23 shown in FIG. 5. Further, for example, information indicating that work 1 was not performed normally is an example of the first work content.

As described above, the work instruction device 10 effectively sets the detection area and prohibited area according to the work, and based on the image data of only the camera that photographs the detection area and the camera that photographs the prohibited area. , determine whether the work was performed normally. Further, the work instruction device 10 disables detection of areas other than the detection area and prohibited area according to the work.

Note that, as shown in FIG. 2B, in tasks for which prohibited cameras are not set (for example, tasks 3 and 4), the process of step S142 may not be performed. In this case, if the answer is Yes in step S36, the process advances to step S37. For example, if the result in step S36 is Yes, the control unit 12 executes the determination process in step S142 based on the detection table T2 (for example, based on whether a prohibited camera is set in the work). You may also make a determination as to whether or not to do so.

As explained above, the work instruction method (an example of a work display method) according to the present modification has a third area different from the first area and the second area, and the mounting of the first component is prohibited in the first work. In the first display step, the detection of the first part in the third area is further enabled (S134), and the image data (sensing If the first part is not detected in the third area based on the result example), information indicating that the first work was performed normally is output to the display device 70 as the first work content (S37). .

Thereby, it is possible to determine whether or not the work has been performed normally based on whether or not the first part is detected in the prohibited area. For example, if the first part is detected only in the first area of the first area and the third area, the operator is performing the correct work. Therefore, according to the work display method according to this modification, if the first part is detected in the first area but not in the third area, it is determined that the work has been performed normally. Therefore, it is possible to more accurately determine whether the work has been performed normally.

Furthermore, in the first display step, if the first part is detected in the third area (Yes in S142), information indicating that the first work was not performed normally is displayed as the first work content. It is output to the device 70 (S143).

This allows the operator to be informed that the work has not been performed properly. For example, if the first part is detected in both the first area and the third area, the operator may be performing the wrong work (for example, attaching the first part to the third area). There is sex. Therefore, according to the work display method according to this modification, when the first part is detected in the third area, that is, when there is a possibility that the wrong work is being performed, the first part is detected in the first area. Even if it is detected, it is determined that the work was not performed normally. Therefore, it is possible to more accurately determine whether the work has been performed normally.

Additionally, if the information indicating that the work was not performed normally includes information indicating that the first part was detected in the third area (prohibited area), the operator may detect that the first part was detected in the third area. It is possible to know that it has been determined that the work was not performed normally by detecting this. For example, if the third area is an area where a part similar to the first part is to be mounted, the operator can check whether he or she has misplaced the part.

(Modification 2 of embodiment)
Next, a work instruction method according to a second modification of the embodiment will be described with reference to FIG. 10. FIG. 10 is a flowchart showing the operation of the work instruction device 10 according to the second modification of the embodiment. Note that each step shown in FIG. 10 is an example of a display step for displaying work details to the worker.

Note that the configuration of the work instruction device according to this modification is the same as the work instruction device 10 according to the embodiment, and although the explanation is omitted, each of the cameras a to d is, for example, a color camera. Further, in the flowchart in FIG. 10 as well, processes similar to those in the flowchart in FIG. 6 are designated by the same reference numerals, and the description thereof will be omitted or simplified.

As shown in FIG. 10, after validating the detection area for work N in step S34, the control unit 12 acquires the detection result for work N from the imaging device 40 (S335). The control unit 12 obtains, for example, image data and analysis results. When the work N is the work 1, the control unit 12 acquires the image data of the camera a and the analysis result of the image data. The analysis result here includes, for example, information regarding the color of the component attached to the detection area. That is, the control unit 12 acquires information regarding the color of the component as the analysis result in step S335. The analysis result may be obtained from the imaging device 40, or may be obtained by performing image processing on the image data obtained from the imaging device 40 using the own device. Step S335 corresponds to step S21 shown in FIG.

Next, the control unit 12 determines whether a component has been mounted in the detection area based on the image data (S36). When the control unit 12 determines that the component is attached to the detection area (S36: Yes), the control unit 12 further determines whether a predetermined color is detected based on the analysis result (S242). For example, the control unit 12 determines whether a predetermined color is detected based on the analysis result and the detection table T2.

Taking Work 1 as an example, when the color of the screw A attached to the detection area Ra corresponding to Work 1 is the same as the detected color "blue" shown in the detection table T2, the control unit 12 selects a predetermined value. It is determined that a color has been detected. Steps S36 and S242 correspond to step S22 shown in FIG. 5.

When the control unit 12 detects the predetermined color (Yes in S242), the control unit 12 proceeds to step S37 and executes subsequent processing. Further, if the predetermined color is not detected (No in S242), the control unit 12 outputs information indicating that work N was not performed normally (S243). Then, the control unit 12 ends the process. Note that steps S37, S41, and S243 correspond to step S23 shown in FIG. 5.

As described above, the work instruction device 10 enables the detection of the detection area according to the work, and determines whether a part is attached to the detection area according to the work and whether a predetermined color is detected. Based on this, control is performed to determine whether or not the work has been performed normally.

In addition, in a work in which a detection color is not set, if it is determined Yes in step S36, the process advances to step S37. The control unit 12 determines whether or not to execute the determination process in step S242, for example, based on the detection table T2 (for example, based on whether a detection color is set in the task). Good too.

As explained above, the image in the work instruction method (an example of the work display method) according to this modification includes information regarding the colors in the first area and the second area.

Thereby, it is possible to determine whether the work has been performed normally using the color information, so that the above-mentioned determination can be made more accurately. Further, for example, when a prohibited area is set, if a predetermined color is not detected in the prohibited area, it is determined that the work has been performed normally, so that false detection can be further suppressed.

Further, the information regarding color includes the color of a part used in at least one of the first work and the second work.

With this, for example, if a worker mistakenly attaches a component that is similar in shape to a component used in work but has a different color, it can be detected based on the color of the component. Therefore, it is possible to more accurately determine whether the work has been performed normally.

(Other embodiments)
Although the embodiment and each modified example (hereinafter also referred to as embodiment, etc.) have been described above, the present invention is not limited to such embodiment, etc.

For example, in the above embodiments, an example is described in which there is one detection area for one task, but there may be a plurality of detection areas.

Further, in the above embodiments, an example has been described in which a detection area other than the current work among the detection areas is set as the prohibited area, but the invention is not limited to this. The prohibited area may be a separate area from the detection area. That is, the prohibited area and the detection area may be different areas from each other.

Further, the operation of the work instruction device 10 in the above embodiments etc. (the operation shown in FIG. 6, FIG. 8, or FIG. 10) may be switched for each task based on the information included in the detection table T2. good. For example, in tasks 1 and 2, prohibited cameras are set, so the determination process shown in FIG. 8 (processing after S36) is executed, and in tasks 3 and 4, no prohibited cameras are set, so The determination process shown in FIG. 6 (the process after S36) may be executed.

Further, in the above embodiments, an example has been described in which the camera control unit 50 performs image analysis processing on image data, but the control unit 12 of the work instruction device 10 may perform the image analysis processing.

Furthermore, in the embodiments described above, the work instruction device 10 outputs the read information of the work N to the display device 70 (for example, see S33 shown in FIG. 6). It is not necessary to perform the process of outputting to 70.

Further, in the above embodiments, an example has been described in which it is determined that the work has been performed normally when a component corresponding to the detection area is detected in the detection area, but the invention is not limited to this. If the part is a screw, in step S36, if the number of pulses in the screw tightening work is within the upper and lower limit pulse numbers (see FIG. 2A) and the screw is detected in the detection area, the part is not placed in the detection area. may be determined to have been attached, that is, the work has been performed normally.

Further, in the above embodiments and the like, an example has been described in which one part is attached for one work, but the present invention is not limited to this. A plurality of parts (for example, a plurality of types of parts) may be attached to one work. Further, although an example has been described in which one component is associated with one detection area, the present invention is not limited to this. A plurality of parts may be associated with one detection area. In the first operation, for example, a first part including two or more parts may be attached to the first region. The camera a may be provided so as to be able to detect the first component including two or more components.

Further, in the above embodiments, an example has been described in which the detection areas Ra to Rd are photographed using different cameras, but the present invention is not limited to this. The work instruction system 1 has, for example, one camera that is rotatably supported in the horizontal and vertical directions, and rotates the camera according to the detection area to separately detect the detection areas Ra to Rd. You may take pictures. Thereby, the required number of cameras in the work instruction system 1 can be reduced.

Furthermore, in the above embodiments, etc., an example has been described in which mounting a component means tightening a screw and attaching a plate, but the present invention is not limited to this. Attaching a component means, for example, that the component is joined to the detection area of the main body 100 with a joining member (for example, solder, etc.), and that the component is attached to the detection area of the main body 100 with a fastening member such as a screw. Including being fixed.

Further, in the embodiments described above, an example has been described in which the number of display devices 70 connected to the work instruction device 10 is one, but the present invention is not limited to this. For example, when a worker performs each of tasks 1 to 4 on different workbenches, the display device 70 may be placed on each of the workbenches.

Further, in the above embodiments, the work performed by the worker includes work using a tool (for example, a screwdriver) and work without using a tool, but the work is not limited thereto. The work performed by the worker may be, for example, only work without using tools.

Further, in the above embodiments and the like, an example in which the tool is a screwdriver has been described, but the present invention is not limited to this. The tool may be any tool as long as it is communicably connected to the work instruction device 10 and can be controlled by control information from the work instruction device 10. The tool may be, for example, a tool other than a screwdriver (for example, an electric tool) or a jig (for example, an inspection jig).

Furthermore, the division of functional blocks in the block diagram shown in Figure 1 is just an example; multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple functional blocks, or some functions may be implemented as other functional blocks. It may be moved to a functional block. Further, functions of a plurality of functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

Further, in the embodiments described above, each of the work instruction device 10 and the imaging device 40 is realized by a single device, but they may be realized by a plurality of devices connected to each other.

Further, there is no particular limitation on the communication method between the devices included in the work instruction system 1 in the above embodiments and the like. Wireless communication or wired communication may be performed between the devices. As the wireless communication method, for example, a wireless communication method such as WiFi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), etc., which are standardized by IEEE802.15.1, is used.

Further, the order in which the steps in the flowchart are executed is for illustrative purposes to specifically explain the present invention, and may be in an order other than the above. For example, some of the above steps may be executed simultaneously (in parallel) with other steps, or may be executed in a switched order with other steps.

Further, some or all of the components included in the work instruction device 10 and the imaging device 40 in the above embodiments may be configured from one system LSI (Large Scale Integration). . For example, it may be configured from a system LSI having processing units such as the control unit 12 of the work instruction device 10 and the camera control unit 50 of the imaging device 40.

A system LSI is a super-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically includes a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), etc. A computer system that includes: A computer program is stored in the ROM. The system LSI achieves its functions by the microprocessor operating according to a computer program.

Note that although it is referred to as a system LSI here, it may also be called an IC, LSI, super LSI, or ultra LSI depending on the degree of integration. Further, the method of circuit integration is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used.

Furthermore, if an integrated circuit technology that replaces LSI emerges due to advancements in semiconductor technology or other derived technology, then of course the functional blocks may be integrated using that technology. Possibilities include the application of biotechnology.

Further, one aspect of the present invention may be a computer program that causes a computer to execute each characteristic step included in the work instruction method. Further, one aspect of the present invention may be a computer-readable non-transitory recording medium on which such a computer program is recorded.

Note that in the above embodiments and the like, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Other embodiments may be obtained by making various modifications to the embodiments that those skilled in the art may think of, or may be realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of the present invention. The form is also included in the present invention.

1 Work instruction system (work display system)
10 Work instruction device (work display device)
11 Communication section (output section)
12 Control unit 13 Storage unit 20 First driver 30 Second driver 40 Imaging device 50 Camera control unit 60 Imaging unit (sensor unit)
70 Display device 100 Main unit P Work instruction image p1 Work content p2 Notes p3 Explanation image p4 Part name p5 Selection image p6 Elapsed time display bar p7 Performance information p8 Operation image p9 Work list Ra to Rd Detection area T1 Work table T2 Detection table for

Claims (8)

A first work is a work in which a first part is installed in a first area, and the first part is not installed in a second area different from the first area, and a second part is installed in the second area. and a second work in which the second part is not mounted in the first area, and a work display method for displaying work details to a worker who performs a plurality of works, including a second work in which the second part is not attached to the first area,
a first display step for displaying a first work content in the first work, and a second display step for displaying a second work content in the second work,
In the first display step,
Detection of the first part in the first area is enabled, and detection of the second part in the second area is disabled,
When the first part is detected in the first area based on the sensing result obtained from the sensor unit, a display device displays information indicating that the first work has been performed normally as the first work content. Output to
In the second display step,
enabling detection of the second part in the second region and disabling detection of the first part in the first region;
When the second part is detected in the second region based on the sensing result obtained from the sensor unit, information indicating that the second work has been performed normally is set as the second work content. Output to display device,
How to display your work. a third area different from the first area and the second area, including a third area where mounting of the first component is prohibited in the first operation;
In the first display step, further:
enabling detection of the first part in the third area;
When the first part is not detected in the third area based on the sensing result obtained from the sensor unit, information indicating that the first work has been performed normally is set as the first work content. output to the display device;
The work display method according to claim 1. In the first display step, when the first part is detected in the third area, information indicating that the first work was not performed normally is displayed on the display device as the first work content. output to,
The work display method according to claim 2. The sensor unit includes an imaging device that captures an image including the first area and the second area as the sensing result.
The work display method according to any one of claims 1 to 3. The image includes information regarding colors in the first area and the second area,
The work display method according to claim 4. The information regarding the color includes the color of a part used in at least one of the first work and the second work.
The work display method according to claim 5. The second work is a work that is performed continuously after the first work,
In the first display step, further displaying an image for performing the first task on the display device,
In the second displaying step, further, in the first displaying step, if the first component is detected in the first area, displaying an image for performing the second work on the display device.
The work display method according to any one of claims 1 to 6. A first work is a work in which a first part is installed in a first area, and the first part is not installed in a second area different from the first area, and a second part is installed in the second area. and a work display device for displaying work details to a worker who performs a plurality of works including a second work in which the second part is not mounted in the first area,
When the first work is performed, the detection of the first part in the first region is enabled, the detection of the second part in the second region is disabled, and the second work is performed. a control unit that enables detection of the second part in the second region and disables detection of the first part in the first region when
When the first work is performed, when the first part is detected in the first region based on the sensing result obtained from the sensor unit, it indicates that the first work has been performed normally. When information is output to a display device as a first work content and the second work is performed, the second part is detected in the second area based on the sensing result obtained from the sensor unit. and an output unit that outputs information indicating that the second work has been performed normally as second work content to the display device.
Work display device.

Images