JP7611093B2 - Work support system and work support method - Google Patents

Description

The present invention relates to a work support system and a work support method.

Conventionally, there are various technologies that support the work of workers on work objects such as workpieces. For example, Patent Document 1 discloses a technology in which "the work time proficiency is calculated by comparing the actual work time, from which invalid work not caused by the worker is removed based on data obtained from the image capture device 11 and the work video analysis device 21, with the standard work time previously stored in the standard work time DB 22. The quality proficiency is calculated from the results of inputting the number of quality defects caused by the worker that occurred in the assembly and inspection processes during manufacturing in the assembly process quality defect input unit 31 and inputting the number of quality defects caused by the worker that occurred in the final inspection process in the final process quality defect input unit 34. Furthermore, the overall work proficiency is calculated from the sum of the work time proficiency and the quality proficiency."

For example, Patent Document 2 discloses a technology in which "the variance calculation unit 103 of the proficiency assessment device 100 divides three-dimensional posture data showing the three-dimensional posture of the trainee when the trainee attempts a movement multiple times into movement phases based on changes in the position of a specified body part, normalizes the duration of the phase in the three-dimensional posture data of the multiple attempts for each phase, and resamples the normalized three-dimensional posture data at specified intervals. The variance calculation unit 103 calculates a value that quantitatively represents the variance of the movement among multiple attempts from one or more of the variance of the duration for each phase in the three-dimensional posture data of the multiple attempts, the variance of the trajectory for each phase of the specified body part obtained from the resampled three-dimensional posture data, and the variance of the posture obtained from the posture matrix in which the three-dimensional posture data resampled for each phase are arranged. The proficiency calculation unit 104 calculates the proficiency based on the value that represents the variance of the movement."

JP 2004-086322 A JP 2015-150226 A

In the above Patent Document 1, proficiency is determined from the working time and the number of defects, but there is no mention of determining the proficiency of a worker from the variance in the movements of the parts of the body that act on the work object. In addition, in the above Patent Document 2, proficiency is determined from the variance in the time series of three-dimensional data, but there is no disclosure of taking into account the variance in the movements when the worker repeats the movements of the parts of the body that act on the work object.

For example, consider a case where a worker repeatedly picks up a tool from a shelf or table used for work setup and manually screws a part onto a steel plate, which is the workpiece being worked on. An experienced worker will repeatedly perform the series of actions of picking up the tool and screwing the part onto the same part at the same time and in the same position (e.g., the same tempo, same hand position), whereas an unskilled worker will often have temporal and spatial variations in their actions. Such variations in actions during repetitive tasks can lead to reduced work efficiency and accuracy. For this reason, it is important to understand the variations in actions during repetitive tasks and support the worker's work so as not to lead to reduced work efficiency and accuracy.

The present invention aims to provide a work support system and a work support method that can understand the variation in movements during repetitive work and support the work of workers without causing a decrease in work efficiency or accuracy.

In order to solve the above problems and achieve the object, the work support system of the present invention comprises a section division unit that uses sensor data of a specific part of a worker performing work on a work object, including repetitive movements, obtained by analyzing image data of the worker, and work sequence information that defines the steps or steps of the work, to output comparative work data that associates the sensor data with the steps or steps of the work, and a section division unit that selects an identification area for the worker to be subjected to proficiency assessment based on the image data and reference area information that defines the work position, which is the position of the work object, or the work position where the worker performs the work, and selects an identification area for the worker to be subjected to proficiency assessment based on the image data, and selects an identification area for the worker to be subjected to proficiency assessment based on the image data and reference area information that defines the work position, which is the position of the work object, or the work position where the worker performs the work ... The work support system is characterized by having an identification area selection unit that matches a worker in the selected identification area with the comparative work data of a specific part of the worker, reference work data that is determined using standard sensor data that serves as a standard for determining the worker's proficiency and the work sequence information, and that matches the standard sensor data with the work process or procedure, and a variance calculation unit that uses the comparative work data of a worker in the selected identification area and the work sequence information to determine the worker's proficiency based on the difference in variance in repetitive actions in the work process or procedure, and presents the result of the determination.

The present invention makes it possible to understand the variation in movements during repetitive tasks and support the worker's work so as not to cause a decrease in work efficiency or accuracy.

FIG. 1 is a diagram illustrating an example of the configuration of a work support system. FIG. 1 is a diagram showing an outline of a computer. FIG. 2 is a diagram showing an example of imaging data including images and videos captured by a sensor. FIG. 11 is a diagram illustrating an example of sensor data. FIG. 11 is a diagram showing an example of comparison task data. FIG. 11 is a diagram showing an example of work sequence information (work process). FIG. 11 is a diagram showing an example of work sequence information (work procedure). FIG. 11 is a diagram showing an example of reference area information. FIG. 11 is a diagram showing an example of specific reference area information. FIG. 13 is a diagram showing an example of time comparison variation data. FIG. 13 is a diagram showing an example of joint position comparison variation data. FIG. 13 is a diagram showing an example of a skill determination result screen. FIG. 13 is a diagram showing an example of a variation determination result details screen. 13 is a flowchart showing a processing procedure of a skill level determination process. FIG. 13 is a diagram illustrating a configuration example of a work support system (embodiment 2). FIG. 11 is a diagram showing an example of comparison operation data (Example 2); FIG. 11 is a diagram illustrating an example of work sequence information (Example 2). FIG. 13 is a diagram showing an example of a skill determination result screen (Example 2); 11 is a flowchart showing a procedure of a skill level determination process according to a second embodiment;

The following describes an embodiment of the present invention with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some parts have been omitted or simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

In the following explanation, various types of information may be explained using expressions such as "database," "table," and "list," but the various types of information may be expressed in data structures other than these. To indicate independence from data structure, "XX table," "XX list," and the like may be referred to as "XX information." When explaining identification information, when expressions such as "identification information," "identifier," "name," "ID," and "number" are used, these are interchangeable.

When there are multiple components with the same or similar functions, they may be described using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

In the following description, the processing performed by executing a program may be described, but the program is executed by a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) to perform a specified process using storage resources (e.g., memory) and/or interface devices (e.g., communication ports) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit that performs specific processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

The program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Also, in the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

1 is a diagram showing an example of the configuration of a work support system 1000 in this embodiment. As shown in FIG. 1, the work support system 1000 uses work sequence information 601, which is data related to the work process and work procedure of the work performed by the worker, to output sensor data of the part (e.g., joint position) where the worker acts on the work object, which is included in the image data output from a sensor S such as a camera, as comparison work data 501, which is time-series work data divided into work processes or work procedures, and uses reference area information 701, which defines the work position, which is the position of the work object and the work position where the worker performs the work, for each work process and work procedure, to filter the part where the worker acts on the work object and identify the worker. The system has an identification area selection unit 200 that selects an identification area for the work support system 1000, a variation calculation unit 300 that calculates the variation in work time and the variation in the parts acting on the work target in the repetitive motion of each work process or each work procedure based on the divided reference work data 501A in which reference work data, which is past time-series work data output from the sensor S, is divided into work processes or work procedures and the comparison work data 501, and outputs the proficiency level and improvement points of the work based on the variation for each work process or each work procedure to a display device D such as a display, and a work sequence controller 400 that controls each of these units. The specific processing performed by each of these units of the work support system 1000 and the specific configuration of the data used will be described later.

The sensor S and the work support system 1000 are connected via a general communication network. The display device D and the work support system 1000 are connected via a general communication network or by various cables of a specified standard. In the following, the explanation is given on the assumption that one work process is composed of one or more work steps, but one work procedure may be composed of one or more work steps.

The work support system 1000 shown in FIG. 1 can be realized, for example, by a general computer 1600 equipped with a CPU 1601, memory 1602, an external storage device 1603 such as a HDD (Hard Disk Drive), a reading/writing device 1607 for reading and writing information from a portable storage medium 1608 such as a CD (Compact Disk) or USB memory, an input device 1606 such as a keyboard or mouse, an output device 1605 such as a display, a communication device 1604 such as a NIC (Network Interface Card) for connecting to a communication network, and an internal communication line (called a system bus) 1609 such as a system bus that connects these devices, as shown in FIG. 2 (schematic diagram of a computer).

Various data stored in the above system or used for processing can be realized by the CPU 1601 reading it from the memory 1602 or the external storage device 1603 and using it. In addition, each functional unit of the above system (e.g., the section division unit 100, the identification area selection unit 200, the variation calculation unit 300, and the work sequence controller 400) can be realized by the CPU 1601 loading a specific program stored in the external storage device 1603 into the memory 1602 and executing it.

The above-mentioned specific program may be stored (downloaded) in the external storage device 1603 from the storage medium 1608 via the reading device 1607 or from the network via the communication device 1604, and then loaded onto the memory 1602 and executed by the CPU 1601. Alternatively, the program may be directly loaded onto the memory 1602 from the storage medium 1608 via the reading device 1607 or from the network via the communication device 1604, and then executed by the CPU 1601.

The following describes an example in which the work support system 1000 is configured by a single computer, but similar functions may be realized by distributing all or part of these functions across one or more computers, such as a cloud, and communicating with each other via a network.

Figure 3 is a diagram showing an example of imaging data including images and videos captured by the sensor S. As shown in Figure 3, the imaging data 301 records the state of a worker working on a workpiece, which is the work object. For example, in Figure 3, multiple workers W1, W2, and W3 are working within the imaging range of the sensor S. Of these, worker W2 is working on a workpiece WK, which is the work object, placed on a work table T1. In addition, in the imaging data 301, tools TO used for working on the workpiece WK are placed on the setup table T2. Worker W2 selects a tool that is appropriate for the work process from among the multiple tools TO and works on it. Workers W1 and W3 in the imaging data 301 are workers who are not involved in the work on the workpiece WK. The section division unit 100 obtains sensor data on various parts of the worker acting on the work object included in the imaging data 301 by using various techniques such as image analysis that have been known in the past.

Figure 4 is a diagram showing an example of sensor data. The sensor data is data before the comparison work data 501 described below is divided into work processes or work procedures. As shown in Figure 4, the sensor data 401 is output as time-series work data including sensor values of various parts acting on the work object (e.g., hand joints, elbow joints). In this embodiment, joint parts are shown as examples of parts of the worker acting on the work object, but this is not limited to this, and other parts such as the upper arm, forearm, thigh, and lower leg may also be parts of the worker acting on the work object.

Figure 4 shows, for example, sensor data when worker W2 shown in Figure 3 moves the joints of his hand, elbow, and shoulder to perform work on the work object WK. In this case, hand sensor data 401a, elbow sensor data 401b, and shoulder sensor data 401c are plotted on a graph with time t on the horizontal axis and sensor value on the vertical axis. In this way, the sensor data stores the sensor values for the parts acting on the work object in chronological order. The sensor data is output for each worker included in the imaging data 301.

Figure 5 is a diagram showing an example of comparison work data. The comparison work data is data obtained after dividing the above-mentioned sensor data 401 into work processes or work procedures. As shown in Figure 5, the comparison work data 501 associates the sensor data 401 with work sequence information 601. In Figure 5, for example, the sensor data 401 divided into work processes P1 to P9 is stored as the comparison work data 501. In this way, the comparison work data 501 stores the sensor data 401 and the above-mentioned work sequence information 601 in association with each other.

Figures 6A and 6B are diagrams showing an example of work sequence information 601. Work sequence information 601 includes work process sequence information in which standard work times are defined for each work process, and work procedure sequence information in which standard procedure times are defined for procedures included in each work process.

As shown in FIG. 6A, in the work process sequence information 601A, the standard work time and process content are defined for each work process. The section division unit 100 can detect the transition of the sensor value in each work process corresponding to the standard work time by overlaying the work process sequence information 601A with the sensor data 401. In FIG. 6, for example, the standard work time for work process 0 is 18 minutes (the end time of the work finish 18 minutes after the start time of the work 0:00, which is 0:18), and the transition of the sensor value corresponding to the standard work process or standard operating procedure can be detected. In addition, as shown in FIG. 6A, the process content indicating what kind of work is to be performed is stored in association with the standard work time. For example, it is indicated that work a0 and b0 are to be performed in work process 0.

As shown in FIG. 6B, in the work procedure sequence information 601B, a standard procedure time is defined for each work procedure for each work process. For example, the section division unit 100 can detect the transition of the sensor value in the standard work procedure for each work procedure constituting each work process by superimposing the work procedure sequence information 601B and the sensor data 401. In FIG. 6, for example, the procedures constituting work process 0 are procedure A to procedure K, and the standard procedure time for procedure A is 2 minutes (the end time of the work end 2 minutes after the start time of the work 0:00, which is 0:02), and the transition of the sensor value corresponding to the standard procedure section can be detected. Also, as shown in FIG. 6B, the procedure contents indicating what kind of work is to be performed are stored in association with the standard work time. For example, in work procedure A, it is indicated that procedures a0 and b0 are performed.

As described below, the identification area selection unit 200 acquires the above-mentioned comparison task data 501 and the reference area information 701, and selects an identification area for identifying the worker. More specifically, the identification area selection unit 200 acquires the comparison task data 501 for each of the workers W1, W2, and W3 included in the imaging data 301, which was created by the section division unit 100. Furthermore, the identification area selection unit 200 compares the imaging data 301 with the reference area information 701, selects the overlapping range of both as an identification area, and sets the comparison task data 501 for the worker included in the selected identification area (worker W2 in this example) as the data to be used for proficiency assessment for the worker.

Furthermore, the identification area selection unit 200 acquires the divided reference work data 501A stored in advance as a reference and the reference area information 701, and selects an identification area for identifying the worker. More specifically, the identification area selection unit 200 acquires the reference work data 501A for the expert contained in the past image data 301 created by the section division unit 100. Furthermore, the identification area selection unit 200 compares the past image data 301 with the reference area information 701, selects the overlapping range of both as an identification area, and sets the reference work data 501A for the expert contained in the selected identification area as the data to be used for proficiency assessment for the expert.

The divided reference work data 501A is data in which reference work data obtained from the past comparative work data 501 of a predetermined expert is associated with a work process or work procedure. The divided reference work data 501A has the same structure as the comparative work data 501 divided into work processes or work procedures, so a detailed explanation is omitted here.

In this embodiment, the divided reference work data 501A of the expert is used, but instead, the divided reference work data 501A of the worker himself may be used. In this case, if the past comparison work data 501 of the same worker is used as the divided reference work data 501A, it becomes possible to confirm on the proficiency assessment result screen how much the worker to be assessed for proficiency has grown compared to past work data and whether there is any variability in repetitive actions. The user may specify whether to use the divided reference work data 501A of the expert or the divided reference work data 501A of the worker himself.

Here, we will explain repetitive actions. Repetitive actions refer to actions that are repeated in time and/or space, such as actions when the same work procedure is performed multiple times in succession in a certain work process, or actions when the same task is performed multiple times in a certain work procedure. For example, consider a case where a series of actions is repeated, such as picking up a tool from a shelf or table and screwing a part, which are used to set up a work process. In this case, the action of picking up a tool and screwing a part is performed every few seconds in the space between the position of the worker, the position of the tool, and the position of the part, so the action can be said to be repetitive.

The work sequence information 601 shown in Figs. 6A and 6B includes the repetitive actions described above. For example, if a procedure is one in which a worker repeatedly picks up a tool from a shelf or table for setting up the work and manually screws a part onto a steel plate, the workpiece to be worked on, the procedure is performed multiple times in succession (for example, two procedures, Procedures E and F, shown in Fig. 6B). In such a procedure, an expert repeatedly performs a series of actions, such as picking up a tool and screwing a part, at the same time and in the same position (for example, at the same tempo and in the same hand position), so the temporal and spatial variations in each action are often below a certain standard. On the other hand, an unskilled worker performs unnecessary actions and the like, which causes temporal and spatial variations in the actions compared to an expert. The greater the variation in the repetitive actions in the work, the more the efficiency and accuracy of the work are reduced.

In this embodiment, the system not only detects the variation in the worker's actions in a certain work process or work procedure, but also detects the variation in such repetitive actions, and supports the worker's work so that work efficiency and accuracy are not reduced. What actions are considered repetitive actions is determined in advance. For example, in FIG. 6B, work procedures E and F are the same procedure, and are repeated twice. An example of a work procedure is given below, but the same can be considered when a work process is repeated.

Figure 7 is a diagram showing an example of the reference area information 701. As shown in Figure 7, in the reference area information 701, the work position, which is the position of the work object, and the work position where the worker performs the work are predefined for each work process and work procedure. In Figure 7, as an example, the range in which the worker performs the work in work process 0 is the range (x, y) = (300 to 450, 150 to 400) on the coordinates defined in the imaging range of certain imaging data. The identification area selection unit 200 compares the imaging data 301 shown in Figure 3 with the above-mentioned reference area information 701 defined for each work process and work procedure, and identifies the worker included in the reference area information 701 as the worker to be subject to proficiency judgment. The identification area selection unit 200 associates the part (e.g., joint position) corresponding to the sensor data acquired by the section division unit 100 with the worker W2 included in the reference area information 701 for the identified worker. In FIG. 7, each part of the body, including the top of the head, both hands, elbows, shoulders, hips, knees, and ankles, is associated with the body parts 702 of worker W2 that act on the work object.

In this way, the identification area selection unit 200 selects the range in which the workers included in the imaging range of the imaging data 301 perform work for each work process defined as the reference area information 701 as the identification area in which work for that work process is to be performed. In FIG. 7, it can be seen that the coordinate range (x, y) = (300-450, 150-400) in which worker W2 is located among workers W1, W2, and W3, is the range in which work for work process 0 is performed, and an image of that range is output.

Furthermore, the identification area selection unit 200 identifies, among the parts that act on the work target of the worker and are included in the identification area, the above-mentioned parts that have a predetermined positional relationship with the workpiece WK and the work tables T1 and T2 (for example, a relationship within 1 meter, which is a certain distance from the worker) as parts that directly act on the workpiece WK. This will be explained in detail below.

Figure 8 is a diagram showing an example of specific reference area information 801 after the parts of the parts 702 shown in Figure 7, namely the left and right hands, elbows, shoulders, and waist, which are parts that are a certain distance from the workpiece WK or the worktable T1, have been identified as parts that directly act on the workpiece WK. In Figure 8, it can be seen that, of the parts that act on the worker's work target, the parts that are at least a certain distance from the workpiece WK and the worktables T1 and T2 have been identified as parts that directly act on the workpiece WK.

The above-mentioned parts may be identified by, for example, identifying the parts that directly act on the workpiece WK when the parts operated by the worker W2 while performing work change by a certain amount or more. For example, in FIG. 8, assume that the worker W2 performs movements M1 and M2 to perform work process 0. In this case, if the movements of the left and right hands, elbows, shoulders, and waist are greater than those of the top of the head, knees, ankles, etc., and the amount of movement is equal to or greater than a certain threshold, the parts can be identified as the parts that directly act on the workpiece WK.

In Figures 7 and 8, we have described workers who are included in the image data 301 and in the identification area, but the same can be said for skilled workers. That is, as described above, the identification area selection unit 200 identifies skilled workers who are included in the past image data 301 and in the identification area as being parts that directly act on the workpiece WK.

The reference area information 701 shown in FIG. 7 and the specific reference area information 801 shown in FIG. 8 are stored in association with the comparison task data 501 and the divided reference task data 501A shown in FIG. 5. Such association and storage are performed by the identification area selection unit 200. In this way, the identification area selection unit 200 reads out the reference area information 701 and selects an identification area from the imaging data where the worker will work for each work process, and identifies the parts of the worker who works in the selected identification area that directly affect the work as the parts to be subject to proficiency assessment. The identification area selection unit 200 also selects the above identification area and identifies the parts to be subject to proficiency assessment for the skilled workers included in the reference task data 501A in the same way as for the workers.

In this way, the identification area selection unit 200 selects an identification area including a worker to be subjected to the variance calculation, and further specifies the parts of the worker's joints included in the identification area that directly act on the work object as the proficiency judgment target parts. Therefore, the variance calculation unit 300, which will be described later, can reduce noise when calculating the variance using various conventionally known algorithms, including DTW (Dynamic Time Warping). In other words, when comparing the whole body of a worker with the whole body of a skilled worker, the motion of parts that are not related to the work is included, resulting in a lot of noise and reduced accuracy. Therefore, in this embodiment, the parts for proficiency judgment as described above are limited to improve accuracy. In other words, in conventional image analysis, the whole body of the worker is imaged, so there are always points that should be focused on and points that should not be focused on, resulting in a decrease in accuracy, but this system can solve such problems. For example, there are points to focus on in work procedures, such as pre-setup actions, actions to retrieve parts, crimping actions, and post-setup actions. By evaluating the deviations around these points, it is possible to prevent a deterioration in accuracy caused by incorporating variations in the worker's body parts (for example, head, waist, elbows, and other body parts unrelated to the work) into the proficiency evaluation.

As described below, the variance calculation unit 300 uses the comparison task data 501 for the worker included in the selected identification area (in this example, worker W2) and the divided reference task data 501A for the skilled worker, together with the task sequence information 601, to compare the variance in the repetitive actions in each task process and each task procedure, thereby determining the worker's proficiency.

Figure 9 is a diagram showing an example of the variation in work time (time comparison variation data) calculated by the variation calculation unit 300. In Figure 9, the work process or work procedure shown in Figure 5 is omitted.

9, the time comparison variation data 901 shows that there is a time variation in the positional fluctuation (in this case, the positional fluctuation of the shoulder) when performing work including the repetitive movement of the part 401c shown in FIG. 4. Specifically, in the comparative work data 901c (graph at the bottom of the drawing) of the worker's part 401c, the time required to perform the repetitive movement of the work from steps E, F to G in a certain process is the work time TW1. On the other hand, in the comparative work data 901c' (graph at the top of the drawing) of the skilled worker's part 401c, the time required to perform the repetitive movement of the work from steps E, F to G in a certain process is recorded as the work time TW2 in the divided reference work data 501A. Therefore, it is shown that there is a difference ΔT1 in the time variation due to the comparison of the work times between the two. This shows that, for example, the comparative work data 901c takes longer to perform the repetitive movement, and there is a possibility that each of the repetitive movements is wasteful. The variance calculation unit 300 expands or contracts the time comparison variance data in the time axis direction using various conventionally known algorithms, including DTW, and scores the difference in the above-mentioned time variance. In FIG. 9, the time comparison variance data was calculated for a certain work procedure, but the same can be considered for a work process.

In this way, the variance calculation unit 300 calculates the variance (variance in the time direction) of the work time of each action in the repetitive action performed by the worker in the comparative work data of the worker, and calculates the variance (variance in the time direction) of each action in the repetitive action performed by the expert in the divided reference work data of the expert. The variance calculation unit 300 then calculates the difference in the time variance between the two as a score. By calculating such a score and presenting it to the user on the screen as described below, the user can infer the reason for the difference in time variance, such as the worker's lack of preparation, trial and error due to an undefined order of actions, or a small mistake in the middle of an action leading to a retry.

Figure 10 is a diagram showing an example of the variance (joint position comparison variance data) of the parts acting on the work target calculated by the variance calculation unit 300. In Figure 10, as in the case described in Figure 9, the description of the work process or work procedure is omitted.

In FIG. 10, the joint part ratio variation data 1001 shows that there is spatial variation in the positional fluctuation (in this case, the positional fluctuation of the shoulder) when performing work including the repetitive motion of the part 401c shown in FIG. 4. Specifically, the joint position fluctuation is compared in the time width where the waveforms in a certain process match in the comparative work data 1001c (graph at the bottom of the drawing) and the comparative work data 1001c' (graph at the top of the drawing). In FIG. 10, the waveform TW3 of a certain time width in the comparative work data 1001c matches the waveform TW4 of a certain time width in the comparative work data 1001c'. The time width is, for example, the same time width as the time width of steps E and F, which are the repetitive motions shown in FIG. 6B, so it can be seen that steps E and F, which are the repetitive motions, are performed in TW3 and TW4. That is, in the joint part comparison variance data 1001, when the work positions of both are compared using the joint positions in procedures E and F, which are repetitive movements recorded over the same time width, it is shown that there is a positional deviation of the part 401c for a time width ΔT2 at a certain point in time, that is, a spatial variation. Therefore, in this example, when the repetitive movements of procedures E and F are performed, it can be said that the repetitive movements of worker W2 are shifted in position by a time period of ΔT2 compared to the skilled worker. As in the case shown in FIG. 9, the variance calculation unit 300 scores the difference in the spatial variation of the joint position comparison variance data using various conventionally known algorithms including DTW. In FIG. 10, the joint position comparison variance data is calculated for a certain work procedure, but as in the case of FIG. 9, the same can be considered for work processes.

In this way, the variance calculation unit 300 calculates the variance (spatial variance) of the movements of the worker and the worker's body parts for each of the repetitive movements performed by the worker in the comparative work data of the worker, and calculates the variance (spatial variance) of the movements of the worker and the worker's body parts for each of the repetitive movements performed by the expert in the divided reference work data of the skilled worker. The variance calculation unit 300 then calculates the difference in spatial variance between the two as a score. By calculating such a score and presenting it to the user on the screen as described below, it is possible to infer the reason for the difference in spatial variance, such as insufficient preparation by the worker, trial and error due to an undefined order of movements, or small mistakes during the movements that result in unusual movements during production.

The idea behind finding the difference in spatial variation is, for example, to calculate the difference in the coordinate positions of the joint angles between waveforms matched by DTW, and then calculate the coordinate deviation from the matching results. The advantage of calculating the coordinate deviation from the matching results in this way is that even if there are differences in the phase of the waveforms of both data, the start coordinates of the comparison range, or the work speed, if the work is the same, the time difference can be absorbed and the joint position deviation can be generated.

9 and 10, the variance calculation unit 300 calculates time comparison variance data and joint position comparison variance data that indicate the variance when performing work including repetitive motions in a certain work process or work procedure, and displays the proficiency assessment result obtained by scoring the difference in variance between the worker and the expert on a display device such as a display. The variance calculation unit 300 refers to the work sequence information 601 shown in FIGS. 6A and 6B, and outputs the work process or work procedure in which repetitive motions are performed in association with the proficiency assessment result. As will be described below, by presenting the proficiency assessment result to the user, the points in the work process or work procedure where attention should be paid to for work improvement can be selected and indicated by points picked up from the work contents in the work procedure manual.

Figure 11 is a diagram showing an example of a proficiency judgment result screen displayed by the variance calculation unit 300 on a display device such as a display. The proficiency judgment result screen 1101 shown in Figure 11 includes a time variance judgment result display area 1102 that displays the proficiency judgment result for the time comparison variance data 901 shown in Figure 9, and a joint part variance judgment result display area 1103 that displays the proficiency judgment result for the joint part comparison variance data 1001 shown in Figure 10.

More specifically, in the time variation judgment result display area 1102, the judgment result of the proficiency level based on the difference in time variation for each of the work processes P1 to P9 performed by the worker is displayed. In this example, the greater the difference in the variation in work time between the skilled worker and the worker (i.e., the more inexperienced), the darker the color, and the smaller the difference in variation (i.e., the more skilled), the lighter the color. For example, in work process P1, the difference in the variation in work time between the skilled worker and the worker is smaller than the difference in the variation in work time between the skilled worker and the worker in work process P6, so it can be seen that it is displayed in a lighter color. Therefore, at a glance, the user can determine that the proficiency level of the worker in work process P6 is low. Also, for example, in FIG. 11, since the judgment result of the proficiency level is displayed in chronological order, for example, if the later work processes are displayed in a darker color, it can be inferred that the difference in variation is due to an increase in work delays caused by the worker's fatigue.

Similarly, in the joint part variation judgment result display area 1103, the judgment result of the proficiency level based on the difference in spatial variation is displayed for each of the work steps P1 to P9 of the work performed by the worker. In this example, the greater the difference in the variation of the joint position between the skilled worker and the worker (i.e., the more inexperienced), the darker the color, and the smaller the difference in the variation (i.e., the more skilled), the lighter the color. For example, in work step P1, the difference in the variation of the joint position between the skilled worker and the worker is smaller than the difference in the variation of the joint position between the skilled worker and the worker in work step P2, so it can be seen that it is displayed in a lighter color. At a glance, the user can determine that the proficiency level of the worker in work step P2 is low. Also, for example, in FIG. 11, since the judgment result of the proficiency level is displayed in chronological order, for example, if the later work steps are displayed in a darker color, it can be inferred that the cause of the difference in the variation is a decrease in the work accuracy due to the worker's fatigue.

Furthermore, the user can check the relationship between the time variation judgment result display area 1102 and the joint part variation judgment result display area 1103 to perform various analyses. For example, in the work process P2 displayed in the time variation judgment result display area 1102, the difference in the variation in the work time with the skilled worker is small, while in the joint part variation judgment result display area 1103 of the same work process P2, the difference in the variation in the joint position with the skilled worker is large, so it can be seen that the accuracy of the work is low. In addition, in the work process P6 displayed in the time variation judgment result display area 1102, the difference in the variation in the work time with the skilled worker is large, while in the joint part variation judgment result display area 1103 of the same work process P6, the difference in the variation in the joint position with the skilled worker is small, so it can be seen that the accuracy of the work is high. In this way, the correlation between the work delay and the work accuracy in each work process or work procedure can be easily grasped. Furthermore, the details of the work process displayed in each area displayed on the proficiency judgment result screen shown in FIG. 11 can be confirmed. The screen for checking the details will be explained later.

Furthermore, the variance calculation unit 300 displays the repeat action display areas 1104 and 1105, which indicate that the work process or work procedure includes a repeat action, in association with the work process or work procedure. In FIG. 11, the variance calculation unit 300 displays "RP2" in the repeat action display areas 1104 and 1105, which indicates that the work process P2 includes a work procedure including a repeat action and that the number of times is two. In this way, the variance calculation unit 300 displays information indicating that the work process or work procedure includes a repeat action in the repeat action display areas 1104 and 1105, so that the user can infer the reason for the variance at a glance. The variance calculation unit 300 may determine which work process or work procedure includes a repeat action by, for example, referring to the work sequence information 601 shown in FIG. 6A and FIG. 6B.

Figure 12 is a diagram showing an example of a variance judgment result details screen showing details of the proficiency judgment result screen shown in Figure 11. When a work process or work procedure displayed in each display area of the proficiency judgment result screen 1101 is selected by the user, the variance calculation unit 300 displays the variance judgment result details screen on a display device such as a display. In the following, as an example, a case where an image of specific reference area information 801 is displayed is illustrated, but reference area information 701 may also be displayed.

In the variation judgment result details screen 1201 shown in FIG. 12, as an example, a detailed screen of the proficiency judgment result in the work process P6 displayed in the time variation judgment result display area 1102 and the joint part variation judgment result display area 1103 is illustrated. In FIG. 12, a reference specific reference area information display area 1202 is displayed, which displays the image or video of the specific reference area information 801 stored in association with the divided reference work data 501A. The slide bar B1 is an adjustment bar that can change the time when the image or video was captured. For example, when the user clicks the right end arrow of the slide bar B1, the variation calculation unit 300 plays the image or video of the skilled worker working in the future. This allows the user to visually check the image or video of the skilled worker working, which changes from moment to moment in the future. When the user clicks the left end arrow of the slide bar B1, the variation calculation unit 300 plays the image or video of the skilled worker working in the past. This allows the user to visually check images and videos of an expert performing work that change from moment to moment over the course of the past. In particular, for repetitive movements, the same movement is performed repeatedly with the same tempo and the same hand position, so by playing back the images and videos, the user can easily understand the variations in each movement in the repetitive movements.

In addition, in FIG. 12, a specific reference area information display area 1203 is displayed, which displays the image or video of the specific reference area information 801 stored in association with the comparison work data 501. The slide bar B2 is an adjustment bar that can change the time when the image or video was captured. For example, when the user clicks the right end arrow of the slide bar B2, the variation calculation unit 300 plays the image or video of the worker working in the future. This allows the user to visually check the image or video of the worker working, which changes from moment to moment in the future. When the user clicks the left end arrow of the slide bar B2, the variation calculation unit 300 plays the image or video of the worker working in the past. This allows the user to visually check the image or video of the worker working, which changes from moment to moment in the past. In particular, for repetitive actions, as in the case of the reference specific reference area information display area 1202, it is easy to understand what kind of variation there is in each action in the repetitive action. The user can then check the difference in variation by comparing the specific reference area information display area 1202 for reference with the specific reference area information display area 1203.

In addition, the variation calculation unit 300 displays a difference range D in the reference area information display area 1203, which indicates which range on the image is different, by comparing it with the image in the reference specific reference area information display area 1202. Furthermore, the variation calculation unit 300 displays a marking part M in which the joint part included in the difference range D is marked as a joint part whose proficiency is lower than that of the expert by a certain threshold or more. For example, the variation calculation unit 300 compares the positions of the same joint part (e.g., hand, elbow) on the image at the same time for the image displayed in the reference area information display area 1203 and the image displayed in the reference specific reference area information display area 1202 to identify the difference range D. Then, among the joint parts included in the difference range D, the part whose proficiency is lower by a certain threshold or more (e.g., the part whose position has a large difference in variance between the expert and the worker by a certain threshold or more) is displayed as the above-mentioned marking part M. In FIG. 12, it can be seen that the joint parts of the hand and elbow of the worker W2 are highlighted as the marking part M.

In this way, on the variation judgment result details screen 1201, the reference specific reference area information display area 1202 and the reference area information display area 1203 are displayed on one screen, and the differences between the two images are displayed as the marking area M. Therefore, the user can grasp at a glance which work process or work procedure the worker's work differs from that of an expert. In particular, for repetitive actions, the areas that cause variation in each action in the repetitive action can be easily grasped.

Furthermore, the variance judgment result details screen 1201 displays a delay status graph display area 1204 that shows the extent to which the comparison work data 501 is delayed (or accelerated) in time relative to the divided reference work data 501A. As shown in the delay status graph display area 1204, the horizontal axis is time and the vertical axis is the difference, which is the delay status, and the change in the temporal difference between the comparison work data 501 and the divided reference work data 501A is displayed for each time. In this example, it shows that the work is continuously delayed during a certain time period t1-t2 of a certain work procedure or work process. In this way, the variance calculation unit 300 calculates the temporal difference between the comparison work data 501 and the divided reference work data 501A and displays the calculated difference in the delay status graph display area 1204 as the delay status of the work, so that the user can grasp at a glance the progress of the delay of the work of the worker relative to the skilled worker.

In addition, the variation judgment result details screen 1201 displays a variation status graph display area 1205 that shows the degree of variation in the position of the part of the comparison work data 501 compared to the divided reference work data 501A in a time series. As shown in the variation status graph display area 1205, the horizontal axis is time and the vertical axis is the difference in joint position, and the change in the spatial difference between the comparison work data 501 and the divided reference work data 501A is displayed for each time. In this example, it is shown that the variation in the joint position of a certain joint position (finger joint) becomes large at certain times t3 and t4. In this way, the variation calculation unit 300 calculates the spatial difference between the comparison work data 501 and the divided reference work data 501A for each time, and displays the calculated difference in the variation status graph display area 1205 as the degree of variation in the position of the work, so that the user can grasp at a glance the change in the variation in the position of the part of the worker compared to the skilled worker.

In this way, on the variation judgment result details screen 1201, the delay status graph display area 1204 and the variation status graph display area 1205 are displayed on a single screen. Therefore, the user can grasp at a glance the difference in temporal variation and the difference in spatial variation in a certain work process or work procedure. Furthermore, by displaying the delay status graph display area 1204 and the variation status graph display area 1205 on a single screen together with the above-mentioned reference specific reference area information display area 1202 and reference area information display area 1203, the user can easily grasp the correlation between them.

In addition, if a worker's work process or procedure includes repetitive actions, the user can easily understand what kind of variability has occurred in the repetitive actions by comparing the specific reference area information display area 1202 for reference and the reference area information display area 1203 on the variability judgment result details screen 120. For example, if work process P2 includes two repetitive actions, the user can check in detail the delay status of the first and second actions by worker W2, the progress of variability in parts between the first and second actions, etc.

Next, the processing performed by the work support system 1000 will be described. FIG. 13 is a flowchart showing the processing steps of the processing (proficiency assessment processing) performed by the work support system 1000. In the proficiency assessment processing, first, the section division unit 100 receives image data 301 output from a sensor S such as a camera, and acquires sensor data 401 about the worker contained in the received image data 301 (S1301). The section division unit 100 outputs comparison work data 501 that associates the acquired sensor data 401 with the work sequence information 601 shown in FIGS. 6A and 6B (S1302).

Then, the identification area selection unit 200 uses the comparison task data 501 and the reference area information 701 to select an identification area for identifying the worker, and also uses the divided reference task data 501A and the reference area information 701 to select an identification area for identifying the expert (S1303). The identification area selection unit 200 identifies the parts of the worker and expert who perform the task in the selected identification area that directly affect the work as the parts to be subject to proficiency assessment (S1304).

The variance calculation unit 300 uses the comparative work data 501 for the worker included in the selected identification area, the divided reference work data 501A for the expert included in the selected identification area, and the work sequence information 601 to determine the proficiency of the worker relative to the expert based on the difference in variance in each work process and work procedure, and the difference in variance in the repetitive actions included therein (S1305), and displays the proficiency determination result screen shown in FIG. 11 (S1306). That is, the identification area selection unit 200 identifies, among the specified parts of the worker in the selected identification area, the parts that have a specified positional relationship with the work object as parts that directly act on the work object, and the variance calculation unit 300 determines the proficiency of the worker based on the difference in variance in the repetitive actions for the specified parts that directly act, and presents the result of the determination.

In this way, in this embodiment, the section division unit 100 uses sensor data 401 of a specific part (e.g., a joint part) of the worker obtained by analyzing image data 301 capturing an image of a worker performing work on a work object, including repetitive movements, and work sequence information 601 defining the steps or steps of the work to output comparison work data 501 in which the sensor data 401 is associated with the steps or steps of the work, and selects an identification area for a worker to be subjected to proficiency assessment based on the image data 301 and reference area information 701 defining the work position, which is the position of the work object, or the work position where the worker performs the work, and the image data 301, and then selects an identification area for the worker to be subjected to proficiency assessment, and determines the location of the worker in the selected identification area and the location of the worker. The system includes an identification area selection unit 200 that matches the comparative task data of a fixed area with the reference sensor data (e.g., sensor data 301 of an expert) that is a standard for determining the proficiency of the worker, reference task data (e.g., divided reference task data 501A of an expert) that matches the reference sensor data with the process or procedure of the task, which is determined using the work sequence information 601 and the reference sensor data (e.g., sensor data 301 of an expert), and a variance calculation unit 300 that uses the comparative task data 501 of a worker in the selected identification area and the work sequence information 601 to determine the proficiency of the worker based on the difference in variance of repetitive actions in the process or procedure of the task, and presents the result of the determination.

Therefore, it is possible to grasp not only the difference in the variation in the movements of the worker in each work process or work procedure compared to an expert worker, but also the difference in the variation in the movements in repetitive work, and to support the work of the worker so as not to cause a decrease in work efficiency or work accuracy. That is, based on the imaging data 301 acquired by the imaging unit and the work sequence information 601 indicating the work process or work procedure, the identification area of the worker is selected, the proficiency is judged from the difference in the degree of variation in the joint positions related to the repetitive movements of the worker in the identification area, and the judgment result is output as an improvement point after identifying the work defined in the work sequence information, so that the above-mentioned support is possible. Conventionally, the proficiency of the worker was not judged taking into consideration the relationship between the positions of parts and products related to the work process or work procedure such as an assembly diagram and the worker's movements, that is, the relationship between the position of the work object on which the worker performs the work according to the work process or work procedure and the worker's movements. However, according to this system, as described above, work support that takes into consideration the variation in movements in the work process or work procedure, including repetitive work, is possible.

In the first embodiment, the work process and the work procedures included in the work process are determined in advance, and the proficiency level is determined by calculating and scoring the difference in the variation of the movements and repetitive movements in these work processes and work procedures, and the result is presented to the user. However, the variation in movements is not necessarily limited to those explicitly included in the work process and work procedures. For example, when repeating a series of movements such as picking up a tool and screwing a part, the difference in the movements of the preparation before picking up the tool (for example, the movement of placing multiple tools in the same direction before work) between an experienced worker and an unskilled worker may cause variation in the subsequent movements. Therefore, in the following, by determining the proficiency not only in the movements included in the work process and work procedures, but also in the approach movements, which are movements for preparation performed between the work processes and work procedures, it is possible to present the user with more objective judgment materials for the variation in movements. As described above, the approach movements are movements in preparation for starting work, but the approach movements may include repetitive work as in the case of the first embodiment. For example, if the action of placing multiple tools in the same direction before starting work is repeated, this action can be considered an approach action that includes a repetitive action.

The work support system 2000 in Example 2 differs from the work support system 1000 in Example 1 in that it has comparison work data 2001, divided reference work data 2001A, and work sequence information 2002 that are different from the comparison work data 501, divided reference work data 501A, and work sequence information 601 in Example 1. Therefore, the following mainly describes these differences from Example 1, and the other components are given the same reference numerals and their descriptions are omitted.

Figure 14 is a diagram showing an example of the configuration of a work support system 2000 in this embodiment. As shown in Figure 14, the work support system 2000 has comparison work data 2001, divided reference work data 2001A, and work sequence information 2002 that are different from those in the first embodiment.

Figure 15 is a diagram showing an example of the comparative work data 2001 divided into work steps or work procedures. As shown in Figure 15, the comparative work data 2001 is data in which the sensor data 401 is associated with the work sequence information 2002, which is data on the work steps and work time of the work performed by the worker. Unlike the case of Example 1, the comparative work data 2001 in Figure 15 includes a predetermined spare time between a certain work step or work procedure and the next work step or work procedure. For example, Figure 15 shows that the spare time P12 is included between work steps P1 and P2, and the spare time P56 is included between work steps P5 and P6. In this way, the sensor data 401 divided into the sections of work steps P1 to P9, including the spare time P12 and the spare time P56, is stored as the comparative work data 2001, and the comparative work data 2001 stores the sensor data 401 and the above-mentioned work sequence information 2002 in association with each other. As in the first embodiment, the divided reference task data 2001A has the same structure as the comparison task data 2001 divided into task steps or task procedures, and so a detailed description is omitted here. Although FIG. 15 illustrates an example in which there is spare time between task steps, the same can be considered when there is spare time between task procedures.

Figure 16 is a diagram showing an example of work sequence information 2002. Work sequence information 2002 includes work process sequence information in which a standard work time is defined for each work process, and work procedure sequence information in which a standard procedure time is defined for the procedures included in each work process.

As shown in FIG. 16, in the work process sequence information 2002A, a standard work time is set for each work process. The section division unit 100 can detect the transition of the sensor value in each work process corresponding to the standard work time by overlapping the work process sequence information 2002A with the sensor data 401. Unlike the case of the first embodiment, FIG. 16 shows that there is a two-minute reserve time P12 between work process 1 and work process 2, and there is a two-minute reserve time P56 between work process 5 and work process 6. In other words, it shows that the reserve time is the difference between the end time of the previous work process and the start time of the next process. The section division unit 100 detects the transition of the sensor value in the reserve time as in each work process. During such reserve time, for example, an action is performed in preparation before starting work, such as arranging multiple tools in the same direction before work. Although not shown below, the reserve time between work processes may be considered in the same way for the work process sequence information 2002B as for the work process sequence information 2002A.

Figure 17 is a diagram showing an example of a proficiency judgment result screen displayed by the variance calculation unit 300 on a display device such as a display. The proficiency judgment result screen 1701 shown in Figure 17 includes a time variance judgment result display area 1702 similar to the time variance judgment result display area 1102 for the time comparison variance data 901 shown in Figure 9, and a joint part variance judgment result display area 1703 similar to the joint part variance judgment result display area 1103 for the joint part comparison variance data 1001 shown in Figure 10.

In FIG. 17, unlike the first embodiment, the difference in time variation in the reserve time P12 and the reserve time P56, and the difference in joint part variation are displayed. It can be seen that the reserve time P12 and the reserve time P56 are both displayed in dark colors because the difference in variation is large. In this way, in this embodiment, the proficiency judgment results are displayed for the approach movement, which is an operation performed in the reserve time between work steps P1 and P2 and between work steps P5 and P6, and for the repeating operation if the approach movement includes a repetitive operation. Therefore, the proficiency is judged for these operations performed in the reserve time for which the process content or procedure content is not explicitly defined in the work process sequence information 601, and the results are displayed. In FIG. 17, the variation calculation unit 300 displays the approach movement (AP12) in the same way as the repeating operation (RP2) so that it can be visually recognized as an approach movement.

The variance calculation unit 300 displays the approach motion display areas 1704 and 1705, which indicate that the approach motion is included in the reserve time between the work steps or work procedures, in association with the reserve time. In FIG. 17, the variance calculation unit 300 displays "AP12" in the approach motion display areas 1704 and 1705, which indicates that the reserve time between work steps P1 and P2 includes the approach motion. The approach motion "AP56", which is the reserve time between work steps P5 and P6, is also displayed in the same manner. If the approach motion includes a repetitive motion, these can be displayed side by side. In this way, the variance calculation unit 300 displays information indicating that the approach motion performed between the work steps or work procedures is included in the approach motion display areas 1704 and 1705, so that the user can see at a glance that there is a difference in the variance in the motion in the reserve time where no specific work content is specified. Although not shown below, the variation judgment result details screen 1201 shown in FIG. 12 is also displayed for the approach movement during the above-mentioned reserve time, as in the case of Example 1.

Figure 18 is a flowchart showing the procedure of the process (proficiency assessment process) performed by the work support system 2000. In the proficiency assessment process in this embodiment, the processes from S1301 to S1304 are performed as in the first embodiment. After that, the variance calculation unit 300 uses the comparison work data 501 for the workers included in the selected identification area, the divided reference work data 501A for the skilled workers included in the selected identification area, and the work sequence information 2002 to determine whether there is spare time between work processes or work procedures (S1801). For example, the variance calculation unit 300 refers to the work sequence information 2002 and determines whether there is discontinuous time between each work process or each work procedure, thereby determining whether there is spare time between those work processes or work procedures.

When the variance calculation unit 300 determines that there is a discontinuous time between each work process or work procedure and that there is spare time between those work processes or work procedures (S1801; Yes), it determines the proficiency of the worker relative to an expert based on the difference in variance in the approach movement of the spare time, the difference in variance in the repetitive movements included in the approach movement, the difference in variance in each work process or work procedure, and the difference in variance in the repetitive movements included therein (S1802).

On the other hand, if the variance calculation unit 300 determines that there is no discontinuous time between each work process or each work procedure and that there is no spare time between those work processes or work procedures (S1801; No), it performs the same processing as S1305 in Example 1.

The variation calculation unit 300 displays the result determined in S1802 or S1305 as the proficiency determination result screen shown in FIG. 17 (S1803).

In other words, the identification area selection unit 200 identifies, among the specified body parts of the worker in the selected identification area, those parts that have a specified positional relationship with the work object as parts that directly act on the work object, and the variance calculation unit 300 judges the proficiency of the worker based on the difference in variance in the approach movement for the identified directly acting parts, and presents the result of the judgment.

In this way, in this embodiment, the section division unit 100 outputs the comparison work data 2001 using the work sequence information 2002 including the spare time between the work steps or procedures and the sensor data 401, and the identification area selection unit 200 uses the reference work data 2001A determined using reference sensor data (e.g., the sensor data 301 of an expert) that serves as a standard for determining the proficiency of the worker and the work sequence information 2002 including the spare time to determine the proficiency of the worker based on the difference in variance in the approach movements for preparation performed by the worker during the spare time between the work steps or procedures, and presents the result of the determination.

Therefore, even if there is an approach motion between work processes or work steps, the judgment result of the proficiency of the approach motion can be presented. For example, in the case of repeating a series of actions of picking up a tool and screwing a part, if the step of "screwing the part" is defined in the work sequence information, but there is a time that is discontinuous with the previous work process or work step, that time is determined to be preparation time, and the proficiency of the worker is judged for the approach motion at that time (for example, an action taken when preparing before picking up a tool, such as placing multiple tools in the same direction before work) and even for that repeated motion, and the judgment result is output. Therefore, it is possible to grasp the worker's proficiency and the difference in motion from an expert for the approach motion during the preparation time to perform a task defined as a work process or work step.

Although each embodiment has been described above, various modifications are possible as shown below. For example, in the above embodiment, the divided reference work data 501A of the expert is used as reference data for determining the proficiency of the worker. However, since the body structure and skeleton are different between the expert and the worker, in order to improve accuracy, the divided reference work data 501A of the expert may be compared with the comparative work data 501 of the worker after skeleton correction. For example, the section division unit 100 compares the skeleton obtained by image analysis of the worker included in the imaging data 301 with the skeleton obtained by image analysis of the expert included in the past imaging data 301, and calculates the degree of enlargement or reduction of the image or video according to the magnitude of the difference in the same joint position. The area selection unit 200 corrects the position of the joint part of the worker included in the identification area according to the calculated degree of enlargement or reduction, and further identifies the part that directly acts on the work object among the joint parts at the corrected position as the part to be subjected to proficiency judgment. That is, the section division unit 100 corrects the position of a specific part of the expert included in the image data 301 that is the analysis source of the reference sensor data (e.g., the sensor data 301 of the expert) that is the basis for determining the proficiency of the worker to the position of the specific part of the worker obtained by analyzing the image data 301, and the variance calculation unit 300 uses the reference work data 2001A determined using the reference sensor data after the correction and the work sequence information 2002 to determine the proficiency of the worker based on the difference in variance of the repetitive movements in the process or procedure of the work, and presents the result of the determination. In this way, it is possible to identify the part for proficiency determination taking into account the difference in bone structure between the expert and the worker, and present the result of the proficiency determination.

Furthermore, in each of the above-mentioned embodiments, the divided reference work data 501A of the skilled worker is assumed to be the work data of the skilled worker at a certain point in the past. However, the same skilled worker will not necessarily be better than the worker in the future. Therefore, if the proficiency level of the worker determined in S1305 of FIG. 13, S1802 or S1305 of FIG. 18 is superior to that of the worker (for example, the score of the difference in variance is negative), the variation calculation unit 300 may register the comparative work data 501 of the worker as the divided reference work data 501A of the new skilled worker. In other words, if the proficiency level of the worker is superior to that of the skilled worker as a result of determining the proficiency level of the worker based on the difference in variance, the variation calculation unit 300 registers the comparative work data 501 of the worker determined to be superior as the new reference work data 501A. As a result, if the work level of the worker exceeds the work level of the skilled worker, the comparative work data of the worker can be automatically registered as reference data.

In addition, in each of the above-mentioned embodiments, the divided reference work data 501A of the expert is explained on the premise that it is the work data of the expert at a certain point in the past. However, one expert is not necessarily good at all work processes and work procedures, and there are cases where expert A is good at one task and expert B is good at another task. Therefore, the variation calculation unit 300 determines which work process or work procedure the worker is better at (for example, the score of the difference in variation is negative) from the proficiency determined in S1305 of FIG. 13 and S1802 or S1305 of FIG. 18, and updates the data of the work process or work procedure determined to be superior in the divided reference work data 501A using the data of the work process or work procedure determined to be superior. If the proficiency of the worker is judged based on the difference in variance and the worker is found to be more skilled than the expert, the variance calculation unit 300 uses the data on the work process or work procedure judged to be superior (e.g., data on work process P2 included in FIG. 5 or FIG. 15) to update the data on the work process or work procedure judged to be superior among the reference work data (e.g., data corresponding to work process P2). As a result, when the work level of the worker exceeds the work level of the expert, the data is updated for each work process or work procedure, and new reference data that, for example, results in the shortest work time can be automatically registered.

Furthermore, the variation calculation unit 300 may register the comparative work data 501 of the worker as the divided reference work data 501A of the skilled worker when the work process or work procedure corresponding to the comparative work data 501 of the worker and the work time of the work process or work procedure corresponding to the divided reference work data 501A of the skilled worker are within a predetermined range (for example, ±20%) with respect to the MOD value calculated by the MODAPTS (modular arrangement of predetermined timestandards) method. That is, the variation calculation unit 300 may register the comparative work data 501 of the worker as the reference work data 501A of the skilled worker when the work time of the work process or work procedure corresponding to the comparative work data 501 of the worker calculated by the MODAPTS method is within a predetermined range of the MOD value as a result of judging the proficiency of the worker based on the difference in the variation. In this case, the divided reference work data 501A of the skilled worker can be asymptotically approached to the data of the work time calculated by the PTS method.

In the above examples, the positions of the work table and the workpiece are fixed. However, in production lines, the positions of these items are within a certain range (e.g., 3 meters square), and the range is somewhat fixed. The same ideas as in the above examples can also be applied.

1000, 2000 Work support system 100 Section division unit 200 Identification area selection unit 300 Variation calculation unit 400 Work sequence controller 501, 2001 Comparison work data 501A, 2001A Divided reference work data 601, 2002 Work sequence information 701 Reference area information

Claims (9)

a section division unit that uses sensor data of a predetermined part of a worker obtained by analyzing image data of a worker performing a task on a work object, including a repetitive motion, and task sequence information that defines a process or procedure of the task, to output comparative task data in which the sensor data is associated with the process or procedure of the task;
an identification area selection unit that selects an identification area of a worker whose proficiency level is to be determined based on the image data and reference area information that defines a work position, which is the position of the work object, and a work position where the worker performs work, and associates the worker in the selected identification area with the comparative work data of a predetermined part of the worker;
a variance calculation unit that uses reference work data, which is determined using reference sensor data that serves as a standard for judging the proficiency of the worker and the work sequence information, to associate the reference sensor data with a process or procedure of the work, the comparative work data of a worker in the selected identification area, and the work sequence information, to judge the proficiency of the worker based on a difference in variance of repetitive actions in the process or procedure of the work, and presents the result of the judgment;
A work support system comprising: The section dividing unit outputs the comparison work data using the work sequence information including spare times between the steps or procedures of the work and the sensor data,
the identification area selection unit uses the reference task data determined using the reference sensor data and the task sequence information including the spare time to determine the proficiency of the worker based on a difference in variation in a preparatory approach motion performed by the worker during the spare time between the steps or procedures of the task, and presents a result of the determination.
2. The work support system according to claim 1 . the identification area selection unit identifies, among predetermined body parts of the worker in the selected identification area, a body part that has a predetermined positional relationship with the work object as a body part that directly acts on the work object;
The variation calculation unit determines a proficiency level of the worker based on a difference in variation of the repetitive motion for the specified directly acting part, and presents a result of the determination.
2. The work support system according to claim 1 . the identification area selection unit identifies, among predetermined body parts of the worker in the selected identification area, a body part that has a predetermined positional relationship with the work object as a body part that directly acts on the work object;
The variation calculation unit determines a proficiency level of the worker based on a difference in variation of the approach motion for the specified directly acting part, and presents a result of the determination.
3. The work support system according to claim 2. the section dividing unit corrects a position of a predetermined part of the expert included in the imaging data that is the basis for analyzing the reference sensor data to a position of the predetermined part of the expert obtained by analyzing the imaging data;
the variation calculation unit uses the reference work data determined using the corrected standard sensor data and the work sequence information to determine the proficiency of the worker based on a difference in variation of a repetitive motion in a process or procedure of the work, and presents a result of the determination.
2. The work support system according to claim 1 . When the proficiency of the worker is superior to that of an expert as a result of judging the proficiency of the worker based on the difference in the variance, the variation calculation unit updates the data of the work process or work procedure judged to be superior among the reference work data using the data of the work process or work procedure judged to be superior.
2. The work support system according to claim 1 . When the proficiency of the worker is superior to that of the expert as a result of judging the proficiency of the worker based on the difference in the variance, the variation calculation unit registers the comparative task data of the worker judged to be superior as new reference task data.
2. The work support system according to claim 1 . The variance calculation unit, as a result of judging the proficiency of the worker based on the difference in the variance, registers the comparative work data of the worker as reference work data of a skilled worker when the work time of the work process or work procedure corresponding to the comparative work data of the worker, calculated by the MODAPTS method, falls within a predetermined range of MOD values.
2. The work support system according to claim 1 . A work assistance method for assisting a worker in a work, which is carried out by a computer, comprising the steps of:
the section dividing unit uses sensor data of a predetermined part of the worker obtained by analyzing image data of a worker performing a task on a work object, including a repetitive motion, and task sequence information that defines a process or procedure of the task, to output comparative task data in which the sensor data is associated with the process or procedure of the task;
an identification area selection unit selects an identification area of a worker whose proficiency is to be judged based on reference area information that defines a work position, which is the position of the work object, and a work position where the worker performs work, and the image data, and associates the worker in the selected identification area with the comparison work data of a predetermined part of the worker;
a variation calculation unit uses reference work data, which is determined using reference sensor data serving as a standard for determining the proficiency of the worker and the work sequence information, and which associates the reference sensor data with the work process or procedure, the comparative work data of the worker in the selected identification area, and the work sequence information to determine the proficiency of the worker based on a difference in variation in repetitive actions in the work process or procedure, and presents the result of the determination;
A work support method comprising the steps of:

Images