JP7777936B2 - Work situation analysis system and work situation analysis method - Google Patents

Description

The present invention primarily relates to a work situation analysis system that analyzes the work situation of workers.

Patent Document 1 discloses a field work implementation status management device that creates accurate work record information for a workplace. Specifically, the field work implementation status management device includes a work data acquisition unit, a work location information acquisition unit, a work time information acquisition unit, and an imaging unit. The work data acquisition unit is attached to the worker's equipment or the like and acquires data related to the work. The work location information acquisition unit acquires the location where the work is being performed by communicating with an external server. The work time acquisition unit acquires the time when the work is being performed by communicating with an external server. The imaging unit acquires video information related to the work. The field work implementation status management device creates work record information by associating a collection of multiple pieces of information in different formats with a single task by adding information acquired by the work data acquisition unit, work location information acquisition unit, and work time information acquisition unit based on the video information acquired by the imaging unit.

Japanese Patent Application Laid-Open No. 2008-84068

The method disclosed in Patent Document 1 does not allow the device to determine whether a worker's work is appropriate, so a manager must make the determination based on video footage, etc. Furthermore, video information related to work can be difficult to handle due to the large amount of data.

The present invention was made in consideration of the above circumstances, and its main purpose is to provide a work status analysis system that can automatically determine whether a worker's work is appropriate while minimizing the amount of data handled.

The problem that this invention aims to solve is as described above. Next, we will explain the means for solving this problem and its effects.

According to a first aspect of the present invention, there is provided a work status analysis system having the following configuration. That is, the work status analysis system includes a communication device and a processing device. The communication device acquires photographic data capturing the work status of a worker. The processing device analyzes the work status based on the photographic data. The processing device has a work estimation model constructed by performing machine learning using still images capturing the work of the worker and the work indicated in the still images as training data. The processing device inputs still images based on the photographic data acquired by the communication device into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device performs second work estimation processing to correct the first work estimation data so that estimation results for the same work are consecutive, thereby creating second work estimation data. The processing device performs work order estimation processing to estimate the order of work based on the second work estimation data. The processing device performs a determination process to determine whether the order of tasks performed by the worker is appropriate by comparing the order of tasks estimated by the task order estimation process with predetermined criteria for the order of tasks. The processing device performs a work time estimation process to estimate the work time for each task based on the second task estimation data. The processing device determines that the work time for each task is appropriate when the work time for each task estimated by the work time estimation process is equal to or greater than the minimum time and equal to or less than the maximum time of the predetermined work time for each task.
According to a second aspect of the present invention, there is provided a work status analysis system having the following configuration. That is, the work status analysis system includes a communication device and a processing device. The communication device acquires photographic data capturing the work status of a worker. The processing device analyzes the work status based on the photographic data. The processing device has a work estimation model constructed by performing machine learning using still images capturing the work of the worker and the work indicated in the still images as training data. The processing device inputs still images based on the photographic data acquired by the communication device into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device performs second work estimation processing to correct the first work estimation data so that estimation results for the same work are consecutive, thereby creating second work estimation data. The processing device performs work order estimation processing to estimate the order of work based on the second work estimation data. The processing device performs a determination process to determine whether the order of the tasks performed by the worker is appropriate by comparing the order of the tasks estimated by the task order estimation process with predetermined criteria for the order of the tasks. When the estimation result of the task estimation model, which uses a still image based on the photographic data acquired by the communication device as input, cannot be classified into any of the tasks learned in advance, the processing device determines that the task corresponds to other tasks.

According to a third aspect of the present invention, there is provided a work status analysis method in which a computer performs the following processes : Acquire photographic data capturing the work status of a worker; Estimate the work indicated by the still images by inputting still images based on the photographic data into a work estimation model constructed by inputting still images of the worker's work and the work indicated by the still images as training data; Perform a first work estimation process to create first work estimation data that chronologically arranges the work estimation results; Perform a second work estimation process to create second work estimation data by correcting the first work estimation data so that estimation results for the same work are consecutive; Perform a work order estimation process to estimate the order of work based on the second work estimation data; Perform a determination process to determine whether the order of work performed by the worker is appropriate by comparing the order of work estimated by the work order estimation process with predetermined criteria for the order of work; Perform a work time estimation process to estimate the work time for each work based on the second work estimation data. If the task time for each task estimated in the task time estimation process is equal to or greater than the minimum task time and equal to or less than the maximum task time determined in advance, the task time for that task is determined to be appropriate.
According to a fourth aspect of the present invention, there is provided a task status analysis method in which a computer performs the following processes : Acquire photographic data capturing the task status of a worker; Infer the tasks represented by the still images by inputting still images based on the photographic data into a task estimation model constructed by inputting still images of the task performed by the worker and the tasks represented by the still images as training data and performing machine learning; Perform a first task estimation process to create first task estimation data that chronologically arranges the task estimation results; Perform a second task estimation process to create second task estimation data by correcting the first task estimation data so that estimation results for the same task are consecutive; Perform a task order estimation process to estimate the task order based on the second task estimation data; Perform a determination process to determine whether the task order estimated by the task order estimation process is appropriate by comparing the task order estimated by the task order estimation process with predetermined criteria for the task order; If the estimation results of the task estimation model, which inputs still images based on the acquired photographic data, cannot be classified into any of the tasks previously learned, determine that the task corresponds to other task.

As a result, the tasks performed by the worker are estimated using a task estimation model, making it possible to determine the order in which the worker performed their tasks with high accuracy. In particular, by using still images instead of video as input, the amount of data required for machine learning and the amount of data required for the task estimation model can be reduced.

This invention makes it possible to realize a work status analysis system that can automatically determine whether a worker's work is appropriate while minimizing the amount of data handled.

1 is a block diagram showing the configuration of a work situation analysis system according to an embodiment of the present invention; FIG. 10 is a diagram showing a process of constructing an operation estimation model for each process. 10A and 10B are diagrams conceptually showing mapping results obtained by classifying training data through distance learning and mapping results obtained by classifying actual work. 10A and 10B are diagrams showing a process of determining a work situation based on a video captured by a photographing device. FIG. 10 is a diagram showing an example of detecting the occurrence of an omission of work. FIG. 10 is a diagram showing an example of detecting that a work time has exceeded the limit. FIG. 10 is a diagram showing an example of detecting whether an operation is normal.

Next, an embodiment of the present invention will be described with reference to the drawings. The work situation analysis system 1 shown in Figure 1 is a system that automatically determines whether the work performed by a worker is appropriate.

The work situation analysis system 1 can be applied to, for example, work performed in a factory to manufacture or maintain products, specifically, assembly, painting, cleaning, disassembly, application of lubricants, etc. The work situation analysis system 1 is not limited to work performed in a factory, but can be applied to any work performed by people, robots, machine tools, etc.

As shown in Figure 1, the work in this embodiment is divided into three stages: Process A, Process B, and Process C, and each process includes one or more tasks. For example, if Process A is applying grease to a first part, then applying grease to a first location on the first part is Work A-1, and applying grease to a second location on the first part is Work A-2.

The work situation analysis system 1 includes a camera device 11, a video management device 12, and a computer 13.

The camera device 11 is a video camera. The camera device 11 is positioned so that its lens faces the workplace, and captures images of workers performing their work to generate video. The camera device 11 transmits the generated video (captured data) to the video management device 12. Figure 1 shows one workplace and one camera device 11. Alternatively, a camera device 11 may be placed in each of multiple workplaces. The camera device 11 is not limited to a video camera, and may also be a camera that captures still images (captured data).

The video management device 12 is an information processing device such as a PC. The video management device 12 processes the video received from the image capture device 11. For example, the video management device 12 stores the video received from the image capture device 11 and then converts the video into multiple still images. The video management device 12 transmits the multiple still images to the computer 13.

The computer 13 is an information processing device such as a PC, server, or workstation. The computer 13 includes a communication device 21, a processing device 22, and a display device 23. The communication device 21 is a communication module or the like that performs wired and wireless communication. The communication device 21 is capable of communicating with external devices. For example, the communication device 21 can communicate with the video management device 12 to exchange data, with an external server to exchange data, or with a system administrator's terminal via a network. The processing device 22 includes a calculation device such as a CPU and a storage device such as an HDD or SSD. The calculation device executes a program stored in the storage device to perform processing to determine whether the work performed by the worker was appropriate based on still images received from the imaging device 11 or the video management device 12. The display device 23 is a display such as an LCD or organic EL display, and is capable of displaying information related to the analysis system 1. Information related to the analysis system 1 includes, for example, the current operating status of the analysis system 1 and any abnormality occurrences.

The processing device 22 has a task estimation model. The task estimation model is a model constructed through machine learning that takes a still image of the task situation as input and outputs the task content of the worker.

Next, the process for constructing an activity estimation model will be explained with reference to Figures 2 and 3.

In this embodiment, a task estimation model is constructed for each process. First, a video is generated by capturing footage of the worker and workplace performing process A. It is preferable to generate multiple videos to learn various worker movements. The videos may be generated using the imaging device 11 or a separate imaging device.

Next, learning data is generated based on the video. As shown in Figure 2, the learning data is data made up of multiple still images, and is associated with the tasks that the still images represent. For example, by watching the video and specifying the time period during which task A-1 is being performed, learning data for task A-1 can be generated by extracting still images from the video during this time period at predetermined intervals. Similarly, learning data for process A can be generated by specifying the time period during which task A-2 is being performed and the time periods during which other tasks are being performed. Other tasks are tasks other than tasks A-1 and A-2, and are tasks that would normally occur in process A. For example, it could be the work of a worker preparing for task A-2 after task A-1 is completed.

The generated learning data is then input into a machine learning computer to perform machine learning, thereby constructing an operation estimation model for process A. The machine learning computer may be the same as computer 13, or it may be a different computer. The machine learning in this embodiment is supervised learning, and the above-mentioned learning data is the training data. There are no limitations on the format of the machine learning model, but for example, a model using a neural network with a general configuration can be used.

The task estimation model is constructed based on the learning data by analyzing the conditions under which each task in process A is performed, specifically the characteristics and trends of the worker's position and posture, the workpiece's position and posture, and the tool's position and posture.

More specifically, the task estimation model is constructed by performing distance learning based on the characteristics of each task. Distance learning is a technique in which learning data is plotted on virtual coordinates according to the characteristics of each category (in this embodiment, each process) and a range is determined for each category. The position of the virtual coordinates is determined according to the characteristics of the data, and data with similar characteristics are plotted in close positions on the virtual coordinates. In other words, as shown in Figure 3, data representing the same task is plotted within a certain range on the virtual coordinates.

A work estimation model for process A can be constructed in this way. A similar method can be used to construct work estimation models for processes B and C. In this embodiment, a work estimation model is created for each process, but a work estimation model that encompasses multiple processes may also be constructed.

When using a task estimation model to estimate the task of unknown data (specifically, a still image whose task is unknown), the relevant still image is first input into the task estimation model. The task estimation model analyzes the characteristics of this still image and identifies its position on the virtual coordinate system. For example, if the still image is located at or near the center of the range of task A-1, the task estimation model will output a highly likely result that task A-1 is the task. The closer the still image is to the center of the range of task A-1, the higher the probability of task A-1.

On the other hand, if the still image is located between the ranges of task A-1 and task A-2, the difference between the probability of task A-1 and the probability of task A-2 will be small. In this way, when the estimation results output by the task estimation model suggest both the possibility of task A-1 and the possibility of task A-2, the processing device 22 determines that the task indicated by the still image is another task. Specifically, if the difference between the most probable estimation result and the second most probable estimation result is less than a threshold, the processing device 22 determines that the task is another task. This is because if the task closest in the virtual coordinates (the task with the highest probability) is estimated, there is a possibility that the task estimation will be incorrect, which could result in an incorrect determination of the task status. Furthermore, in such cases, it is preferable to perform additional learning as needed. Additional learning is a process of updating the task estimation model by performing additional machine learning using data that is not located within the ranges of either task. By performing additional learning, the range indicated by each task is corrected, thereby improving the accuracy of task estimation.

Next, with reference to Figure 4, we will explain how to use a task estimation model to determine whether the task performed by a worker is appropriate.

The camera device 11 captures images of the worker performing the work, generates a video, and transmits the video to the video management device 12. It is preferable that the orientation and shooting range of the camera device 11 are the same as when the learning data was generated.

Next, the video management device 12 performs image extraction processing on the video received from the image capture device 11 to generate multiple still images. The image extraction processing is a process of cutting out the video at a predetermined time interval to generate still images. The time interval for generating still images may be a fixed value, or the time interval may be changed depending on the situation. For example, if there is little change in the worker and surrounding situation shown in the video, the time interval may be longer than usual. The image capture device 11 may also send still images to the computer 13 at the time interval. In other words, in this case, the image extraction processing can be omitted.

The video management device 12 also associates the still image with the shooting time and process and transmits it to the computer 13. For example, a still image based on a video taken while a worker is performing process A is associated with the shooting time and process A and transmitted to the computer 13.

There are various ways in which the video management device 12 can identify processes, but processes can be identified as follows, for example. For example, if only process A is performed in a certain workplace, the video management device 12 associates process A with a still image based on video captured by a camera 11 located in that workplace. Alternatively, if processes A and B are performed in a certain workplace, the video management device 12 associates process A or process B with a still image based on video, based on a predetermined schedule or a timestamp from a work instruction device. Note that if a work estimation model that encompasses all processes is created, process association is not necessary.

Next, the processing device 22 of the computer 13 performs a first estimation process based on the received multiple still images and their associated shooting times and processes. The first estimation process is a process in which the received still images are input into a task estimation model to obtain an estimation result for the task indicated by the still images. The still images are also associated with shooting times and processes. The processing device 22 inputs the still images into a task estimation model corresponding to the process associated with the still images. Specifically, if the still image is associated with process A, the processing device 22 inputs the still image into the task estimation model for process A. Note that instead of the video management device 12 associating processes with still images, the computer 13 may estimate the task associated with the still images based on the task estimation model, and switch the task estimation model to be applied to the still images depending on the task estimation result.

The processing device 22 creates the first task estimation data shown in Figure 4 based on the estimation results of the task estimation model and the shooting time associated with the still image. The first task estimation data is a plot of the estimated results of the worker's task according to the shooting time. In the example shown in Figure 4, the number of plots per task is around five to make the first task estimation data easier to understand, but in reality, it is likely that a larger number of plots will be required.

Next, the processing device 22 performs a second estimation process. The second estimation process is a process for correcting the first task estimation data. The estimation results for each still image may contain estimation errors. For example, the area surrounded by a dashed square in the first task estimation data shown in Figure 4 is an estimation error. Generally, it is not possible to imagine that a worker will perform a different task for just a moment while performing a certain task.

The processing device 22 performs the following process, for example. First, the processing device 22 selects an arbitrary task and identifies the location where the number of consecutive plots for this task is below a threshold. For example, the processing device 22 selects task A-2 and identifies the location where the number of consecutive plots for task A-2 is 1 (the third plot). Next, the processing device 22 corrects the identified plot to a different task so that the estimation results for the same plot are consecutive. In the example shown in Figure 4, the estimation results before and after the third plot are task A-1, so task A-2 is corrected to task A-1. The processing device 22 performs this process for all tasks. As a result, the estimation results for the same task become consecutive. Note that if the processing device 22 considers whether or not a correction is necessary and determines that a correction is not necessary, it does not perform the correction.

Note that the above-described process is just one example, and other processes may be performed as long as they result in consecutive estimation results for the same task. For example, the processing device 22 selects an arbitrary task and identifies a time period in which the proportion of this task is equal to or greater than a threshold. For example, the processing device 22 selects task A-1 and identifies a time period in which the proportion of task A-1 is equal to or greater than a threshold. In the example shown in Figure 4, four of the first to fifth plots are task A-1, so the first to fifth plots are identified. Next, the processing device 22 identifies an estimation result for the identified time period that differs from the initially selected task and corrects this estimation result to the initially selected task. In the example shown in Figure 4, the third plot is task A-2, so it is corrected to task A-1. This results in consecutive estimation results for the same task.

Next, the processing device 22 performs a task order estimation process and a task time estimation process based on the second task estimation data. The task order estimation process is a process for estimating the order of tasks performed by the worker. Since the second task estimation data contains consecutive estimation results for the same task, the order of tasks performed by the worker can be estimated by arranging the tasks indicated by this series of estimation results.

The task time estimation process is a process for estimating the task time for each task performed by a worker. In the second task estimation data, estimation results for the same task are consecutive, so the task time for each task can be estimated based on the number of consecutive plots for this task and the interval between the capture times of the still images.

The processing device 22 performs a task order estimation process and a task time estimation process to create the judgment data shown in Figure 4. The judgment data lists the tasks performed by the worker in order and describes the task time for each task. Based on the judgment data, the processing device 22 determines whether the tasks performed by the worker were appropriate.

If the processing device 22 determines that the work performed by the worker is inappropriate, it notifies a higher-level control device or a manager's terminal, etc., to that effect. Furthermore, the processing device 22 allows the manager to check still images or videos taken during the time period when the work was determined to be inappropriate, so that the manager can confirm the specific work performed by the worker. Specifically, the processing device 22 may send still images or videos taken during the time period when the work was determined to be inappropriate to the manager's terminal, or may store these still images or videos on a server and send link information for accessing the server to the manager's terminal.

Next, with reference to Figures 5 to 7, we will explain a specific example of determining whether the work performed by a worker is appropriate.

Figures 5 to 7 show the second task estimation data, reference data, judgment data, and judgment results, respectively. Reference data is data that indicates the criteria for determining whether the tasks performed by a worker are appropriate. The procedures and task details in the reference data describe the appropriate order of tasks. The minimum time is the minimum time set for each task. The maximum time is the maximum time set for each task. In other words, if each task is less than the minimum time or exceeds the maximum time, the task performed by the worker is determined to be inappropriate. In the examples shown in Figures 5 to 7, there is only one reference data for each task. However, if, for example, the task order is allowed to be changed, there may be multiple reference data. Specifically, in the example shown in Figure 5, if the task order of tasks A-1 and A-2 is allowed to be changed, reference data that places task A-1 first and reference data that places task A-2 first are used. If the task performed by the worker satisfies at least one reference data, the task is determined to be appropriate.

Figure 5 shows the data for each case where an omission occurs. Steps 4, 5, and 6 in the reference data each contain task A-1, task A-2, and others. However, in the judgment data, task A-1 is followed by others, and task A-2 in step 5 is missing. In this case, the processing device 22 determines that an omission has occurred. Note that when an omission occurs, the priority level is set to "high" because it affects product quality. If the priority level is "high," a notification is immediately sent to the higher-level control device or manager's terminal, as described above. This notification includes information that an "omission" has occurred, which is the reason why the worker's work was determined to be inappropriate.

Figure 6 shows the data for each case when a time overrun occurs. Step 5 in the reference data states that the maximum time for task A-2 is "5 seconds." However, the judgment data states that the task time for step 5 is "6 seconds." In this case, the processing device 22 determines that a time overrun has occurred. Note that tasks A-1 and A-2 shown in Figure 6 are tasks for which the quality of the product will not be affected even if a time overrun occurs, so the time overrun priority level is set to "medium." Therefore, the processing device 22 does not immediately notify the higher-level control device or the manager's terminal, etc., but rather notifies them collectively at a specified notification timing. However, for tasks for which the quality of the product will be affected if a time overrun occurs, the time overrun priority level is set to "high." In this case, when a time overrun occurs, the processing device 22 immediately notifies the higher-level control device or the manager's terminal, etc. This notification includes information about the occurrence of a "time overrun," which is the reason why the worker's work was determined to be inappropriate.

Figure 7 shows the data for cases where the work performed by the worker is appropriate. The order of work and work time of the judgment data in Figure 7 both meet the reference data. Therefore, the processing device 22 determines that the work performed by the worker is appropriate.

In this way, by performing the above-mentioned processing, it is possible to automatically determine whether the work performed by the worker is appropriate. Furthermore, if it is determined to be inappropriate, the reason for the determination (i.e., the items that do not meet the standard data) can also be notified to the administrator. Furthermore, compared to systems that learn from videos, create models, and make judgments based on the videos, the amount of data handled can be reduced.

The processing device 22 stores the judgment data in the processing device 22 or an external server, regardless of whether the work performed by the worker is appropriate or not. This allows the work of the worker to be recorded. Note that still images input into the work estimation model may also be stored in addition to the judgment data. Since still images have a smaller data volume than videos, the amount of data to be stored can be reduced. Furthermore, by storing judgment data, videos, or still images only when the work is determined to be inappropriate, the amount of data to be stored can also be reduced.

As described above, the work status analysis system 1 of this embodiment includes a communication device 21 and a processing device 22, and performs the following work status analysis method. The communication device 21 acquires photographic data capturing the work status of the worker. The processing device 22 analyzes the work status based on the photographic data. The processing device 22 has a work estimation model constructed by inputting still images capturing the work of the worker and the work indicated by the still images as training data and performing machine learning. The processing device 22 performs a first work estimation process by inputting still images based on the photographic data acquired by the communication device 21 into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device 22 performs a second work estimation process by correcting the first work estimation data so that estimation results for the same work are consecutive, thereby creating second work estimation data. The processing device 22 performs a work order estimation process by estimating the order of work based on the second work estimation data. The processing device 22 performs a determination process to determine whether the order of tasks performed by the worker is appropriate by comparing the order of tasks estimated in the task order estimation process with predetermined criteria for the order of tasks.

As a result, the tasks performed by the worker are estimated using a task estimation model, making it possible to determine the order in which the worker performed their tasks with high accuracy. In particular, by using still images instead of video as input, the amount of data required for machine learning and the amount of data required for the task estimation model can be reduced.

In the work status analysis system 1 of this embodiment, if the number of consecutive estimation results for the same task in the first task estimation data is below a threshold, the estimation result is corrected to a different task, making the estimation results for the same task consecutive.

Since it is unlikely that the work a worker performs will change for even a moment, errors in work estimation can be corrected.

In the work status analysis system 1 of this embodiment, the processing device 22 performs a work time estimation process to estimate the work time for each task based on the second task estimation data. The processing device 22 compares the work time for each task estimated in the work time estimation process with a predetermined work time for each task to determine whether the work time for the task performed by the worker is appropriate.

This makes it possible to determine whether the task sequence and task time are appropriate. Therefore, for example, if the task time significantly exceeds the standard time, work efficiency can be improved by identifying and resolving the cause.

In the work status analysis system 1 of this embodiment, the processing device 22 determines that the work time for a task is appropriate if the work time for each task estimated in the work time estimation process is equal to or greater than the minimum time and equal to or less than the maximum time for each predetermined task.

This allows for flexible determination of whether work time is appropriate or not.

In the work status analysis system 1 of this embodiment, if the estimation results of the work estimation model, which uses as input a still image based on photographic data acquired by the communication device 21, cannot be classified into any of the previously learned tasks, the processing device 22 determines that the task falls under other tasks.

If a still image is input and the task estimation model outputs the most likely task as the estimated result, there is a possibility of an erroneous judgment. In such cases, additional learning can be performed as needed to further improve the accuracy of task estimation.

In the work status analysis system 1 of this embodiment, the processing device 22 determines whether the work performed by the worker is appropriate, and if it is determined to be inappropriate, notifies the worker of the reason for the determination that it is inappropriate.

This allows managers to review workers' work while taking into account the reasons for its inappropriateness. This reduces the effort required to review workers' work.

The above describes a preferred embodiment of the present invention, but the above configuration can be modified, for example, as follows:

The processing performed by the video management device 12 may be performed by the imaging device 11 or the computer 13, and the video management device 12 may be omitted.

The imaging device 11 and computer 13 do not need to be located in the same factory; the computer 13 may be located in a facility away from the factory.

In the above embodiment, the processing device 22 performs both the work order estimation process and the work time estimation process, but the work time estimation process may be omitted.

REFERENCE SIGNS LIST 1 work situation analysis system 11 imaging device 12 video management device 13 computer 21 communication device 22 processing device 23 display device

Claims (7)

a communication device for acquiring photographic data of the worker's working status;
a processing device that analyzes the work situation based on the photographed data;
Equipped with
the processing device has a task estimation model constructed by performing machine learning using input of still images of a worker's task and the task shown in the still images as training data,
the processing device performs a first task estimation process to estimate tasks indicated by each still image by inputting still images based on the photographic data acquired by the communication device into the task estimation model, and to create first task estimation data in which task estimation results are arranged in chronological order;
the processing device performs a second task estimation process for correcting the estimation results of the first task estimation data so that estimation results of the same task are continuous, thereby creating second task estimation data;
the processing device performs a task order estimation process to estimate a task order based on the second task estimation data;
the processing device performs a determination process to determine whether the order of the tasks performed by the workers is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks ;
the processing device performs a task time estimation process to estimate a task time for each task based on the second task estimation data;
The processing device determines that the work time for each task is appropriate if the work time for each task estimated by the work time estimation process is greater than or equal to the minimum time and less than or equal to the maximum time of the predetermined work time for each task . a communication device for acquiring photographic data of the worker's working status;
a processing device that analyzes the work situation based on the photographed data;
Equipped with
the processing device has a task estimation model constructed by performing machine learning using input of still images of a worker's task and the task shown in the still images as training data,
the processing device performs a first task estimation process to estimate tasks indicated by each still image by inputting still images based on the photographic data acquired by the communication device into the task estimation model, and to create first task estimation data in which task estimation results are arranged in chronological order;
the processing device performs a second task estimation process for correcting the estimation results of the first task estimation data so that estimation results of the same task are continuous, thereby creating second task estimation data;
the processing device performs a task order estimation process to estimate a task order based on the second task estimation data;
the processing device performs a determination process to determine whether the order of the tasks performed by the workers is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks ;
The processing device is configured to determine that the estimation result of the task estimation model, which receives as input a still image based on the photographic data acquired by the communication device, falls under other tasks when the estimation result cannot be classified into any of the tasks learned in advance . The work situation analysis system according to claim 1 or 2 ,
A work status analysis system characterized in that, when the number of consecutive estimation results for the same work in the first work estimation data is below a threshold, the estimation result is corrected to a different work, thereby making the estimation results for the same work consecutive. The work situation analysis system according to claim 2 ,
the processing device performs a task time estimation process to estimate a task time for each task based on the second task estimation data;
The processing device determines whether the work time for each task performed by the worker is appropriate by comparing the work time for each task estimated by the work time estimation process with a predetermined work time for each task. The work situation analysis system according to any one of claims 1 to 4 ,
The processing device determines whether the work performed by the worker is appropriate, and if it determines that the work is inappropriate, notifies the worker of the reason why it is determined to be inappropriate. The computer acquires the photographic data of the worker's work situation,
A still image of a worker's work and the work shown in the still image are input as training data to a work estimation model constructed by machine learning. A still image based on the photographed data is then input to the work estimation model, and the computer estimates the work shown in the still image.
a computer performs a first task estimation process to create first task estimation data in which task estimation results are arranged in chronological order;
a computer performs a second task estimation process to correct the first task estimation data so that estimation results of the same task are consecutive, thereby creating second task estimation data;
a computer performs a task sequence estimation process to estimate a task sequence based on the second task estimation data;
a computer performs a determination process to determine whether the order of the tasks performed by the worker is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks;
a computer performs a task time estimation process to estimate a task time for each task based on the second task estimation data;
A work status analysis method characterized in that a computer determines that the work time for a task is appropriate if the work time for each task estimated by the work time estimation process is equal to or greater than a predetermined minimum time and equal to or less than a predetermined maximum time for each task. The computer acquires the photographic data of the worker's work situation,
A still image of a worker's work and the work shown in the still image are input as training data to a work estimation model constructed by machine learning. A still image based on the photographed data is then input to the work estimation model, and the computer estimates the work shown in the still image.
a computer performs a first task estimation process to create first task estimation data in which task estimation results are arranged in chronological order;
a computer performs a second task estimation process to correct the first task estimation data so that estimation results of the same task are consecutive, thereby creating second task estimation data;
a computer performs a task sequence estimation process to estimate a task sequence based on the second task estimation data;
a computer performs a determination process to determine whether the order of the tasks performed by the worker is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks;
a computer determining that the estimation result of the task estimation model, which receives as input a still image based on the acquired photographic data, corresponds to other tasks when the estimation result cannot be classified into any of the tasks learned in advance.