JP7836967B2 - Work analysis apparatus and work analysis method - Google Patents

Description

This invention relates to a work analysis device and a work analysis method that use a processor to perform a process of outputting analysis result information regarding a worker's work status based on video footage of the worker's work.

In factories, analyzing workers' work performance and revising work procedures based on the analysis results can improve work efficiency. Therefore, technology that can efficiently analyze workers' work performance is desired. Furthermore, technology that can present the analysis results regarding workers' work performance to users in an easy-to-understand manner is also desired.

Technologies related to these requirements include, conventionally, technologies that use cameras to film workers performing tasks, analyze the footage to visualize the workers' work status, and create Gantt charts that visualize the progress of each process, as well as technologies that present the video footage of the workers to the user (see Patent Document 1).

Japanese Patent Publication No. 2019-16226

Conventional technology allows users to relatively easily understand the work status of workers at each stage of a process by comparing Gantt charts with video footage. However, if the video footage of workers is long, the analysis results of the workers' work status will also be long. Therefore, there is a need for an efficient way to review these long-duration analysis results. However, conventional technology has not taken this need into consideration.

Therefore, the main objective of the present invention is to provide a work analysis device and work analysis method that enable users to efficiently review analysis results, even when the analysis of a worker's work situation takes a long time.

The present invention relates to a work analysis device that uses a processor to perform a process of outputting analysis result information regarding the worker's work status based on video footage of the worker's work. The processor recognizes the object handled by the worker and the actions that constitute the worker's work based on the video footage, and acquires analysis result information including the recognition result of the object and the recognition result of the actions. The device includes a first display unit that displays first analysis result display information that visualizes the analysis result information for a first period, and a second period included in the first period, and the first analysis result display information. The system generates a display screen that includes a second display unit that displays second analysis result display information, which visualizes the analysis result information by expanding the time axis; a first operation unit that selects one of a plurality of predetermined periods included in the first period; and a second operation unit that specifies an arbitrary period from the first period. When an arbitrary period is specified as the second period by the second operation unit, the period corresponding to the second period is highlighted in the first analysis result display information. When a predetermined period is selected as the second period by the first operation unit, the specified state of the second operation unit is set to correspond to that predetermined period .

Furthermore, the present invention relates to a work analysis method in which a processor performs a process to output analysis result information regarding the worker's work status based on video footage of the worker's work, the method includes: a first display unit that recognizes the object handled by the worker and the actions constituting the worker's work based on the video footage, acquires analysis result information including the recognition result of the object and the recognition result of the actions, and displays first analysis result display information that visualizes the analysis result information for a first period, and a second period included in the first period, which is calculated from the first analysis result display information for a time A display screen is generated that includes a second display unit that displays second analysis result display information which visualizes the analysis result information by enlarging the axis, a first operation unit for selecting one of a plurality of predetermined periods included in the first period, and a second operation unit for specifying an arbitrary period from the first period. When an arbitrary period is specified as the second period by the second operation unit, the period corresponding to the second period is highlighted in the first analysis result display information. When a predetermined period is selected as the second period by the first operation unit, the specified state of the second operation unit is set to correspond to that predetermined period .

According to the present invention, a user can view first analysis result display information covering a first period , thereby gaining an overview of the worker's work status over a long period. Furthermore, a user can set a portion of the first period using the first or second operation unit and view second analysis result display information, thereby gaining a detailed view of the worker's work status. This allows the user to efficiently review the long-term analysis results, even when the analysis results of the worker's work status extend over a long period, using the two operation units .

Overall configuration diagram of the work analysis system according to this embodiment Block diagram showing the general configuration of the server. Block diagram showing the contents of the analysis processing performed on the server. Diagram illustrating the status of the work cycle detection process performed on the server. Block diagram showing the contents of the analysis result visualization process performed on the server. Diagram illustrating the analysis results display screen shown on the administrator's terminal. This diagram illustrates the process selection section, target period specification section, work video display section, playback operation section, and work cycle selection section of the analysis results display screen. Detailed diagram showing the analysis results display section of the analysis results display screen. An explanatory diagram showing the key parts of the detailed analysis result display unit in the event of a work error (abnormality). An explanatory diagram showing the key parts of the detailed analysis result display unit in the event of a work error (abnormality). An explanatory diagram showing another example of the detailed analysis results display section. Diagram illustrating the overall analysis results display section of the analysis results display screen. An explanatory diagram showing another example of the overall analysis results display section.

The first invention made to solve the aforementioned problems is a work analysis device that uses a processor to perform a process of outputting analysis result information regarding the worker's work status based on video footage of the worker's work, wherein the processor recognizes the object handled by the worker and the actions constituting the worker's work based on the video footage, acquires analysis result information including the recognition result of the object and the recognition result of the actions, and displays first analysis result display information that visualizes the analysis result information for a first period, and sets a second period included in the first period and displays the first analysis result A display screen is generated that includes a second display unit that displays second analysis result display information, which visualizes the analysis result information by expanding the time axis from the displayed information; a first operation unit that selects one of a plurality of predetermined periods included in the first period; and a second operation unit that specifies an arbitrary period from the first period. When an arbitrary period is specified as the second period by the second operation unit, the period corresponding to the second period is highlighted in the first analysis result display information. When a predetermined period is selected as the second period by the first operation unit, the specified state of the second operation unit is set to correspond to that predetermined period .

According to this system, users can view the first analysis results display information covering a first period , allowing them to get an overview of the workers' work status over a long period. Furthermore, by setting a portion of the first period using either the first or second control unit and viewing the second analysis results display information, users can examine the workers' work status in detail. This allows users to efficiently review even long-term analysis results using the two control units, even when the analysis of workers' work status spans a long period.

Furthermore, the second invention is configured such that the processor outputs the video in a format that allows playback, with the video associated with the analysis result display information.

According to this, users can easily understand the worker's work status by comparing the analysis results display information with the video. In this case, the video may be associated with the second analysis results display information, which presents the worker's detailed work status, so that the video included in the second period is played. Alternatively, for example, a timeline cursor indicating the current playback position may be displayed on the second analysis results display information.

Furthermore, the third invention is configured such that the display screen further includes a period specification unit for inputting the start time and end time of the period in which the video is stored, in order to specify the first period .

According to this, the user can input the start time and end time in text as information about the first period for which the first analysis result display information is to be displayed. In this case, the date may be specified by the user's operation .

Furthermore, the fourth invention is configured such that the first operating unit is a work cycle selection unit that selects one of a plurality of work cycles consisting of a predetermined series of actions included in the first period .

According to this, the analysis results for each cycle of a task consisting of a predetermined series of actions can be easily confirmed by the second analysis result display information.

Furthermore, the fifth invention is the work analysis apparatus according to claim 1, characterized in that the second operating unit is a slider bar for specifying an arbitrary period on the time axis of the first analysis result display information included in the first period .
This will be the structure.

According to this, the user can appropriately specify the second period of the second analysis result display information using a slider bar on the time axis of the first analysis result display information, which covers a broad first period .

Furthermore, the sixth invention is configured such that, upon detecting an abnormality in the worker's work status, the processor outputs the analysis result display information, which includes information regarding the abnormality detection result.

According to this, users can easily identify abnormalities in the worker's work status. In this case, for example, abnormalities related to excesses or deficiencies in actions within a predetermined work cycle, the order of actions, and the duration of the work cycle may be detected.

Furthermore, the seventh invention provides a configuration in which the second analysis result display information includes, as display information relating to the anomaly detection result, at least one of the following: highlighting of the display field for the target object, displaying a mark indicating the behavior related to the anomaly, and displaying characters representing the content of the anomaly.

According to this, users can more easily identify anomalies in the workers' work status. In particular, in the second analysis result display information, which uses a shorter time axis, anomaly detection results are displayed on an action-by-action basis, allowing users to examine the anomaly in detail.

Furthermore, the eighth invention is configured such that the first analysis result display information includes, as display information relating to the anomaly detection result, either a display of the number of anomaly detections detected in the specified period included in the first period , or a highlighting of the specified period in which the anomaly was detected .

This allows users to more easily identify abnormalities in the workers' work status. In particular, in the first analysis result display information, which covers a long time period, the number of abnormality detections within a specified period is displayed , allowing users to easily confirm the status of the abnormality. For example, the number of abnormality detections may be presented for each of the multiple work cycles included in the first period.

Furthermore, the ninth invention is a work analysis method in which a processor performs a process to output analysis result information regarding the worker's work status based on video footage of the worker's work, the method includes: a first display unit that recognizes the object handled by the worker and the actions constituting the worker's work based on the video footage, acquires analysis result information including the recognition result of the object and the recognition result of the actions, and displays first analysis result display information that visualizes the analysis result information for a first period, and sets a second period included in the first period to extend the time axis from the first analysis result display information. The system generates a display screen that includes a second display unit for displaying second analysis result display information which visualizes the aforementioned analysis result information, a first operation unit for selecting one of a plurality of predetermined periods included in the first period, and a second operation unit for specifying an arbitrary period from the first period. When an arbitrary period is specified as the second period by the second operation unit, the period corresponding to the second period is highlighted in the first analysis result display information. When a predetermined period is selected as the second period by the first operation unit, the specified state of the second operation unit is set to correspond to that predetermined period .

According to this, similar to the first invention, even when the analysis results of analyzing the worker's work status take a long time, the user can efficiently check the long-term analysis results using two operating units .

The embodiments of the present invention will be described below with reference to the drawings.

Figure 1 is an overall diagram of the work analysis system according to this embodiment.

This system analyzes the work status of workers in factories and other similar settings, and presents the analysis results to the user. The system comprises a camera (1), a recorder (2), a server (3 - work analysis device), and an administrator terminal (4).

Camera 1 films the workers performing their tasks at their assigned positions.

Recorder 2 records the video output from camera 1.

Server 3 acquires video from Recorder 2, performs work analysis based on that video to analyze the workers' work status, and outputs the analysis results. Server 3 is installed in a facility (such as a factory) and performs work analysis targeting that facility. Alternatively, Server 3 may be configured as a cloud computer and perform work analysis targeting multiple facilities.

Administrator terminal 4 is used by system administrators and operational managers (users) and consists of PCs, tablet devices, etc. On administrator terminal 4, system administrators perform various settings related to the processing performed on server 3. Furthermore, administrator terminal 4 displays the analysis results output from server 3 for operational managers to view.

Next, we will describe the general configuration of Server 3. Figure 2 is a block diagram showing the general configuration of Server 3.

Server 3 comprises a video input unit 11, a screen output unit 12, a storage unit 13, and a processor 14.

The video input unit 11 receives video recorded by the recorder 2.

The screen output unit 12 outputs the analysis result display screen generated by the processor 14, and the analysis result display screen (see Figure 6) is displayed on the administrator terminal 4.

The memory unit 13 stores programs executed by the processor 14. It also stores video footage for analysis acquired from the recorder 2. Furthermore, the memory unit 13 stores the object behavior recognition model (object behavior estimation model) used by the processor 14, the analysis result video generated by the processor 14, and the analysis result information.

The processor 14 performs various processes by executing the program stored in the memory unit 13. In this embodiment, the processor 14 performs analysis processing P1 and analysis result visualization processing P2, etc.

In analysis processing P1, the processor 14 analyzes the worker's work status based on video footage of the worker performing the task and generates analysis result information.

In the analysis result visualization process P2, the processor 14 visualizes the analysis result information acquired in the analysis process P1 and presents it to the user. In this embodiment, an analysis result display screen (see Figure 6) is generated.

Next, we will explain the analysis process P1 performed on server 3. Figure 3 is a block diagram showing the contents of analysis process P1.

On server 3, processor 14 performs analysis processing P1. Analysis processing P1 includes video acquisition processing P11, object behavior recognition processing P12, analysis result video generation processing P13, work cycle detection processing P14, anomaly detection processing P15, and output processing P16.

In the video acquisition process P11, the processor 14 acquires the video to be analyzed from the storage unit 13.

In the object and action recognition process P12, the processor 14 performs object recognition to recognize the type of object (person, part, and work tool) from the video to be analyzed, and also performs action recognition (action estimation) to recognize the type of person's action. Action recognition recognizes the person's action from the person's movement and the type of object the person came into contact with. The object and action recognition process P12 obtains the action ID assigned to the action, the type of object (target object) associated with the action, and the start and end times of the action as recognition result information. The object and action recognition process P12 can use an object and action recognition model (machine learning model) constructed using machine learning such as deep learning.

In the analysis result video generation process P13, the processor 14 generates an analysis result video with added recognition result information. Specifically, it generates the analysis result video by superimposing a rectangular frame surrounding the object (person, part, work tool) recognized in the object action recognition process P12, characters representing the type of recognized object, and characters representing the type of recognized action onto the original video being analyzed.

In the work cycle detection process P14, the processor 14 detects a work cycle (specified period) consisting of a predetermined series of actions from among the worker actions recognized in the object action recognition process P12. The work cycle detection process P14 obtains the cycle ID assigned to the work cycle, as well as the start and end times of the work cycle, as work cycle detection result information.

In the anomaly detection process P15, the processor 14 detects work errors (work anomalies) by comparing the worker's work procedure with the standard work procedure. Specifically, the processor 14 detects work errors by comparing the actions performed by the worker with the actions included in the standard work procedure for the target work cycle. Work errors include, for example, excessive actions, insufficient actions, and changes in the order of actions. An excessive action is a state where an action is performed extraneously at a time when it should not be performed. An insufficient action is a state where an action that should be performed is not performed.

Furthermore, in the anomaly detection process P15, the processor 14 detects anomalies in a work cycle by comparing the worker's work time for one work cycle with the standard time. Specifically, for example, if the work cycle time is longer than the standard time by a predetermined limit, such as when the line is temporarily stopped due to some trouble, it is determined to be an anomaly in the work cycle. Also, for example, if the work cycle is canceled midway, it is determined to be an anomaly in the work cycle if the work cycle time is shorter than the standard time by a predetermined limit.

In output processing P16, the processor 14 outputs analysis result information, which includes information about the worker's actions (type of action, start time, and end time, etc.) acquired in object action recognition processing P12, information about the work cycle (start time and end time, etc.) acquired in work cycle detection processing P14, and detection results information from anomaly detection processing P15 (work anomalies (work errors), work cycle anomalies). This analysis result information is stored in the storage unit 13.

Next, we will explain the work cycle detection process P14 performed on server 3. Figure 4 is an explanatory diagram showing the status of the work cycle detection process P14.

Server 3 performs a work cycle detection process P14 to detect a work cycle consisting of a predetermined series of actions. Specifically, the processor 14 detects a trigger action from among the recognized actions that defines the start timing of the work cycle. The timing at which the trigger action is detected is set as the start timing of the work cycle, and the timing at which the action immediately preceding the next trigger action is detected is set as the end timing of the work cycle.

Figure 4 shows an example of actions included in one work cycle. In this case, first, as shown in Figure 4(A), the worker grasps the main body with their hand. Next, as shown in Figure 4(B), the worker grasps the screwdriver with their hand. Next, as shown in Figure 4(C), the worker takes the screw out of the parts box. Next, as shown in Figure 4(D), the worker tightens the screw. Next, as shown in Figure 4(E), the worker carries the main body. Next, as shown in Figure 4(F), the worker grasps the main body with their hand.

Here, as shown in Figures 4(A) and 4(F), the action of the worker grasping the main unit with their hand becomes the trigger action that defines the start timing of the work cycle. The timing at which this trigger action begins is the start timing of the work cycle. Furthermore, the timing at which the action immediately preceding the next trigger action is completed is the end timing of the work cycle.

Furthermore, for each recognized action, the recognition result information includes the action ID assigned to the action, the type of object (target object) associated with the action, and the start and end times of the action. Additionally, the work cycle detection result information includes the cycle ID assigned to the work cycle and the start and end times of the work cycle.

Next, we will explain the analysis result visualization process P2 performed on server 3. Figure 5 is a block diagram showing the contents of the analysis result visualization process P2.

On server 3, processor 14 performs analysis result visualization processing P2. Analysis result visualization processing P2 includes video acquisition processing P21, analysis result acquisition processing P22, and screen generation processing P23.

In the video acquisition process P21, the processor 14 acquires the analysis result video from the storage unit 13. The analysis result video consists of a rectangular frame surrounding the recognized object (person, part, work tool), characters representing the type of recognized object, and characters representing the type of recognized action, superimposed on the original video being analyzed.

In the analysis result acquisition process P22, the processor 14 acquires analysis result information from the storage unit 13. The analysis result information includes information about the worker's actions (type of action, start time, and end time, etc.), information about the work cycle (start time and end time, etc.), and information about the anomaly detection results (work anomalies (work errors), work cycle anomalies).

In screen generation process P23, the processor 14 generates an analysis result display screen (see Figure 6) that visualizes the analysis result information. This analysis result display screen is output from the screen output unit 12 to the administrator terminal 4, and the analysis result display screen is displayed on the administrator terminal 4. In this embodiment, the analysis result display screen displays the analysis result information in a visualized manner, as well as an analysis result video.

Next, the analysis results display screen shown on the administrator terminal 4 will be explained. Figure 6 is an explanatory diagram showing the analysis results display screen. Figure 7 is an explanatory diagram showing the process selection unit 102, target period specification unit 103, work video display unit 104, playback operation unit 105, and work cycle selection unit 106 of the analysis results display screen. Figure 8 is an explanatory diagram showing the detailed analysis results display unit 107 of the analysis results display screen. Figures 9 and 10 are explanatory diagrams showing the main parts of the detailed analysis results display unit 107 when a work error (abnormality) occurs. Figure 11 is an explanatory diagram showing another example of the detailed analysis results display unit 107. Figure 12 is an explanatory diagram showing the overall analysis results display unit 108 of the analysis results display screen. Figure 13 is an explanatory diagram showing another example of the overall analysis results display unit 108.

As shown in Figure 6, the administrator terminal 4 displays the analysis results display screen 101. The analysis results display screen 101 includes a process selection unit 102, a target period specification unit 103, a work video display unit 104, a playback operation unit 105, a work cycle selection unit 106, a detailed analysis results display unit 107 (a unit for displaying the second analysis results information), and an overall analysis results display unit 108 (a unit for displaying the first analysis results information).

As shown in Figure 7, the process selection unit 102 is equipped with buttons 111 for each process (picking, pressing, welding, assembly, inspection). When the user operates a button 111 to select a process, the analysis results for the selected process are displayed on the analysis results display screen 101.

The target period specification unit 103 allows the user to input the date and time (start time and end time) in text as information regarding the target period (first period) for which the analysis results will be displayed. Furthermore, the target period specification unit 103 allows the user to specify any period included in the period during which video footage is stored in the recorder 2, that is, the period during which work analysis results based on the video footage can be obtained.

The work video display unit 104 displays the analysis result video 121. In the analysis result video 121, a rectangular frame 122 surrounding the recognized object (person, part, work tool), text 123 representing the type of recognized object, and text 124 representing the type of recognized action are superimposed on the video footage of the worker performing the task. In this example, a rectangular frame 122 and text 123 representing a person, a rectangular frame 122 and text 123 representing a screwdriver, and text 124 representing the screw-tightening action are displayed.

The playback control unit 105 is equipped with a button 131 that serves both as a play and pause button, and a button 132 for double speed playback.

The work cycle selection unit 106 is equipped with buttons 141 for each work cycle (1 to 6), a previous skip button 142, and a next skip button 143. When the user selects a work cycle by operating button 141, the analysis results for the selected work cycle are displayed on the work video display unit 104 and the detailed analysis results display unit 107. When the user operates the previous skip button 142, the system advances to the beginning of the next work cycle. When the user operates the next skip button 143, the system returns to the beginning of the currently displayed work cycle.

As shown in Figure 8, the detailed analysis results display unit 107 is provided with a first action list display unit 201. The first action list display unit 201 visualizes and presents short-term analysis results regarding the status of actions related to each object to the user. In the first action list display unit 201, action display fields 202 are arranged vertically for each object related to the worker's actions. In this example, action display fields 202 are provided for each object related to the worker's actions: parts (main body, part #1, part #2, and part #3), and work tools (screwdriver, tweezers, and marker).

The action display area 202 displays an action bar 203. The action bar 203 is an image representing the period during which the action was performed. That is, in the action display area 202, the horizontal axis represents time, and the action bar 203 is drawn within the range corresponding to the period during which the action was performed. This allows the status of actions performed for each type of object to be represented chronologically.

The action bar 203 displays different patterns depending on the type of action, specifically, actions such as grasping parts or work tools in the hand, or tightening screws with a screwdriver. In this example, the shading pattern (drawing pattern) drawn on the action bar 203 differs according to the type of action. Alternatively, the fill color of the action bar 203 could also differ according to the type of action.

Furthermore, in the first action list display unit 201, if a work cycle is selected in the work cycle selection unit 106, the analysis results for the entire period of that selected work cycle are displayed. Also, if an arbitrary period is specified in the overall analysis result display unit 108, the analysis results for that specified period are displayed in the first action list display unit 201.

Furthermore, the first action list display unit 201 displays a timeline cursor 204 indicating the current playback position (playback time) of the analysis result video 121 displayed on the work video display unit 104. The timeline cursor 204 moves horizontally (along the time axis) as the playback time of the analysis result video 121 on the work video display unit 104 progresses.

Furthermore, the detailed analysis results display unit 107 is equipped with a time display unit 205. The time display unit 205 displays the start time, end time, and duration of the analysis results range (display period) shown on the first action list display unit 201. Here, when a work cycle is selected in the work cycle selection unit 106, and the entire duration of one work cycle is displayed on the first action list display unit 201, the displayed duration represents the work time of the selected work cycle.

Furthermore, in the detailed analysis results display unit 107, when a work error (abnormality) is detected, the action display field 202 related to the object (tool, part) associated with the detected work error is highlighted. Specifically, the display pattern of the background of the corresponding action display field 202 (e.g., fill color, shading pattern, etc.) changes. Note that work errors (abnormalities) include situations where actions are excessive, insufficient, or where the order of actions is reversed.

Furthermore, the detailed analysis results display unit 107 displays an alert mark 206 when a work error (abnormality) is detected. The alert mark 206 is displayed in the action display section 202 related to the object involved in the detected work error, at a position corresponding to the timing of the action related to the work error. For example, if an action is insufficient, the alert mark 206 is displayed at the position where that action should have been performed. If an action is excessive, the alert mark 206 is displayed at the position where that excessive action was performed.

Alternatively, instead of, or in addition to, the alert mark 206, the display of the action bar 203 related to the work error may be changed to highlight it. For example, the border of the action bar 203 may be drawn as a dotted line, or the transparency of the action bar 203 may be changed.

Here, Figures 9(A-1) and (A-2) show examples of actions related to part #3. In this case, the part handling operation consists of actions of grasping the part with the hand. In the worker's procedure shown in Figure 9(A-2), the action of grasping the part with the hand is one step excessive compared to the normal procedure shown in Figure 9(A-1). This is detected as a work error (abnormality), and alert mark 206 is displayed.

Figures 9(B-1) and (B-2) show examples of actions related to a screwdriver. In this case, the screw-tightening operation consists of the actions of grasping the screwdriver with the hand, taking the screw out of the parts box, and tightening the screw. The actions of taking the screw out of the parts box and tightening the screw are repeated three times. In the worker's procedure shown in Figure 9(B-2), the action of taking the screw out of the parts box is performed one less time compared to the normal procedure shown in Figure 9(B-1). This is detected as a work error (abnormality), and alert mark 206 is displayed.

Figures 10 (A-1) and (A-2) show examples of actions related to markers. In this case, the marking process consists of the actions of grasping the marker with the hand, opening the cap, applying the mark, and closing the cap. In the worker's procedure shown in Figure 10 (A-2), the action of closing the cap is missing compared to the standard procedure shown in Figure 10 (A-1). This is detected as a work error (abnormality), and alert mark 206 is displayed.

Furthermore, as shown in Figure 11, the detailed analysis result display unit 107 may be configured to display an error content display unit 211 when a work error (anomaly) is detected. The error content display unit 211 displays characters representing the detected work error. In this case, it displays that the action related to part #3 was performed one extra time, the screw tightening action related to the driver was performed one less time, and the cap was not closed as an action related to the marker.

As shown in Figure 12, the overall analysis results display unit 108 is equipped with a second action list display unit 301. The second action list display unit 301 visualizes and presents to the user the long-term analysis results regarding the status of actions related to each object. Similar to the first action list display unit 201, the second action list display unit 301 provides action display fields 302 arranged vertically for each object related to the worker's actions. Similar to the first action list display unit 201, action bars 303 are drawn in the action display fields 302.

The second action list display unit 301 displays the analysis results for the period specified in the target period specification unit 103 (target period).

Furthermore, the overall analysis results display unit 108 is equipped with a slider bar 304. The slider bar 304 allows the user to specify an arbitrary period for the analysis results to be displayed on the detailed analysis results display unit 107 by operating the left and right sliders 305. Here, the left slider 305 defines the start time, and the right slider 305 defines the end time. As a result, once a period is specified on the overall analysis results display unit 108, the analysis results for that specified period are displayed on the detailed analysis results display unit 107 in an enlarged state along the time axis.

Furthermore, when a user selects a work cycle in the work cycle selection unit 106, the left and right sliders 305 move to correspond to the duration (start time and end time) of the selected work cycle, and the analysis results of the selected work cycle are displayed in the detailed analysis results display unit 107.

Furthermore, the overall analysis result display unit 108 displays a count mark 306 when a work error (abnormality) is detected. The count mark 306 represents the number of work errors (abnormalities) detected in one work cycle. The count mark 306 is displayed in the display area of the work cycle in which the work error was detected. In this example, one work error was detected in work cycle #1, and three work errors were detected in work cycle #4.

Furthermore, in the overall analysis results display unit 108, if an abnormality in a work cycle is detected, the work cycle in which the abnormality was detected is highlighted. Specifically, a rectangular image 307 is displayed for the duration of the relevant work cycle. The rectangular image 307 is highlighted, for example, by a fill color or a shading pattern. In this example, an abnormality has been detected in work cycle #4. Specifically, an abnormality in a work cycle refers to a situation where the work cycle time is longer than the standard time by a predetermined limit, or where the work cycle time is shorter than the standard time by a predetermined limit.

Furthermore, as shown in Figure 13, the overall analysis results display unit 108 highlights the period specified by the slider bar 304, that is, the period during which the analysis results were displayed in the detailed analysis results display unit 107. Specifically, a rectangular image 311 is displayed during the corresponding period. The rectangular image 311 is highlighted, for example, by a fill color or a shading pattern. The rectangular image 311 is displayed in a different format from the rectangular image 307, which highlights the work cycle in which an anomaly was detected, so that it can be distinguished from the rectangular image 307. In this example, work cycle #5 is selected, and the rectangular image 311 is displayed for work cycle #5.

In this embodiment, the user can arbitrarily specify the period (target period, first period) for the analysis results to be displayed on the overall analysis results display unit 108 using the target period specification unit 103. The user can also arbitrarily specify the period (part of the target period, second period) for the analysis results to be displayed on the detailed analysis results display unit 107 using the slider bar 304. Furthermore, the user can display the analysis results for a single work cycle on the detailed analysis results display unit 107 using the work cycle selection unit 106.

The first period covered by the overall analysis results display unit 108 is the entire target period extracted from the total period during which the worker's work was performed, while the second period covered by the detailed analysis results display unit 107 is a portion of that target period. In other words, the time axis scale differs between the detailed analysis results display unit 107 and the overall analysis results display unit 108. A portion of the analysis results displayed in the overall analysis results display unit 108 is displayed in the detailed analysis results display unit 107 in an enlarged state along the time axis. This allows the user to gain an overview of the worker's work status over a broad period by viewing the first analysis results display information. Furthermore, the user can gain a detailed view of the worker's work status by viewing the second analysis results display information.

The specific procedure involves, for example, the user first using the work cycle selection unit 106 to display and view the analysis results for one work cycle on the detailed analysis results display unit 107. If necessary, the user can adjust the period of analysis results displayed on the detailed analysis results display unit 107 using the slider bar 304, thereby viewing analysis results for various ranges. For example, the user can use the slider bar 304 to display only the analysis results for a specific range of interest within a single work cycle on the detailed analysis results display unit 107. Furthermore, the user can use the slider bar 304 to display analysis results spanning two or more work cycles on the detailed analysis results display unit 107. While the explanation focused on work processes in a factory, this method can also be applied to work outside of factories, i.e., work that occurs regularly within various facilities (e.g., product loading/unloading, cleaning, patrol security).

As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to modified, replaced, added, or omitted embodiments. Furthermore, it is possible to combine the components described in the above embodiments to create new embodiments.

The work analysis device and work analysis method according to the present invention have the effect of allowing users to efficiently review analysis results that extend over a long period of time, even when analyzing the work status of an employee. They are useful as work analysis devices and methods that use a processor to output analysis result information regarding the employee's work status based on video footage of the employee's work.

1. Camera 2. Recorder 3. Server (work analysis device)
4 Administrator terminal 11 Video input unit 12 Screen output unit 13 Memory 14 Processor 101 Analysis result display screen 102 Process selection unit 103 Target period specification unit 104 Work video display unit 105 Playback operation unit 106 Work cycle selection unit 107 Detailed analysis result display unit 108 Overall analysis result display unit 141 Button for selecting work cycle 201 First action list display unit 202 Action display field 203 Action bar 204 Timeline cursor 206 Alert mark 211 Abnormal content display unit 301 Second action list display unit 302 Action display field 303 Action bar 304 Slider bar 305 Slider 306 Count mark

Claims (9)

A work analysis device that uses a processor to perform a process that outputs analysis result information regarding the worker's work status based on video footage of the worker's work,
The aforementioned processor,
Based on the aforementioned video, the system recognizes the object being handled by the worker and the actions constituting the worker's work, and obtains the analysis result information including the recognition result of the object and the recognition result of the actions.
A display screen is generated that includes: a first display unit that displays first analysis result display information which visualizes the analysis result information for a first period ; a second display unit that sets a second period included in the first period and displays second analysis result display information which visualizes the analysis result information with an expanded time axis compared to the first analysis result display information; a first operation unit for selecting one of a plurality of predetermined periods included in the first period; and a second operation unit for specifying an arbitrary period from the first period.
The work analysis apparatus is characterized in that, when an arbitrary period is specified as the second period by the second operating unit, the period corresponding to the second period is highlighted in the first analysis result display information, and when a predetermined period is selected as the second period by the first operating unit, the specified state of the second operating unit is made to correspond to the predetermined period . The aforementioned processor,
The work analysis apparatus according to claim 1, characterized in that the aforementioned video is output in a manner that allows playback in association with the aforementioned analysis result display information. The aforementioned display screen is
Furthermore, the work analysis apparatus according to claim 1 is characterized by including a period specification unit for inputting the start time and end time of the accumulation of the video for specifying the first period . The first operating unit is,
The work analysis apparatus according to claim 1, characterized in that it is a work cycle selection unit that selects one of a plurality of work cycles consisting of a predetermined series of actions included in the first period . The second operating unit described above is:
The work analysis apparatus according to claim 1, characterized in that it is a slider bar for specifying an arbitrary period on the time axis of the first analysis result display information included in the first period . The aforementioned processor,
The work analysis device according to claim 1, characterized in that when it detects an abnormality in the worker's work status, it outputs analysis result display information including information regarding the abnormality detection result. The second analysis result display information is, as display information relating to the anomaly detection result,
Highlighting of the display area for the aforementioned object,
Display of a mark indicating the behavior related to the aforementioned abnormality,
The work analysis apparatus according to claim 6, characterized in that it includes at least one of the following: a display of characters indicating the nature of the abnormality. The first analysis result display information is, as display information relating to the anomaly detection result,
The work analysis apparatus according to claim 6, characterized in that it includes either a display of the number of abnormality detections detected in the specified period included in the first period, or a highlighting of the specified period in which the abnormality was detected. A work analysis method in which a processor performs a process to output analysis result information regarding the worker's work status based on video footage of the worker's work,
Based on the aforementioned video, the system recognizes the object being handled by the worker and the actions constituting the worker's work, and obtains the analysis result information including the recognition result of the object and the recognition result of the actions.
A display screen is generated that includes: a first display unit that displays first analysis result display information which visualizes the analysis result information for a first period ; a second display unit that sets a second period included in the first period and displays second analysis result display information which visualizes the analysis result information with an expanded time axis compared to the first analysis result display information; a first operation unit for selecting one of a plurality of predetermined periods included in the first period; and a second operation unit for specifying an arbitrary period from the first period.
A work analysis method characterized in that, when an arbitrary period is specified as the second period by the second operation unit, the period corresponding to the second period is highlighted in the first analysis result display information, and when the specified period is selected as the second period by the first operation unit, the specified state of the second operation unit is made to correspond to the specified period .