JP7666022B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing device.

Patent Document 1 discloses a learning model that, when an image including a dangerous spot at a work site is input, outputs an event corresponding to the dangerous spot. Specifically, for example, when an image including a "crane" installed at the work site as a dangerous spot is input, the learning model outputs the event "hanging load falling."

JP 2018-81549 A

A device that implements the learning model disclosed in Patent Document 1 can mechanically detect dangerous locations at a work site. However, even if dangerous locations at a work site can be identified, accidents may occur at the work site, for example, if the equipment used by workers is not maintained or if workers behave dangerously at the work site. Therefore, in order to determine whether the situation at a work site is dangerous, it is necessary to consider not only the dangerous locations at the work site but also other risk factors.

The present invention was made in consideration of the above-mentioned problems in the past, and its purpose is to provide an information processing device that can detect multiple risk factors in a work site.

The information processing device of the present invention that solves the above-mentioned problem includes an acquisition unit that acquires an image of a work site where a specified piece of equipment is installed, a processing unit that inputs the acquired image and causes a learning model that has been trained to output, when an image of a work site is input, a first group of items corresponding to risk factors of the specified equipment at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed subject at the work site in the input image to output an item corresponding to the risk factor in the acquired image from the first to third groups of items, and an output unit that outputs a level of risk for an item corresponding to a risk factor in the acquired image using data including a level of risk for each item in the first group of items; an addition unit that adds up, among the levels of risk for items corresponding to risk factors in the acquired image, a first level of risk for each item included in the first group of items, a second level of risk for each item included in the second group of items, and a third level of risk for each item included in the third group of items, the third level of risk being output together with the first and second levels of risk and held for a predetermined period of time; and a determination unit that determines that the situation at the work site in the acquired image is dangerous when a result of the addition by the addition unit is greater than a predetermined value .

The information processing method of the present invention includes the steps of: acquiring an image of a work site where a predetermined facility is installed; inputting the acquired image to a learning model that has been trained to output, when an image of a work site is input, a first group of items corresponding to risk factors of the predetermined facility at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed object at the work site in the input image, and outputting items corresponding to the risk factors in the acquired image from the first to third groups of items; the step of outputting a degree of risk for an item corresponding to a risk factor in the acquired image using data including a degree of risk for each item; the step of adding up, among the degrees of risk for items corresponding to risk factors in the acquired image, a first degree of risk for each item included in the first group of items, a second degree of risk for each item included in the second group of items, and a third degree of risk for each item included in the third group of items, which is output together with the first and second degrees of risk and held for a predetermined period of time; and the step of determining that the situation at the work site in the acquired image is dangerous if the result of the addition is greater than a predetermined value .

The present invention provides an information processing device that can detect multiple risk factors at a work site.

FIG. 1 illustrates an example of an information processing system 10. FIG. 2 illustrates an example of a learning device 21. FIG. 2 is a diagram showing an example of a functional block realized in the learning device 21. FIG. 11 is a diagram illustrating an example of teacher data. FIG. 11 is a diagram illustrating an example of teacher data. FIG. 11 is a diagram illustrating an example of teacher data. FIG. 11 is a diagram illustrating an example of teacher data. 13 is a flowchart showing an example of a generation process of a learning model 41. FIG. 2 is a diagram showing an example of items output from the learning model 41. FIG. 2 is a diagram illustrating an example of a determination device 22. FIG. 13 is a diagram showing an example of risk level data 70. FIG. 2 is a diagram illustrating an example of a functional block realized in a determination device 22. 13 is a flowchart showing an example of a determination process S30. 13 is a flowchart showing an example of a risk score calculation process S44. FIG. 13 is a diagram showing an example of a transition of a risk score. FIG. 11 is a diagram illustrating an example of teacher data.

The present specification and accompanying drawings make clear at least the following:

== ...
1 is a diagram showing the configuration of an information processing system 10 according to an embodiment of the present invention. The information processing system 10 is a system that detects a plurality of risk factors in a work site 15 and determines whether the situation in the work site 15 is dangerous or not, and is configured to include a camera 20, a learning device 21, a determination device 22, and a terminal 23.

<<<Worksite 15>>>
The work site 15 is, for example, a room in an office building under construction, and is provided with a door 200 in the wall and an opening 201 for accessing the space under the floor. The work site 15 is also provided with a portable work platform 210 (hereinafter referred to as the "work platform 210").

The workbench 210 is equipment used by workers when installing ceiling boards or ceiling piping, for example, on a ceiling (not shown), and has a top platform 220, support legs 221, and handrails 222. The top platform 220 is a part that constitutes the scaffolding on which the worker works, and is positioned at a higher position (above) than the work floor when the workbench 210 is installed on the work floor.

The support legs 221 are ladders that support the platform 220 while allowing the worker to ascend and descend to the platform 220, and are provided at both ends of the platform 220 in the longitudinal direction. The handrails 222 are members that prevent the worker from falling from the platform 220, and a pair of handrails are provided on both sides of the platform 220 in the width direction. The work platform 210 corresponds to the "specified equipment." In this embodiment, the "specified equipment" refers to equipment used at the work site, and is not limited to the work platform 210, and may be, for example, a ladder, a stepladder, a hand lift, a light base, a high-altitude work vehicle, etc.

<<<<Overview of Information Processing System 10>>>
The camera 20 is a small, portable device used to capture images of the conditions at the work site 15. In this embodiment, the camera 20 is attached to, for example, a tripod (not shown) and is disposed at a position in the work site 15 where it can capture images of the area where workers are working. The camera 20 may also be attached using a hook or the like to, for example, a curtain rail (not shown) attached to a wall surface at the work site.

The learning device 21 is an information processing device that constructs a learning model for detecting risk factors in the work site 15, and the judgment device 22 is an information processing device that judges whether the situation in the work site 15 is dangerous or not based on an image from the camera 20 and the learning model constructed by the learning device 21. In this embodiment, a server is used as the information processing device, but this is not limited to this. Specifically, all or a part of the information processing device may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, etc., such as a virtual server provided by a cloud system. In addition, as will be described in detail later, all or a part of the functions provided by the information processing device may be realized by a service provided by the cloud system via an API (Application Programming Interface), etc.

The terminal 23 is a tablet on which the site supervisor performs a predetermined operation when the judgment device 22 detects a risk factor. The details of the terminal 23 will be described later, but if the work site 15 is judged to be in a dangerous situation, an alarm is displayed on the display screen of the terminal 23. In addition, the camera 20, the learning device 21, the judgment device 22, and the terminal 23 are each connected via a network 24.

<<<<Details of learning device 21>>>
2 is a diagram showing an example of a hardware configuration of the learning device 21. The learning device 21 is a computer including a CPU (Central Processing Unit) 30, a memory 31, a storage device 32, an input device 33, a display device 34, and a communication device 35.

The CPU 30 executes programs stored in the memory 31 and storage device 32 to realize various functions of the learning device 21.

Memory 31 is, for example, a RAM (Random-Access Memory) and is used as a temporary storage area for programs, data, etc.

The storage device 32 is a non-volatile storage device that stores the learning program 40 and the learning model 41. When the learning device 21 is realized using a cloud server, the storage area of the storage device 32 corresponds to the storage area of the cloud server.

The learning program 40 is a program that is executed when constructing the learning model 41, and the learning model 41 is a model for detecting risk factors at the work site 15 from images of the work site 15 captured by the camera 20. Details of the learning model 41 will be described later.

The input device 33 is a device that accepts commands and data input by the user, and includes input interfaces such as a keyboard and a touch sensor that detects the touch position on a touch panel display.

The display device 34 is, for example, a device such as a display, and the communication device 35 exchanges various programs and data with the determination device 22 and other devices via the network 24.

<<<Functional blocks of learning device 21>>>
3 is a diagram showing an example of functional blocks realized in the learning device 21 when the CPU 30 of the learning device 21 executes the learning program 40. In the learning device 21, an acquisition unit 50 and a learning unit 51 are realized.

The acquisition unit 50 acquires the teacher data D1 to D3 that are prepared in advance and are used to construct the learning model 41.

<<Teacher data D1>>
The teacher data D1 in Fig. 3 is data in which a predetermined correct answer label (or correct answer data) is attached to an image including equipment used at a work site. Fig. 4(a) is an example of teacher data D1 including a workbench that does not pose a safety problem, and Fig. 4(b) is an example of teacher data D1 including a workbench that poses a safety problem.

The target location in the training data D1 in FIG. 4(a) is labeled with a correct answer label 300 of "work platform" indicating the target equipment, a correct answer label 301 of "ladder" used to estimate the height of the equipment, and a correct answer label 302 of "handrail safety" indicating a safe handrail.

In this embodiment, six correct labels 301 are attached to each of the first to third steps of the left and right support legs of the workbench, but for example, it is also possible to attach correct labels to only any of the locations. However, by attaching correct labels 301 to each of the first to third steps of the support legs, the learning model can be trained with greater accuracy. Also, although details are omitted, the correct labels in this embodiment include the coordinates of the range of the target area (i.e., the bounding box).

The training data D1 in FIG. 4(b) is data for learning about an unsafe work platform that does not have a handrail along the longitudinal direction of the platform. The target location of the training data D1 in FIG. 4(b) is labeled with the above-mentioned correct answer label 300 of "work platform", correct answer label 301 of "ladder", and correct answer label 303 of "handrail unsafe" indicating an unsafe handrail (unsafe handrail). Note that FIG. 4(b) is an example of training data for learning about unsafe work platforms, and various images can be used as training data, for example, images with part of a bent handrail.

<<Teacher data D2>>
The teacher data D2 in Fig. 3 is data that associates an image including factors that are dangerous to workers in the environment of the work site with the contents of the factors. In the environment of the work site, factors that are dangerous to workers include, for example, a door through which workers enter and exit, and an opening in the floor. For this reason, although not shown here for convenience, in the image of the teacher data D2, the door area is given the correct answer label of "door", and the opening area is given the correct answer label of "opening".

In this embodiment, doors and openings are given as two factors that pose a danger to workers, but this is not limited to these. For example, steps, stacked blocks, ducts, etc. can also be dangerous factors. Therefore, images that include these dangerous factors may be used as training data D2.

<<Teacher data D3>>
The teacher data D3 in Fig. 3 is data in which a predetermined correct answer label is attached to an image including the motion of a worker to be observed. Fig. 5(a) is an example of teacher data D3 including a worker performing appropriate lifting and descending motion, and Fig. 5(b) is an example of teacher data D3 including a worker performing dangerous (inappropriate) lifting and descending motion.

In the training data D3 in FIG. 5(a), the worker's area is labeled with a correct answer label 310 of "appropriate ascent and descent." In the training data D3 in FIG. 5(b), the worker's area is labeled with a correct answer label 311 of "dangerous ascent and descent."

In addition, FIG. 6(a) is an example of teacher data D3 that includes a worker working on the platform in an appropriate posture, and FIG. 6(b) is an example of teacher data D3 that includes a worker working on the platform in an inappropriate posture (with the upper body protruding from the handrail).

In the training data D3 in FIG. 6(a), the worker's region is labeled with a correct answer label 312 of "appropriate posture." In the training data D3 in FIG. 5(b), the worker's region is labeled with a correct answer label 313 of "upper body protruding."

Each of Figures 7(a) and 7(b) is an example of training data D3 that includes a worker working on the platform in an inappropriate posture (feet overhanging the handrail). In the training data D3 of Figures 7(a) and 7(b), the worker's area is labeled with a correct answer label 314 of "feet overhanging."

In the present embodiment, for the sake of convenience, the image of the teacher data D3 is labeled with only the correct answer level of the worker, but the above-mentioned correct answer labels 300 to 303 of the equipment may be labeled on the workbench included in the image. Such an image is an image that includes the teacher data D1 and D3.

<<Learning Model Generation Process S10>>
8 is a flowchart showing an example of a generation process S10 for generating a learning model. The learning unit 51 of this embodiment uses the teacher data D1 to D3 to detect an object with a correct answer label when an image is input, and generates a learning model that outputs a risk factor item according to the detection result. In this embodiment, a model using a convolutional network (CNN: Convolutional Neural Network) (not shown) is used as the learning model.

First, the acquisition unit 50 in FIG. 3 acquires the training data D1 to D3 (S20). Then, the learning unit 51 uses the training data D1 to D3 to tune the parameters of the CNN so that the target corresponding to the correct label can be detected from the image input to the CNN (S11).

As described above, the teacher data D1 is assigned four correct labels 300-303 ("work platform," "ladder," "handrail safe," and "handrail unsafe"). The teacher data D2 is assigned two correct labels ("door" and "opening"), and the teacher data D3 is assigned five correct labels 310-314 ("appropriate ascent," "dangerous ascent," "appropriate posture," "upper body protruding," and "feet protruding"). As a result, the CNN is able to detect objects corresponding to the 11 correct labels from the input image.

Then, the learning unit 51 adjusts the output layer of the CNN so that the CNN can output multiple items according to risk factors as shown in FIG. 9 from the detection results of the target corresponding to the 11 correct labels (S12). Specifically, the learning unit 51 adjusts the CNN so that item 1A is output when a five-step support leg (i.e., a "work platform" with five "ladders") is detected, and item 2A is output when "handrail unsafe" is detected.

In this embodiment, the height of one stage of the support legs on the workbench is approximately 30 cm, so a workbench with five stages of support legs has a platform that is approximately 1.5 m above the floor. When using such a workbench, there is a high risk of injury if a worker falls from the platform. For this reason, when the CNN of this embodiment detects a workbench with five or more stages of support legs, it outputs item 1A, which indicates that the equipment is dangerous.

In this embodiment, when the CNN detects five support legs, it outputs item 1A. However, this is not limited to this, and item 1A may be output based on other factors that pose a danger to workers (e.g., a support leg is broken).

The learning unit 51 also adjusts the CNN so that if a "door" is detected, item 2A is output, and if an "opening" is detected, item 2B is output. The learning unit 51 also adjusts the CNN so that if a "dangerous ascent or descent" is detected, item 3A is output, if an "upper body protruding" is detected, item 3A is output, and if a "leg protruding" is detected, item 3C is output.

After processing S12 is executed, the learning unit 51 stores the adjusted CNN in the storage device 32 as the learning model 41 (S13). Therefore, the learning model 41 of this embodiment is a model that outputs items 1A and 1B corresponding to risk factors of the equipment in the image, items 2A and 2B corresponding to risk factors of the work site environment in the image, and items 3A to 3C corresponding to risk factors of the actions of the worker who is the object of observation in the image. Therefore, by using such a learning model 41, various risk factors at the work site can be detected.

Note that items 1A and 1B correspond to the "first item" corresponding to risk factors of the equipment, and the two items 2A and 2B correspond to the "second item" corresponding to risk factors in the work site environment. The three items 3A to 3C correspond to the "third item" corresponding to risk factors in the actions of the worker. In this embodiment, the number of "first items" and "second items" is two, and the number of "second items" is three, but this is not limited to these and it is sufficient that there is at least one.

In addition, the CNN of this embodiment outputs two items, items 1A and 1B, as items indicating that the equipment is dangerous, but it may output an item indicating that the equipment is dangerous only if, for example, there are five support legs (i.e., the height of the top platform is approximately 1.5 m or more from the floor) and "handrail unsafe" is detected. In such a case, the CNN will be able to appropriately detect equipment that is more dangerous.

<<<<Details of the Determination Device 22>>>
Fig. 10 is a diagram showing an example of the hardware configuration of the determination device 22 in Fig. 1. The determination device 22 is a computer including a CPU 60, a memory 61, a storage device 62, an input device 63, a display device 64, and a communication device 65. Note that each configuration of the CPU 60 and the like of the determination device 22 is similar to the CPU 30 and the like of the learning device 21, so detailed explanations are omitted here and the following description will focus on the information stored in the storage device 62.

The storage device 62 stores the learning model 41 generated by the learning device 21, risk data 70, and a judgment program 71. As shown in FIG. 11, the risk data 70 is data indicating the risk for each of the seven items 1A to 3C. In this embodiment, for example, the risk level of "1" is associated with item 1A indicating "five or more support legs." The judgment program 71 is a program that is executed when judging whether the situation at the work site is dangerous.

The degree of danger for items 1A to 3C generally varies depending on the work site, the worker, etc. For example, if the distance between the work platform and the door at a work site is very far, the worker working at the work platform will be hardly affected by the door itself or by other workers entering and exiting through the door. In such a situation, the degree to which the door poses a danger at the work site will be low.

Furthermore, if the worker assigned to the work site is an experienced worker and has a high level of safety awareness, the worker is less likely to take dangerous actions. On the other hand, if an inexperienced worker is assigned to the work site, the worker is more likely to cause an accident. In this way, it is preferable to determine the value of the risk level data 70 taking into consideration the environment of the work site, the experience of the worker, and the like. As will be described in detail later, the risk level data 70 is set, for example, by the site supervisor operating the terminal 23. For this reason, in this embodiment, the risk levels of items 1A to 3C can be set to appropriate values according to the actual degree of risk, in accordance with the work site, etc.
In this embodiment, the risk level data 70 is set by the site supervisor, but this is not limited to this. For example, risk level data 70 of a predetermined value may be stored in the storage device 62 in advance.

<<<Functional blocks of the determination device 22>>>
12 is a diagram showing an example of functional blocks realized in the determination device 22 when the CPU 60 of the determination device 22 executes the determination program 71. The determination device 22 includes a receiving unit 80, an acquiring unit 81, a processing unit 82, an output unit 83, an adding unit 84, a determination unit 85, and an alarm unit 86. In the following description, an image of the work site 15 captured by the camera 20 is referred to as an image 400.

The receiving unit 80 receives the risk level data 70 set by the terminal 23 and stores it in the storage device 62. In this embodiment, the storage device 62 in which the risk level data 70 is stored corresponds to the "data storage unit." Note that although the receiving unit 80 stores the risk level data 70 in the storage device 62, it may also store it in the memory 61 or in a storage area of a cloud server. In this case, the portion in which the risk level data 70 is stored corresponds to the "data storage unit."

The acquisition unit 81 acquires the image 400 transmitted from the camera 20. The processing unit 82 inputs the image 400 into the learning model 41, and causes the learning model 41 to output an item corresponding to the risk factor in the image 400 from among the seven items 1A to 3C.

The output unit 83 uses the risk data 70 stored in the storage device 62 to output the risk level for the item in the image 400 according to the risk factor. In this embodiment, the risk level of item x (x is 1A to 3C) is referred to as risk level R(x). The adder 84 adds the risk levels R(x) output from the output unit 83 to calculate a risk score.

Incidentally, among the risk level R(x), the risk levels R(3A) to R(3C) caused by the worker's behavior only occur when the worker commits problematic behavior. Therefore, if the worker repeatedly commits problematic behavior within a specified period (e.g., 30 minutes), the risk score will decrease each time the problematic behavior ceases.

However, in an actual work site, if a worker repeatedly engages in problematic behavior, the likelihood of an accident occurring increases, and so this condition needs to be reflected in the risk score. Therefore, the addition unit 84 of this embodiment holds and adds the risk levels R(3A) to R(3C) attributable to the worker's actions that are output during a predetermined period (e.g., 30 minutes) out of the seven risk levels R(1A) to R(3C). By performing such calculations, the addition unit 84 can more accurately output a risk score that corresponds to the degree of risk at the work site.

When the danger score, which is the result of the addition by the addition unit 84, is greater than a predetermined value, the determination unit 85 determines that the situation at the work site 15 is dangerous. In addition, when the warning unit 86 determines that the situation at the work site 15 is dangerous, it transmits a warning to the terminal 23.

<<<<Example of risk determination process S30>>>
Fig. 13 is a flowchart showing an example of the risk determination process S30 executed mainly by the determination device 22. Fig. 14 is a flowchart showing an example of the risk score calculation process 44. Fig. 15 is a diagram showing an example of the transition of the risk score at the work site 15. Fig. 15 illustrates the transition of the risk score at the work site 15 between 9:00 and 16:00.

In the following, in this embodiment, a worker climbs onto the work platform 210 at the work site 15 shown in FIG. 1, for example, and performs piping work on the ceiling (not shown).

First, the site supervisor of the work site 15 operates the terminal 23 to input the risk level data 70 before the start of work (for example, 8:50). The terminal 23 accepts the input risk level data 70 (S40). In this embodiment, it is assumed that the risk level data 70 shown in FIG. 11 has been input by the site supervisor. As described above, the site supervisor can freely set the value of the risk level data 70, taking into consideration the environment of the work site 15, the skill level of the workers, and the like. For example, in the work site 15 of FIG. 1, the distance between the door 200 and the workbench 210 is short, and the area of the opening 201 is small. For this reason, the site supervisor sets the risk level of the door item 2A to "2" and the risk level of the opening item 2B to "1", which is lower than the risk level of the door.

Then, the terminal 23 transmits the received risk level data 70 to the determination device 22 (S41). The receiving unit 80 of the determination device 22 receives the risk level data 70 (S42) and stores the risk level data 70 in the storage device 62 (S43). Thereafter, the worker turns on the camera 20 at 9:00, which is the work start time, and starts work at the work site 15. When the camera 20 is turned on, the determination device 22 executes a process (FIG. 14) to calculate a risk score for the work site 15 based on the image 400 of the work site 15 transmitted from the camera 20 (S44).

First, the acquisition unit 81 acquires the image 400 transmitted from the camera 20 (S50). Then, when the image 400 is acquired, the processing unit 82 inputs the image 400 into the learning model 41, and causes the learning model 41 to output items corresponding to risk factors at the work site 15 (S51). Here, for example, at 9:00, the worker has not yet performed piping work. Also, in this embodiment, the number of support legs of the workbench 210 is three, there is no defect in the handrail 222, and it is in a safe state. Therefore, at this timing, the learning model 41 will output only item 2A for "door" and item 2B for "opening" out of the seven items 1A to 3C in FIG. 11.

Next, the output unit 83 outputs the risk levels of the output items (S52). For example, at the time of 9 o'clock, the risk levels R(2A) and R(2B) of the two items 2A and 2B are output. The addition unit 84 then adds the output risk levels to calculate a risk score (S53). Here, the risk level R(2A) is "2" and the risk level R(2A) is "1", so for example, the risk score at 9 o'clock is "3". The addition unit 84 then stores the calculated risk score in the storage device 62 (S54).

As shown in FIG. 13, the determination unit 85 compares the danger score with a predetermined value indicating that the work site 15 is dangerous (S45). In this embodiment, the "predetermined value" is a value ("7") that is set in advance by the site supervisor and is stored in the storage device 26. Therefore, for example, at the time of 9 o'clock, the danger score "3" is smaller than the predetermined value "7" (S45: No), and the danger score calculation process S44 described above is repeated.

After that, for example, at 13:00, the worker extends the upper half of his body far outside the handrail 222 of the work platform 210. In such a case, the learning model 41 outputs item 2A for "door" and item 2B for "opening", as well as item 3B for "upper body protruding" (S51). Therefore, the output unit 83 outputs the risk levels R(2A), R(2B), and R(3B) for the three items 2A, 2B, and 3B (S52).

In this case, the adding unit 84 adds the three output risk levels (S53), so the risk score becomes "6." In this example, the adding unit 84 holds the risk level caused by the worker's actions for 30 minutes. Therefore, the risk score value based on risk level R (3B) (here, "3") will be held until 1:30 p.m.

Then, for example, at 13:10, the worker puts his/her feet far out over the handrail 222 of the work platform 210. In such a case, the learning model 41 outputs item 2A for "door" and 2B for "opening", as well as item 3C for "feet sticking out" (S51). Therefore, the output unit 83 outputs the risk levels R(2A), R(2B), and R(3C) of the three items 2A, 2B, and 3C (S52). At this time, the adding unit 84 adds the risk level R(3B) related to behavior among the risk levels output at 13:00 to the three risk levels output this time (S53), so the risk score becomes "8".

When the danger score becomes "8", the determination unit 85 determines that the danger score is greater than a predetermined value ("7") (S45: Yes). Therefore, the alarm unit 86 sends an alarm to the terminal 23 (S46). When the terminal 23 receives the alarm, it displays the alarm on a display screen (not shown) of the terminal 23. As a result, the site supervisor can understand that the work site 15 is in a dangerous state. After "Yes" is selected in process S45, process S44 is executed. Therefore, the determination device 22 continues to calculate the danger score of the work site 15.

In this embodiment, when the predetermined period of 30 minutes has passed since 13:00, when the worker puts his or her upper body far out, the danger level of item 3C is cleared, and the danger score is reduced from "8" to "5." Furthermore, when the predetermined period of 30 minutes has passed since 13:10, when the worker puts his or her legs far out, the danger level of item 3B is cleared, and the danger score is reduced from "5" to "3."

In this way, the determination device 22 of this embodiment can objectively determine whether the situation at the work site 15 is dangerous or not, based on the image 400 from the camera 20. Note that the support legs 221 of the work platform 210 at the work site 15 have three steps, which is fewer than five steps, and therefore the height of the work platform 210 is, for example, less than 1.5 m. For this reason, item 1A is not output here. Also, the handrails 222 of the work platform 210 are safe, and therefore item 1B is not output.

However, if the support legs 221 of the work platform 210 have five or more steps and the handrails 222 are in an unsafe state, items 1A and 1B are output. In such a case, the risk level for each of items 1A and 1B corresponds to the "first risk level." The risk level for each of items 2A and 2B corresponds to the "second risk level," and the risk level for each of items 3A to 3C corresponds to the "third risk level."

<<Other Examples of Teacher Data D1>>
Fig. 16 is a diagram showing an example of teacher data D1 including a work platform that does not pose a safety problem. The target location of teacher data D1 in Fig. 16 is labeled with a correct answer label 300 of "work platform" indicating the target equipment, a correct answer label 302 of "handrail safe" indicating a safe handrail, and a correct answer label 320 of "ladder" used to estimate the height of the equipment.

In FIG. 16 and FIG. 4 described above, the objects with the same reference numerals are the same, so here, the correct label 320 will be explained. Here, the correct label 320 "ladder" is attached to a pair of objects, which are the two steps of the left and right support legs of the workbench. When the teacher data D1 with such a correct label 320 is used, the determination device 22 can, for example, more accurately distinguish between rectangular steel materials and support legs, which are common in general construction sites.

====Summary====
The above describes the information processing system 10 according to the present embodiment. The determination device 22 uses the learning model 41 that is trained using the teacher data D1 to D3. Therefore, the determination device 22 can detect multiple risk factors in a work site.

The output unit 83 also uses the risk data 70 to output the risk level of the item in the image 400 according to the risk factor (for example, S52 in FIG. 14). Therefore, by referring to such risk levels, the site supervisor can more objectively determine whether the work site 15 is in a dangerous situation.

The determination unit 85 also determines whether the work site 15 is dangerous or not based on the degree of danger of the items corresponding to the risk factors in the image 400 (for example, S45 in FIG. 13). Therefore, the site supervisor can immediately determine whether the work site 15 is in a dangerous situation based on the determination result.

As described above, the addition unit 84 retains and adds the risk level corresponding to the worker's behavior among the items corresponding to risk factors in the image 400 for 30 minutes. In general, if the number of times a worker performs a dangerous behavior increases in a specified period, the possibility of the worker being injured increases. The determination device 22 of this embodiment can reflect such behavior that is likely to lead to injury to the worker in the risk score, and can therefore more accurately detect that the work site 15 is in a dangerous state.

The risk level according to items 1A to 3C varies greatly depending on where the equipment used by the worker is located at the work site, as well as the experience of the worker. In this embodiment, the site supervisor can set the risk level before work begins, so the determination device 22 can evaluate the risk of the work site 15 based on a risk level that is more suited to the actual situation at the work site.

In addition, in this embodiment, the determination unit 85 determines that the work site 15 is dangerous when the danger score is greater than a predetermined value, but this is not limited to the above. For example, the determination unit may immediately determine that the work site 15 is dangerous when a predetermined item (e.g., item 3B) out of the seven items 1A to 3C is output. Even with this configuration, the site supervisor can grasp the danger of the work site 15.

In addition, in this embodiment, the judgment device 22 assumes that the "observation target" is a worker and outputs items 3A to 3C corresponding to risk factors based on the worker's actions, but this is not limited to this. For example, the "observation target" may be equipment or tools (e.g., drills, gas burners) that are installed and operated at the work site 15. By using training data that includes safe and unsafe actions as correct answer labels for such equipment, etc. and generating a learning model, the judgment device can detect risk factors caused by the operation of the equipment. Even when such equipment, etc. is the observation target, the judgment device can detect risk factors at the work site.

In this embodiment, the work site is a room in a building, but is not limited to this and may be outdoors. In such a case, for example, examples of "equipment" at the work site include cranes, excavators, concrete mixers, etc. Furthermore, when the work site is outdoors, examples of risk factors according to the "environment" of the work site include utility poles, high-voltage lines, trees, roads, etc. Therefore, when a determination device similar to this embodiment is used at an outdoor work site, risk factors at the outdoor work site can be detected.

The above embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the scope of the present invention. Furthermore, the present invention may be modified or improved without departing from the spirit of the present invention, and it goes without saying that the present invention includes equivalents.

10 Information processing system 15 Work site 20 Camera 21 Learning device 22 Determination device 23 Terminal 30, 60 CPU
31, 61 Memory 32, 62 Storage device 33, 63 Input device 34, 64 Display device 35, 65 Communication device 40 Learning program 41 Learning model 50 Acquisition unit 51 Learning unit 70 Risk level data 71 Determination program 80 Reception unit 81 Acquisition unit 82 Processing unit 83 Output unit 84 Addition unit 85 Determination unit 86 Alarm unit 200 Door 201 Opening 210 Portable workbench (workbench)
220 Platform 221 Support leg 222 Handrail 300-303, 310-315, 320 Correct label D1-D3 Teacher data

Claims (4)

An acquisition unit that acquires an image of a work site where the predetermined equipment is installed;
a processing unit which inputs an acquired image to a learning model which has been trained to output , when an image of a work site is input, a first group of items corresponding to risk factors of a specific piece of equipment at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed object at the work site in the input image, and causes the learning model to output an item corresponding to the risk factor in the acquired image from the first to third groups of items;
an output unit that uses data including a risk level for each of the items included in the first to third item groups to output a risk level for each of the items corresponding to a risk factor in the acquired image;
an adder that adds together a first risk level for each of the items included in the first group of items, a second risk level for each of the items included in the second group of items, and a third risk level for each of the items included in the third group of items, among the risk levels for items corresponding to risk factors in the acquired image, the third risk level being output together with the first and second risk levels and held for a predetermined period of time;
a determination unit that determines that a situation of the work site in the acquired image is dangerous when a result of the addition by the addition unit is greater than a predetermined value;
An information processing device comprising: 2. The information processing device according to claim 1 ,
A receiving unit that receives the data transmitted from the terminal and stores the data in a data storage unit.
Information processing device. 3. The information processing device according to claim 1 ,
The observation target is a worker performing work at the work site.
Information processing device. acquiring an image of a work site where a predetermined facility is installed;
a step of inputting the acquired image to a learning model that has been trained to output , when an image of a work site is input, a first group of items corresponding to risk factors of a specific piece of equipment at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed object at the work site in the input image, and outputting an item corresponding to the risk factor in the acquired image from among the first to third groups of items;
outputting a degree of risk for each item corresponding to a risk factor in the acquired image by using data including a degree of risk for each item included in the first to third item groups;
a step of adding up a first risk level for each of the items included in the first group of items, a second risk level for each of the items included in the second group of items, and a third risk level for each of the items included in the third group of items, among the risk levels for items corresponding to risk factors in the acquired image, the third risk level being output together with the first and second risk levels and held for a predetermined period of time;
determining that the situation of the work site in the acquired image is dangerous when the sum is greater than a predetermined value;
An information processing method comprising:

Images