JP7722224B2 - Information processing device and lighting system - Google Patents

Description

Embodiments of the present invention relate to an information processing device and a lighting system.

Technology has been proposed to detect the work being done by a worker based on images captured by an imaging unit.

Japanese Patent Application Laid-Open No. 2019-120577

However, conventional technology leaves room for improvement in the method for setting the analysis scope for checking worker work.

The problem that this invention aims to solve is to provide an information processing device and lighting system that can accurately set the analysis range.

An information processing device according to an embodiment includes an acquisition unit and a setting unit. The acquisition unit acquires captured images captured by a camera. The setting unit sets an analysis range for analyzing movements related to a predetermined task based on markers included in the acquired captured images. The setting unit sets the analysis range to a range that includes locations where the worker's hand movements are greater than a preset threshold, based on the markers and a heat map corresponding to the worker's movements.

The present invention allows the analysis range to be set with high precision.

FIG. 1 is a perspective view showing an example of the appearance of a lighting device included in a lighting system according to an embodiment. FIG. 2 is a diagram illustrating an overview of a lighting system according to an embodiment. FIG. 3 is a block diagram showing the configuration of the information processing apparatus according to the embodiment. FIG. 4 is a diagram showing an example of setting an analysis range in a captured image. FIG. 5 is a diagram showing an example of setting an analysis range in a captured image. FIG. 6 is a flowchart illustrating the process of setting the analysis range according to the embodiment. FIG. 7 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. FIG. 8 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. FIG. 9 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. FIG. 10 is a diagram showing an example of setting an analysis range in a captured image according to a modified example.

The information processing device 20 according to the embodiment described below includes an acquisition unit 231 and a setting unit 234. The acquisition unit 231 acquires a captured image P captured by the camera 16. The setting unit 234 sets an analysis range R for analyzing movements related to a specified task, based on a marker M included in the acquired captured image P.

In addition, the setting unit 234 in the embodiment described below sets the analysis range R based on the coordinates of the marker M.

In addition, the setting unit 234 in the embodiment described below sets the analysis range R based on the marker M placed on the workbench.

In addition, the setting unit 234 in the embodiment described below sets the analysis range R based on the marker M, whose installation location on the workbench can be changed.

In addition, the setting unit 234 in the embodiment described below sets the analysis range R based on multiple markers M.

In addition, the setting unit 234 according to the embodiment described below sets an analysis range R for each of the multiple markers M.

In addition, the setting unit 234 in the embodiment described below sets the analysis range R2 based on the markers and a heat map corresponding to the worker's movements.

In addition, the setting unit 234 in the embodiment described below can switch the setting of the analysis range between automatic setting based on the marker and manual setting.

The lighting system 1 according to the embodiment described below includes an information processing device 20 and a lighting device. The lighting device includes a camera 16 and a lighting device.

[Lighting equipment]
First, the configuration of a lighting device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a perspective view showing an example of the appearance of the lighting device 10 included in a lighting system 1 according to an embodiment.

As shown in FIG. 1, the lighting device 10 comprises an illumination unit 11, an imaging unit 12, and a main body unit 13. The lighting device 10 is a ceiling-mounted lighting device in which the main body unit 13 is installed on a ceiling surface or the like, and light output from the illumination unit 11 is irradiated onto the floor surface. The lighting device 10 is primarily used indoors, for example, for monitoring production lines in factories or the status of employees in offices.

For ease of understanding, Figure 1 illustrates a three-dimensional Cartesian coordinate system including a Z axis, with the positive direction being vertically downward and the negative direction being vertically upward. The X axis is illustrated as running along the length of the lighting device 10, and the Y axis is illustrated as running along the width of the lighting device 10.

The lighting unit 11 is a lighting bar that has multiple light-emitting elements (not shown) arranged at predetermined intervals on a long chassis or substrate (not shown) that is arranged along the Y-axis direction, and a diffusion cover 14 is provided on the floor side, i.e., on the positive Z-axis side, so that the light-emitting elements are housed between the chassis and the lighting bar.

The diffusion cover 14 is made of a translucent material such as acrylic or polycarbonate. The diffusion cover 14 is frosted to diffuse the light emitted from the multiple light-emitting elements. The diffusion cover 14 may also contain diffusing materials or colorants as appropriate.

The imaging unit 12 has a light-shielding cover 15 and a camera 16. The imaging unit 12 is arranged adjacent to the illumination unit 11 on the negative side of the X axis.

The main body 13 holds the lighting unit 11 and the imaging unit 12. The main body 13 also serves as a mounting member for attaching the lighting device 10 to a predetermined position, such as a ceiling.

The light-shielding cover 15 is disposed on the positive Z-axis side of the main body 13 so as to cover the camera 16 between the main body 13 and the light-shielding cover 15. The light-shielding cover 15 also has a through-hole provided in a position facing the lens of the camera 16. By disposing such a light-shielding cover 15, it is possible to make it difficult for light emitted from the diffusion cover 14 of the lighting unit 11 to enter the lens of the camera 16.

[Lighting System]
Next, the configuration of the lighting system 1 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an overview of the lighting system 1 according to the embodiment. Note that the following description will be given taking as an example a case where the lighting system 1 is installed in a factory.

As shown in FIG. 2, the lighting system 1 according to the embodiment includes a plurality of lighting devices 10 and an information processing device 20. The example shown in FIG. 2 illustrates a case in which each lighting device 10 is installed at a factory work site (e.g., a production line).

For example, multiple lighting devices 10 are installed above the workbenches 40 (see Figure 4) of each worker 50 on a production line, and a camera 16 (see Figure 1) captures images of each workbench 40 from above.

The information processing device 20 is a device that manages each lighting device 10 installed at the work site. For example, the information processing device 20 remotely controls the lighting mode of the lighting unit 11 of each lighting device 10 and the imaging processing of the camera 16 via the network N.

The information processing device 20 also collects and stores captured images P (see FIG. 4) captured by the camera 16 of each lighting device 10. For example, the captured images P collected by the information processing device 20 are used to confirm and analyze the work content, monitor the work site, and so on. Note that in the present disclosure, the information processing device 20 may acquire captured images P as still images or as moving images.

[Information processing device]
Next, the configuration of the information processing device 20 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 20 according to the embodiment. As shown in Fig. 3, the information processing device 20 includes a communication unit 21, a storage unit 22, and a control unit 23. The information processing device 20 is, for example, a cloud server.

The communication unit 21 is realized, for example, by a network interface card (NIC). The communication unit 21 transmits and receives information to and from multiple lighting devices 10 via, for example, the network N.

The storage unit 22 functions as a storage unit in the cloud server. The storage unit 22 is realized, for example, by a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disk.

For example, the memory unit 22 stores a captured image P captured by the camera 16. The memory unit 22 stores an analysis range R in the captured image P.

The control unit 23 functions as a control unit in the cloud server. The control unit 23 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the storage device inside the information processing device 20 using RAM as a working area. The control unit 23 is also, for example, a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 23 includes an acquisition unit 231, a detection unit 232, a determination unit 233, and a setting unit 234, and realizes or executes the functions and actions of the control processing described below. Note that the internal configuration of the control unit 23 is not limited to the configuration shown in FIG. 3, and may be any other configuration that performs the control processing described below.

The acquisition unit 231 acquires captured images P captured by the cameras 16 (see Figure 1). The acquisition unit 231 acquires captured images P from each camera 16 via the communication unit 21.

The detection unit 232 performs various detections. For example, the detection unit 232 detects a marker M from the captured image P. The detection unit 232 detects the coordinates of the marker M in the captured image P. The marker M is placed on the workbench 40. The placement location of the marker M on the workbench 40 can be changed. The marker M is placed by, for example, an administrator. The marker M may also be placed by a worker 50 performing work on the workbench 40.

The marker M is, for example, a symbol written on a card. For example, the marker M is a QR code (registered trademark) written on a card. The symbol may be a circle, a square, a cross, or the like. The marker M may be written on an object other than a card. The marker M may be written on an object with a three-dimensional shape. The marker M may be the shape of the card itself or the three-dimensional shape of the object itself.

The marker M is placed on the workbench 40, for example, via a member that can be attached to and detached from the workbench 40. It is desirable that the marker M be made of a material and have a shape that makes it difficult for it to fall off the workbench 40.

The detection unit 232 performs various detections by performing predetermined image processing on the acquired captured image P. The predetermined image processing is performed, for example, by using a detection model. The detection model is obtained by learning using a machine learning algorithm such as a deep neural network (DNN). The detection model may be obtained, for example, by supervised learning. The detection model may be configured to use a machine learning algorithm such as an SVM (Support Vector Machine) that uses HOG (Histogram Of Gradient) features. The detection unit 232 may also be configured to perform various detections using, for example, template matching, without using a detection model that has undergone machine learning. The detection unit 232 also detects the movement of the worker 50 from the captured image P. The detection unit 232 detects the positions of both hands, for example.

The determination unit 233 determines whether a marker M has been detected. For example, the determination unit 233 determines whether a marker M has been detected for each captured image P obtained.

The setting unit 234 sets an analysis range R for analyzing movements related to a predetermined task from the captured image P. The predetermined task is a task that is set in advance. For example, the predetermined task is a task performed on the workbench 40. Examples of predetermined tasks include attaching parts to an object, packaging an object, and picking up parts or work tools.

The setting unit 234 sets the analysis range R based on a marker M included in the captured image P. The setting unit 234 sets the analysis range R based on a marker M placed on the workbench 40. The setting unit 234 sets the analysis range R based on the coordinates of the marker M in the captured image P. For example, the setting unit 234 sets the coordinates of the analysis range R in the captured image P.

The setting unit 234 sets the analysis range R according to the marker M detected by the detection unit 232. The setting unit 234 sets the analysis range R according to the type of marker M. The marker M has information for setting the analysis range R. The information for distinguishing the method for setting the analysis range R includes the type of marker M, the shape of the marker M, and the pattern of the marker M.

For example, if the marker M detected by the detection unit 232 is a QR code, the setting unit 234 sets an analysis range R with a shape and size specified by the QR code.

For example, if the marker M is a QR code and the QR code indicates an analysis range R that extends above the captured image P and to the right of the captured image P, the analysis range R is set for the captured image P as shown in Figure 4. Figure 4 is a diagram showing an example of setting the analysis range R for the captured image P.

For example, if the marker M is a circle symbol and indicates an analysis range R having a predetermined radius relative to the circle symbol, the analysis range R is set for the captured image P as shown in FIG. 5. FIG. 5 is a diagram showing an example of setting the analysis range R for the captured image P. Note that the method for setting the analysis range R is not limited to the above method. For example, the analysis range R may be rectangular with the marker M at its center. For example, the analysis range R is set relative to the marker M in terms of the distance from the marker M and the extension direction relative to the marker M.

The information processing device 20, for example, collects work information of workers 50 within the set analysis range R and performs analysis to determine whether the work was performed according to a pre-registered work procedure. The information processing device 20 also measures the takt time of the work, etc.

[Analysis range setting process]
Next, the process of setting the analysis range R according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating the process of setting the analysis range R according to the embodiment.

The information processing device 20 acquires from the camera 16 a captured image P captured by the camera 16 (S100). The information processing device 20 performs predetermined image processing on the acquired captured image P to detect markers M (S101).

The information processing device 20 determines whether or not a marker M is present in the acquired captured image P (S102). If a marker M is not present in the captured image P (S102: No), the information processing device 20 ends the current processing.

If a marker M is present in the captured image P (S102: Yes), the information processing device 20 sets an analysis range R according to the marker M (S103).

[effect]
The information processing device 20 includes an acquisition unit 231 and a setting unit 234. The acquisition unit 231 acquires a captured image P captured by the camera 16. The setting unit 234 sets an analysis range R for analyzing movements related to a predetermined task, based on a marker M included in the acquired captured image P.

This allows the information processing device 20 to accurately set the analysis range R for the captured image P. For example, even if the work content on the workbench 40 changes or the layout of the workbench 40 changes, the information processing device 20 can accurately set the analysis range R. The information processing device 20 can also shorten the time required to set the analysis range R. For example, compared to when the analysis range R is set manually by an administrator or the like, the information processing device 20 can shorten the time required to set the analysis range R and can also easily set the analysis range R.

The setting unit 234 sets the analysis range R based on the coordinates of the marker M.

This allows the information processing device 20 to accurately set the analysis range R for the marker M. The information processing device 20 can suppress deviations and variations in the analysis range R. As a result, the information processing device 20 can accurately detect the analysis results in the analysis range R, such as the movements and work content of the worker 50, and improve the accuracy of the analysis results of the movements and work content of the worker 50.

The setting unit 234 sets the analysis range R based on the marker M set on the workbench 40.

This allows the information processing device 20 to accurately set the analysis range R for the workbench 40. Therefore, for example, even if the work content on the workbench 40 changes or the layout of the workbench 40 changes, the information processing device 20 can accurately set the analysis range R for the workbench 40.

The setting unit 234 sets the analysis range R based on a marker M whose installation location on the workbench 40 can be changed.

As a result, even if the work content on the workbench 40 changes, for example, the marker M can be moved in accordance with the work content, allowing the information processing device 20 to accurately set the analysis range R. Even if the work content on the workbench 40 changes, the information processing device 20 can easily set an appropriate analysis range R.

[Modification]
The information processing device 20 according to the modified example may automatically set the analysis range R for the captured image P after the analysis range R has been set once, until a reset operation for the analysis range R is performed. For example, even if the analysis range R is set and the marker M is removed from the workbench 40, the information processing device 20 may set the same analysis range R for the captured image P. The reset operation is performed, for example, by an administrator. The reset operation may also be performed at a predetermined timing. The predetermined timing is, for example, the timing when work begins. The predetermined timing includes a predetermined time.

The information processing device 20 according to the modified example may set the analysis range R based on multiple markers M. The information processing device 20 according to the modified example may set the analysis range R according to the type of multiple markers M. For example, as shown in FIG. 7, the information processing device 20 may set the analysis range R based on the coordinates of multiple related markers M. In other words, the information processing device 20 may set the analysis range R according to the positional relationship of the multiple markers M. FIG. 7 is a diagram showing an example of setting the analysis range R in a captured image P according to the modified example. The multiple markers M in FIG. 7 are markers M for which the range of a polygon with the multiple markers M as vertices is set as the analysis range R.

This allows the information processing device 20 to accurately set the analysis range R based on multiple markers M. Furthermore, the information processing device 20 can set an analysis range R with a complex shape.

The information processing device 20 according to this modification may set an analysis range R for each of multiple markers M, as shown in FIG. 8. FIG. 8 shows an example of setting the analysis range R in a captured image P according to this modification.

This allows the information processing device 20 to set multiple analysis ranges R. For example, the information processing device 20 can analyze work for each analysis range R. For example, if the work procedures on the workbench 40 have been determined, the information processing device 20 can easily confirm the work procedures by setting the analysis range R for each range in which the worker's 50 hands move according to the work procedures. In addition, for example, the information processing device 20 can easily measure the takt time for each work procedure.

As shown in FIG. 9, the information processing device 20 according to this modification may set an analysis range R that includes the worker 50. FIG. 9 is a diagram showing an example of setting the analysis range R in a captured image P according to this modification. In other words, the marker M may include the worker 50.

The marker M may also include a box containing parts to be used in the work. For example, the information processing device 20 may detect information about the parts placed on the workbench 40 from the captured image P. For example, the information processing device 20 may detect part information from the part's model number written on the box containing the part, or a QR code written on the box. The information processing device 20 may also detect the type of part based on the captured image P.

The information processing device 20 according to the modified example may be able to adjust the analysis range R set based on the marker M. For example, when a worker 50 is performing work with his or her hands outside the analysis range R set based on the marker M, the information processing device 20 may set the analysis range R to include the range in which the worker 50's hands move. For example, the information processing device 20 according to the modified example may set the analysis range R based on a heat map.

In a modified example, the information processing device 20 may detect the hand movements of the worker 50 from the captured image P and create a heat map based on the hand movements of the worker 50. The heat map may also be created by another information processing device. The heat map is created based on the number of hand movements of the worker 50 within the field of view of the captured image P captured by the camera 16, etc.

For example, in the heat map, areas where the worker 50 moves a lot and areas where the worker 50 moves less are distinguished by different colors, etc. The information processing device 20 according to the modified example may set the analysis range R based on the marker and the heat map. The information processing device 20 according to the modified example sets the analysis range R to a range that includes a predetermined range that includes areas in the heat map where the worker 50's hand movements are more than a preset threshold. For example, the information processing device 20 according to the modified example sets the analysis range R2 by adding an analysis range based on the heat map to the analysis range R that is set based on the marker M.

For example, as shown in FIG. 10, the information processing device 20 sets the analysis range R2 by adding a range including a predetermined range R1 that includes locations in the heat map where the worker's 50 hand movements are greater than a preset threshold to the analysis range R set based on the marker M. FIG. 10 is a diagram showing an example of setting the analysis range R2 in a captured image P of a modified example. In FIG. 10, the analysis range R2 to which the range including the predetermined range R1 has been added is indicated by a dashed line.

As a result, the information processing device 20 according to the modified example can set the analysis range R2 based on a heat map corresponding to the movements of the worker 50, thereby accurately setting the analysis range R2 for the work of the worker 50. As a result, the information processing device 20 can improve the accuracy of the analysis results in the analysis range R2.

In a modified example, the information processing device 20 may be able to switch the setting of the analysis range R between automatic setting based on the marker M and manual setting. This allows the information processing device 20 to switch the setting method of the analysis range R depending on, for example, the purpose of use.

Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be embodied in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

REFERENCE SIGNS LIST 1 Lighting system 10 Lighting device 11 Lighting section 12 Imaging unit 16 Camera 20 Information processing device 21 Communication section 22 Storage section 23 Control section 40 Workbench 50 Worker 231 Acquisition section 232 Detection section 234 Setting section M Marker P Captured image R Analysis range R2 Analysis range

Claims (8)

an acquisition unit that acquires an image captured by the camera;
a setting unit that sets an analysis range for analyzing a movement related to a predetermined task based on a marker included in the acquired captured image;
Equipped with
The setting unit is an information processing device that sets an analysis range to a range that includes locations where the worker's hand movements are greater than a predetermined threshold based on the markers and a heat map corresponding to the worker's movements . The information processing device according to claim 1 , wherein the setting unit sets the analysis range based on coordinates of the marker. The information processing device according to claim 1 , wherein the setting unit sets the analysis range based on the marker placed on a workbench. The information processing apparatus according to claim 3 , wherein the setting unit sets the analysis range based on the marker, the installation location of which on the workbench can be changed. The information processing device according to claim 1 , wherein the setting unit sets the analysis range based on a plurality of markers. The information processing device according to claim 1 , wherein the setting unit sets the analysis range for each of a plurality of markers. The information processing device according to claim 1 , wherein the setting unit is capable of switching the setting of the analysis range between automatic setting based on the marker and manual setting. An information processing device according to any one of claims 1 to 7 ;
an illumination device having the camera and an illumination unit;
A lighting system comprising: