JP7809162B2 - surveillance system - Google Patents

Description

The present invention relates to a monitoring system that monitors people, things, and machines at manufacturing sites.

Systems that monitor the people involved in manufacturing, the products being manufactured, and the machines that carry out the manufacturing process (manufacturing equipment, inspection equipment, packaging equipment, etc.) are known at manufacturing sites, and further improvements to these systems are being made (see, for example, Patent Documents 1 to 3).

For example, at manufacturing sites in the assembly industry, multiple parts are aggregated and combined to produce a product through tasks (e.g., welding, cutting, assembly, etc.) in accordance with prescribed assembly drawings and work procedures. In the case of complex products, predetermined tasks are carried out at multiple work sites, and each part (unit) is combined to create a more complex product. In such work, when a large number of parts (items) are aggregated, the work procedures, site layout, and part aggregation work become complicated. This can lead to issues such as waiting for tasks, parts being backed up, timing discrepancies in part aggregation, and worker (human) placement errors, resulting in reduced assembly work efficiency. However, particularly in the assembly industry, where equipment and machinery are rarely used and assembly work is primarily performed by humans, a sensor-based logging environment is not yet in place. For this reason, there is a demand for monitoring systems that can accurately grasp and monitor the status of people and items at manufacturing sites. In other words, there is a need to build a digital twin of the assembly industry and utilize it to improve and monitor management figures.

However, when quantifying and recording a person's work status, the worker must stop what they are doing and record their work status, or another worker must observe and record the worker's movements, which poses problems of time and cost.

Furthermore, with regard to monitoring the status of an object, when the object is inside a machine, it is possible to record and monitor the machine's operations, but when an object is moved from a machine at one work site to a machine at another work site, its status cannot be adequately monitored.

Patent No. 5416322 Japanese Patent Application Laid-Open No. 2019-139570 Japanese Patent Application Laid-Open No. 2019-16226

The present invention provides a monitoring system that can accurately grasp the movements of people and objects, accurately monitor the manufacturing process, and contribute to improving work efficiency.

To solve the above problems, the monitoring system of the present invention is a monitoring system that monitors people, objects, and equipment at a manufacturing site, and includes a camera that monitors the people, objects, or equipment, an analysis unit that analyzes video data obtained by the camera, and a display unit that displays the analysis results by the analysis unit. The analysis unit detects change points in the video data and analyzes the status of the people, objects, or equipment based on the change points.

The monitoring system according to the present invention can accurately monitor the manufacturing process and contribute to improving work efficiency.

1 is a schematic diagram illustrating an example of the configuration of a monitoring system 1 according to an embodiment. 1 is a schematic diagram for explaining the operation of a human monitoring camera 11. FIG. 1 is a schematic diagram for explaining the operation of the object monitoring camera 12 and the device monitoring camera 13. FIG. 1 is a schematic diagram for explaining the operation of the object monitoring camera 12 and the device monitoring camera 13. FIG. 4 is a flowchart illustrating the operation of the monitoring system 1 according to the embodiment. 10 is an example of a display screen of a display PC 17. 10 is an example of a display screen of a display PC 17. 10 is an example of a display screen of a display PC 17.

The present embodiment will be described below with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be denoted by the same numerals. Note that the accompanying drawings show embodiments and implementation examples in accordance with the principles of the present disclosure, but these are intended to aid in understanding the present disclosure and should not be used to interpret the present disclosure in any limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

In this embodiment, the present disclosure has been described in sufficient detail to enable those skilled in the art to implement the present disclosure. However, it should be understood that other implementations and forms are possible, and that changes to the configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited to this.

The following describes an embodiment of the process for assembling products (final products and units as intermediate products) composed of multiple parts. Parts ordered and delivered from the core system or ordering system are brought in and stored in a parts warehouse. Meanwhile, when a production number is issued in accordance with the product's manufacturing plan, the parts are transported to the assembly shop in accordance with the manufacturing plan (including assembly drawings and assembly procedures created by the design department) predefined for each production number. In this assembly shop, assembly personnel and the necessary equipment and jigs are allocated in accordance with the manufacturing plan. Assembly personnel perform the specified assembly work using the necessary equipment and jigs depending on the case, in accordance with the manufacturing plan.

After the required assembly work is completed, the product is transported to the next assembly shop. This process is repeated and integrated to complete the final product, which undergoes the required shipping inspection and is then stored in a shipping warehouse. This embodiment provides a means and method for recording the assembly process and work progress as it progresses within a factory. Specifically, the identification information for each part, each piece of equipment/jig, and each worker, as well as the start/end times and takeoff/departure times, are recorded for each warehouse, serving shelf, and assembly shop. In conventional technology, in order to detect the presence or absence of product parts, sensors had to be individually installed for each task, each shop, and each shop's configuration, depending on the work content and shop configuration. This required the installation location of the sensor to be changed whenever the work content or the size and shape of the part changed, for example, due to a product change. According to the embodiment described below, the camera's installation location does not need to be changed as long as the part, jig, or worker is within the camera's imaging range. This significantly reduces the installation costs when introducing the system.

An example of a monitoring system 1 according to an embodiment will be described with reference to Figure 1. As an example, this monitoring system 1 may be composed of a people monitoring camera 11, an object monitoring camera 12, a device monitoring camera 13, a transfer computer (PC) 14, a collection and storage server 15, an analysis server 16, and a display computer (PC) 17.

The human monitoring camera 11 is a camera used to monitor the movements of people (workers) performing tasks at a manufacturing site. The human monitoring camera 11 stores information about, for example, human shapes and movement characteristics in advance, and can be configured to quickly and reliably detect human movements.

The object monitoring camera 12 is a camera used to monitor the movement of objects manufactured and transported at a manufacturing site. As an example, the object monitoring camera 12 can be a camera that captures and identifies identification information (one-dimensional code, two-dimensional code, color chart, etc.) attached to a cart to monitor the movement of the cart on which the object is placed and transported. By assigning different identification information to each of multiple carts, the type of cart can be identified. However, this is merely an example, and the object monitoring camera 12 is not limited to monitoring the movement of carts using identification information. For example, the shape of the object itself may be recognized, or various identification information attached to the object may be identified. Furthermore, instead of recognizing carts, the object monitoring camera 12 may monitor the movement of trays placed on a conveyor belt, for example. If the object monitoring camera 12 is configured to read two-dimensional codes, it can be equipped with an application for analyzing two-dimensional codes. Of course, the application for analyzing two-dimensional codes may also be stored on the cloud.

The equipment monitoring camera 13 is a camera used to monitor the operation of various devices (manufacturing equipment, inspection equipment, packaging equipment, etc.) installed at the manufacturing site. The equipment monitoring camera 13 can be different depending on the characteristics of the device being imaged. For example, an equipment monitoring camera 13 for a device that requires human operation can be a camera that images the device's operation panel and monitors human operation. By imaging the operation panel, it is possible to determine whether the device is operating or stopped.

Today's facilities and equipment can detect parameters that indicate the state of the equipment using sensors placed within the equipment. However, if the equipment does not have these sensors, the equipment must be modified to install them. However, skilled workers who work with the equipment can visually and audibly determine the process stage of the equipment's processing, assembly, or other processes, and monitor the equipment for abnormalities. Therefore, TV cameras can also be used to monitor the equipment's condition. Introducing TV camera monitoring eliminates the need for costly monitoring systems, such as modifications to install sensors and monitoring by skilled workers.

In the case of an automated device that does not require human operation, the device monitoring camera 13 for that device can be a camera for monitoring, for example, the input port for items (parts) or the movement of moving parts (for example, the blade of a cutting device). Furthermore, the device monitoring camera 13 can also be used to inspect the content of various tasks within the device. For example, in a device for connecting harnesses, a harness connection confirmation inspection can be performed, and from the images, it can be determined whether the connections have been made correctly, and if not, errors such as incorrect wiring can be detected.

In addition, in this embodiment, cameras 11-13 may be equipped with an inspection object detection means for detecting inspection objects and a defect detection means for detecting defects, either within the cameras 11-13 or in an external controller. The inspection object detection means has the function of detecting the inspection object part in an image of a non-defective product. As an example, the inspection object detection means may be composed of a deep learning network that has previously learned the inspection object part. By providing this means, it is possible to avoid falsely detecting parts that are significantly deformed but not considered abnormal (for example, parts such as cables) as defective.

Then, by inputting the part detected as the part to be inspected into defect detection means, defects in the object to be inspected can be detected. As an example, this means can be configured by having an autoencoder learn multiple normal parts (images of normal products) in advance. By creating a difference image between the image reconstructed by the autoencoder and the input image, if there is an abnormal part, the image signal at that position will have a large value and will be detected as an abnormality through threshold processing. Detected defect information (defect images, product information, etc.) can be stored in defect information storage means.

When a new detection target is provided, its normal image can be trained by the inspection target detection means and defect detection means (autoencoder). This training may be performed using 3D CAD information generated during product design.

In addition, the information (network parameters) learned about new detection targets can be reflected by performing a weighted average with the corresponding parameters of an already trained network. This method also accommodates cases where users do not want to upload information about new detection targets to the cloud. In other words, by sending the trained network parameters shared on the cloud to the user's location and performing a weighted average at that location, it is possible to configure and provide a highly accurate defect detection method and inspection target detection method specific to the user.

The device monitoring camera 13 can use a variety of lenses (ultra-close-up lens, fisheye lens, wide-angle lens, telephoto lens, etc.) depending on the characteristics of the object to be imaged and the distance to the object.

The progress of assembly work at an assembly process site can be identified and recorded by visual inspection by skilled personnel or managers with specific skills. From this perspective, this embodiment is configured to calculate the takeoff and landing times to be recorded by performing appropriate data processing, as described below, using only image information captured by a TV camera using visible light.

In addition, when assembly work is completed at each assembly shop during the assembly process, not only is the completion time recorded, but the start and completion times for each assembly process, as well as any defects in the intermediate products that have been assembled, are also detected and recorded. The sampling time of this TV camera does not need to be the usual 30 fps or 60 fps, but can be thinned to a value such as 1 fps or 0.2 fps, as necessary. Conversely, high-speed imaging such as 120 fps or 2000 fps is also acceptable. The former has the effect of reducing the total amount of data stored, while the latter is necessary when collecting data involving high-speed movement.

The transfer PC 14 has the function of reading video data (which may include both video and still images) captured by the various cameras 11-13 and transferring it to the collection and storage server 15. The transfer PC 14 can also be configured to perform data compression using well-known methods prior to the transfer operation. In addition to data compression, it can also perform image processing and manipulation (for example, image processing such as leaving only the image portion of the person or object being monitored and deleting unnecessary images, or image processing such as increasing the resolution of the image of the monitored object and decreasing the resolution of the image of other objects). The transfer PC 14 may be connected to the collection and storage server 15 wirelessly or via a wired connection.

The collection and storage server 15 is a computer that collects and stores the video data transferred from the transfer PC 14 in a storage unit (not shown). The analysis of the collected video data is performed by the analysis server 16, but it is also possible for the collection and storage server 15 to be responsible for part of the analysis process, for example, part of the analysis of the video data.

The analysis server 16 analyzes the various types of video data stored in the collection and storage server 15 to analyze the movements of people, objects, and machines, and generates analysis data as the results of the analysis. The generated analysis data is output to the display PC 17 and displayed in a visualized form that can be recognized by the operator. The analysis server 16 can also be configured to compare the analysis data resulting from the analysis of the movements of people, objects, and machines with work plan data and analyze the differences.

The analysis items in the analysis server 16 can include, but are not limited to, items that a person (e.g., a site supervisor) might discover in the course of their daily supervisory duties. For example, signal A contained in video data acquired and accumulated (learned) over a long period of time in the collection and storage server 15 can be compared with signal A' contained in newly acquired video data, and the results of this comparison can be used as the analysis items. If there is a difference between signals A and A' obtained in this way that is greater than a predetermined value, the analysis server 16 will identify the location where this difference occurred as a "change point" that indicates a significant change in the behavior of a person, object, or machine.

Identifying such change points makes it possible to detect when significant behavior occurs in people, objects, and machines that would otherwise be overlooked by human monitoring, and can serve as an indicator of factors impeding work efficiency in the manufacturing process. Furthermore, when monitoring by humans, it is necessary to separate data showing correct behavior from data showing abnormal behavior. While correct data is easy to prepare, preparing incorrect data is time-consuming. This system eliminates the need to prepare incorrect data, allowing for easy system implementation in a short space of time. The analysis server 16 links the time and location at which such change points occur, and mathematically defines this as "work time at a specific work location."

Generally, simply having people and things in a specific place does not necessarily mean that work is progressing properly. However, by comparing and analyzing the work time data obtained as described above, it is possible to, for example,
(a) If the work currently being done deviates from the average work time,
(b) When there is a large variation in work time between multiple tasks,
(c) It is possible to "mechanically" detect when people or things are staying outside of a specific work location. These are different from the "work time" defined or recognized by humans, but it is possible to detect factors that hinder manufacturing lead time. That is, in (a), the hindering factor is "deviation from the average time," and in (b), the hindering factor is "large variation" in the work itself. In (c), the hindering factors are "waiting for work" and "stagnation of things."

The operation of the human monitoring camera 11 will be explained briefly with reference to Figure 2. Figure 2 shows a manufacturing site where multiple machines (1 to 9) are arranged, and how a human (worker) H moves along the arrows, captured by the human monitoring camera 11. As an example, the human monitoring camera 11 can store information on the physique of the human H, the characteristics (color, shape, etc.) of the clothing typically worn by the human H, and movement characteristics (average walking speed, stride length). The human monitoring camera 11 can identify the image of the human H contained in the video data by comparing this information with the video data. The identified image of the human H is surrounded by a bounding box BB, and the video data can be recorded in a form that includes the bounding box BB.

In this way, once video data is obtained by the human monitoring camera 11, the video data is analyzed by the analysis server 16, and it is possible to analyze, based on the identified change points in human behavior, which tasks human H performed and at what time periods (residence time), and which time periods he was not performing tasks but was moving around (movement time) or was in a position outside the captured range. Change points in human behavior can be identified based on the time the person arrived in front of a specific machine, the time they left, and other information. The bar display in the lower right of Figure 2 indicates the task times (including start and end times) for tasks (4) to (8). Such bar displays can also be displayed on the display screen of the display PC 17.

In this embodiment, the worker detection means is composed of a people monitoring camera 11 or an external controller. The people monitoring camera 11 as worker detection means detects workers in captured images and may include a deep learning network that has previously learned images of workers. This worker detection means detects workers in captured images. From the detected position, it is determined whether the worker is present at a specified location, i.e., the time of entry and exit from the work area. In this embodiment, captured images are captured from multiple directions, the worker's location (viewpoint direction) in each two-dimensional image is determined, and the intersection of these directions is detected as the worker's location in three-dimensional space. This method of detecting the worker's location from multiple images is effective in cases where high depth detection accuracy is desired. Furthermore, using multiple images also has the effect of enabling detection in cases where the worker is hidden in the shadow of equipment or other objects and does not appear in the image.

The operation of the object monitoring camera 12 and machine monitoring camera 13 will be explained generally with reference to Figure 3. Figure 2 illustrates an example in which three object monitoring cameras 12 (12A-12C) and three machine monitoring cameras 13 (13A-13C) are arranged, but this is merely an example, and the number of object monitoring cameras 12 and machine monitoring cameras 13 is not limited to a specific number. Furthermore, the object monitoring camera 12 is not limited to a specific type, but preferably can be a camera that photographs a two-dimensional code 22 attached to a cart 21 on which an object (part, etc.) is placed, as shown generally in Figure 3.

As shown in Figure 3, the cart 21 moves between multiple machines (11) to (13). For example, when the cart 21 completes a process prior to machine (11) and moves to the front of machine (11), the two-dimensional code 22 attached to the cart 21 is captured by the object monitoring camera 12A. This detects the arrival of the cart 21 at machine (11).

When work on machine (11) is completed and cart 21 leaves machine (11) and moves toward the next machine (12), object monitoring camera 12A detects that the two-dimensional code 22 is no longer captured, thereby detecting that cart 21 has left machine (11). When cart 21 arrives in front of machine (12), object monitoring camera 12B captures the two-dimensional code 22 attached to cart 21. This detects that cart 21 has arrived at machine (12). By repeating this operation, the movement of cart 21, i.e., the object (part, etc.), can be detected based on the video data from object monitoring camera 12.

Change points in the operation of an object can be identified by the start and end times of a certain machine (see Figure 4). The travel time (waiting time) from a higher-level machine (process) to a lower-level machine (process) can be calculated from the difference between the departure time from the higher-level machine and the arrival time at the lower-level machine. Change points in machine operation can also be identified by the start and end times of a certain machine's specific operation.

Furthermore, when parts are placed on the cart 21 on the serving shelf, the intermediate part products placed on the cart 21 are assigned a correspondence with the cart number, part number, and product serial number linked to the part number. The correspondence is also assigned as information to the two-dimensional code 22, and by reading the two-dimensional code 22, this correspondence information is automatically collected. This series of automatic collections links the cart 21, part, worker, shop, and change time. This linking is also performed between other shops, other products, and other workers. As a result, the locations of parts, workers, facilities, equipment, and jigs at a specific time are recognized, and this vector data is stored in the storage means. Furthermore, detection of the time of a status change does not necessarily have to be based on images captured by a TV camera, but can also be combined with information from other sensors, already installed sensors, etc.

The operation of the monitoring system 1 according to this embodiment will be explained with reference to the flowchart in Figure 5. In this system, the people monitoring camera 11, the object monitoring camera 12, and the machine monitoring camera 13 each operate independently to monitor the operation of people, objects, and machines. The obtained video data is transferred to and read by the transfer PC 14, which then performs a predetermined data compression operation on the video data before transferring the video data to the collection and storage server 15. The collection and storage server 15 receives and stores the transferred data. The data stored in the collection and storage server 15 is sent to the analysis server 16 as appropriate and is used for analysis.

Based on the stored data, the analysis server 16 identifies the aforementioned change points in people, objects, and machines, determines the time at which the change points occurred, and even the location at which the change points occurred, and stores this information in a memory unit (not shown). By identifying multiple change points, it is possible to determine the start and end times of significant actions for people and each machine object.

By identifying change points and the times and locations at which they occurred, the status of people and things in each process can be determined, which not only determines the time required for each process but also determines which of multiple processes has room for shortening lead time (room for margin). In other words, the analysis server 16 can function as a slack determination means that determines slack areas in each process based on the identified change points and data from the time when significant actions occurred. Room for margin can be visually displayed on the display PC 17.

Figure 6 is an example of a display screen when the analysis results from the analysis server 16 are displayed on the display of the display PC 17. In the display example of Figure 6, the horizontal axis is the time axis, and it is a Gantt chart that uses bars to compare the timeline of the manufacturing plan and the timeline of each process (processes 1 to 3) in the actual work (performance) for each different production number. This display makes visible the differences between the manufacturing plan and the actual work on each work day. This screen display makes it clear which production number is in which process, and allows the progress of each sub-process within a large process that takes place over several days to be understood.

As shown here, the monitoring system of this embodiment is highly effective in monitoring the progress of a manufacturing plan. Delays in the progress of a manufacturing plan result in increased labor costs, equipment depreciation costs, and other expenses, leading to increased manufacturing costs. This is why progress management is so effective. Furthermore, when a significant delay occurs, appropriate manufacturing output can be achieved by revising the manufacturing plan (manufacturing schedule). This is why rescheduling of the manufacturing plan is necessary. In this case, the magnitude of the delay can be preset, for example, a 20% or 30% increase from the initial lead time. Alternatively, the takt time of a specific process that is causing a bottleneck can be preset, for example, a 20% or 30% increase. In other words, the monitoring system of this embodiment plays an important role in determining when rescheduling is necessary. In other words, combining the monitoring system of this invention with a scheduler can produce even greater benefits. In this case, a certain degree of effectiveness can be expected whether the scheduler being tried out is an automatic stacking scheduler or a scheduler equipped with some kind of optimization algorithm.

Figure 7 is another example of a display screen when the analysis results from the analysis server 16 are displayed on the display of a display PC. In the display example at the top of Figure 7, the horizontal axis represents elapsed time, and the breakdown of the work time for each process (processes 1 to 3) for each different production number is displayed as the length of the bar. This display visualizes the variation in total work time for each production number, as well as the variation in the time required for each process. This makes it easy to identify production numbers that require long work times and to investigate the causes.

In addition, in the example display at the bottom of Figure 7, the horizontal axis represents elapsed time, but the travel time between different processes for each of multiple carts is displayed as the length of the bar. This display makes it possible to visualize which carts and which processes have long travel times, making it easier to investigate the causes.

Figure 8 is a graph of the operating time and non-operating time for each process (processes 1 to 3). When focusing on a single process, the higher the ratio of operating time to the total (operating rate), the higher the work efficiency. However, if one process has a high operating rate while an adjacent process has a low operating rate, the work efficiency of the entire process cannot be improved. In this way, when there is a difference in operating rate between adjacent processes, it can be determined that a bottleneck has occurred in the work. When a bottleneck occurs, work efficiency can be improved by changing the manufacturing plan, including personnel allocation.

In the embodiment described above, for security reasons, there may be cases where the user wishes to avoid uploading on-site information (such as the processing of image data after acquisition in this embodiment, the time detection of changes in the status of people, objects, and equipment, and the information resulting from subsequent data processing) to the cloud. In this case, the change time, product information, and other information can be encoded and decoded in advance using a password set by the user. This allows this information to be treated as text information whose content has meaning to different people within the site, and as information that has no meaning without a password on the cloud.

Furthermore, by using a hash function to identify data on the cloud before and after use, it is possible to ensure that the data on the cloud has not been tampered with, thereby ensuring the security of user data. This method may also utilize blockchain technology, which adds a previous hash value to the stored data to generate the next hash value, making any tampering operations take a long time, effectively making tampering impossible and ultimately further improving security.

Furthermore, in systems that use images from TV cameras, workers may be resistant to being photographed, or may be concerned about their personal information being leaked. To address this, a controller or other device can be provided with a means for performing image transformations, such as a Fourier transform, on the captured images at an early stage, converting them into information that cannot be interpreted by the human eye. By using such transformed images in subsequent learning and other processes, similar image transformations can be performed in subsequent processes such as worker detection.

By acquiring various time data for each product, including the detected work completion time, the lead time from start to completion of the product can be analyzed as the work time for each task. Furthermore, for example, at the end of a day's work, a report can be created that compiles information such as the progress of work and the placement of parts, etc.

A system and method for utilizing various time data such as take-off and arrival times and start and completion times detected in this way will be explained below. With this system, it is possible to describe the progress time of each task by focusing on a specific product. When the variance in the progress time is calculated for a specific product or a specific shop, if the variance is large due to the worker or other disturbances, it can be determined that the task is not mature. In such a case, for example,
- Changing workers in charge of specific tasks to members with higher work skills. - Analyzing the causes of variation and taking countermeasures such as preparing jigs to separate tasks.

In addition, in the above case, by monitoring the variability in start-to-completion times, it is possible to know in real time if the quality of work is deteriorating.

Furthermore, by preparing an environment on the cloud that includes the monitoring system, production planning system, and display system shown in Figures 6 to 8 of this embodiment, the following new value can be created. For example, when an algorithm is developed that can detect human change points with greater accuracy and higher time resolution, the value of the entire system can be improved by simply changing this algorithm. For example, because a process can be broken down into smaller parts when it is changed, changing the combination of these parts can make it possible to achieve even greater accuracy in reducing lead time through process redesign. In this way, having a monitoring system, production planning system, or display system makes it possible to easily and quickly apply new technologies and programs to the workplace. In other words, the present invention has the aspect of providing a "platform" for improving efficiency at the production site.

Furthermore, by counting the usage time of each technology and program applied on-site, it is possible to measure the total usage time of those technologies. The system provider's compensation can be automatically calculated based on this usage time. This is the mechanization of administrative processes, but even in manufacturing sites, the mechanization and efficiency of such administrative processes is important in terms of cost reduction. Furthermore, by having this type of usage time measurement system and paying compensation according to usage time, one-time cash payments can be reduced compared to introducing a system with a lump-sum payment, which is expected to have the effect of improving cash flow.

In addition, this monitoring system can detect and alert any deviations from average work time or expected work completion time. With a remote monitoring system that simply uses surveillance cameras, the monitor must continuously monitor the images being captured, whereas with this system, the monitor can receive a report only when there is a deviation from the expected time.

[effect]
As described above, according to the system of this embodiment, the movements of people, objects, and machines are monitored by cameras 11-13. Significant changes in the movements of people, objects, and machines are detected based on change points in the video data, analyzed by analysis server 16, and the results are visualized and presented. Therefore, the movements of people, objects, and machines can be monitored visually by an operator, eliminating the need for manual input and the time and effort required for condition setting. This allows for accurate and low-cost monitoring of the manufacturing process, improving work efficiency. In other words, matters that are currently judged intuitively by a supervisor can be handled with the same level of accuracy by simply capturing images using cameras 11-13. As a result, this system allows for efficient monitoring of the manufacturing site without the need for a dedicated system technician. Furthermore, visualization of the manufacturing site allows for visualization, which in turn enables one-stop solutions to management issues.

The cameras 11-13 used can be installed in different locations, with different types or even increased in number as needed, making it easy to adapt to changes in the manufacturing site. Specifically, this system can record the status or results of worksites where it is not easy to keep records, such as human assembly work, human visual inspection, human transport of items, and human machine operation. As a result, it is possible to record the status of human worksites in the same way as a factory site made up of automated machines that have a mechanism for recording data indicating that status.

[others]
The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

1...Monitoring system, 11...Human monitoring camera, 12...Object monitoring camera, 13...Device monitoring camera, 14...Transfer computer (PC), 15...Collection and storage server, 16...Analysis server, 17...Display computer (PC), 21...Cart, 22...Two-dimensional code.

Claims (6)

In a monitoring system that monitors people, things, and equipment in the manufacturing process,
a camera that photographs a plurality of work processes in a plurality of work locations and monitors the people, objects , and devices;
an analysis unit that analyzes the video data obtained by the camera;
a display unit that displays the analysis results by the analysis unit;
Equipped with
The analysis unit
Detecting a change point in the video data;
Analyzing the states of the person, object , and device based on the change points;
comparing signals contained in newly acquired video data with signals contained in machine-learned video data to detect the change points, and detecting and linking the positions and times of the change points to define work time at a specific work location, thereby monitoring work errors in the manufacturing process ;
the analysis unit detects the change points in the movement of the person according to the time when the person arrives in front of a predetermined machine and the time when the person leaves the machine,
the analysis unit detects the change point in the operation of the item according to a start time and a stop time of a predetermined machine for the item;
the analysis unit detects the change points in the operation of the device according to the time when the device starts and the time when the device finishes a specific operation;
The analysis unit identifies the change points and the times and places at which the change points occurred, thereby identifying the states of the people and the things in each of the work processes, identifying the time required for each of the work processes, and further determining whether there is room for shortening the lead time in any of the work processes.
A monitoring system characterized by: The analysis unit
2. The monitoring system according to claim 1, further comprising an inspection object detecting means for detecting an inspection object that is a candidate for the change point. The analysis unit
The monitoring system according to claim 1, further comprising a means for sending the machine-learned video data parameters to a manufacturing process base and performing a weighted average with the video data parameters learned at the base. The monitoring system according to claim 1, characterized in that a plurality of cameras are installed to monitor the people, objects , and devices, thereby detecting the positions of the people, objects , and devices in three-dimensional space. In a monitoring system that monitors people, things, and equipment in the manufacturing process,
sensors that monitor the people, objects , and devices in a plurality of work processes at a plurality of work locations;
an analysis unit that analyzes sensing data obtained by the sensor;
a display unit that displays the analysis results by the analysis unit;
Equipped with
The analysis unit
Detecting a change point in the sensing data;
Analyzing the states of the person, object , and device based on the change points;
The signal included in the newly acquired sensing data is compared with the signal included in the machine-learned sensing data to detect the change point, and the time and location of the change point are detected and linked to define the work time at a specific work location, thereby monitoring the manufacturing process .
the analysis unit detects the change points in the movement of the person according to the time when the person arrives in front of a predetermined machine and the time when the person leaves the machine,
the analysis unit detects the change point in the operation of the item according to a start time and a stop time of a predetermined machine for the item;
the analysis unit detects the change points in the operation of the device according to the time when the device starts and the time when the device finishes a specific operation;
The analysis unit identifies the change points and the times and places at which the change points occurred, thereby identifying the states of the people and the things in each of the work processes, identifying the time required for each of the work processes, and further determining whether there is room for shortening the lead time in any of the work processes.
A monitoring system characterized by: The monitoring system described in claim 5, characterized in that the display unit has a scheduler.