JP7608501B2 - Dust generation monitoring system and dust generation monitoring device - Google Patents

Description

The present invention relates to a dust monitoring system and a dust monitoring device.

Dust emissions from chimneys in factories and other facilities need to be reduced from an environmental protection perspective, and immediate measures are required if dust is generated. To achieve this, it is necessary to constantly monitor for dust generation. Conventionally, the presence or absence of dust was identified by having several workers take turns checking recorded video, but as this is not real-time monitoring, there is a delay in identifying dust generation and there is also the possibility of dust being overlooked during the video check. Furthermore, due to personnel constraints, it is impossible to monitor all chimneys on a large site in real time.

Patent Document 1 discloses a technology for generating dust or detecting dust generation through image processing. Patent Document 2 discloses a technology for restricting production in a factory when the measured value of soot and smoke emitted from the factory exceeds a control value.

Japanese Patent Application Publication No. 5-10737 JP 2009-199237 A

Problems with human dust monitoring include overlooking dust due to the inability to monitor in real time, personalizing evaluations due to differences in how each person perceives dust, and delays in taking action against dust due to delayed notifications. Even with the technologies disclosed in Patent Documents 1 and 2, it is difficult to perform dust monitoring automatically and in real time.

The present invention has been made in consideration of the above, and aims to provide a dust monitoring system and a dust monitoring device that can automatically monitor dust generation in factories and the like in real time.

In order to solve the above-mentioned problems and achieve the objective, the dust monitoring system of the present invention is a dust monitoring device that monitors the dust situation based on an image captured by a camera that captures an area including a dust source as its shooting range, and the dust monitoring device determines whether or not the dust is a dust that requires notification based on the concentration and area of the dust area identified by applying a dust determination program to the image, and issues a specified notification if it is determined that the dust is a dust that requires notification.

In addition, in the dust monitoring system according to the present invention, in the above invention, when dust is detected across multiple images that are consecutive in time, the dust monitoring device identifies the dust source based on the first image in which dust is detected first, identifies a dust rank indicating the scale of dust based on the second image in which dust is detected last, and determines the continuity of dust based on one or more images that exist between the first image and the second image.

In addition, in the dust monitoring system according to the present invention, when the dust monitoring device detects multiple dust emissions from different dust sources in multiple temporally consecutive images, it determines that the dust emission with the longer dust emission period is the dust emission to be notified.

In addition, in the dust monitoring system according to the present invention, in the above invention, when the dust monitoring device detects multiple dust emissions from different dust sources in multiple temporally consecutive images, it determines that the dust generated from the dust source at a lower position is the dust to be notified.

In order to solve the above-mentioned problems and achieve the objective, the dust monitoring device of the present invention is a dust monitoring device that monitors the dust situation based on an image captured by a camera that captures an area including a dust source as its shooting range, and determines whether or not the dust is a dust that should be notified based on the concentration and area of the dust area identified by applying a dust determination program to the image, and if it is determined that the dust is a dust that should be notified, outputs data indicating that the dust is a dust that should be notified by the alarm device.

The dust monitoring system and dust monitoring device of the present invention can automatically monitor dust in factories and other locations in real time.

FIG. 1 is a block diagram showing a schematic configuration of a dust monitoring system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a dust monitoring range in the dust monitoring system according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of a dust monitoring range in the dust monitoring system according to the embodiment of the present invention. FIG. 4 is an explanatory diagram for explaining a conventional dust generation detection method. FIG. 5 is an explanatory diagram for explaining three-step authentication by the dust monitoring system according to the embodiment of the present invention. FIG. 6 is a diagram showing a case where feedback correction is not performed when dust is detected across a plurality of temporally consecutive images. FIG. 7 is a diagram showing a case where feedback correction is performed when dust is detected across a plurality of temporally consecutive images. FIG. 8 is a flowchart showing the flow of a dust monitoring method executed by the dust monitoring system according to the embodiment of the present invention. FIG. 9 is an embodiment of a dust generation monitoring system and a dust generation monitoring device according to the present invention, and is a graph showing an example of a dust generation detection rate and a false detection rate according to the present invention. FIG. 10 is an embodiment of a dust monitoring system and a dust monitoring device according to the present invention, and is a graph showing an example of the working hours of a worker before and after the introduction of the present invention.

A dust monitoring system and a dust monitoring device according to an embodiment of the present invention will be described with reference to the drawings. Note that the components in the following embodiments include those that are easily replaceable by a person skilled in the art, or those that are substantially identical.

(Dust monitoring system)
The dust generation monitoring system according to the embodiment is a system for monitoring the dust generation status inside and outside a factory such as a steelworks, etc. The dust generation monitored by the dust generation monitoring system includes smoke, soot, water vapor, etc. generated from chimneys, etc. of the factory.

As shown in FIG. 1, the dust monitoring system 1 includes a plurality of cameras 10, an AI (Artificial Intelligence) server 20, a data server 30, and a mail server 40. The cameras 10, the AI server 20, the data server 30, and the mail server 40 all have communication functions and are configured to be able to communicate with each other via a network N. This network N is configured, for example, from an Internet network, a mobile phone network, WiFi (registered trademark, Wireless Fidelity), BLE (Bluetooth (registered trademark) Low Energy), etc. Also, the dust monitoring device according to the embodiment is configured from the AI server 20, or the AI server 20 and the mail server 40, which are components of the dust monitoring system 1.

Camera 10 is installed inside or outside a factory, such as a steelworks, and captures images of an area that includes one or more dust sources. This camera 10 is realized by an imaging device equipped with, for example, a general CCD (Charge Coupled Device) sensor or CMOS (Complementary Metal Oxide Semiconductor) sensor.

In dust monitoring using cameras 10, for example, as shown in FIG. 2, cameras 10 are installed in multiple directions within the factory site and images are continuously captured. In FIG. 3, (a) is the shooting range of camera 10 facing the north side of FIG. 2, (b) is the shooting range of camera 10 facing the center side of FIG. 2, and (a) is the shooting range of camera 10 facing the south side of FIG. 2. Of the shooting ranges shown in FIG. 3, the range that includes the chimney, which is a dust source, is the dust monitoring range. In FIG. 2 and FIG. 3, the indications such as "**CO (extrusion)", "**chimney", and "**CDQ" indicate facilities that may be dust sources. In addition, "CDQ" indicates coke dry quenching equipment.

Since the installation position of the camera 10 is always fixed, the position coordinates of the dust source, such as a chimney, are known in advance on the system side (AI server 20 side). In addition, the images captured by the camera 10 are sequentially transmitted to the AI server 20 via the network N.

The AI server 20 monitors the dust generation situation based on the images captured by the camera 10. This AI server 20 is realized by a general-purpose computer such as a workstation or a personal computer, or a server device arranged on the cloud. The AI server 20 may also be installed in a factory. The AI server 20 also includes a dust generation determination unit 21.

The dust determination unit 21 determines whether or not the dust is a dust that is to be notified, based on the density and area of the dust region identified by applying a dust determination program to the image captured by the camera 10. If the dust determination unit 21 determines that the dust is a dust that is to be notified, it outputs data indicating that the dust is a dust that is to be notified by the mail server 40, which functions as a notification device, to the mail server 40. The dust determination program is an AI model that learns about dust by deep learning using, for example, several thousand dust images (images that contain dust) as training data.

In the AI model, for example, several thousand dust images are classified into large, medium and small dust scales (hereinafter referred to as "dust ranks") based on the area and concentration of the dust region, and each dust image is learned as teacher data along with the dust rank (e.g., large, medium and small). The dust determination unit 21 then applies the above-mentioned dust determination program to an image newly captured by the camera 10 to determine whether the dust contained in the image is dust that should be notified, and whether the dust level is large, medium or small. An example of "dust that should be notified" is dust with a dust level of "medium" or higher.

In addition, the AI model may classify, for example, several thousand dust images into density categories (e.g., dark, normal, light), and learn each dust image together with this density category information as teacher data. In this case, the dust determination unit 21 applies the above-mentioned dust determination program to an image newly captured by the camera 10, thereby determining whether the dust contained in the image is dust to be notified and which of the above-mentioned concentrations it has.

The area of the dust-generating region can be recognized by using a segmentation AI model to label the pixels of the dust-generating portion in the image and quantifying the number of pixels, or by using an object detection technique to extract the dust-generating region in the image and quantifying the number of pixels. It is also possible to determine whether the dust-generating level is large, medium, or small based on the dust concentration and the area of the dust-generating region. Note that "large, medium, and small" as a classification of dust-generating levels is just one example, and the levels may be classified into two or less, or four or more, as necessary.

Now consider a case where multiple dust sources (e.g., chimneys) are included in one image, as shown in FIG. 4. In this figure, the dust is located above multiple dust sources, so if an attempt is made to identify both the dust source and the dust rank from such a single image, there is a possibility that the dust source cannot be accurately identified. Therefore, it is preferable for the dust determination unit 21 to identify the dust source and the dust rank, etc., based on multiple images.

In this case, when dust generation is detected across multiple temporally consecutive images by the above-mentioned dust generation determination program, the dust generation determination unit 21 identifies the dust source based on the first image in which dust generation was first detected, and identifies the dust generation rank indicating the scale of dust generation based on the second image in which dust generation was last detected. The dust generation determination unit 21 then determines the continuity of dust generation based on one or more images present between the first and second images.

The dust generation determination unit 21 recognizes dust generation by dividing it into early, middle, and late stages, for example, as shown in FIG. 5. That is, the dust generation determination unit 21 identifies the dust generation source based on an image of the early dust generation stage. The dust generation determination unit 21 also identifies the dust generation rank (e.g., large, medium, or small) based on an image of the later dust generation stage. The dust generation rank can be identified based on the area and concentration of the dust generation region. The dust generation determination unit 21 also determines the continuity of dust generation based on an image of the middle dust generation stage. In this manner, the process of identifying the dust generation source, the continuity of dust generation, and the dust generation rank based on images of the early, middle, and late dust generation stages is defined as "three-stage authentication" in this embodiment.

For ease of explanation, FIG. 5 only shows an image corresponding to the later period of the dust generation period (the second image described above), but for example, an "image of the early stage of dust generation" used to identify the dust source does not include dust generation in the middle or later stages. Similarly, an "image of the middle stage of dust generation" used to determine the continuity of dust generation does not include dust generation in the later stages.

In addition, for example, as shown in FIG. 6, when multiple dust sources (e.g., chimneys) are included in one image, the actual dust source is "6CO extrusion", but since the dust area also overlaps with "56CDQ", "56CDQ" may be identified as the dust source. In the figure, for example, after dust is detected in both "6CO extrusion" and "56CDQ", "56CDQ" may be identified as the dust source.

In the dust generation determination unit 21, it is preferable to retain logic and perform feedback correction when multiple dust occurrences are detected by the above-mentioned dust generation determination program. Note that "retaining logic" means that when multiple dust occurrences are detected in one image, for example, the detection results of the multiple dust occurrences are retained. In this case, when multiple dust occurrences from different dust sources are detected in multiple temporally consecutive images, the dust generation determination unit 21 determines that the dust occurrence with the longer dust generation period is the dust occurrence to be notified.

As shown in FIG. 7, for example, when multiple dust events are detected in an image, if the interval between the two (two dust events) is within a predetermined time (for example, one minute), the dust determination unit 21 retains logic that dust events have been detected from both (see (1) in the figure). Then, after the dust generation period of "56CDQ" has ended, the dust determination unit 21 determines that the dust generation from "6CO extrusion" with the long dust generation period (dust generation during dust generation period X) is dust generation that should be reported (see (2) in the figure).

In addition, as shown in FIG. 7, for example, when multiple dust emissions from different dust sources ("6CO extrusion", "56CDQ") are detected in multiple temporally consecutive images, the dust determination unit 21 determines that the dust (dust emission in dust emission period X) generated from the dust source at a lower position ("6CO extrusion") is the dust emission to be reported. In this manner, when multiple dust emissions from different dust sources are detected, the process of identifying the actual dust source is defined as "feedback correction" in this embodiment.

The data server 30 is realized by a general-purpose computer such as a workstation or a personal computer, or a server device arranged on the cloud. The data server 30 may also be installed in a factory. In addition, while the data server 30 and the AI server 20 are illustrated in FIG. 1 as separate configurations, the data server 30 may be provided within the AI server 20. The data server 30 also includes a classification storage unit 31 and a memory unit 32.

The classification storage unit 31 classifies and stores the dust generation determination results from the dust generation determination unit 21. This "dust generation determination result" is data indicating that the dust generation is the target for a specified notification by the mail server 40 functioning as an alarm device. The dust generation determination results from the dust generation determination unit 21 include, for example, an image containing dust (dust generation image), the dust generation source, the dust generation level (e.g. large, medium, small), the dust generation area, and the dust generation concentration classification (e.g. dark, normal, light). The classification storage unit 31 classifies the above-mentioned dust generation determination results and stores them in the memory unit 32.

The storage unit 32 is composed of recording media such as an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), and removable media. Examples of removable media include disk recording media such as a Universal Serial Bus (USB) memory, a Compact Disc (CD), a Digital Versatile Disc (DVD), and a Blu-ray (registered trademark) Disc (BD). The storage unit 32 can store an operating system (OS), various programs, various tables, various databases, and the like. The storage unit 32 stores, for example, the dust generation determination results from the dust generation determination unit 21.

The mail server 40 is realized by a general-purpose computer such as a workstation or a personal computer, or a server device located on the cloud. The mail server 40 may also be installed in a factory. The mail server 40 also includes a notification unit 41.

When the dust determination unit 21 determines that the dust contained in the image is dust that is to be notified (for example, dust with a dust level of "medium" or higher), the notification unit 41 acquires the dust determination result from the dust determination unit 21. The notification unit 41 then issues a predetermined notification based on the acquired dust determination result. Examples of "predetermined notification" include sending an email to relevant parties, issuing an alarm to relevant parties, etc. The notification unit 41 may also issue a predetermined notification when the number of images containing dust that is to be notified reaches a predetermined threshold.

In the dust monitoring system and dust monitoring device according to the embodiment having the above configuration, for example, consider a case where one dust source is monitored by one camera 10. In this case, one camera 10 is installed so that one dust source whose position coordinates are known is included in the shooting range, and then the AI server 20 captures images at a rate of, for example, "1 image/sec." Next, the dust determination unit 21 of the AI server 20 detects dust in the image using a dust determination program. When dust is detected, for example, the concentration and scale of the dust are calculated and identified as the dust rank.

Then, the image in which dust was detected and the judgment result including the dust rank are stored in the data server 30, and when, for example, the number of images containing dust to be notified reaches a predetermined threshold, the notification unit 41 of the mail server 40 sends an email to the relevant parties requesting correction. By carrying out such processing, dust generation can be suppressed and the environmental burden can be reduced.

In addition, in the dust monitoring system and dust monitoring device according to the embodiment having the above-mentioned configuration, for example, a case where two or more dust sources are monitored by one camera 10 is considered. In this case, one camera 10 is installed so that two or more dust sources whose position coordinates are known are included in the shooting range, and then the AI server 20 captures images at a rate of, for example, one image per second. Next, the dust determination unit 21 of the AI server 20 detects dust generation in the images using a dust generation determination program. When dust generation is detected, the dust generation images are tracked, for example, until the dust generation disappears, and are classified into images of the first, middle, and last stages of dust generation.

Next, the dust generation determination unit 21 determines the dust generation source from the image of the dust generation, the continuity of dust generation from the image of the middle stage of dust generation, and the dust generation rank from the image of the later stage of dust generation (three-stage authentication). Since the installation position of the camera 10 is fixed, the position coordinates of the multiple dust generation sources in the image are also known in advance. In addition, the continuity of dust generation between multiple images can be determined, for example, by whether or not the dust generation areas between the images overlap. In addition, the dust generation rank can be determined based on the range of dust generation from when dust generation begins to be detected until dust generation disappears.

Furthermore, when dust generation is detected in two or more places for a preset dust source, the dust generation determination unit 21 performs the above-mentioned three-stage authentication for each dust generation, and adopts, for example, the dust generation source with the longest time since dust generation began as the dust generation source for the current dust generation (feedback correction). In other words, when dust generation is detected, the dust generation determination unit 21 performs three-stage authentication and feedback correction for a group of images (tracked image group) from the start of the detection of the dust generation to the end of the dust generation.

Then, the dust generation images and the judgment results including the dust generation rank are stored in the data server 30, and when, for example, the number of images including dust to be notified reaches a predetermined threshold, the notification unit 41 of the mail server 40 sends an email to the relevant parties requesting correction. By carrying out such processing, dust generation can be suppressed, and the environmental burden can be reduced.

In this way, the dust generation monitoring system and dust generation monitoring device according to the embodiment detects dust generation using a pre-trained dust generation determination program, and performs predetermined post-processing (three-step authentication, feedback correction) as necessary. This allows dust generation monitoring in factories and the like to be performed automatically and in real time.

(Dust generation monitoring method)
The flow of the dust generation monitoring method executed by the dust generation monitoring system and the dust generation monitoring device according to the embodiment will be described with reference to FIG.

First, the dust generation determination unit 21 reads the image captured by the camera 10 (step S11). Next, the dust generation determination unit 21 uses a dust generation determination program to detect dust generation in the image (step S12).

Then, the dust generation determination unit 21 identifies the dust generation source based on the image of the early dust generation stage (step S13). The dust generation determination unit 21 then determines the continuity of dust generation based on the image of the middle dust generation stage (step S14). The dust generation determination unit 21 then identifies the dust generation rank based on the image of the later dust generation stage (step S15). Note that the order of steps S13 to S15 is not particularly limited, and may be performed in an order different from that shown in FIG. 8.

Next, in step S12, if dust generation is detected in two or more locations for a preset dust source, the dust generation determination unit 21 performs feedback correction of the dust source (step S16). Next, the classification storage unit 31 classifies the dust generation determination results in steps S12 to S16 and stores them in the memory unit 32 (step S17). Next, the notification unit 41 sends an email to relevant parties, for example, informing them of the time of dust generation, the dust source, and the dust generation rank (step S18), and this process is completed.

(Example)
An embodiment of a dust generation monitoring system and a dust generation monitoring device according to the present invention will be described with reference to FIGS.

Figure 9 shows an example of the dust detection rate and false positive rate before and after the introduction of the present invention. In the figure, the vertical axis represents the detection rate (%) and false positive rate (%), and the horizontal axis represents the dust source.

The three bar graphs in FIG. 9(a) show, from the left, the dust detection rate of invention 1 (before the introduction of three-step authentication and feedback correction), the dust detection rate of invention 2 (after the introduction of three-step authentication), and the dust detection rate of invention 3 (after the introduction of three-step authentication and feedback correction). The "detection rate" was calculated by "(number of dust detections by the system)/(number of dust detections by humans)×100". As shown by the thick line in the figure, the target value of the detection rate is "100%", and the closer the detection rate is to 100%, the higher the performance (dust can be detected accurately). As shown in the rightmost column of the figure (item "average"), the methods of inventions 2 and 3 have detection rates closer to the target value and higher performance than invention 1. Note that in invention 1, depending on the dust source, the detection rate may be far from the target value (many false detections), but the false detection itself is on the safe side, and dust detection is better than not performing it.

The three bar graphs in FIG. 9(b) show, from the left, the false detection rate of dust generation in the present invention 1 (before the introduction of three-step authentication and feedback correction), the false detection rate of dust generation in the present invention 2 (after the introduction of three-step authentication), and the false detection rate of dust generation in the present invention 3 (after the introduction of three-step authentication and feedback correction). The "false detection rate" was calculated by "(number of false detections of dust generation by the system) / (number of dust generation detections by humans + number of dust generation detections by the system) x 100". As shown by the thick line in the figure, the target value of the false detection rate is "20%", which means that the lower the false detection rate is below 20%, the higher the performance (the less likely false detection of dust generation occurs). As shown in the rightmost column of the figure (the "average" item), it can be seen that the false detection rate of the methods of the present inventions 2 and 3 is lower than the target value. It can also be seen that the bias in the false detection rate for each dust source is eliminated in the methods of the present inventions 2 and 3 compared to the first invention. In addition, in the present invention 1, depending on the dust source, the false detection rate may be far from the target value (there may be many false detections), but the false detection itself is on the safe side, and performing dust detection is better than not performing it at all.

Figure 10 shows an example of a dust monitoring system and dust monitoring device according to the present invention, and shows an example of worker working hours before and after the introduction of the present invention. A worker's working hours are, for example, the total time required for "checking for large and medium rank dust", the time required for "contacting relevant departments", and the time required for "checking for small rank dust". As shown in the figure, by introducing the present invention, dust monitoring can be automated, and it can be seen that working hours have been reduced by "2.0 h/day".

The dust monitoring system and dust monitoring device according to the present invention have been specifically described above using the mode and examples for carrying out the invention, but the spirit of the present invention is not limited to these descriptions and must be interpreted broadly based on the claims. Furthermore, it goes without saying that various changes and modifications based on these descriptions are also included in the spirit of the present invention.

REFERENCE SIGNS LIST 1 Dust generation monitoring system 10 Camera 20 AI server 21 Dust generation determination unit 30 Data server 31 Classification storage unit 32 Memory unit 40 Mail server 41 Notification unit

Claims (2)

A dust generation monitoring system including a dust generation monitoring device that monitors the state of dust generation based on an image captured by a camera that captures an area including a dust source as a capture range,
The dust generation monitoring device applies a dust generation determination program to the image to determine whether or not the dust generation is a dust that should be notified, based on the density and area of the dust generation region identified, and when it is determined that the dust generation is a dust that should be notified, issues a predetermined notification,
When dust generation is detected across a plurality of temporally consecutive images, the dust generation monitoring device identifies a dust source based on a first image in which dust generation is first detected, identifies a dust generation rank indicating the scale of dust generation based on a second image in which dust generation is last detected, and determines the continuity of dust generation based on one or more images present between the first image and the second image.
Dust monitoring system. A dust generation monitoring device that monitors the state of dust generation based on an image captured by a camera that captures an area including a dust source as a capture range,
A dust generation determination program is applied to the image to determine whether or not the dust generation is a dust generation that should be notified, based on the density and area of the dust generation region identified, and when it is determined that the dust generation is a dust generation that should be notified, data is output indicating that the dust generation is a dust generation that should be notified by a predetermined notification device,
When dust generation is detected across a plurality of temporally consecutive images, a dust source is identified based on a first image in which dust generation is first detected, a dust rank indicating the scale of dust generation is identified based on a second image in which dust generation is last detected, and the continuity of dust generation is determined based on one or more images present between the first image and the second image.
Dust monitoring device.

Images