JP7528613B2 - Water quality determination device - Google Patents

Description

The present invention relates to a water quality determination device.

In recent years, there has been a labor shortage at sewage treatment plants, and there is a demand for more efficient maintenance and management of sewage treatment facilities.
For this reason, there are considerations being given to replacing some of the work of staff with artificial intelligence (AI).

Patent Document 1, which is an example of the conventional technology, discloses a water quality analysis device that includes a hue analysis unit that learns a convolution AE (Auto Encoder) on image data of a sedimentation tank to extract hue features of the sedimentation tank, classifies the image data into hue clusters by clustering from the hue features, and draws a hue feature space and cluster distribution, and a floating matter analysis unit that learns a convolution AE on the image data to extract feature values of floating matter in the sedimentation tank, classifies the image data into floating matter clusters by clustering from the floating matter features, and draws the floating matter feature space and cluster distribution.
According to the technology disclosed in Patent Document 1, unsupervised learning and clustering enable learning and water quality judgment without assigning correct labels to a large number of water quality images.

JP 2019-136664 A

However, the above conventional technology had the problem that the water quality state and clusters had to be evaluated during the learning period, making it difficult to detect unknown abnormal conditions.

The present invention has been made in consideration of the above, and aims to provide a technology that can determine water quality even when an unknown abnormal condition occurs.

The present invention, which solves the above problems and achieves the object, is a water quality assessment device that includes an image data storage unit that stores image data acquired from a water quality assessment target, an image processing unit that processes the image data stored in the image data storage unit and outputs processed image data, a convolution AE learning unit that learns image data when the water quality of the water quality assessment target is in a normal state as teacher data and outputs learned parameters, a convolution AE parameter storage unit that stores the learned parameters output by the convolution AE learning unit, a background estimation unit that uses the learned parameters stored in the convolution AE parameter storage unit to construct a convolution AE to generate a convolution AE estimated background image, and generates and outputs estimated background image data from a background estimation value based on a representative value of the convolution AE estimated background image and a past image, and a water quality assessment unit that compares the processed image data output by the image processing unit with the estimated background image data output by the background estimation unit to generate differential image data and performs water quality assessment based on the differential image data.

In the water quality determination device having the above configuration, it is preferable that the background estimation unit weights each of the images used to generate the convolution AE estimated background image and the estimated background image data.

In the water quality determination device having the above configuration, it is preferable that the background estimation unit automatically performs the weighting using a histogram of pixel values.

The present invention has the effect of making it possible to determine water quality even when an unknown abnormal condition occurs.

FIG. 1 is a block diagram showing the configuration of a water quality determining device according to the first embodiment. FIG. 2 is a diagram showing an image of learning in the convolution AE learning unit. FIG. 3 is a block diagram showing the configuration of the background estimation unit. FIG. 4 is a diagram for explaining background estimation by the median method. FIG. 5 is a diagram illustrating the background estimation unit. FIG. 6 is a diagram illustrating a background estimation unit included in the water quality determining device according to the second embodiment. FIG. 7 is a block diagram showing the configuration of a water quality determining device according to the third embodiment. FIG. 8 is a diagram showing a histogram of pixel values generated by the statistical processing unit.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the drawings.
However, the present invention should not be construed as being limited to the following embodiments.

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of a water quality determining device 100 according to this embodiment.
The water quality judgment device 100 shown in Figure 1 includes an image data input unit 101, an image data storage unit 102, an image processing unit 103, a convolution AE learning unit 104, a convolution AE parameter storage unit 105, a background estimation unit 106, a water quality judgment unit 107, and a judgment result output unit 108.

The image data input unit 101 is an interface into which image data obtained by a camera (not shown) installed on the subject of water quality assessment is input.
Examples of the water quality to be assessed include settling ponds or tanks in sewage treatment facilities.

The image data storage unit 102 stores the image data input to the image data input unit 101 .
The image data stored in the image data storage unit 102 is output to the image processing unit 103 .
The image data storage unit 102 can be realized by a recording medium such as a semiconductor memory or a magnetic disk.
This image data includes both still images and moving images made up of a number of successive still images.

The image processing unit 103 receives the image data stored in the image data storage unit 102, processes the image data, and outputs the processed image data.
The processing performed by the image processing unit 103 is trimming, normalization, or filtering performed on the image data so that the image data can be input to the convolution AE learning unit 104 .
The image processing unit 103 can be realized by a processor such as an MPU (Micro-Processing Unit) and a CPU (Central Processing Unit).

The convolution AE learning unit 104 receives the processed image data output by the image processing unit 103 as input, learns using the input image data, that is, normal processed image data, as teacher data, generates learned parameters, and outputs them to the convolution AE parameter memory unit 105.
Here, the learning is performed by extracting features of the inputted normal-state processed image data and generating a restored image so as to reproduce the inputted normal-state processed image data.
The convolution AE learning unit 104 can be realized by a processor such as an MPU or a CPU.

FIG. 2 is a diagram showing an image of learning in the convolution AE learning unit 104. As shown in FIG.
As shown in FIG. 2, the convolutional AE learning unit 104 is a neural network that uses teacher data as an input image, extracts features of the input image using a convolutional layer that compresses the dimensions, and outputs a restored image that reproduces the input image using a deconvolutional layer that restores the dimensions.
Here, the learning performed by the convolution AE learning unit 104 is performed when the water quality of the subject of water quality assessment is in a normal state, so that a model can be constructed that restores a normal image, which is an image taken under normal conditions.
The normal state refers to a state in which the system including the subject of water quality assessment is operating without any malfunctions.
The convolution AE learning unit 104 automatically extracts features of normal-state processed image data, which is input image data that is input to the image data storage unit 102 when the water quality of the subject of water quality assessment is in a normal state and processed by the image processing unit 103, generates restored image data to reproduce the input image data, performs learning, and generates learned parameters.

The convolution AE learning unit 104 may have one model or multiple models.
When preparing multiple models for the convolution AE learning unit 104, multiple different models are constructed by changing the period of the image data to be learned or by performing processing such as adding noise to the image.

The convolution AE parameter storage unit 105 receives the learned parameters output by the convolution AE learning unit 104 and stores the learned parameters.
The convolution AE parameter storage unit 105 can be realized by a recording medium such as a semiconductor memory or a magnetic disk.

FIG. 3 is a block diagram showing the configuration of the background estimation unit 106. As shown in FIG.
The background estimation unit 106 shown in FIG. 3 includes a convolution AE background estimation unit 1061, a median calculation unit 1062, and an estimated background image generation unit 1063, and effectively performs background estimation of image data by coordinating background estimation using the convolution AE and background estimation using the median method.
The background estimation unit 106 can be realized by a processor such as an MPU or a CPU, and a recording medium such as a semiconductor memory or a magnetic disk.

The convolution AE background estimation unit 1061 estimates a background by a convolution AE constructed using the learned parameters stored in the convolution AE parameter storage unit 105, and outputs a first group of background images.
The convolution AE background estimation unit 1061 has N models and outputs N restored images.
Here, N is a natural number.
However, the present invention is not limited to this, and may have a configuration in which one model is provided, and the output of this model is replicated to output N restored images.
Here, since the convolution AE has already been learned by the convolution AE learning unit 104 using normal images, even if an image in an abnormal state is input, the convolution AE restores and outputs a pseudo normal image.
That is, the convolution AE background estimation unit 1061 generates and outputs a plurality of (N) estimated background images by removing the foreground from the input image.
Here, the foreground is an image that appears in the image, such as scum (solid matter or sludge floating to the water surface) and flocs, and is caused by an abnormal state to be detected.
The background is an image that does not contain any abnormal conditions to be detected.

However, in the case of convolution AE, when a change occurs in the background that did not occur during learning, it is difficult to correctly generate a background image.
In particular, when the water quality assessment subject is a harsh shooting environment such as a final sedimentation tank, changes in lighting or external light that did not occur during learning often occur, and background estimation using convolution AE can result in false positives.

Another method for estimating the background is the median method.
FIG. 4 is a diagram for explaining background estimation by the median method.
In FIG. 4, the horizontal axis represents time, with the left side representing the past and the right side representing the present.
In the median method, an estimated background image is generated from N past frame images captured between the past and the present.
Specifically, the median value of each pixel is taken for the past N frame images from the past to the present, that is, N images, and the median value is used as the background estimation value for each pixel to generate an estimated background image.
According to the median method, no false colors appear in the estimated background image, and a background image can be flexibly generated even for a dynamically changing background.

However, with the median method, if a change occurs even for an instant with respect to the past N frame images due to a change in lighting or the like, it is difficult to generate a correct background image.
Therefore, in this embodiment, the median method is applied to 2N images, which are the N estimated background images generated by the convolution AE background estimation unit 1061 and the past N frame images, i.e., N images, thereby performing background estimation of the image data by cooperation between the convolution AE and the median method.

The median calculation unit 1062 receives the N estimated background images generated by the convolution AE background estimation unit 1061 and the past N frame images, calculates the median, which is the representative value of each pixel value, for a total of 2N images, and outputs this median as the background estimation value.

The estimated background image generating unit 1063 generates an estimated background image using the background estimation value output by the median calculation unit 1062.

In this way, the background estimation unit 106 estimates the background by coordinating background estimation using convolution AE and background estimation using the median method, so that a background image can be generated correctly without any problems even if a lighting change occurs that is not present during the learning period or if a very momentary change occurs over N frames.

FIG. 5 is a diagram illustrating the background estimation unit 106. As shown in FIG.
The upper part of FIG. 5 shows the generation of N estimated background images by convolution AE, and the lower part of FIG. 5 shows N frames of images from the present to the past.
As described above, an estimated background image is generated by calculating the median value of each pixel of a total of 2N images, including N restored images generated by the convolution AE and the past N frame images.

The water quality assessment unit 107 receives as input the processed image data from the image processing unit 103 and the estimated background image data output by the background estimation unit 106, compares the processed image data with the estimated background image data to generate a difference image, performs water quality assessment based on the difference image, and outputs water quality assessment data indicating the water quality assessment result.
The water quality determination unit 107 can be realized by a processor such as an MPU or a CPU.

The water quality determination unit 107 calculates the difference between the processed image data and the estimated background image data for each pixel to generate foreground image data.
When abnormal scum or flocks are captured in the input image, the scum or flocks appear as the foreground in the foreground image.
The water quality determination unit 107 performs image processing such as binarization, morphological transformation, or labeling processing on the foreground image to quantitatively evaluate the abnormal area.
Furthermore, the water quality determination unit 107 evaluates the number and area of abnormalities, and when the number and area of abnormalities exceed a predetermined threshold value, determines that an abnormality has occurred, and outputs the determination result.

The water quality judgment by the water quality judgment unit 107 is specifically intended to judge the amount of floating matter such as scum that floats to the object to be judged, or to judge the change in color of the object to be judged that changes due to the activated sludge settling rate, etc., but the present invention is not limited to this, and any abnormality can be detected as long as a difference from the normal state is displayed on the image, regardless of the cause of the abnormality.

The judgment result output unit 108 outputs the judgment result of the water quality judgment unit 107 to the outside.
The judgment result output unit 108 may be a display device that can be visually recognized by the operator, or may be an interface connected to the water quality control device.
Based on the judgment result outputted to the outside, the operator or the water quality control device performs operation on the water quality judgment target.

In this embodiment, the median is used as the representative value, but the present invention is not limited to this, and instead of the median, the average value of each pixel value may be used as the background estimation value.

As described above, according to this embodiment, by using an estimated background image generated by learning from image data when the water quality of the subject of water quality assessment is in a normal state, and the past N frame images, it is possible to detect abnormalities in the water quality of the subject of water quality assessment.
Therefore, according to this embodiment, it is possible to detect an abnormality in the water quality assessment subject without acquiring image data at the time of the abnormality.
Therefore, according to this embodiment, even if an unknown abnormality occurs in the water quality of the water being assessed, the abnormality can be detected and the water quality can be assessed.

<Embodiment 2>
In the first embodiment, a configuration has been described in which all image data used to calculate the representative value is treated equally.
In this embodiment, a form will be described in which a weighting factor w is assigned to each piece of image data to adjust the contribution of each piece of image data to the estimated background image.

The water quality determination device according to this embodiment differs from embodiment 1 only in the background estimation unit, and the other configurations are the same as those of embodiment 1.

FIG. 6 is a diagram illustrating the background estimation unit included in the water quality determination device according to this embodiment.
The pixel value of pixel (x, y) of the image at the current time is a _xy ⁰ , and the pixel value of pixel (x, y) of the image at time t before is a _xy ^t .
Note that t = 1, 2, ..., N-1.
Then, the pixel value at pixel (x, y) of the restored image of the k-th convolution AE is denoted by b _xy ^k .
Note that k=0, 1, ..., N-1.
In this case, the pixel value l _xy of a pixel (x, y) of the estimated background image is expressed by the following equation (1).
However, this is based on the following formula (2).

Here, if the pixel values are c _xy ⁱ without distinguishing between the images of the past N frames and the N restored images of the convolution AE, the above formula (1) can also be expressed by the following formula (3).
Here, i=0, 1, ..., 2N-1.

According to the past image priority u _t , the priority of the images of the past N frames is set.
In many cases, past data can be set to be forgotten by giving a high priority to images from a few hours ago and giving a low priority to older images.
However, some ingenuity is required to satisfy the constraint of the above formula (2).
When the priority of each image is logarithmically decreased, it is advisable to use a probability distribution known as Benford's law, as shown in the following equation (4).

Furthermore, when the past image priority u _t is set individually and independently, if the past image priority u _t does not satisfy the above formula (2), the above formula (2) can be satisfied by using a transformed priority u _t ′ that has been variable-transformed using the softmax function shown in the following formula (5).

Furthermore, the priority of the convolution AE image is set according to the convolution AE image priority _vt .
For example, by increasing the convolution AE image priority _vt of a model trained in the most recent learning period and decreasing the convolution AE image priority _vt of a model trained using data with added noise, it becomes possible to place emphasis on the most recent learning and perform background estimation without taking noise into account.

Furthermore, according to the image generation method priority w _i , it is possible to set whether to give priority to the image of the convolution AE or the image of the past N frames.
When it is determined that the learning of the convolution AE is insufficient due to a short learning period, _w0 is increased, and when it is determined that the learning of the convolution AE has been performed appropriately, _w1 is increased.
Here, it is also possible to set the number of frames of past images and the number of models of the convolution AE to different values.
At this time, in order to make the contribution rate for each image uniform, it is preferable that the following formula (6) be satisfied using the number of frames _N0 of past images and the number of models _N1 of the convolution AE.

For example, when N ₁ >N ₀ , if w ₀ =0.5 and w ₁ =0.5, it should be noted that the contribution rate per restored image decreases because the number of restored images of the convolution AE model is large.

If the past image priority u _t , the convolution AE image priority v _t , and the image generation method priority w _i are regarded as a probability mass function, then the above equation (1) estimates the background by the expected value.
When estimating the background using the median, there is a method of virtually increasing or decreasing the number of pixel values a _xy ^t according to the probability of the past image priority u _t .
For example, when N=3, u ₀ =0.5, u ₁ =0.3, and u ₂ =0.2, a _xy ⁰ is replicated five times, a _xy ¹ is replicated three times, and a _xy ² is replicated two times, and the median is obtained from a total of 10 pixel values a _xy .
Increasing or decreasing the number of each pixel value a _{x y} is equivalent to virtually increasing or decreasing the occurrence probability of each pixel value a _{x y} , and the occurrence probability of a pixel value with a large past image priority u is virtually increased and becomes more likely to be adopted as the median value.
The same applies to the convoluted AE image priority v and priority w. However, since there are only two priorities w, _w0 and _w1 , the larger one will always be adopted.
To avoid this, it is preferable to use ω _i in the above equation (3) to evaluate both the convolution AE and the past N frames without distinguishing between them.

According to this embodiment, by weighting each image, it is possible to adjust the contribution of each image to the background image, which allows tuning such as prioritizing an image that is closer to the current image or prioritizing the restored image of the convolutional autoencoder.

<Embodiment 3>
In this embodiment, a form in which the weighting in the second embodiment is automatically determined based on a histogram of pixel values will be described.

FIG. 7 is a block diagram showing the configuration of a water quality determining device 100a according to this embodiment.
The water quality determination device 100a shown in Figure 7 differs from the water quality determination device 100 shown in Figure 1 only in that it is equipped with a statistical processing unit 111 and a background histogram memory unit 112, and the other configurations are the same as those of the water quality determination device 100 shown in Figure 1.

The statistical processing unit 111 models the probability distribution of pixel values, and in order to automatically determine the weighting coefficients described in the second embodiment, counts up the pixel values of all images captured during a learning period, generates a histogram of the pixel values, and generates a density function f(a _xy ) by estimation.

FIG. 8 is a diagram showing a histogram of pixel values generated by the statistical processing unit 111. As shown in FIG.
The density function f(a _xy ) is generated by kernel density estimation.

The background histogram storage unit 112 stores the density function f(a _xy ) generated by the statistical processing unit 111 .

When estimating unknown image data, the background estimation unit 106 a automatically determines the weight ω _i in the above equation (3) using the density function f(a _xy ) stored in the background histogram storage unit 112 .
For example, the pixel values of the estimated background images output from the two convolution AEs are denoted as a^ _xy ¹ and a^ _xy ² , respectively.
Whether the estimated background image is close to the actual background image is examined based on whether it is close to a representative value of a histogram of pixel values, that is, the average value, the median value, or the mode value.

In FIG. 8, if the representative value is the mode a _xy ^mode , it is estimated that the pixel value a ^ _xy ¹ is more likely to be closer to the actual background image than the pixel value a ^ _xy ² .
Therefore, the background estimation unit 106a automatically determines the weight ω _i in the above equation (3) by the following equation (7) so that the weight for a^ _xy ¹ is large and the weight for a^ _xy ² is small.

According to this embodiment, the contribution rate weighting coefficient is automatically set based on the histogram of past normal images, eliminating the need for cumbersome adjustment of the weighting coefficient compared to embodiment 2.

100, 100a Water quality determination device 101 Image data input unit 102 Image data storage unit 103 Image processing unit 104 Convolution AE learning unit 105 Convolution AE parameter storage unit 106, 106a Background estimation unit 107 Water quality determination unit 108 Determination result output unit 111 Statistical processing unit 112 Background histogram storage unit 1061 Convolution AE background estimation unit 1062 Median calculation unit 1063 Estimated background image generation unit

Claims (3)

an image data storage unit that stores image data acquired from a water quality assessment target;
an image processing unit that processes the image data stored in the image data storage unit and outputs processed image data;
a convolution AE learning unit that learns image data when the water quality of the water quality judgment target is in a normal state as teacher data and outputs learned parameters;
a convolution AE parameter storage unit that stores the learned parameters output by the convolution AE learning unit;
a background estimation unit that constructs a convolution AE using the learned parameters stored in the convolution AE parameter storage unit to generate a convolution AE estimated background image, and generates and outputs estimated background image data from a background estimation value based on a representative value including a median or average value of each pixel of the convolution AE estimated background image and a past image;
A water quality determination unit that compares the processed image data output by the image processing unit with the estimated background image data output by the background estimation unit to generate differential image data, and determines water quality based on the differential image data. The water quality determination device according to claim 1, characterized in that the background estimation unit weights each of the images used to generate the convolution AE estimated background image and the estimated background image data. The water quality determination device according to claim 2, characterized in that the background estimation unit automatically performs the weighting using a histogram of pixel values.

Images