JP7670261B2 - Float meter reading program, float meter reading device, and float meter reading method - Google Patents

Description

The present invention relates to a float meter reading program, a float meter reading device, and a float meter reading method.

For example, factories that manufacture semiconductors and the like generally have many analog meters installed to measure the temperature, pressure, and other aspects of each piece of equipment. Specifically, factories such as the one described above are equipped with, for example, float meters that measure the flow rate of liquids or gases flowing through each piece of equipment.

The worker then, for example, patrols the factory at regular intervals and checks the values indicated by each analog meter. After that, the worker, for example, performs necessary calculations using the values of each analog meter and determines whether or not an abnormality (e.g., equipment failure, etc.) has occurred within the factory. As a result, if it is determined that an abnormality has occurred within the factory, the worker, for example, makes arrangements to replace the equipment with the abnormality (see Patent Documents 1 and 2).

Japanese Patent Application Publication No. 7-06552 Chinese Patent Publication No. 106289426

The reading of the analog meter as described above is generally performed by the worker's visual inspection. Therefore, when reading the analog meter as described above, there is a possibility that errors may occur in reading the value or inputting the value into the management system (not shown).

In particular, reading a float meter, whose scales are not spaced uniformly, is usually more difficult than reading other analog meters. As a result, reading float meters is more prone to errors in reading or inputting values than reading other analog meters.

The object of the present invention is to provide a float meter reading program, a float meter reading device, and a float meter reading method that enable the reading and input of float meter values to be performed automatically.

The float meter reading program of the present invention for achieving the above object is characterized in that it causes a computer to execute the following process: extracting first partial image data including a float meter from verification image data including a float meter having a float that moves vertically inside the float; extracting second partial image data including the float from the extracted first partial image data; identifying first and second positions in the vertical direction for the first and second parts included in the float from the extracted second partial image data; identifying the top end position of the float in the vertical direction from the identified first and second positions; and calculating the float meter scale value corresponding to the identified top end position by using a relational equation between each position in the vertical direction of the float meter and each scale value of the float meter.

The float meter reading program, float meter reading device, and float meter reading method of the present invention make it possible to automatically read and input float meter values.

FIG. 1 is a diagram showing an example of a configuration of an information processing device 1 according to the first embodiment. FIG. 2 is a diagram for explaining a specific example of the float meter 10. As shown in FIG. FIG. 3 is a diagram for explaining an outline of the float meter reading process in the first embodiment. FIG. 4 is a diagram for explaining an outline of the float meter reading process in the first embodiment. FIG. 5 is a flow chart illustrating details of the float meter reading process in the first embodiment. FIG. 6 is a flow chart illustrating details of the float meter reading process in the first embodiment. FIG. 7 is a flow chart illustrating details of the float meter reading process in the first embodiment. FIG. 8 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 9 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 10 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 11 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 12 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 13 is a diagram for explaining details of the float meter reading process in the first embodiment. FIG. 14 is a diagram for explaining details of the float meter reading process in the first embodiment.

Below, an embodiment of the present invention will be described with reference to the drawings. However, such an embodiment does not limit the technical scope of the present invention.

First, a configuration example of the information processing device 1 (hereinafter also referred to as the float meter reading device 1) in the first embodiment will be described. FIG. 1 is a diagram showing a configuration example of the information processing device 1 in the first embodiment.

The information processing device 1 is a computer device, for example, a general-purpose PC (Personal Computer). The information processing device 1 performs a process for automatically reading the value indicated by a float meter (not shown) (hereinafter, also referred to as a float meter reading process).

The information processing device 1 has the hardware configuration of a general-purpose computer device, and for example, as shown in FIG. 1, has a processor, a CPU 101, a memory 102, a communication interface 103, and a storage medium 104. Each part is connected to each other via a bus 105.

The storage medium 104 has, for example, a program storage area (not shown) that stores a program (not shown) for performing a float meter reading process.

The storage medium 104 also has a storage unit 110 (hereinafter also referred to as storage area 110) that stores information used when performing the float meter reading process. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

The CPU 101 executes the program loaded from the storage medium 104 to the memory 102 to perform the float meter reading process.

The communication interface 103 communicates with an operation terminal 2 such as a smartphone via a network NW such as the Internet.

[Example of a float meter]
Next, a specific example of the float meter 10 will be described.

As shown in Fig. 2, the float meter 10 has, for example, a cylindrical glass tube 11 that extends vertically, and a float 12 that moves vertically inside the glass tube 11. Note that the intervals between the scales 11a added to the side of the glass tube 11 increase vertically from the top to the bottom, as shown in Fig. 2.

Then, the operator measures the flow rate of the fluid, such as gas or liquid, that has flowed into the glass tube 11, for example, by reading the value (hereinafter also referred to as the scale value) of the scale 11a that corresponds to the position 12a (hereinafter also referred to as the upper end position 12a) that corresponds to the upper end edge of the float 12.

2, the floatmeter 10 has four circular first reference areas 13a, 13b, 13c, and 13d (hereinafter, these are also collectively referred to as the first reference areas 13), and two rectangular second reference areas 14a and 14b (hereinafter, these are also collectively referred to as the second reference areas 14). Each of the first reference areas 13 is, for example, a circular sticker (e.g., a blue sticker) affixed to the floatmeter 10, and each of the second reference areas 14 is, for example, a rectangular sticker (e.g., a yellow sticker) affixed to the floatmeter 10. The uses of the first reference areas 13 and the second reference areas 14 will be described later.

In the following description, the reading of the scale 11a is performed based on the top end position 12a of the float 12, but the reading of the scale 11a may be performed based on a position other than the top end position 12a of the float 12.

[Outline of the first embodiment]
Fig. 3 and Fig. 4 are diagrams for explaining an outline of the float meter reading process in the first embodiment. Specifically, Fig. 3 is a diagram for explaining the learning stage process of the float meter reading process in the first embodiment. Also, Fig. 4 is a diagram for explaining the inference stage process of the float meter reading process in the first embodiment.

First, we will explain the learning stage of the float meter reading process in the first embodiment.

As shown in FIG. 3, the first data generation unit 111 of the information processing device 1 generates learning data (hereinafter also referred to as first learning data) by adding information (hereinafter also referred to as position information) about the position of the floatmeter 10 in the first learning image data to the learning image data (hereinafter also referred to as first learning image data) showing the floatmeter 10.

Specifically, the first data generator 111 generates the first learning data by adding position information of the float meter 10 in the first learning image data to the first learning image data that includes at least the scale 11a and the four first reference areas 13, for example.

The first data generation unit 111 may generate further first learning data by changing the size, color, etc. of the first learning data that has already been generated.

Then, the first model generation unit 112 of the information processing device 1 generates a learning model (hereinafter also referred to as the first learning model) by, for example, performing machine learning using the multiple first learning data generated by the first data generation unit 111. After that, the first model generation unit 112 stores the generated first learning model in the memory area 110.

In other words, the first model generation unit 112 uses multiple pieces of first learning data to generate a first learning model that is used when extracting partial image data (hereinafter also referred to as first partial image data) showing the float meter 10 from the detection image in the inference stage.

The second data generation unit 113 of the information processing device 1 generates learning data (hereinafter also referred to as second learning data) by adding position information of the float 12 in the second learning image data to learning image data (hereinafter also referred to as second learning image data) that shows the float 12 moving inside the float meter 10.

Specifically, the second data generating unit 113 generates the second learning data by adding position information of the float 12 in the second learning image data to the second learning image data that shows the float 12 and an area vertically above the upper end position 12a of the float 12 (e.g., an area whose vertical length is the same as the length of the float 12).

The second data generation unit 113 may generate further second learning data by changing the size, color, etc. of the second learning data that has already been generated.

The second model generation unit 114 of the information processing device 1 then generates a learning model (hereinafter also referred to as the second learning model) by, for example, performing machine learning using the multiple pieces of second learning data generated by the second data generation unit 113. After that, the second model generation unit 114 stores the generated second learning model in the memory area 110.

In other words, the second model generation unit 114 uses multiple second learning data to generate a second learning model that is used when extracting partial image data (hereinafter also referred to as second partial image data) about the float 12 from the first partial image data in the inference stage.

Next, we will explain the inference stage of the float meter reading process in the first embodiment.

The first image extraction unit 121 of the information processing device 1 extracts, for example, first partial image data about the float meter from the verification image data. The verification image data is, for example, image data that shows the same float meter 10 as the first learning data.

Specifically, the first image extraction unit 121 identifies location information indicated by a value output in response to input of the verification image data to the first learning model stored in the memory area 110. Then, the first image extraction unit 121 extracts the first partial image data corresponding to the identified location information from the verification image data.

Then, the image correction unit 122 of the information processing device 1 performs, for example, homography transformation on the first partial image data extracted by the first image extraction unit 121.

Next, the second image extraction unit 123 of the information processing device 1 extracts second partial image data for the float 12 from the first partial image data extracted by the first image extraction unit 121.

Specifically, the second image extraction unit 123 identifies location information indicated by a value output in response to input of the first partial image data to the second learning model stored in the memory area 110. Then, the second image extraction unit 123 extracts the second partial image data corresponding to the identified location information from the first partial image data.

Then, the position identification unit 124 of the information processing device 1 identifies, from the second partial image data extracted by the second image extraction unit 123, the first position and the second position in the vertical direction for the first part and the second part constituting the float 12 reflected in the second image data.

Specifically, the position identification unit 124 calculates, for example, the sum of pixel values in a direction perpendicular to the vertical direction (hereinafter also referred to as the horizontal direction) at each of a plurality of positions in the vertical direction of the second partial image data extracted by the second image extraction unit 123. Then, the position identification unit 124 identifies, for example, any two of the positions in the vertical direction of the second partial image data extracted by the second image extraction unit 123 where the calculated sum is equal to or less than a threshold value as the first position and the second position, respectively.

Then, the top end identification unit 125 of the information processing device 1 identifies the top end position 12a of the float 12 reflected in the second image data by using the first position and the second position identified by the position identification unit 124.

Specifically, the top end determination unit 125 determines the top end position 12a of the float 12 reflected in the second image data, for example, from the ratio of the length from the top end position 12a to the second position to the length from the top end position 12a to the first position, and the first position and the second position in the second partial image data.

Furthermore, the value calculation unit 126 of the information processing device 1 calculates the scale value of the float meter 10 corresponding to the upper end position 12a identified by the upper end identification unit 125, for example, by using a relational equation between each vertical position on the float meter 10 and each scale value on the scale 11a of the float meter 10.

Specifically, the operator generates an approximation equation by using multiple combinations of the vertical position of the floatometer 10 shown in the first learning image and the scale value on the scale 11a of the floatometer 10 shown in the first learning image. The value calculation unit 126 then calculates the scale value corresponding to the upper end position 12a identified by the upper end identification unit 125, for example, by using the generated approximation equation. In the following description, the approximation equation used to calculate the scale value is a quadratic approximation equation, but this approximation equation may be, for example, a cubic approximation equation, a quartic approximation equation, or a logarithmic approximation equation.

That is, in a typical float meter, the shape of the part of the float used to read the scale (e.g., the top end position of the float) may be similar to the shape of other parts of the float. Therefore, for example, when using a single learning model that outputs the top end position 12a of the float 12 in response to input of verification image data showing the float meter 10, the information processing device 1 may identify a position other than the top end position 12a as the top end position 12a, and may output image data that does not include part of the float 12 as the second partial image data.

In this embodiment, the information processing device 1 generates a second learning model in the learning stage by using second learning data to which position information is added not only for the float 12 but also for the area vertically above the top end position 12a of the float 12. Then, in the inference stage, the information processing device 1 identifies the top end position 12a of the float 12 from the second partial image data output from the second learning model.

As a result, the information processing device 1 in this embodiment is able to generate a second learning model capable of outputting second partial image data including the entire float 12, even when a position other than the top end position 12a is identified as the top end position 12a. Therefore, the information processing device 1 in this embodiment is able to accurately read the scale 11a, for example, even for a float 12 in which the shape of the part used to read the scale 11a is similar to the shape of the other parts.

The first learning model and the second learning model generated in this embodiment may be, for example, a learning model based on SSD (Single Shot Multibox Detector) or a learning model based on YOLO (You Only Look Once).

[Details of the First Embodiment]
Next, the details of the float meter reading process in the first embodiment will be described. Figures 5 to 7 are flow charts for explaining the details of the float meter reading process in the first embodiment. Figures 8 to 14 are diagrams for explaining the details of the float meter reading process in the first embodiment.

[Processing in the learning stage]
First, the process in the learning stage will be described in detail in the float meter reading process in the first embodiment. FIG 5 is a diagram for explaining the process in the learning stage.

As shown in FIG. 5, the first data generation unit 111 waits, for example, until the learning timing arrives (NO in S11). The learning timing may be, for example, the timing when the worker inputs the first learning image data via the operation terminal 2. The learning timing may also be, for example, the timing when the worker inputs information to generate each learning model.

Then, when the learning timing arrives (YES in S11), the first data generation unit 111 generates the first learning data by adding the position information of the float meter 10 in each image data to each of the first learning image data (S12).

Specifically, as shown in FIG. 8, the first data generation unit 111 adds position information of the float meter 10 to the first learning image data DT11 by using, for example, a bounding box B1 including the float meter 10 (a bounding box B1 specified by the operator).

More specifically, the worker prepares image data corresponding to each frame included in video data captured by an imaging device (not shown), for example, as the first learning image data DT11. The worker then adds position information of the float meter 10 to the first learning image data DT11 by using a bounding box B1. After that, the first data generator 111 generates the first learning data, for example, by associating the first learning image data DT11 with the coordinates (X and Y coordinates) of the top left corner of the bounding box B1 and the vertical and horizontal lengths of the bounding box B1.

The imaging device may be, for example, a non-fixed camera. The imaging device may also be, for example, built into the operation terminal 2.

Then, the first model generation unit 112 generates a first learning model by performing machine learning using the first learning data generated in the processing of S12 (S13).

Next, the second data generation unit 113 generates second learning data by adding position information of the float 12 in each piece of image data to each piece of second learning image data (S14).

Specifically, as shown in FIG. 9, the second data generation unit 113 adds position information of the area including the float 12 to the second learning image data DT12 by using, for example, a bounding box B2 including the float 12 (a bounding box B2 specified by the worker).

More specifically, the second data generation unit 113 acquires, for example, image data output in response to input of the first learning image data DT11 to the first learning model generated in the process of S13 as the second learning image data DT12. Then, the worker uses the bounding box B2 to add position information of the area that reflects the float 12 and the area vertically above the top end position 12a of the float 12 to the second learning image data DT12. After that, the second data generation unit 113 generates the second learning data by, for example, associating the second learning image data DT12 with the coordinates (X and Y coordinates) of the top left corner of the bounding box B2 and the vertical and horizontal lengths of the bounding box B2.

Then, the second model generation unit 114 generates a second learning model by performing machine learning using the second learning data generated in the processing of S14 (S15).

This allows the worker to generate each learning model to be used in the inference stage described below.

[Processing in the inference stage]
Next, the process in the inference stage of the float meter reading process in the first embodiment will be described in detail with reference to Figs. 6 and 7.

As shown in FIG. 6, the first image extraction unit 121 waits until the judgment timing arrives (NO in S21). The judgment timing may be, for example, the timing when the worker inputs the verification image data via the operation terminal 2. The judgment timing may also be, for example, the timing when the worker inputs information to the effect that a judgment will be made on the verification image data.

Then, when the determination timing arrives (YES in S21), the first image extraction unit 121 acquires verification image data for the float meter 10 (S22).

Specifically, as shown in FIG. 10(A), the first image extraction unit 121 receives, for example, image data corresponding to each frame included in video data captured by an imaging device as verification image data DT21.

Next, the first image extraction unit 121 acquires values output from the first learning model in response to input of the verification image data acquired in the processing of S22 (S23).

Then, the first image extraction unit 121 extracts the first partial image data corresponding to the value obtained in the process of S23 from the verification image data obtained in the process of S22 (S24).

Specifically, as shown in FIG. 10(B), the first image extraction unit 121 acquires, for example, image data corresponding to the portion of the verification image data acquired in the processing of S21 that shows the float meter 10 as the first partial image data DT22.

The image correction unit 122 performs homography transformation on the first partial image data extracted in the processing of S24 (S25).

Specifically, the image correction unit 122 performs a homography transformation so that, for example, each of the second reference areas 14 in the floatometer 10 reflected in the first partial image data DT22 extracted in the processing of S24 moves to a pre-specified position.

That is, the image correction unit 122 normalizes the first partial image data DT22 by performing a homography transformation on the first partial image data DT22, and corrects the positions of each of the second reference areas 14 in the float meter 10 reflected in the first partial image data DT22.

This enables the information processing device 1 to align the position of the second reference area 14 reflected in all of the first partial image data DT22. Therefore, the information processing device 1 can identify the upper end position 12a of the float 12 based on the position of the second reference area 14, etc., as described below.

In addition, the image correction unit 122 may perform, for example, homography transformation on the first partial image data DT22 after emphasizing the first reference region 13 by performing an emphasis process such as binarization on the first partial image data DT22.

This enables the information processing device 1 to, for example, more easily identify the first and second positions described below.

Next, the second image extraction unit 123 acquires values output from the second learning model in response to input of the first partial image data that has been subjected to homography transformation in the processing of S25 (S26).

Then, as shown in FIG. 7, the second image extraction unit 123 extracts the second partial image data corresponding to the value obtained in the process of S26 from the first partial image data that has been subjected to homography transformation in the process of S25 (S31).

Specifically, as shown in FIG. 11(B), the second image extraction unit 123 acquires, for example, image data corresponding to the portion of the first partial image data DT22 (FIG. 11(A)) on which homography transformation has been performed in processing S25, which corresponds to the portion showing the float 12, as the second partial image data DT23.

More specifically, as shown in FIG. 11(B), the second image extraction unit 123 acquires image data that shows the area vertically above the top end position 12a of the float 12 in addition to the float 12 as the second partial image data DT23.

Next, the position identification unit 124 calculates the sum of the horizontal pixel values at each of the multiple vertical positions of the second partial image data extracted in the process of S31 (S32).

Then, the position identification unit 124 identifies, for example, any two of the positions where the total value calculated in the process of S32 satisfies a predetermined condition as the first position and the second position (S33). Specifically, the position identification unit 124 identifies, for example, any two of the positions where the total value calculated in the process of S32 is equal to or less than a threshold value (hereinafter also referred to as the first threshold value) as the first position and the second position. Specific examples of the processes of S32 and S33 are described below.

[Specific example of processing in S32 and S33]
12 and 13 are diagrams for explaining a specific example of the processes of S32 and S33. Specifically, FIG. 12(A) is a diagram for explaining a specific example of the float 12, and FIG. 12(B) is a diagram for explaining a specific example of the second partial image data DT23. Also, FIG. 13 is a graph showing the sum of pixel values at each position in the vertical direction of the second partial image data DT23. The vertical axis in FIG. 13 indicates the position (coordinate) in the vertical direction of the second partial image data DT23, and the horizontal axis in FIG. 13 indicates the sum of pixel values in the horizontal direction at each position. Note that, in the following, the vertical coordinate of each image data increases from the top to the bottom.

First, we will explain a specific example of float 12.

As shown in FIG. 12(A), the float 12 has, for example, an upper end position 12a that serves as a reference when reading the scale 11a. The float 12 also has a recess 12b on the side surface between positions 12b0 and 12b1 in the vertical direction, and further has a tapered portion 12c that extends downward from position 12c0 at the lower end in the vertical direction. Hereinafter, as shown in FIG. 12(B), the vertical coordinate corresponding to the upper end position in the second partial image data DT23 is also called coordinate b0, the vertical coordinate corresponding to the lower end position in the second partial image data DT23 is also called coordinate b1, the vertical coordinate corresponding to position 12b0 in the recess 12b is also called coordinate y1, the vertical coordinate corresponding to position 12b1 in the recess 12b is also called coordinate y2, and the vertical coordinate corresponding to position c1 in the tapered portion 12c is also called coordinate y3.

Next, we will explain specific examples of the processing in S32 and S33.

In the process of S32, the position identification unit 124 calculates, for example, the sum of the horizontal pixel values at each of a plurality of positions in the vertical direction of the second partial image data DT23. The plurality of positions here are, for example, positions corresponding to each pixel in the vertical direction of the second partial image data DT23.

Next, the position identification unit 124 generates a graph showing the sum of pixel values at each position in the vertical direction of the second partial image data DT23, as shown in, for example, FIG. 13.

Specifically, for example, if the vertical length of the second partial image data DT23 (the length from coordinate b0 to coordinate b1) is "80", the position identification unit 124 generates a graph in which, for example, coordinate b0 is set to "0" and coordinate b1 is set to "80", as shown in FIG. 13.

Then, in the process of S33, the position identification unit 124 refers to the generated graph and identifies, among multiple positions in the vertical direction of the second partial image data DT23, a position where the sum of pixel values is equal to or less than a threshold value.

Specifically, the graph shown in FIG. 13 indicates that the sum of pixel values corresponding to each point whose vertical coordinate is between "10" and "20" and the sum of pixel values corresponding to each point whose vertical coordinate is between "60" and "70" are particularly small. Therefore, the position identification unit 124, for example, identifies the start point (e.g., "14") and end point (e.g., "16") of the range with the smallest sum of pixel values among each point whose vertical coordinate is between "10" and "20" as coordinate y1 and coordinate y2, respectively. In addition, the position identification unit 124, for example, identifies the start point (e.g., "68") of the range with the smallest sum of pixel values among each point whose vertical coordinate is between "60" and "70" as coordinate y3.

That is, the sum of the horizontal pixel values in the recess 12b and tapered portion 12c reflected in the second partial image data DT23 is smaller than the sum of the pixel values in other portions of the float 12, for example, due to the way the light hits them when the verification image data DT21 is captured. Therefore, the position identification unit 124 can identify the coordinates y1, y2, and y3, for example, by identifying a position where the sum of the pixel values is equal to or less than a threshold value.

Then, the position identification unit 124 identifies, for example, two of coordinates y1, y2, and y3 as the first position and the second position.

In the process of S25, for example, when the first reference region 13 is enhanced by performing an enhancement process on the first partial image data DT22 and then homography transformation is performed, the sum of the horizontal pixel values in the recess 12b and the tapered portion 12c reflected in the second partial image data DT23 may be greater than the sum of the pixel values in other portions of the float 12. Therefore, in this case, the position identification unit 124 may, for example, identify any two of the positions where the sum calculated in the process of S32 is equal to or greater than a threshold value (hereinafter also referred to as the second threshold value) as the first position and the second position (S33). Specifically, the position identification unit 124 may, for example, identify the positions where the sum of the pixel values is equal to or greater than the threshold value, thereby identifying the coordinates y1, y2, and y3.

Returning to FIG. 7, the upper end determination unit 125 determines the upper end position 12a of the float 12 in the vertical direction from the first position and the second position determined in S33 (S35).

Specifically, the top end determination unit 125 calculates the coordinate y0 corresponding to the top end position 12a by using, for example, any of the following formulas (1), (2), and (3). In the following formulas (1), (2), and (3), c1, c2, and c3 are the length from the top end position 12a to position 12b0, the length from the top end position 12a to position 12b1, and the length from the top end position 12a to position 12c0, respectively, measured using other image data (not shown).

More specifically, for example, if coordinates y3 and y2 are identified as the first position and the second position in the processing of S34, the upper end identification unit 125 calculates coordinate y0 according to, for example, the following formula (1).

That is, for example, the value obtained by dividing the length from the top end position 12a to position 12c0 by the length from the top end position 12a to position 12b1 is the same for all image data showing the float 12. Therefore, the position identification unit 124 calculates the coordinate y0 by using the value obtained by dividing the length from the top end position 12a to position 12c0 measured using other image data by the length from the top end position 12a to position 12b1 measured using other image data.

For example, if the coordinates y3 and y1 are identified as the first and second positions in the processing of S34, the upper end identification unit 125 calculates the coordinate y0 according to, for example, the following formula (2).

For example, if the coordinates y2 and y1 are identified as the first and second positions in the processing of S34, the upper end identification unit 125 calculates the coordinate y0 according to, for example, the following formula (3).

The upper end determination unit 125 may calculate the coordinate y0 of the upper end position 12a by using, for example, all of the above formulas (1), (2), and (3). In this case, the upper end determination unit 125 may calculate the median or average of the coordinate y0 calculated by using, for example, each of the above formulas (1), (2), and (3).

Returning to FIG. 7, the value calculation unit 126 calculates the scale value of the float meter 10 corresponding to the upper end position 12a identified in the processing of S34 (S35).

Specifically, as shown in FIG. 14, the value calculation unit 126 calculates the coordinate in the first partial image data DT22 of the coordinate y0 calculated in the processing of S34 (hereinafter also referred to as the coordinate Y0). Then, the value calculation unit 126 obtains a value indicating the position of the coordinate Y0 in the first partial image data DT22 by dividing the length from the top end position in the vertical direction of the first partial image data DT22 to the top end position 12a of the float 12 (the length from coordinate m0 to coordinate Y0 shown in FIG. 14) by the length from the top end position to the bottom end position in the vertical direction of the first partial image data DT22 (the length from coordinate m0 to coordinate m1 shown in FIG. 14).

More specifically, the value calculation unit 126 calculates the coordinate Y0 by adding the coordinate b0 in the first partial image data DT22 (hereinafter also referred to as the coordinate B0) described in FIG. 12B to the coordinate y0 calculated in the process of S34. Then, the value calculation unit 126 obtains a value indicating the position of the coordinate Y0 in the first partial image data DT22 by following the following formula (4). Note that in the following formula (4), P corresponds to the value indicating the position of the coordinate Y0, y _min corresponds to the coordinate m0 corresponding to the upper end position of the second reference area 14a, and y _max corresponds to the coordinate m1 corresponding to the lower end position of the second reference area 14b.

Then, the value calculation unit 126 calculates the scale value corresponding to the coordinate Y0 (the scale value on the scale 11a) by substituting a value indicating the position of the coordinate Y0 into the quadratic approximation equation corresponding to the float meter 10 that was generated in advance. After that, the value calculation unit 126 outputs the calculated scale value to the operation terminal 2, for example.

For example, when the pixel size of image data (hereinafter also referred to as reference image data) corresponding to each frame included in video data captured by an imaging device is different from that of the verification image data DT21, the value calculation unit 126 may, for example, multiply the verification image data DT21 by a scaling factor of the reference image data relative to the verification image data DT21, and then substitute this into the quadratic approximation equation. In other words, the value calculation unit 126 may substitute a value indicating the position of the coordinate Y0 in the reference image data into the quadratic approximation equation.

The worker may also generate a learning model (not shown) to be used in calculating the scale value on the scale 11a of the floatometer 10 reflected in the first partial image data DT22, for example, by using learning data (not shown) including the first learning image data DT11 and information indicating the scale value on the scale 11a of the floatometer 10 reflected in the first learning image data DT11. The value calculation unit 126 may then identify, for example, a value output in response to input of the first partial image data DT22 to the generated learning model as the scale value on the scale 11a of the floatometer 10 reflected in the first partial image data DT22.

Furthermore, for example, if the correct answer value of the scale value on the scale 11a corresponding to the upper end position 12a of the float 12 or a correct equation that can calculate the scale value on the scale 11a by using the upper end position 12a of the float 12 is known in advance, the value calculation unit 126 may use this correct answer value or correct equation to identify the scale value on the scale 11a of the float meter 10 reflected in the first partial image data DT22.

In this manner, the information processing device 1 in this embodiment uses a first learning model that outputs first partial image data showing the float meter 10 in response to input of verification image data, and a second learning model that outputs second partial image data showing the float 12 and the area vertically above the top end position 12a of the float 12 in response to input of the first partial image data, to sequentially identify the float meter 10 shown in the verification image data and the float 12 shown in the first partial image data. Then, the information processing device 1 identifies the top end position 12a of the float 12 shown in the second partial image data, for example, by using the first and second positions (positions other than the top end position 12a of the float) of the float 12 shown in the second partial image data.

As a result, the information processing device 1 in this embodiment is able to generate a second learning model capable of outputting second partial image data including the entire float 12, even when a position other than the top end position 12a is identified as the top end position 12a. Therefore, the information processing device 1 in this embodiment is able to accurately read the scale 11a, for example, even for a float 12 in which the shape of the part used to read the scale 11a is similar to the shape of the other parts.

The first image extraction unit 121 may continuously acquire verification image data DT21 corresponding to each frame included in the video data captured by the imaging device in the process of S22. The information processing device 1 may then perform processes from S23 to S35 for each of the acquired verification image data DT21. Thereafter, the value calculation unit 126 may output the median and average of each of the scale values calculated in the process of S35 to the operation terminal 2.

In addition, in the processing of S12, the first data generation unit 111 may, for example, automatically determine the position of the bounding box B1 from the positions of each of the reference markers 13.

Specifically, the first data generation unit 111 may, for example, automatically identify a position that includes all of the reference markers 13 as the position of the bounding box B1. The first data generation unit 111 may then automatically generate the first learning data by adding position information of the identified bounding box B1 to the first learning image data DT11.

This enables the information processing device 1 to reduce the workload of workers involved in generating the first learning data.

In addition, in the processing of S14, the second data generation unit 113 may automatically determine the position of the bounding box B2 from, for example, each position of the reference marker 14 and a graph showing the sum of pixel values corresponding to each position in the vertical direction of the second learning image data DT12 (the graph described in FIG. 13).

Specifically, the second data generation unit 113, for example, identifies a range in the vertical direction among the positions in the second learning image data DT12 where the sum of pixel values is equal to or less than a threshold value. Then, the second data generation unit 113 may, for example, automatically identify a position corresponding to an area obtained by expanding the identified range vertically upward and downward by a predetermined length as the position of the bounding box B2. Thereafter, the second data generation unit 113 may automatically generate the second learning data by adding position information of the identified bounding box B2 to the second learning image data DT12.

This enables the information processing device 1 to reduce the workload of workers involved in generating the second learning data.

1: Information processing device 2: Operation terminal 101: CPU
102: Memory 103: Communication interface 104: Storage medium 105: Bus

Claims (11)

Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter;
extracting second partial image data including the float from the extracted first partial image data;
Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively;
Identifying an upper end position of the float in the vertical direction from the identified first position and second position;
calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter;
A float meter reading program that causes a computer to execute a process. In claim 1,
In the process of identifying the first position and the second position,
calculating a sum of pixel values in a horizontal direction at each position of the second partial image data in the vertical direction;
identifying, as the first position and the second position, any two of the positions of the second partial image data in the vertical direction where the calculated sum satisfies a predetermined condition;
A float meter reading program comprising: In claim 2,
In the process of identifying the first position and the second position, any two of positions of the second partial image data in the vertical direction where the sum is equal to or less than a first threshold value are identified as the first position and the second position.
A float meter reading program comprising: In claim 2,
In the process of identifying the first position and the second position, any two of the positions of the second partial image data in the vertical direction where the sum is equal to or greater than a second threshold value are identified as the first position and the second position.
A float meter reading program comprising: In claim 1,
In the process of extracting the second partial image data, partial image data including the float and an area above the float in the vertical direction, the area having a length in the vertical direction equal to or greater than a predetermined length, is extracted as the second partial image data.
A float meter reading program comprising: In claim 1, further comprising:
performing a homography transformation on the first partial image data;
The process is executed by a computer,
In the process of extracting the second partial image data, the second partial image data is extracted from the first partial image data that has been subjected to homography transformation.
A float meter reading program comprising: In claim 1,
In the process of identifying the upper end position,
determining an upper end position of the float in the vertical direction from a ratio of a length from the upper end position to the second position to a length from the upper end position to the first position, and from the first position and the second position in the second partial image data;
A float meter reading program comprising: In claim 1, further comprising:
generating a plurality of first learning data by adding position information of the float meter included in each of the plurality of first learning image data, the position information being included in each of the plurality of first learning image data;
generating a first learning model by performing machine learning using the generated plurality of first learning data;
The process is executed by a computer,
In the process of extracting the first partial image data,
acquiring a value output from the first learning model in response to inputting the verification image data;
extracting the first partial image data corresponding to the acquired value from the verification image data;
A float meter reading program comprising: In claim 1, further comprising:
generating a plurality of second learning data by adding position information of the float included in each of the plurality of second learning image data including the float;
generating a second learning model by performing machine learning using the generated plurality of second learning data;
The process is executed by a computer,
In the process of extracting the second partial image data,
Obtaining a value output from the second learning model in response to input of the first partial image data;
extracting the second partial image data corresponding to the obtained value from the first partial image data;
A float meter reading program comprising: Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter;
extracting second partial image data including the float from the extracted first partial image data;
Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively;
Identifying an upper end position of the float in the vertical direction from the identified first position and second position;
calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter;
A float meter reading device comprising: Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter;
extracting second partial image data including the float from the extracted first partial image data;
Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively;
Identifying an upper end position of the float in the vertical direction from the identified first position and second position;
calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter;
A method for reading a float meter, the method comprising the steps of: executing processing by a computer;

Images