JP7745504B2 - Reading method, reading device, reading system, and program - Google Patents

Description

This disclosure relates to a reading method, a reading device, a reading system, and a program.

A method has been proposed in which an indicator, which shows its state with a pointer, is imaged with a camera and the value indicated by the pointer is read from the image. For example, Patent Document 1 discloses a meter reading device that compares an image of an indicator with a reference image taken when the value indicated by the pointer is known, determines the rotation angle of the pointer shown in the image, and determines the indicated value from the rotation angle and scale information.

Japanese Patent Application Laid-Open No. 2020-166332

In order to read the value indicated by the indicator from the captured image, it is necessary to accurately identify the position indicated by the indicator needle.

This disclosure provides a reading method, reading device, reading system, and program that can solve the above problems.

The reading method disclosed herein includes the steps of: acquiring a captured image of an indicator that indicates an indication value at the position of the indication unit; moving the position of the indicator in a pseudo-image that simulates the indicator; identifying the amount of movement of the indicator in the pseudo-image when the position of the indicator in the pseudo-image is closest to the position of the indicator in the captured image; and estimating the indication value indicated by the indicator based on the identified amount of movement, wherein the indicator is a switch, the indication unit is an operating unit of the switch, and in the estimating step, the setting value of the switch is estimated based on the identified amount of movement of the operating unit.

The reading device of the present disclosure comprises a means for acquiring a captured image of an indicator that indicates an indication value at the position of the indication unit, a means for moving the position of the indicator in a pseudo image that simulates the indicator, a means for identifying the amount of movement of the indicator in the pseudo image when the position of the indicator in the pseudo image is closest to the position of the indicator in the captured image, and a means for estimating the indication value indicated by the indicator based on the identified amount of movement , wherein the indicator is a switch, the indication unit is an operating unit of the switch, and the estimating means estimates the setting value of the switch based on the identified amount of movement of the operating unit.

The reading system of the present disclosure comprises an indicator that indicates an indication value at the position of the indicator, a camera that photographs the indicator, and the above-mentioned reading device that further comprises means for transmitting the indication value.

The program disclosed herein causes a computer to perform the following steps: acquiring a captured image of an indicator that indicates an indication value at the position of the indicator; moving the position of the indicator in a pseudo-image that simulates the indicator; identifying the amount of movement of the indicator in the pseudo-image when the position of the indicator in the pseudo-image is closest to the position of the indicator in the captured image; and estimating the indication value indicated by the indicator based on the identified amount of movement, wherein the indicator is a switch and the indicator is an operating part of the switch, and the estimating step causes the computer to execute a process of estimating the setting value of the switch based on the identified amount of movement of the operating part .

The above-described reading method, reading device, reading system, and program enable the indicated value shown by the indicator to be read with high accuracy.

FIG. 1 is a diagram illustrating an example of an indication value acquisition system according to an embodiment. FIG. 2 is a diagram illustrating an example of an indicator (measuring instrument) according to an embodiment. FIG. 2 is a diagram illustrating an example of an indicator (switch) according to an embodiment. 10A and 10B are diagrams illustrating the movement of an indicator portion of an indicator according to an embodiment and estimation of an indicated value. FIG. 10 is a diagram illustrating weighting coefficients according to the embodiment. 10 is a flowchart illustrating an example of a process for estimating an indication value according to the embodiment. 10A and 10B are diagrams illustrating an outline of a process when the indication value estimation process according to the embodiment is applied to a horizontal measuring instrument. 10A and 10B are diagrams illustrating an outline of a process when the process of estimating an indication value according to the embodiment is applied to a toggle switch. FIG. 10 is a diagram illustrating a conventional example of an instruction value estimation process. FIG. 2 is a diagram illustrating an example of a hardware configuration of a reading device according to an embodiment.

The indication value acquisition system of the present disclosure will be described below with reference to FIGS.
<Embodiment>
(System configuration)
FIG. 1 is a diagram illustrating an example of an indication value acquisition system according to an embodiment.
The indicator value acquisition system 100 includes an indicator 1, a camera 5, a reading device 10, a terminal device 20, and another system 30. The indicator 1 is a measuring instrument or switch that does not have a means for communication or transmission. An example of the indicator 1 is shown in FIGS. 2( a) to 2(c). The indicator 1 is a round analog measuring instrument that indicates a state quantity of a monitored object (e.g., current, voltage, temperature, pressure, flow rate, rotation speed, vibration frequency, etc.) by swinging an indicator needle in the direction of clockwise movement or by touching both sides, as shown in FIG. 2(a). Alternatively, the indicator 1 may be a horizontal analog measuring instrument as shown in FIG. 2(b) or a vertical analog measuring instrument as shown in FIG. 2(c). In the indicator 1 of these measuring instruments, the measurement value is indicated by an indicator unit 2 (pointer needle). The indicator 1 may also be a switch operated by an operating unit such as a lever or knob. Examples of a lever switch, a knob switch, a slide switch, and a dial switch are shown in Figures 3(a), 3(b), 3(c), and 3(d), respectively. In these switch-type indicators 1, the setting value (ON or OFF, low/medium/high, 1-4, brightness/darkness, etc.) is indicated by the position pointed to by the indicator 2 (operation unit). The measured values measured by these measuring instruments and the setting values indicated by the switches can only be confirmed by going to the location where the indicator 1 is installed. The indication value acquisition system 100 reads the indication values displayed by the indicators 1 installed in factories, plants, commercial facilities, etc., converts the read content into digital data, and transmits it to the terminal device 20 or another system 30.

The camera 5 is installed in a position where it can photograph the indicator 1, and continuously photographs the values indicated by the indicator 1, either constantly or at predetermined time intervals. The camera 5 is communicatively connected to the reading device 10 and outputs the photographed images to the reading device 10. The reading device 10 may be installed near the camera 5 or at a distance from the camera 5. The reading device 10 analyzes the images photographed by the camera 5 (referred to as photographed images), reads the measured values indicated by the measuring instruments and the setting values indicated by the switches, and outputs them as digital data. The reading device 10 is communicatively connected to the terminal device 20 and other systems 30 via the network NW, and transmits digital data including the measured values and setting values to the terminal device 20 and other systems 30. The terminal device 20 is, for example, a personal computer (PC) 20a or a mobile terminal 20b such as a smartphone or tablet. The other system 30 is, for example, a monitoring system that monitors the measurement values indicated by the indicator 1, or a system that operates by acquiring the measurement values indicated by the indicator 1. The indication value acquisition system 100 allows the indication values displayed by the indicator 1 to be collected online at a remote location and used.

The reading device 10 includes an image acquisition unit 11, an image processing unit 12, a disturbance removal unit 13, an instruction value estimation unit 14, a deep learning unit 15, a setting reception unit 16, a memory unit 17, and a communication unit 18.
The image acquisition unit 11 acquires the image captured by the camera 5 .
The image processing unit 12 performs image processing such as resizing the captured image and cutting out only the area necessary for reading the indicated value.
The disturbance removal unit 13 removes disturbances such as shadows and halation from the captured image. The disturbance removal unit 13 removes disturbances such as noise using a deep learning model constructed using a DnCNN (Denoising Convolutional Neural Network). The DnCNN is a known CNN for removing disturbances, and by providing learning images to the DnCNN and having it learn, a deep learning model that accurately removes noise can be constructed.

The indicated value estimation unit 14 moves an indicator (called a pseudo needle) simulating an indicator needle, lever, etc., within the movable range of the indicator needle, lever, etc. on an image simulating the indicator 1 (called a pseudo image), and evaluates the degree of overlap with the indicator captured in the captured image. Figure 4(a) shows an example of a captured image 50 of a round measuring instrument (indicator 1). In the captured image 50, the indicator 51 points to the indicated value α. Figure 4(b) shows a pseudo image 60 of the captured image 50 in Figure 4(a). In the pseudo image 60, the pseudo needle 61 corresponding to the indicator 51 is configured to be movable (moved by image processing). The min to max values shown in Figure 4(b) correspond to the movable range of the indicator 2 of the indicator 1. In the pseudo image 60, the indicated value estimation unit 14 moves (rotates) the pseudo needle 61 clockwise from a predetermined starting position (e.g., the min position) at a predetermined resolution. Resolution is the minimum amount of movement, and this amount of movement directly determines the resolution in terms of reading accuracy. For example, the amount of movement by which the tip of the pseudo needle 61 moves by one pixel can be defined as the resolution. The indication value estimation unit 14 superimposes the pseudo image 60 on the captured image 50 and evaluates the degree of overlap between the indicator 51 and the pseudo needle 61. The indication value estimation unit 14 calculates the degree of overlap between the indicator 51 and the captured image 50 each time movement is performed. For example, the signal intensity indicating the degree of overlap between the pseudo needle 61 and the indicator 51 may be calculated using a convolution integral. The indication value estimation unit 14 identifies the amount of movement (movement angle) θ when the signal intensity indicating the degree of overlap between the pseudo needle 61 and the indicator 51 is maximized. Figure 4(c) shows the pseudo image 60 and the movement angle θ when the signal intensity indicating the degree of overlap between the pseudo needle 61 and the indicator 51 is maximized (when the position of the pseudo needle 61 is closest to the position of the indicator 51). The relationship between the movement amount of the indicator 2 in the indicator 1 and the indication value is analyzed in advance, and a table defining the correspondence between the movement amount and the indication value is stored in the memory unit 17. The indication value estimation unit 14 estimates the indication value α from the identified movement amount θ and the table defining the correspondence between the movement amount and the indication value.

When calculating the signal strength indicating the degree of overlap between the pseudo-needle 61 and the indicator 51, the indication value estimation unit 14 may calculate the signal strength by setting a large weighting coefficient for the characteristics of the indicator 2. For example, the indication value estimation unit 14 divides the indicator 51, which has the shape shown in FIG. 5, into three regions, 51a to 51c, and calculates the signal strength of the degree of overlap for each region. The indication value estimation unit 14 then assigns the largest weighting coefficient to region 51a, which has the most distinctive shape, the smallest weighting coefficient to region 51c, which has the least distinctive shape, and a medium weighting coefficient to region 51b to calculate the signal strength indicating the degree of overlap between the pseudo-needle 61 and the indicator 51. For example, if the signal intensities calculated for regions 51a-51c are Sa-Sc, respectively, and the weighting coefficients for regions 51a-51c are ka-kc (ka > kb > kc), respectively, the indication value estimation unit 14 calculates the signal intensity indicating the degree of overlap using ka*Sa + kb*Sb + kc*Sc. Note that any method can be used to calculate the signal intensity indicating the degree of overlap between the pseudo-needle 61 and the indication unit 51. Also, while weighting was explained using the indication unit 51 in the captured image 50 in Figure 5, a weighting coefficient may also be set for the pseudo-needle 61 in the pseudo image 60.

The deep learning unit 15 uses DnCNN to create a deep learning model that removes disturbances. For example, for various types of indicators 1, images with various disturbances added (not necessarily the entire indicator 1, but images with noise added to the scale area and indicator part 2) and images without disturbances are provided to the deep learning unit 15. The deep learning unit 15 trains the DnCNN to learn how to detect and remove disturbances from these images, and creates a deep learning model for noise removal.

The setting acceptance unit 16 is configured using input devices such as a keyboard, mouse, touch panel, and buttons, and accepts various settings related to the reading of indicated values entered by the user using these input devices. For example, the setting acceptance unit 16 accepts settings such as the resolution setting, the starting position of the indicator 2's movement, the movement range, the movement direction, the weighting factor, and the area to which the weighting factor is assigned. For example, if the indicator 1 is a thermometer capable of measuring temperatures from 0 to 100°C and the measured values are known to fall within the range of 45 to 50°C, the user can set the movement starting position to 45°C, the movement range to 45 to 50°C, the movement direction to the direction of increasing temperature, and the resolution to the minimum resolution (e.g., one pixel resolution, or 0.1°C to match the minimum resolution of the original measuring device). If only temperatures in 1°C increments are required, the resolution can be set to the movement amount (e.g., number of pixels) corresponding to 1°C. The user may also set the movement start position to 50°C and the movement direction to a decreasing temperature, or may set the movement start position to 47°C and the initial movement direction to a decreasing temperature, and then move from 47°C to 50°C after reaching 45°C. Furthermore, if there is no significant change in the value measured by the indicator 1, the movement start position may be set to a position corresponding to the previously estimated indicated value, and the movement range may be set to the range of possible change from that start position. The setting reception unit 16 acquires the settings input by the user and records them in the memory unit 17.

The memory unit 17 is configured using a storage device such as a hard disk drive (HDD) or flash memory, and stores captured images, pseudo images, settings accepted by the setting acceptance unit 16, learning images used to construct a deep learning model for noise removal, the deep learning model after learning, a table defining the correspondence between movement amounts and instruction values, etc.

The communication unit 18 communicates with other devices. For example, the communication unit 18 transmits the indication value estimated by the indication value estimation unit 14 to the terminal device 20 or other system 30. The communication unit 18 may transmit the indication value each time an indication value is estimated, or may transmit the indication value in response to a request from the terminal device 20 or other system 30.

(operation)
Next, the operation of the indication value acquisition system 100 will be described with reference to FIG.
As a premise, it is assumed that a deep learning model for removing disturbances that has already been constructed, various settings accepted by the setting acceptance unit 16, a pseudo image 60 that allows the pseudo needle 61 to be moved arbitrarily, and a table that defines the correspondence between the amount of movement and the indicated value are pre-recorded in the memory unit 17.
The image acquisition unit 11 acquires the captured image 50 of the indicator 1 captured by the camera 5 (step S1). The image acquisition unit 11 records the captured image 50 in the storage unit 17.
Next, the image processing unit 12 processes the captured image 50 (step S2). The image processing unit 12 resizes the captured image 50 to an image with a predetermined resolution (for example, the minimum resolution required to read the indicated value) or cuts out only the portion of the image required to read the indicated value. These processes are performed to reduce the processing load. The image processing unit 12 records the captured image 50 after image processing in the storage unit 17. The image processing unit 12 may also create a pseudo image 60 by performing predetermined image processing on the captured image 50 after image processing, and record the created pseudo image 60 in the storage unit 17.

Next, the disturbance removal unit 13 provides the processed captured image 50 to a deep learning model for disturbance removal to remove disturbances such as shadows and halation from the captured image (step S3). The disturbance removal unit 13 records the disturbance-removed captured image 50 in the memory unit 17.

Note that the order of steps S3 and S2 may be reversed. That is, the disturbance removal unit 13 may remove disturbances from the captured image 50 acquired by the image acquisition unit 11 (step S3), and then the image processing unit 12 may perform image processing on the captured image 50 after the disturbances have been removed (step S2).

Next, the indication value estimation unit 14 estimates the indication value of the indicator 1 using the photographed image 50 and pseudo image 60 after the disturbance has been removed. First, the indication value estimation unit 14 adjusts the image so that the photographed image 50 in FIG. 4(a) and the pseudo image 60 in FIG. 4(b) are the same size. Then, in the pseudo image 60 illustrated in FIG. 4(b), the indication value estimation unit 14 moves the pseudo needle 61 from its movement start position in units of a predetermined resolution (in this example, the tip of the pseudo needle 61 is moved, for example, by one pixel in the direction of the clock's hands) (step S4), and each time, calculates the signal strength indicating the degree of overlap between the pseudo needle 61 and the indicator 51 illustrated in FIG. 4(b). For example, if the movement range is set from min to max in FIG. 4(b), the indication value estimation unit 14 starts from the position where the tip of the pseudo needle 61 points to min and moves it in units of resolution to the position where the tip of the pseudo needle 61 points to max. The indication value estimation unit 14 associates the movement amount (in this example, the angle between the pseudo needle 61 at the movement start position and the pseudo needle 61 after movement) with the signal strength and records them in the storage unit 17. When the pseudo needle 61 has completed movement throughout the entire movement range, the indication value estimation unit 14 identifies the movement amount when the indication unit 51 in the captured image 50 and the pseudo needle 61 in the pseudo image 60 are closest (overlapped) (step S5). Specifically, the indication value estimation unit 14 selects the largest signal strength from the signal strengths calculated for each movement and reads out the movement amount recorded in association with the selected signal strength from the storage unit 17. The read-out movement amount is the identified movement amount. Next, the indication value estimation unit 14 estimates the indication value from the identified movement amount (step S6). Specifically, the indication value estimation unit 14 refers to a table that defines the correspondence relationship between movement amounts and indication values stored in the storage unit 17, and reads out the indication value corresponding to the movement amount that maximizes the signal strength. The indication value estimation unit 14 estimates the read indication value as the indication value of the indicator 1. The indication value estimation unit 14 records the estimated indication value in the memory unit 17.

Next, the communication unit 18 reads the estimated indication value from the memory unit 17 and transmits it to the terminal device 20 or the other system 30 (step S7). The terminal device 20 displays the transmitted indication value. The user of the terminal device 20 can check the indication value displayed by the indicator 1 without going to the location where the indicator 1 is installed. The other system 30, for example, compares the transmitted indication value with a threshold value to detect abnormalities. This makes it possible to obtain measurement values in real time and monitor equipment and facilities equipped with measuring instruments (indicators 1) that do not have communication means, even from a remote location such as a monitoring room.

(Application example 1)
Next, referring to FIG. 7 , the process of estimating the indicated value when the indicator 1 is a horizontal analog measuring instrument will be described. FIG. 7( a) illustrates an example of a captured image 50 after image processing (step S2). The captured image 50 includes disturbances such as shadows. The disturbance removal unit 13 removes disturbances from the captured image 50 (step S3) and outputs a captured image 50' after the disturbances have been removed. (The order of steps S2 and S3 may be reversed.) The captured image 50' is shown in FIG. 7( b). Next, the indicated value estimation unit 14 superimposes the disturbance-removed captured image 50' on a pseudo image 60 (not shown) and moves the pseudo needle 61 to estimate the indicated value (step S6). Specifically, the pseudo needle 61 is moved left and right to determine the amount of movement when the overlapping area between the pseudo needle 61 and the indicator 51' is largest. The position corresponding to this amount of movement is used as the needle position, and the indicated value is read. In this case, the distance moved left and right is the amount of movement. 5, a greater weight may be assigned to the tip of the indicator 51' or the pseudo needle 61 to calculate the signal strength of the overlap. By applying a process similar to that described in FIG. 7, the indicated value can be read from the vertical analog measuring instrument shown in FIG. 2(c), and setting values 1 to 4 can be read from the slide switch shown in FIG. 3(c). Furthermore, for the knob switch shown in FIG. 3(b) and the dial switch shown in FIG. 3(d), setting values (low, medium, high, brightness, and darkness) can be read by the same process as for the round measuring instruments described in FIG. 4. The pseudo needle 61 may be moved in a sweep (continuous movement) or stepwise movement (discrete movement) depending on the set resolution. For example, in the case of the switch shown in Fig. 3(b), the indication value estimation unit 14 may move the pseudo needle 61 in three steps of low, medium, and high, calculate the signal strength indicating the degree of overlap at each position, and specify the amount of movement when the signal strength is at its maximum. The same applies to the slide switch shown in Fig. 3(c).

(Application example 2)
Next, with reference to FIG. 8 , a process for estimating a set value when the indicator 1 is a lever-type switch (toggle switch) will be described. FIG. 8( a) shows an example of a captured image of a toggle switch photographed from an oblique direction. Six toggle switches (indicators 51) are shown in the captured image 50. A pseudo image simulating the captured image 50 shown in FIG. 8( a) is prepared. The indicated value estimation unit 14 then moves the tip of the pseudo toggle switch (corresponding to the pseudo needle 61) in the pseudo image while keeping its base fixed, and superimposes it on the captured image 50 shown in FIG. 8( a). The indicated value estimation unit 14 evaluates the degree of overlap between the pseudo toggle switch and the indicators 51 to estimate whether each switch is indicating ON or OFF.

Figure 8(b) shows an example of a captured image of a toggle switch taken from the front. In this case, a pseudo image simulating the captured image 50 shown in Figure 8(b) is similarly prepared. The indication value estimation unit 14 then moves the pseudo toggle switch in the pseudo image vertically, superimposing it on the captured image 50 in Figure 8(b), and evaluates the degree of overlap between the pseudo toggle switch and the indication unit 51 to estimate whether each switch is indicating ON or OFF.

(Compared to conventional methods)
Next, an example of a conventional reading method will be described for comparison with this embodiment. This conventional method performs the following process. First, as shown in FIG. 9, several hundred images are prepared by moving the indicator 2 of the indicator 1 in increments of 1° within the min to max range. Furthermore, for each image, images in which shadows, halation, etc. occur are prepared. These images are then used to perform deep learning and build a CNN model. When the captured image of the evaluation target is evaluated using the CNN model, the image closest to the captured image of the evaluation target is selected. The indicated value is estimated from the position of the indicator needle in the selected image.

With this conventional method, a huge number of training images are required to improve accuracy. For example, the resolution is 1°, the movement interval of images prepared in advance. However, if this resolution is insufficient for accuracy, images must be captured while moving the indicator 2 in 0.1° increments, for example, resulting in enormous processing costs. Furthermore, if an image matching the position of the indicator needle cannot be selected, the likelihood decreases. In contrast, with this embodiment, the pseudo-needle 61 is moved through image processing, so even high resolution can be freely set. Regardless of the resolution, the indicated value can be estimated with relatively light processing load. Furthermore, there is no need to prepare a large number of images (training data) in advance, and no learning process is required for these images. Furthermore, because it does not rely on training data, there is no decrease in accuracy or likelihood.

In addition, to improve reading accuracy, reliable disturbance removal is necessary. However, with conventional methods, improving disturbance removal accuracy requires preparing and training multiple image patterns that take into account factors such as shadows and halation in the image, such as the light source position. This requires not only numerous patterns for the position of the indicator 2, but also as many images as there are combinations of light source position patterns, necessitating a huge amount of training data. In contrast, this embodiment separates disturbance removal from indicated value estimation, and uses DnCNN for disturbance removal. While images are required for training the DnCNN, there is no need to prepare or train as many training data sets as there are combinations of indicator needle position patterns and disturbance patterns, significantly reducing processing costs. Furthermore, with conventional methods, unless training data is generated from actual captured images after camera 5 is installed and training is performed, noise corresponding to the actual environment may not be sufficiently removed, potentially resulting in a long time being required to read indicated values with sufficient accuracy. In contrast, in this embodiment, by using DnCNN, it is possible to construct a general-purpose deep learning model that can remove much of the noise that occurs in various real environments through pre-training alone, making it easy to ensure accuracy and eliminating the need to reconstruct the deep learning model after the start of actual operation.

(effect)
As described above, according to this embodiment, it is possible to accurately read indicated values (measured values and set values) displayed by the indicator 1, such as analog measuring instruments and switches that can only be checked on-site. Furthermore, there is no need to prepare teaching data or perform learning in order to read indicated values, and reading can be performed with a relatively low processing load. Furthermore, by setting the resolution finer, reading accuracy can be improved, so reading accuracy can be improved quickly (without the need for learning, etc.). Furthermore, the user can change the resolution simply by setting the resolution.

Furthermore, the reading device 10 can transmit the estimated indication value to the terminal device 20 or another system 30. Because the data size of the indication value (digital data) is small, there are few communication constraints, making it applicable to many plants, factories, facilities, etc., including plants that impose limits on communication volume, such as nuclear plants. This makes it possible to automate tasks such as visually checking the indication value of the indicator 1 by on-site workers, thereby improving work efficiency.

FIG. 10 is a diagram illustrating an example of a hardware configuration of a reading device according to an embodiment.
The computer 900 includes a CPU 901 , a main memory device 902 , an auxiliary memory device 903 , an input/output interface 904 , and a communication interface 905 .
The above-described reading device 10 is implemented in a computer 900. The above-described functions are stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, loads it into the main storage device 902, and executes the above-described processing in accordance with the program. The CPU 901 also allocates a storage area in the main storage device 902 in accordance with the program. The CPU 901 also allocates a storage area in the auxiliary storage device 903 for storing data being processed in accordance with the program.

A program for implementing all or part of the functions of the reading device 10 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing by each functional unit. The term "computer system" as used here includes hardware such as the OS and peripheral devices. Furthermore, if a WWW system is used, the term "computer system" also includes the homepage provision environment (or display environment). Furthermore, "computer-readable recording medium" refers to portable media such as CDs, DVDs, and USBs, as well as storage devices such as hard disks built into the computer system. Furthermore, if the program is distributed to the computer 900 via a communication line, the computer 900 that receives the program may load the program into the main storage device 902 and execute the above-mentioned processing. Furthermore, the program may be for implementing part of the above-mentioned functions, or may be capable of implementing the above-mentioned functions in combination with a program already stored in the computer system.

As described above, several embodiments of the present disclosure have been described. However, all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments may be embodied in a variety of other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope of the invention and its equivalents as set forth in the claims, as well as within the scope and spirit of the invention.

<Additional Notes>
The reading method, reading device, reading system, and program described in each embodiment can be understood, for example, as follows.

(1) The reading method according to the first aspect includes the steps of: acquiring a photographed image 50 of an indicator 1 (an analog measuring instrument or a switch) that indicates an indication value at the position of an indication part 2 (S1); moving the position of the indication part in a pseudo image 60 that simulates the indicator 1 (S4); identifying the amount of movement of the indication part in the pseudo image when the position of the indication part in the pseudo image is closest to the position of the indication part in the photographed image (ideally, when they completely overlap) (S5); and estimating the indication value indicated by the device based on the identified amount of movement (S6).
This allows the indicated value of the indicator to be read with high accuracy and with a small processing load.

(2) A reading method according to a second aspect is the reading method of (1), further comprising the step of removing disturbance from the captured image.
By eliminating the disturbance, the indicated value can be read with greater accuracy.

(3) A reading method according to a third aspect is a reading method according to (1) to (2), in which in the step of identifying the amount of movement of the indication part, a large weighting coefficient is assigned to the characteristics of the indication part in the captured image, and the amount of movement when the position of the indication part in the pseudo image is closest to the position of the indication part in the captured image is calculated.
This means that even if, for example, the captured image is unclear or contains noise that could not be completely removed by noise removal, so that the shapes of the pseudo needle in the pseudo image and the indicator in the captured image do not match and the signal strength of the overlap is unreliable, by increasing the weighting of characteristic parts and calculating the signal strength, it is possible to accurately detect the position where the pseudo needle in the pseudo image and the indicator in the captured image are closest to each other and read the indication value with high accuracy.

(4) A reading method according to a fourth aspect is the reading method of (1) to (3), in which the unit movement amount is changeable.
The unit movement amount (resolution) can be changed through settings, and by setting a fine unit movement amount (resolution), reading accuracy can be improved simply by changing the unit movement amount in image processing, without the need for the time and effort required to prepare and learn a large number of images of an indicator photographed using a fine resolution.

(5) The reading method according to the fifth aspect is a reading method according to any one of (1) to (4), in which the starting position of the movement is set to a predetermined reference value (for example, the starting position of the movement when it is known that the measured value falls within the range of 45 to 50°C is 45°C, 47°C, 50°C, etc.).
When the approximate value of the instruction value is known, the range of movement can be limited by using the start position of the movement as a reference value, thereby reducing processing costs.

(6) A reading method according to a sixth aspect is the reading method according to any one of (1) to (5), in which any one of the start position, direction, and range of movement can be set.
By setting any one of the start position, direction and range of movement according to the characteristics of the instruction value, it is possible to improve the efficiency of the processing.

(7) A reading method according to a seventh aspect is a reading method according to any one of (1) to (6), wherein the indicator is a pointer, the indicator is a measuring instrument (such as the measuring instrument illustrated in Figures 2(a) to 2(c)) that indicates a measurement value based on the position of the pointer, and the estimating step estimates the measurement value based on the identified amount of movement of the pointer.
This allows measurement values that can only be confirmed by going to the site where the measuring instrument is installed to be confirmed remotely by capturing photographic images.

(8) A reading method according to an eighth aspect is a reading method according to any one of (1) to (7), wherein the indicator is a switch, the indicator unit is an operating unit of the switch, and the estimating step estimates the setting value of the switch based on the identified amount of movement of the operating unit.
This allows the setting value of a switch, which can only be confirmed by visiting the site where the switch is installed, to be confirmed remotely by acquiring a photographed image.

(9) A reading device 10 according to a ninth aspect includes a means for acquiring a captured image of an indicator that indicates an indication value at the position of the indicator, a means for moving the position of the indicator in a pseudo image that simulates the indicator, a means for identifying the amount of movement of the indicator in the pseudo image when the position of the indicator in the pseudo image is closest to the position of the indicator in the captured image, and a means for estimating the indication value indicated by the indicator based on the identified amount of movement.

(10) A reading system (indication value acquisition system 100) according to the tenth aspect includes an indicator that indicates an indication value at the position of the indicator, a camera that photographs the indicator, and a reading device according to (9) that further includes a means for transmitting the indication value.
This allows the indicated value to be read and transmitted to another device without contact. Because the indicated value can be read without contact, for example, it is possible to obtain the indicated value remotely by simply installing the camera 5 and the reading device 10 (if the reading device 10 is installed at a location away from the camera 5, only the camera 5 will suffice), without having to perform any construction work on the plant or the like.

(11) A program according to an eleventh aspect causes a computer to execute the steps of: acquiring a captured image of an indicator that indicates an indication value at the position of the indicator; moving the position of the indicator in a pseudo image that simulates the indicator; identifying the amount of movement of the indicator in the pseudo image when the position of the indicator in the pseudo image is closest to the position of the indicator in the captured image; and estimating the indication value indicated by the indicator based on the identified amount of movement.

1... indicator 5... camera 10... reading device 11... image acquisition unit 12... image processing unit 13... disturbance removal unit 14... indication value estimation unit 15... deep learning unit 16... setting reception unit 17... memory unit 18... communication unit 20... terminal device 20a... PC (personal computer)
20b...Mobile terminal 30...Another system 100...Indication value acquisition system 900...Computer 901...CPU
902: Main storage device 903: Auxiliary storage device 904: Input/output interface 905: Communication interface

Claims (10)

acquiring a captured image of an indicator indicating an indication value at the position of the indication unit;
moving a position of the indicator in a pseudo image that simulates the indicator;
specifying a movement amount of the indicator in the pseudo image when the position of the indicator in the pseudo image is closest to the position of the indicator in the captured image;
estimating an indication value indicated by the indicator based on the identified movement amount;
and
The indicator is a switch, and the indicator unit is an operation unit of the switch,
In the estimating step, a setting value of the switch is estimated based on the identified movement amount of the operation unit.
Reading method. further comprising a step of removing disturbance from the captured image.
The reading method according to claim 1 . In the step of specifying the movement amount of the indication part, a large weighting coefficient is assigned to a feature of the indication part in the photographed image, and the movement amount when the position of the indication part in the pseudo image is closest to the position of the indication part in the photographed image is calculated.
The reading method according to claim 1 or 2. With respect to the movement, the unit movement amount is changeable.
The reading method according to claim 1 or 2. setting the start position of the movement to a predetermined reference value;
The reading method according to claim 1 or 2. Any of the start position, direction, and range of movement can be set.
The reading method according to claim 1 or 2. The indicator is a measuring instrument that indicates a measurement value by the position of the indicator,
In the estimating step, the measurement value is estimated based on the identified movement amount of the indicator needle.
The reading method according to claim 1 or 2. a means for acquiring a photographed image of an indicator that indicates an indication value at the position of the indicator;
a means for moving the position of the indicator in a pseudo image that simulates the indicator;
a means for specifying a movement amount of the indicator in the pseudo image when the position of the indicator in the pseudo image comes closest to the position of the indicator in the captured image;
means for estimating an indication value indicated by the indicator based on the identified amount of movement;
Equipped with
The indicator is a switch, and the indicator unit is an operation unit of the switch,
the estimating means estimates the setting value of the switch based on the identified movement amount of the operation unit.
Reading device. An indicator that indicates an indication value at the position of an indication part; a camera that photographs the indicator;
The reading device according to claim 8 , further comprising means for transmitting the indicated value;
A reading system comprising: On the computer,
acquiring a captured image of an indicator indicating an indication value at the position of the indication unit;
moving a position of the indicator in a pseudo image that simulates the indicator;
specifying a movement amount of the indicator in the pseudo image when the position of the indicator in the pseudo image is closest to the position of the indicator in the captured image;
estimating an indication value indicated by the indicator based on the identified movement amount;
and
The indicator is a switch, and the indicator unit is an operation unit of the switch,
In the estimating step, a program is executed to execute a process of estimating a setting value of the switch based on the identified movement amount of the operation unit .