JP7387346B2 - Image processing device, its control method, program - Google Patents

Description

The present invention relates to a technique for reading numerical values displayed on a display device.

Various techniques are known for recognizing numbers displayed on a display device that expresses letters and numbers using multiple segments. In particular, in order to digitize measurements of old equipment that has been used for a long time, characters and numbers displayed in segments are imaged, and the characters and numbers contained in the captured image are recognized using recognition technology. That is what is being considered. As a technique for performing such recognition, Patent Document 1 discloses an image processing device that extracts a target area from horizontal lines and/or vertical lines and cuts out a number display area to include the target area. .

Japanese Patent Application Publication No. 2017-010170

However, with the technology described in Patent Document 1, when reading a numerical value from a multi-segment number display with a decimal point, it is read for each rectangle that includes one character of the multi-segment characters, so even if the decimal point is included, the decimal point is not read. There were times when it could not be detected correctly.

An image processing device according to the present invention is an image processing device that detects a numerical value indicated by a combination of a plurality of segments lit on a display, and includes an acquisition means for acquiring an image of the display, and an acquisition means for acquiring an image of the display. a specifying means for specifying a detection range in the obtained image; a dividing means for dividing the detection range into regions for each digit; a numerical value detection means for detecting a numerical value from a lighting state for each region for each digit; A decimal point detection means for detecting a decimal point from a lighting state in an area spanning a plurality of the areas for each digit within the range, and a position of the decimal point detected by the decimal point detection means and a position of the area for each digit. determining means for determining the position of the decimal point in the entire numerical value based on the relationship .

According to the present invention, display detection accuracy can be improved.

Overall configuration diagram of a system according to an embodiment of the present invention An example of a user interface of an image processing device according to an embodiment of the present invention An example of a user interface of an image processing device according to an embodiment of the present invention Flowchart diagram illustrating the process of reading a numeric value from a multi-segment numeric display with decimal point according to an embodiment of the present invention. A diagram showing one digit of 7 segments according to an embodiment of the present invention Conceptual diagram of a table showing detection results of lighting and turning off of segments and corresponding output values according to an embodiment of the present invention A diagram showing a decimal point reading range according to an embodiment of the present invention A flowchart diagram showing a process of calculating the position of the decimal point and the number of digits after the decimal point according to an embodiment of the present invention. A flowchart diagram showing a process of reading numerical values from a multi-segment numerical display during operation in an image processing apparatus according to an embodiment of the present invention.

<First embodiment>
In this embodiment, an image processing apparatus that reads numerical values from a multi-segment numerical display with a decimal point will be described. There are various types of multi-segment display devices, but in this embodiment, a display device that displays numerical values with seven segments lit is used as a reading target.

In this embodiment, it is assumed that the image to be subjected to image processing is an uncompressed or decompressed grayscale 8-bit depth image. It is assumed that the origin of the coordinates on the image exists at the upper left, the X coordinate increases as it goes to the right, and the Y coordinate increases as it goes downward. The angle is assumed to be 0 degrees when a straight line is drawn from the center of the image in the positive direction of the X coordinate with the same Y coordinate value, and the angle when a straight line is drawn in the negative direction of the Y coordinate with the same X coordinate value. is 90 degrees. The Y coordinate is the same value, and the angle of a straight line drawn in the negative direction of the X coordinate is 180 degrees. Assume that the X coordinates have the same value and the angle of a straight line drawn in the positive direction of the Y coordinate is 270 degrees. Hereinafter, the display of numerical values in the lighting state of seven segments will be referred to as 7 segments. Further, in the image processing apparatus according to an embodiment of the present invention, there are two modes, a setting mode and an operation mode, in order to realize numerical reading from a 7-segment display. In the setting mode, various settings are made to read numerical values on the 7-segment display. In the driving mode, images are taken at predetermined intervals, and numerical values are read from the image data taken on the 7-segment display based on the information set in the setting mode.

[System configuration]
FIG. 1 is an overall configuration diagram including an image processing apparatus 100 according to an embodiment of the present invention.

In FIG. 1, 100 is a computer that operates as an image processing device. By executing a predetermined control program in a computer, the computer performs the image processing described below and functions as an image processing device.

A CPU 101 controls the entire image processing apparatus 100, and performs display control, recording control, input/output control, various image processing, etc. based on signals from various parts and various programs. Note that the CPU does not necessarily need to be composed of one piece of hardware, and a plurality of pieces of hardware such as a GPU may work together to function as one unit. Further, one piece of hardware may serve as a plurality of means by controlling different processes.

Reference numeral 102 denotes a ROM that stores operational processing procedures for the CPU 101 (for example, programs for computer startup processing, basic input/output processing, etc.).

103 is a RAM, which functions as the main memory of the CPU 101. Various programs including control programs for realizing the processes described below are loaded into the RAM 103 from the hard disk drive 105 and the like, and are executed by the CPU 101. Further, the RAM 103 provides a work area when the CPU 101 executes various processes.

A display 104 performs various displays under the control of the CPU 101.

A hard disk drive (hereinafter referred to as HDD) 105 is used for storing and reading application programs, data, libraries, and the like.

106 is an input device such as a pointing device or a keyboard.

A storage medium mounting unit (media drive) 107 allows a removable storage medium to be mounted and data captured and stored with a digital still camera to be read out.

A network interface 108 is connected to a computer network 111 via a communication line 110. This network interface 108 transmits and receives data from communicable devices. In this embodiment, the CPU 101 connects to the imaging device 112 via this network interface, issues a shooting instruction to the imaging device 112, acquires a shot image from the imaging device 112, and records it on the HDD 105.

Reference numeral 109 denotes a system bus (consisting of an address bus, a data bus, and a control bus) that connects the units described above.

112 is a digital camera that operates as an imaging device. It is connected to the computer network 111 via a communication circuit 113.

114 is a segment display device to be measured. In this embodiment, it is assumed that the display device displays physical quantities such as temperature, pressure, rotation speed, and distance as 7-segment numbers. The digital camera 112 periodically photographs the display 114 and generates grayscale image data. Then, it is transferred to the image processing apparatus 100 via the computer network 111.

In the image processing apparatus of this embodiment, it is assumed that image data to be processed is stored in the HDD 105.

2 and 3 are examples of user interfaces of an image processing apparatus according to an embodiment of the present invention. Both FIGS. 2 and 3 are user interfaces in the setting mode. A setting method and display method for a user to read meter values will be explained using FIGS. 2 and 3. Assume that the state after setting the detection area in FIG. 2 is as shown in FIG. 3. Components that perform the same operations in FIGS. 2 and 3 will be described with reference numbers.

201 and 301 are image display areas. The CPU 101 reads the image of the 7-segment display to be read into the RAM 103 and displays it in response to the user's operation. It also displays lines and detected edges superimposed on the image to assist the user in setting operations.

202 and 302 are shooting buttons. In response to the user's press, the CPU 101 instructs the digital camera 112 to take a photograph via the network interface 108, acquires a photographed image, and stores it in the HDD 105. Then, the CPU 101 reads the captured image from the HDD 105 into the RAM 103 and displays the photographed image in the image display area 201.

203 and 303 are file read buttons. If the user presses the button, the CPU 101 determines that the user has performed a file designation operation, and displays a general file open dialog. Then, the file specified by the user in the file open dialog is read from the HDD 105 into the RAM 103 and displayed in the image display area 201.

204 and 304 are detection area designation buttons. In response to the user's press, the CPU 101 determines that the user has performed an operation to start specifying a detection area, and in response to the operation using the pointing device 106 in the image display area 201, a rectangle for specifying the detection area (detection range) is created. Draw. Figure 3 shows the user interface after specifying the detection area. The area surrounded by a rectangle 320 in FIG. 3 is the detection area.

205 and 305 are character reading angle boxes. After the user specifies the detection area, the angle of the long side of the rectangle is displayed as an initial value. This is an example in which the angle of the arrow 326 in FIG. 3 indicating the reading direction is specified as 0 degrees. When the user changes the numerical value, the CPU 101 assumes that the user has given a change instruction and updates the drawing by changing the reading direction arrow 326 and the angle of the long side of the detection area 320 according to the input value. At this time, it is assumed that the detection area 320 rotates about the center of the rectangle.

206 and 306 are brightness threshold boxes. This is a box for specifying a brightness threshold when performing binarization for number detection (recognition) in the detection area 320. The initial value is 128, and values in the range 0 to 255 are accepted. When the user changes the numerical value, the CPU 101 assumes that the user has given an instruction to change the brightness threshold and changes the threshold for binarization.

207 and 307 are 7-segment digit boxes. The initial value is 4, and an integer value of 1 or more is accepted. Then, the side of the detection area 320 in the direction of the character reading angle box 205 is divided into equal parts, and the divided area is used as a detection area for a one-digit number. When the user changes the numerical value, the CPU 101 assumes that the user has given an instruction to change the number of digits, changes the number of divisions of the detection area 320, and updates the display. FIG. 3 shows the user interface after 8 is specified in the 7-seg digit number box 207. In FIG. 3, the detection area 320 is divided into six equal parts along the sides in the reading direction as indicated by dotted lines 321, 322, 323, 324, and 325.

208 and 308 are segment width threshold boxes. The initial value is 3, and an integer value between 1 and 200 is accepted. Then, when recognizing a number as a numerical value, it is determined whether or not the segment to be detected is in a lighting state based on whether or not a specified number of consecutive pixels are equal to or greater than a binarization threshold. A method of reading numbers and symbols from the lighting state of the 7-segment and replacing them with numerical values will be described later.

209 and 309 are text color list boxes. Lists dark text on a light background and light text on a dark background. In the detection area on the image, it is switched whether the brightness value greater than or equal to the value in the brightness threshold box 206 is to be recognized as a number, or the brightness value less than the value in the brightness threshold box 206 is to be recognized as a number. When the user selects dark characters on a bright background, numbers less than the value in the brightness threshold box 206 are targeted for recognition. When the user selects bright characters on a dark background, numbers equal to or higher than the value in the brightness threshold box 206 are targeted for recognition. In the detection area 320, pixels with a luminance value of 1 after binarization, which are targets for number recognition, are colored and superimposed on the grayscale image and displayed in the luminance threshold box 206 and the character color list box 209. This state is shown at 327 in FIG.

210 and 310 are decimal point processing setting check boxes. Specify whether or not to perform processing related to the decimal point on the output value. The initial value is OFF, and when it is OFF, members 211 to 215 to 311 to 315 related to the decimal point, which will be described later, are disabled and do not accept user operations. When turned on by a user's operation, the CPU 101 determines that the output value may include a decimal point, enables the members 211 to 215 to 311 to 315, and accepts the user's operation. In the case of OFF, the CPU 101 assumes that there is no decimal point in the output value and outputs only integer values.

211 and 311 are radio buttons for selecting the number of digits below the decimal point. The initial value is OFF. When the user turns it on, the CPU 101 determines that there are digits below the decimal point, and outputs the output value as having the number of digits below the decimal point equal to the number specified in the number of digits below the decimal point box 213, which will be described later. Add a decimal point. Then, the decimal point detection radio button 212 or 312, which will be described later, is turned off.

212 and 312 are decimal point detection radio buttons. The initial value is ON. If ON, the CPU 101 determines that it is necessary to perform decimal point detection processing, and performs decimal point detection processing, which will be described later.

213 and 313 are decimal places boxes. The initial value is 1, and an integer value of 1 to 7 segment digit number box 207 or 307 or less is accepted. When 6 is specified in the 7-segment digit box 207 or 307 and the number of digits after the decimal point is 2, the output value is calculated assuming that the decimal point is always between the second and third digit. do. If the 6-digit value is detected as 168753, it is set as 1687.53. If the 6-digit value is 168750, the 6-digit value is 1687.5. If the upper two digits of the six-digit value are not detected and 8753 is detected from the third digit, it is set as 87.53.

214 and 314 are minimum size boxes for decimal points, which are examples of decimal point size designation means. The initial value is 40, and an integer value greater than or equal to 1 and less than the maximum size is accepted.

215 and 315 are the maximum size boxes for the decimal point. The initial value is 300, and an integer value greater than the minimum size and less than or equal to 300 is accepted. Then, the decimal point is detected in the numerical value detection process described later, assuming that the size of the decimal point on the image is equal to the number of pixels in the range between the minimum size and the maximum size.

216 and 316 are setting screen exit buttons. If the user presses the button, the CPU 101 determines that the user has given an instruction to end the settings, and ends the series of settings processing. At the same time, it is assumed that the setting mode is switched to the operation mode.

217 and 317 are notification threshold boxes. The initial value is 1000, and when a numerical value is input by the user, the CPU 101 determines that the user has specified a threshold value to be notified, and stores the value in the RAM 103.

328 is a result output area. The detected results are displayed in the image display area 301 according to various settings on the setting screen. Numerical value detection processing, which will be described later, is executed according to the settings of the various setting members 305 to 315, and if the detection is successful, the numerical value of the detection result is displayed.

[Numeric value detection processing]
FIG. 4 is a flowchart diagram illustrating the process of reading a numeric value from a multi-segment numeric display with decimal point according to one embodiment of the present invention. Note that the process in FIG. 4 is executed by the CPU 101 executing a control program stored in the RAM 103. After reading the image and specifying the detection area 320 in the setting mode, setting of initial values and each time the user changes settings using various members are performed. It is also assumed that this is executed when switching to the driving mode. Common members in FIGS. 2 and 3 will be explained using the numbers assigned to FIG. 3.

In step S400, CPU 101 reads grayscale image data within detection area 320 into RAM 103.

In step S401, the CPU 101 reads into the RAM 103 various settings on the user interface described in FIGS. 2 and 3.

In step S402, the CPU 101 performs binarization processing on the image data held in the RAM 103 according to the value of the brightness threshold box 306 and the setting of the character color list box 309, and generates a numerical value detection target image. Then, it is held in the RAM 103.

In step S403, the CPU 101 assigns 1 to a counter i for executing a process of reading a number and a minus sign for each digit.

In step S404, the CPU 101 determines whether the counter i is less than or equal to the number of digits. If it is determined that it is small, the process advances to step S407. If it is determined that the counter i is larger than the number of digits, it is assumed that the numeric minus sign reading process for each number of digits has been completed, and the process advances to step S410.

In step S405, the CPU 101 divides the numerical value detection target image binarized in step S403 by the number of digits in the reading direction. The width of the image data for numerical detection is imgW, the height is imgH, the reading direction is the positive direction of the x-axis, the number of digits is NumDigits, and the upper left coordinates (sepX0[i], sepY0[ i]) is shown in Equation 1. The lower right coordinates (sepX1[i], sepY1[i]) are shown in Equation 2. The center coordinates (sepCenterX[i], sepCenterY[i]) are shown in Equation 3.
sepX0[i]=(imgW÷NumDigits)×(i-1)
sepY0[i]=0…. Formula 1
sepX1[i]=(imgW÷NumDigits)×(i)
sepY1[i]=imgH…. Formula 2
sepCenterX[i] = (sepX0[i]+sepX1[i])÷2
sepCenterY[i] = (sepY0[i]+sepY1[i])÷2... Formula 3

A counter i is attached to the image divided into each digit and the image is stored in the RAM 103. At this time, the numerical value detection target image is not discarded but continues to be held in the RAM 103.

In step S406, the CPU 101 stores in the RAM 103 the center coordinates (sepCenterX[i], sepCenterY[i]) of the divided area for each digit in the numerical value detection target image.

In step S407, the CPU 101 reads a number and a minus sign from the i-th digit divided image data. The numeric minus sign reading process will be described later using FIGS. 5 and 6.

In step S420, the CPU 101 determines whether the reading was successful. If the reading is successful, the process advances to step S408. If unsuccessful, the process advances to step S419.

In step S408, the CPU 101 assigns the detection result to the i-th array that holds the results for each digit, and stores it in the RAM 103.

In step S409, the CPU 101 increments the counter i, and returns to step S404.

In step S410, the CPU 101 determines whether the decimal point setting is turned on. If the decimal point processing setting check box 310 is ON, the CPU 101 determines that the decimal point setting is ON, and proceeds to step S411. If the decimal point processing setting check box 310 is OFF, the CPU 101 determines that the decimal point setting is not ON, and proceeds to step S421.

In step S411, the CPU 101 determines whether or not to perform decimal point detection. If the decimal point detection radio button 312 is ON, the CPU 101 determines that the decimal point is to be detected, and proceeds to step S412. If the decimal point detection radio button 312 is OFF, the CPU 101 determines that decimal point detection is not to be performed, and proceeds to step S416.

In step S412, the CPU 101 performs decimal point detection without dividing the numerical value detection target image binarized in step S402 into digits. The decimal point detection process will be described later using FIG. The output values of the decimal point detection process are the number of detected decimal points and the XY coordinates of the center of the decimal point in the detection area 320.

In step S413, the CPU 101 determines whether a plurality of decimal points have been detected. If the number of decimal points detected is 2 or more, the CPU 101 determines that decimal point detection has not been executed normally and proceeds to step S419.

In step S414, the CPU 101 determines whether the number of decimal points detected is one. If the decimal point detected is 1, the process advances to step S415. If the decimal point detected is 0, the CPU 101 determines that no decimal point has been detected, and proceeds to step S417.

In step S415, the CPU 101 calculates the number of digits below the decimal point. The calculation method will be described later using FIG.

In step S416, the CPU 101 reads the value in the number of digits below the decimal point box 313 as the number of digits below the decimal point.

In step S421, the CPU 101 sets the number of digits below the decimal point to zero.

In step S417, the CPU 101 determines whether the result of arranging the numerical value of each digit and the minus sign detected in step S408 constitutes a numerical value. If a minus sign is detected, there must be only one at the beginning. Even if there is a NULL digit (described later) in FIG. 6, it must be continuous only at the beginning or end of the digit. subject to the following conditions. If the value holds true as a numerical value, the process advances to step S418. If the value does not hold true, the process advances to step S419.

In step S418, the CPU 101 converts it into a numerical value. Regarding the NULL digit, we detected the value multiplied by 10 as 0, 1 for the first lower digit, 10 for the second lower digit, 100 for the third lower digit, etc. Multiply by the number of digits and add up. Then divide according to the number of digits after the decimal point. If the number of digits after the decimal point is 1, divide by 10. If the number of digits after the decimal point is 2, divide by 10, and so on. If the number of digits after the decimal point is 0, no division is performed and the value remains as an integer.

In step S419, the CPU 101 assumes that the reading of the numerical value has failed and sets the result as NG. Then, the series of reading processing ends.

In step S419, the CPU 101 determines that the reading of the numerical value has been successful, sets the numerical value in step S418 as an output value, and accepts the result as OK. Then, the series of reading processing ends.

[Number minus sign reading process]
FIG. 5 is a diagram showing one digit of 7 segments according to an embodiment of the present invention. In this embodiment, the image data to be detected is a binarized image divided into digits stored in the RAM 103 in step S405 of FIG. Since the image data has already been binarized, it is assumed that the luminance value of the pixels constituting the image data is either 0 or 1.

FIG. 6 is a conceptual diagram of a table showing output values corresponding to detected lighting statuses and extinguishing statuses of segments, according to an embodiment of the present invention. It is assumed that FIG. 6 is held in the RAM 103.

A method of detecting lighting and extinction of each segment and a method of obtaining an output value from the result will be explained using FIGS. 5 and 6. Common members in FIGS. 2 and 3 will be explained using the numbers assigned to FIG. 3.

In FIG. 5, 501, 502, 503, 504, 505, 506, and 507 are segments forming 7 segments. Thereafter, segments 501 to 507 are assigned segNo. 1~segNo. 7.

508 is a point that divides the upper side of one-digit image data into two equal parts.

Points 509 and 510 are points that divide the right side of one-digit image data into thirds. The upper side is 509 and the lower side is 510.

511 is a point that divides the lower side of one-digit image data into two equal parts.

Points 512 and 513 are points that divide the right side of one-digit image data into thirds. The upper side is 513 and the lower side is 512.

514 is the intersection of the straight line connecting points 508 and 511 and the straight line connecting points 513 and 509.

515 is the intersection of the straight line connecting the points 508 and 511 and the straight line connecting the points 512 and 510.

segNo. When determining whether 1 is on or off, the CPU 101 sequentially reads the pixel values from point 509 to point 514 and determines whether the value of 1 consecutively appears more than the number set in the segment width threshold box 308. Determine whether or not to do so. If it is determined that the segment width threshold box 308 has more than the number of segments set in the segment width threshold box 308, the CPU 101 selects segNo. Assuming that 1 is lit, segNo. 1 and stored in the RAM 103. If the number is less than the number set in the segment width threshold box 308, segNo. Assuming that segNo. 1 is off, segNo. 1 and stored in the RAM 103.

Similarly, segNo. 2, from point 514 to point 515, segNo. 3, from point 510 to point 515, segNo. 4, from point 511 to point 515, segNo. 5, from point 512 to point 515, segNo. 6, from point 513 to point 514, segNo. Regarding 7, the pixel values are sequentially read from point 508 to point 514, and the above method is used to determine whether or not it is lit, and the result is set to segNo. It is stored in the RAM 103 in association with.

601 is a management number string for uniquely identifying table data. 602 is a diagram showing a lighting state.

603 to 609 are columns indicating lighting conditions for each segNo. 603 is segNo. 1, 604 is segNo. 2, 605 is segNo. 3, 606 is segNo. 4, 607 is segNo. 5, 608 is segNo. 6, 609 is segNo. Corresponds to 7.

610 is a column of output values.

Lines 611 to 625 indicate the lighting conditions of each segment and the corresponding output values.

segNo. 1 to segNo. If all rows up to 7 are ON, 622 is the matching row and its output value is 8.

If only segNo. 2 is ON and all other segments are OFF, 624 is the matching row, and its output value is - (minus).

In step S407 of FIG. 4, the CPU 101 stores the lighting results of each segment in the RAM 103 using the detection method of FIG. Then, the table of FIG. 6 is used to search for a row of lighting conditions that match the lighting result. If there is a matching row, the value of the output value column 610 of that row is held in the RAM 103 as the result of reading the numeric minus sign. If it is not found, it is held in the RAM 103 as a reading failure.

[Decimal point detection processing]
FIG. 7 is a diagram showing a decimal point reading range according to an embodiment of the present invention. The method for detecting the decimal point in step S412 in FIG. 4 will be explained using FIG. 7. In this embodiment, the image data to be detected is the binarized image of the detection area 320 held in the RAM 103 in step S402 of FIG. Since the image data has already been binarized, it is assumed that the luminance value of the pixels constituting the image data is either 0 or 1.

The double-lined rectangle formed by the vertices 700, 701, 702, and 703 is the decimal point reading area. In this embodiment, the detection area 320 is assumed to be the lower one-third area.

In step S412, the CPU 101 reads pixel values in the decimal point reading area from the 700 and 701 directions in the reading direction. Then, a search is made for a location where the number of adjacent pixels specified by the minimum decimal point size box 314 and the maximum decimal point size box 315 are consecutively 1, and where the shape approximates a circle. Whether the shape is approximated to a circle is determined based on whether the ratio of the width to the height of a portion that is continuously 1 is within the range of 1:0.8 to 1.2. If a corresponding location is found as a result of the search, the x and y coordinates of the central point are held in the RAM 103. Then, as a result of searching up to 702 and 703, if there are a plurality of corresponding locations, all the x and y coordinates corresponding to the number of locations are held in the RAM 103. Then, the results are used in the processing from step S413 onwards.

[Decimal point position and number of digits below the decimal point calculation process]
FIG. 8 is a flowchart showing the process of calculating the position of the decimal point and the number of digits to the right of the decimal point according to an embodiment of the present invention.

In this embodiment, the explanation will be given assuming that the arrow 326 in the reading direction coincides with the positive direction of the x coordinate.

In step S801, the CPU 101 assigns 1 to a counter i for searching the position of the digit where the decimal point is placed.

In step S802, the CPU 101 assigns the center coordinate x of the decimal point to pX.

In step S803, the CPU 101 determines whether the value of pX is smaller than the center coordinates of the i-th divided region. If it is determined that the number is small, the CPU 101 assumes that a decimal point is inserted between the i-th digit and the i-1 digit, and proceeds to step S807.

In step S804, the CPU 101 determines whether the counter i has a value less than the number of digits. If the counter i is less than the number of digits, the process advances to step S805. If the counter i is greater than or equal to the number of digits, the CPU 101 determines that the decimal point is placed to the right of the last digit, and proceeds to step S806.

In step S805, the CPU 101 determines whether pX falls between the center coordinates of the i-th digit divided area and the center coordinates of the i+1-th digit divided area. If it is determined that it is entered, the process advances to step S806. If it is determined that it does not enter, the process advances to step S808.

In step S806, the CPU 101 substitutes i for n, which holds the position of the decimal point. The position of the decimal point indicates between n and n+1 digits.

In step S807, the CPU 101 assigns 0 to n. In this case, the decimal point position will be to the left of the first digit, that is, before the most significant digit.

In step S808, the CPU 101 increments the counter i. Then, the process returns to step S804.

In step S809, the CPU 101 calculates the number of digits below the decimal point. Assuming that the number of digits specified in the 7-segment number of digits box 307 is NumDigits, the number of digits below the decimal point NumDeci is shown in Equation 4.
NumDeci = NumDigits - n…. Formula 4

[How to read numerical values during operation]
FIG. 9 is a flowchart showing a process of reading numerical values from a multi-segment numerical display during operation in the image processing apparatus according to an embodiment of the present invention. Note that the process in FIG. 9 is executed by the CPU 101 executing a control program stored in the RAM 103. 2 and 3 illustrate the user interface in the setting mode of the image processing apparatus of this embodiment. On the other hand, the process in FIG. 9 is a process in the driving mode. Periodically photograph the multi-segment numeric display to be measured from the same position as when setting, and read the value from the display each time. Then, the read value is recorded in the HDD 105 at any time, and the user is notified if the value is equal to or higher than a threshold value specified by the user.

In step S901, the CPU 101 determines whether the user has performed a reading start operation. It is assumed that the reading start operation is a press of the setting screen end button 316.

In step S902, the CPU 101 issues a shooting instruction to the digital camera 112 connected to the computer network 111 via the communication line 110.

In step S903, the CPU 101 acquires image data taken by the digital camera 112 via the communication line 110, and stores it in the HDD 105.

In step S904, the CPU 101 reads the numerical value and stores the result in the RAM 103. The value reading method is the method explained in FIGS. 4 to 8.

In step S905, the CPU 101 determines whether reading is successful. If the reading is successful, the process advances to step S906. If reading fails, the process advances to step S910.

In step S906, the CPU 101 determines whether the notification threshold value is equal to or greater than the value specified in the notification threshold box 317. If the value is greater than or equal to the value specified in the notification threshold box 317, it is determined that the display indicates a value that requires notification, and the process advances to step S907. If it is less than the value specified in the notification threshold box 317, it is determined that notification is unnecessary, and the process advances to step S909.

In step S907, the CPU 101 notifies the user. The notification is the display of a warning dialog on the display 104. The warning dialog displays the message "A value outside the threshold has been detected. Do you want to continue reading? Do you want to end?" along with a continue button and an end button. Further, the reading result is displayed in the result output area 328, superimposed on the image to be measured under the warning dialog.

In step S908, the CPU 101 determines whether there is a user operation. User operation is defined as pressing the continue button or end button in the warning dialog. If the user has operated a button, the process advances to step S909.

In step S909, the CPU 101 determines whether there is a termination operation from the user. The end operation is pressing the end button in the warning dialog in step S907 or step S910. When the end button is pressed, the CPU 101 determines that the user has performed an end operation, ends the series of processes for reading the display, and shifts to the setting mode. If it is determined that there is no end operation, the process returns to step S902.

In step S910, the CPU 101 notifies the user. The notification is the display of a warning dialog on the display 104. The warning dialog box shows that the value could not be read correctly. Do you want to continue reading? Do you want to quit? The message "Continue button" and "End button" are displayed.

In step S911, the CPU 101 determines whether there is a user operation. User operation is defined as pressing the continue button or end button in the warning dialog. If the user has operated a button, the process advances to step S909.

In addition, in the image processing device of this embodiment, when numerical reading fails in the operation mode, measurement is interrupted and a warning is displayed, but the imaging instruction is issued again without displaying the warning and measurement is continued. You can. This is because depending on the application, it is better not to interrupt measurement and recording of values.

In the image processing apparatus of this embodiment, the decimal point is detected in the lower third of the detection area 320, but it may be detected in the entire detection area, or the decimal point can be displayed in the lower quarter to If it falls within one-fifth of the total, it may be within that range. The smaller the detection area of the decimal point, the shorter the time required for detection and the less likely it is to pick up noise. Furthermore, it is possible to prevent a comma, which is a digit separator sometimes drawn at the top, from being mistakenly detected as a decimal point.

Although the image processing apparatus of this embodiment has been described using a 7-segment display as a processing target, the present invention is applicable even if the number of segments is different. In this case, the table shown in FIGS. 5 and 6 that associates the method of detecting lighting and unlighting of each segment with the output value may correspond to the position and number of segments to be detected.

Furthermore, in the image processing apparatus of this embodiment, a 7-segment display with no tilted vertical segments has been described as a processing target. It is possible to read numerical values by tilting the screen. Furthermore, since the position of the digit where the decimal point is placed is determined by comparing the center coordinates of each digit with the center coordinates of the position where the decimal point is detected, the position is appropriate regardless of the inclination.

In the image processing device of this embodiment, whether the approximation to a circle is determined by determining whether the width and height ratio of consecutive 1 points on the image is in the range of 1:0.8 to 1.2. Judgment was made based on whether it was included or not. On the other hand, if the shape of the decimal point on the display is a circle, the decimal point may be detected by searching in the shape of a circle in the decimal point detection area. If the shape of the decimal point is a comma, the decimal point may be detected by searching the decimal point detection area in the shape of a comma.

In the image processing device of this embodiment, the detection area 320 is determined by the coordinates specified by the user, but it is also possible to detect a mark for specifying the location and change the detection area based on the relative position from the mark. You can. By doing so, even if the relative positions of the display and the imaging device are shifted, it can be corrected as long as the display and the mark are on the subject image.

In the image processing apparatus of this embodiment, the validity of the numerical value is verified by the number of decimal points, and the validity of the numerical value by the relative positional relationship between the number and the minus sign is verified in different steps. On the other hand, the validity may be verified by arranging all the read numbers, minus signs, and decimal points. If there is an existing function that converts characters into numerical values, a number, a minus sign, and a decimal point may be input as input, and the determination can be made based on whether or not a numerical value conversion error occurs. Processing may be simplified or speeded up.

As described above, according to the image processing apparatus of this embodiment, it is determined whether or not a numerical value is valid according to the number of detected decimal points, so that false detection can be prevented.

As explained above, according to the image processing device of the present embodiment, the size range of the decimal point on the image is specified and the decimal point is detected only when the size condition is met, thereby preventing false detection due to noise. be able to.

As described above, according to the image processing apparatus of this embodiment, it is determined whether or not the numerical value is valid based on the relative positional relationship between the read numerical value and the minus sign, so that false detection can be prevented.

As explained above, according to the image processing apparatus of this embodiment, when the decimal point is detected at the end of a digit, the value is treated as an integer value, and when the decimal point is detected at the beginning of the digit, the value is treated as a value less than 1. can be output.

As described above, according to the image processing apparatus of this embodiment, when a decimal point is not detected, the reading result can be output as an integer value.

As described above, according to the image processing apparatus of this embodiment, since the width of the segment is specified in the segment width threshold box 308, it is possible to detect both thick and thin segments according to the user's operation.

As explained above, according to the image processing apparatus of the present embodiment, even if the upper digit is not lit, the lower digit is 0, or the upper digit is given a minus sign, an appropriate value is displayed. It becomes possible to read.

As described above, according to the image processing apparatus of this embodiment, numerical values and minus signs are read in areas divided into digits, and decimal points are detected in areas that are not divided into digits. Therefore, even if the decimal point straddles both divided regions of adjacent digits, duplicate detections and missed detections can be prevented.

As described above, according to the image processing device of the present embodiment, when reading a numerical value from a multi-segment number display with a decimal point, it is possible to read the decimal point and reduce misrecognition.

Although the present invention has been described above in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and the present invention may take various forms without departing from the gist of the present invention. included. Some of the embodiments described above may be combined as appropriate.

Further, the present invention can also be realized by executing the following processing. That is, the software (program) that realizes the functions of the embodiments described above is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

Claims (9)

An image processing device that detects a numerical value indicated by a combination of multiple segments lit on a display,
acquisition means for acquiring an image of the display;
identification means for identifying a detection range in the image obtained by the acquisition means;
dividing means for dividing the detection range into areas for each digit;
Numerical value detection means for detecting a numerical value from a lighting state for each area of each digit;
decimal point detection means for detecting a decimal point from a lighting state in an area spanning a plurality of digit-based areas within the detection range;
determining means for determining the position of the decimal point in the entire numerical value based on the position of the decimal point detected by the decimal point detecting means and the positional relationship between the area for each digit;
An image processing device having: The image according to claim 1, further comprising designation means for designating whether or not to detect a decimal point, and when it is designated to detect the decimal point, the decimal point is detected using the decimal point detection means. Processing equipment. 3. The method according to claim 1, further comprising means for determining the correctness of the numerical value based on the relative positional relationship of the detection results for each region of each digit by the numerical value detection means. The image processing device described. 4. The image processing apparatus according to claim 1, further comprising means for determining that detection has failed when a plurality of decimal points are detected as a result of the decimal point detection means. The image processing apparatus according to any one of claims 1 to 4 , wherein it is determined whether the numerical value displayed on the display device is an integer value according to the number of decimal points detected by the decimal point detection means. . further comprising decimal point size designation means for designating the size of the decimal point on the image;
6. The image processing apparatus according to claim 1, wherein the decimal point detection means controls detection of the decimal point according to the size specified by the decimal point size specification means. 7. The area to be detected by the decimal point detecting means is a lower area of the screen within the detection range specified by the specifying means. Image processing device. A method for controlling an image processing device that detects a numerical value indicated by a combination of a plurality of segments lit on a display, the method comprising:
an acquisition step of acquiring an image of the display;
an identification step of identifying a detection range in the image obtained in the acquisition step;
a dividing step of dividing the detection range into areas for each digit;
a numerical value detection step of detecting a numerical value from a lighting state for each area of each digit;
A decimal point detection step of detecting a decimal point from a lighting state in a region spanning a plurality of digit regions within the detection range ;
a determining step of determining the position of the decimal point in the entire numerical value based on the position of the decimal point detected in the decimal point detection step and the positional relationship with the area for each digit;
A control method having A program executable by a computer that causes the computer to function as the image processing apparatus according to any one of claims 1 to 7 .

Images