JP6524721B2 - Recognition device, recognition system, recognition program and recognition method - Google Patents

Description

  The present invention relates to a recognition device, a recognition system, a recognition program and a recognition method.

  There is a technology for recognizing characters and the like from an image captured by an imaging device. For example, as a car number recognition system, it is performed to automatically read characters from a plate (a number plate) of a vehicle traveling on a road.

  When handling a captured image of a fast moving object such as a vehicle, a decrease in recognition accuracy due to motion blur caused by the movement of the exposure point on the imaging device within the exposure time tends to be a problem. The exposure time is determined by the shutter speed when the shutter is provided in the imaging apparatus that outputs a moving image, and is determined by the time of one frame when the shutter is not provided. In an imaging apparatus that outputs a still image, the exposure time is determined by the shutter speed.

  FIG. 1 is a view showing an example of motion blur, and (a), (b) and (c) show cases where the exposure time is short, and (d), (e) and (f) show cases where the exposure time is long. In each of FIGS. 1A and 1D, each lattice (cell) indicates a pixel of the image pickup device, and a gray line 71 indicates the movement of an image of one point of the object on the image pickup device. It is movement within the exposure time. In the case of (a) where the exposure time is short, as shown in (b), one pixel is detected as a clear difference in exposure. However, when the exposure time is long (d), as shown in (e), the exposure amount is detected over a plurality of pixels in a form extending in the moving direction (the color density corresponds to the exposure amount) A blur occurs. When the entire vehicle is imaged, when the exposure time is short, the vehicle is clearly imaged as shown in (c), but when the exposure time is long, the vehicle is generally along the movement direction as shown in (f) And the image is blurred.

  In recognition of characters and the like included in the captured image, the captured image is collated with a dictionary image prepared in advance, and is recognized as a specific character from the matching rate. FIG. 2 shows an example of matching with a dictionary image, where (a) shows the case where there is no motion blur and (b) shows the case where there is motion blur. In addition, a captured image is converted into the image seen from the front, when it is the distorted image imaged from the diagonal direction, before collating with a dictionary image.

  In the case of (a) where there is no motion blur, it is determined that the character “A” in the dictionary image clearly matches and “B” does not clearly match in recognizing the character enclosed by the white frame of the captured image. If there is motion blur (b), the motion blur will increase the difference from the dictionary image, so even if it matches “A” in the dictionary image, it may not be possible to determine if it does not match “B”. Incurs a decrease in accuracy.

JP, 2009-71821, A

  In order to prevent the occurrence of the motion blur described above, it is effective to shorten the exposure time at the time of imaging. However, since the exposure amount is reduced by shortening the exposure time, there is a problem that the exposure amount is insufficient particularly at the time of imaging at night. In order to prevent an insufficient amount of exposure, it is conceivable to increase the sensitivity of the imaging device or to increase the light amount of the illumination device. However, there is a cost limit to improving the sensitivity of the imaging device, and it is difficult to increase the light quantity of the lighting device because the number of lighting devices and the power consumption increase.

  An object of the present invention is, in one aspect, to suppress a decrease in recognition accuracy of characters and the like even in a state where motion blur occurs in a captured image.

  In one aspect, the recognition device calculates the moving direction on the image of the recognition target from a plurality of images obtained by capturing the recognition target at different imaging times, and generating the motion blur generating the movement direction on the image An estimation unit for estimating a direction, a generation unit for generating a dictionary image in which a blur of a predetermined size with respect to the generation direction is added to the dictionary image, any of the plurality of images and the dictionary image generated by the generation unit And a recognition unit that recognizes the recognition target.

  As one aspect, even in a state where motion blur occurs in a captured image, it is possible to suppress a decrease in recognition accuracy of characters and the like.

It is a figure which shows the example of a motion blur. It is a figure which shows the example of collation with a dictionary image. It is a figure showing an example of composition of a system concerning one embodiment. It is a figure which shows the structural example of a recognition apparatus. It is a figure which shows the other structural example of a recognition apparatus. It is a figure showing the example of hardware constitutions of a recognition device. It is a flowchart (the 1) which shows the process example of embodiment. It is a figure which shows the example of the captured image temporarily held by vehicle detection. It is a figure which shows the example of extraction of a plate candidate. It is a figure which shows the example of image conversion. It is a figure which shows the example of recognition of a serial number. It is a figure which shows the example of the movement amount and motion blur amount of a vehicle. It is a figure which shows the example of affine transformation with respect to movement amount or motion blur amount. It is a figure which shows the example of estimation of the amount of motion blur by the back projection to real space. It is a figure (the 1) showing an example of generation of a motion blur filter. It is a figure which shows the example of xy plane and a straight line. It is a figure which shows the example of calculation of the length which each grating | lattice and a line segment overlap. It is a figure (the 2) showing an example of generation of a motion blur filter. It is a figure which shows the example of xy plane and a straight line. FIG. 16 is a diagram (part 3) illustrating an example of generation of a motion blur filter. It is a figure which shows the example of xy plane and a straight line. It is a figure which shows the example of a motion blur filter. It is a figure which shows the example of a production | generation of a temporary dictionary. It is a figure which shows the example of collation with the temporary dictionary image which added the motion blur. It is a flowchart (the 2) which shows the process example of embodiment.

  Hereinafter, preferred embodiments of the present invention will be described.

<Configuration>
FIG. 3 is a diagram showing an example of the configuration of a system according to an embodiment. In FIG. 3, the lighting device 4 and the imaging device 5 are disposed at predetermined positions with the vehicle 2 and the plate 3 as a field of view with respect to the road 1 on which the vehicle 2 travels. After being input, the recognition device 6 is configured to output the recognition result through predetermined processing. The illumination device 4 is, for example, a light emitting diode (LED) illumination device, and the imaging device 5 is, for example, a moving image digital camera. The recognition device 6 is, for example, an information processing device such as a personal computer.

  FIG. 4 is a view showing a configuration example of the recognition device 6. In FIG. 4, the recognition device 6 includes a vehicle detection unit 601, a plate detection unit 603, a plate candidate extraction unit 604, an image conversion unit 605, a serial number recognition unit 607, and a plate determination unit 609 which constitute the plate detection unit 603. Is equipped. The recognition device 6 further includes a motion blur amount estimation unit 610, a motion blur filter generation unit 611, a temporary dictionary generation unit 612, a small letter recognition unit 614, and a recognition result output unit 615.

  The vehicle detection unit 601 has a function of inputting a captured image of a moving image (image in which frames of a still image are continuous) from the imaging device 5. In addition, the vehicle detection unit 601 has a function of temporarily holding at least two still images accompanied by imaging time information as the imaged image 602 when the imaged image includes an object determined to be a vehicle from the appearance. ing. Note that at least two still images accompanied by imaging time information are still images of continuous frames or still images of frames picked up every few frames. Moreover, the vehicle detection part 601 has a function which outputs the vehicle detection result to the effect of having detected the vehicle. The imaging device 5 may or may not include the shutter. In addition, vehicle detection is intended to efficiently carry out subsequent processing such as plate detection etc. If it is not necessary to consider the processing efficiency so much, plate detection etc directly without passing through vehicle detection from the captured image etc. You can also do

  The plate detection unit 603 has a function of detecting the plate of the vehicle (specifying the plate position) based on the captured image 602 when the vehicle detection unit 601 detects the vehicle based on the vehicle detection result.

  The plate candidate extraction unit 604 extracts an image portion of an outline of a predetermined shape (for example, a rectangle or a parallelogram) having a possibility of being a plate as a candidate from the captured image 602, and coordinates positions on the image (for example, The coordinate position of the central part of the lower part of the plate is output as a plate candidate position.

  The image conversion unit 605 has a function of cutting out a plate candidate image from the captured image 602 based on the plate candidate position input from the plate candidate extraction unit 604. Further, the image conversion unit 605 has a function of converting into a front image by affine conversion or the like based on the inclination of the contour, aspect ratio, etc. and temporarily holding it as a plate candidate image 606. Further, the image conversion unit 605 has a function of outputting an image conversion result indicating that a plate candidate image has been generated. Generally, since the image is taken from the oblique direction of the vehicle, the plate portion is distorted in a parallelogram shape, but by converting it into a front image, matching with the dictionary image created based on the front image Is possible.

  When the image conversion result is input from the image conversion unit 605, the serial number recognition unit 607 recognizes from the plate candidate image 606 the serial number, which is a large number at the center of the plate, using the dictionary 608 for serial numbers. It has a function of outputting a recognized serial number or a serial number recognition result indicating that it could not be recognized. In this configuration example, processing of the dictionary image is not performed on the sequence number which is a large number, on the assumption that the influence of motion blur is small.

  The plate determination unit 609 determines whether the plate is a plate according to the format (for example, a 4-digit number string etc.) of the plate based on the serial number recognition result input from the serial number recognition unit 607, and indicates that It has a function of outputting the plate determination result shown.

  The motion blur amount estimation unit 610 estimates the direction and size of occurrence of motion blur present on the captured image after conversion to the front image, when it indicates that the plate determination result is determined to be a plate. It has a function to output as a blur parameter. The occurrence direction and the magnitude of motion blur collectively refer to both the occurrence direction and the magnitude of motion blur, also referred to as “motion blur amount” or simply “blur amount”. In this estimation, the plate candidate position output by the plate candidate extraction unit 604 (which is “plate position” when it is determined to be a plate) and parameters used for image conversion of the image conversion unit 605 (matrix value of affine transformation Etc.). If the magnitude of motion blur can be given as a predetermined value from the representative velocity of the target vehicle, only the direction of motion blur occurrence may be estimated.

  The motion blur filter generation unit 611 has a function of generating and outputting a motion blur filter based on a convolution matrix or the like based on the motion blur parameter input from the motion blur amount estimation unit 610.

  The temporary dictionary generation unit 612 has a function of generating and outputting a temporary dictionary by applying the motion blur filter input from the motion blur filter generation unit 611 to the dictionary image of the dictionary 613 for lower case (convolution operation etc.) doing.

  The lower case recognition unit 614 uses the temporary dictionary generated by the temporary dictionary generation unit 612 to recognize lower case letters from lower case parts excluding the serial number part from the plate candidate image 606, and recognized lower case letters or could not be recognized It has a function to output a lower case recognition result indicating that.

  The recognition result output unit 615 has a function of outputting the recognition result of the entire plate including the serial number recognition result of the serial number recognition unit 607 and the small letter recognition result of the small letter recognition unit 614.

  FIG. 5 is a view showing another configuration example of the recognition device 6. In the recognition device 6 shown in FIG. 4, processing of the dictionary image is not performed on the sequence number having a large size, since the influence of motion blur is small, but in the configuration example of FIG. In preparation for the case where the serial number can not be recognized because of the influence, the dictionary image is also processed when recognizing the serial number. As the implementation, either the configuration of FIG. 4 or FIG. 5 may be used, or the configuration of both may be included and may be selected by the user, or automatically switched according to the frequency of plate detection failure or the like. You may do so.

  5 differs from the configuration of FIG. 4 in that the motion blur amount estimation unit 610 and the motion blur filter generation unit 611 in FIG. 4 are moved immediately after the image conversion unit 605 and the temporary dictionary is This is a point at which the generation unit 612-1 is provided. The temporary dictionary generation unit 612 in FIG. 4 is a temporary dictionary generation unit 612-2 to distinguish it from the newly provided temporary dictionary generation unit 612-1.

  In FIG. 5, the motion blur amount estimation unit 610 and the motion blur filter generation unit 611 have the same functions as those in FIG. 4 except that the processing timing is different. The temporary dictionary generation unit 612-1 generates and outputs a temporary dictionary by applying the motion blur filter input from the motion blur filter generation unit 611 to the dictionary image of the dictionary 608 for serial numbers (convolution operation etc.) It has a function. The serial number recognition unit 607 recognizes the serial number from the plate candidate image 606 using the temporary dictionary generated by the temporary dictionary generation unit 612-1, and recognizes the serial number recognized or the serial number recognition indicating that it could not be recognized. It has a function to output the result.

  In FIG. 5, the plate is detected by the recognition of the serial number, and the small letters are recognized after the plate is detected. However, the detection of the plate can be omitted and the serial number and the small letters can be sequentially or simultaneously recognized. .

  FIG. 6 is a diagram showing an example of the hardware configuration of the recognition device 6. In FIG. 6, the recognition device 6 includes a central processing unit (CPU) 6002, a read only memory (ROM) 6003, a random access memory (RAM) 6004, and a non-volatile random access memory (NVRAM) 6005 connected to the system bus 6001. Is equipped. The recognition device 6 also includes an I / F (Input / Output Device) 6007, an HDD (Hard Disk Drive) 6008, and an NIC (Network Interface Card) 6009 connected to the I / F (Interface) 6006 and the I / F 6006. And a monitor 6010 connected to the I / O 6007, a keyboard 6011, a mouse 6012 and the like. A CD / DVD (Compact Disk / Digital Versatile Disk) drive or the like can also be connected to the I / O 6007. An OS (Operating System) operates on the illustrated hardware, and the units shown in FIGS. 4 and 5 operate based on a computer program or the like.

<Operation>
FIG. 7 is a flowchart showing an example of the process of the above embodiment, which is an example of the process of the recognition device 6 shown in FIG.

  In FIG. 7, when the vehicle detection unit 601 receives a captured image from the imaging device 5 (step S101), the vehicle detection unit 601 determines whether the vehicle is detected based on whether the captured image includes an object that is determined to be a vehicle from the appearance. (Step S102). When it is determined that the vehicle is not detected (No in step S102), the process returns to the input of the captured image (step S101). When it is determined that a vehicle is detected (Yes in step S102), at least two still images (still images of continuous frames or still images of frames picked up every several frames) accompanied by imaging time information are set as the imaged image 602 It temporarily stores (step S103). Then, the vehicle detection result indicating that the vehicle has been detected (the steering wheel may be provided to the captured image temporarily stored together) is output (step S104), and the process returns to the input of the captured image (step S101). FIG. 8 is a view showing an example of a captured image 602 temporarily held by vehicle detection, where (a) shows a captured image at time T, and (b) shows a captured image at time T + dt.

  Referring back to FIG. 7, the plate detection unit 603 determines whether or not the vehicle is detected based on the vehicle detection result of the vehicle detection unit 601 (step S105), and when not detecting the vehicle (No in step S105), stands by . When a vehicle is detected (Yes in step S105), the plate candidate extraction unit 604 extracts an image portion of an outline of a predetermined shape (for example, a rectangle or parallelogram) which may be a plate from the captured image 602. It is extracted as a candidate, and the coordinate position on the image (for example, the coordinate position of the central part of the lower part of the plate) is output as a plate candidate position (step S106). FIG. 9 is a view showing an example of extraction of a plate candidate, and for each of the captured image at time T in (a) and the captured image at time T + dt in (b), the coordinate value x on the image at the lower center of the plate y is extracted as a plate candidate position.

  Next, referring back to FIG. 7, the image conversion unit 605 cuts out a plate candidate image from the captured image 602 based on the plate candidate position output from the plate candidate extraction unit 604. Further, the image conversion unit 605 converts the image into a front image by affine conversion or the like based on the inclination of the contour, aspect ratio, etc. and temporarily holds the image as a plate candidate image 606, and indicates that the plate candidate image is generated. The result is output (step S107). FIG. 10 is a diagram showing an example of image conversion, and shows a state in which the image of the distorted plate included in the captured image 602 of (a) is converted into a rectangular image seen from the front of (b).

  Next, referring back to FIG. 7, when the image conversion result is input from the image conversion unit 605, the sequence number recognition unit 607 uses the dictionary 608 for the sequence number from the plate candidate image 606 and uses the large number in the plate central size. A certain serial number is recognized, and a recognized serial number or a serial number recognition result indicating that it could not be recognized is output (step S108). FIG. 11 is a diagram showing an example of recognition of the serial number, and shows a state in which the serial number "2345" is recognized.

  Next, referring back to FIG. 7, based on the serial number recognition result of the serial number recognition unit 607, the plate judgment unit 609 judges whether the plate is a plate according to the format of the serial number in the plate (for example, 4-digit numerical string etc.) If the plate is not a plate (No in step S109), the process returns to the vehicle detection standby (step S105).

  If it is determined to be a plate (Yes in step S109), the motion blur amount estimation unit 610 estimates the direction and size of occurrence of motion blur present on the captured image after conversion to the front image. This estimation is performed based on the plate candidate position (plate position) output from the plate candidate extraction unit 604 and the parameter (affine transformation matrix value, etc.) used for image conversion of the image conversion unit 605. Also, the motion blur amount estimation unit 610 outputs, as a motion blur parameter, the generation direction and the size of the motion blur which is the estimation result (step S110). If the magnitude of motion blur can be given as a default value from the representative speed of the target vehicle, the motion blur amount estimation unit 610 may estimate only the direction of motion blur occurrence. . If the size of the motion blur is also estimated from the captured image, the recognition accuracy can be further improved.

  FIG. 12 is a diagram showing an example of the amount of movement of the vehicle and the amount of motion blur. FIG. 12 (a) shows the captured image at time T and the captured image at time T + dt overlapped for the sake of explanation, and the plate position at time T (FIG. 9 (a)) and the plate position at time T + dt (FIG. 9) From the difference from (b), the amount of movement dx in the x direction on the image at time dt and the amount of movement dy in the y direction are calculated. The amount of movement includes both the movement direction as a whole and the movement distance.

Then, if the shutter value (for example, if the shutter speed is "1/1000", the shutter value is "1000"; the shutter value is the reciprocal of the exposure time) is s, as shown in FIG. The magnitude of blur in the x direction is calculated by dx / (s · dt), and the magnitude of blur in the y direction is calculated (estimated) by dy / (s · dt). The blur generation direction can be calculated (estimated) from the arc tangent (tan ⁻¹ ) with respect to the ratio of the blur size in the x direction to the blur size in the y direction.

  Note that the above-mentioned blur amount is a value on the image before image conversion, so it is converted to a value on the front image at the time of matching with the dictionary image. FIG. 13 is a diagram showing an example of affine transformation with respect to movement amount or motion blur amount, which is used for image transformation in the image transformation unit 605 to blur amount vector on the image before image transformation shown in (a) By applying an affine transformation using the same parameters, it is possible to obtain a blur amount vector (p, q) on the front image as shown in (b). The components p and q of the blur amount vector are output as motion blur parameters.

  In the above description, the movement amount is obtained from the position of the object (plate) on the captured image, but in order to further improve the accuracy, back projection to the real space can be used. In FIG. 14, when the movement amount is calculated from the image of the plate at position A and the image of the plate at position B, since the positions A and B are different in distance from the imaging device, they are not exact movement amounts. . Therefore, the physical distance and angle between the imaging device and each position in the field of view are measured in advance, the positions A and B are back-projected to the real space, and the movement amount is calculated in the real space. By projecting onto the image space, it is possible to calculate a more accurate amount of movement. The process after calculating the movement amount is the same as that described above.

  Next, referring back to FIG. 7, the motion blur filter generation unit 611 generates and outputs a motion blur filter such as a convolution matrix based on the motion blur parameters output from the motion blur amount estimation unit 610 (step S111). The motion blur filter generation method will be described below.

  FIG. 15 is a diagram showing an example of generation of a motion blur filter. In FIG. 15, the motion blur filter generation unit 611 inputs a motion blur parameter (x-direction blur amount p, y-direction blur amount q) from the motion blur amount estimation unit 610 (step S211).

  Next, the motion blur filter generation unit 611 calculates the length W corresponding to the size of the motion blur and the angle θ 1 corresponding to the generation direction from the motion blur parameter (step S212). Each equation is as illustrated.

  Next, the motion blur filter generation unit 611 sets a straight line on the virtual xy plane such that the line segment L of length W overlaps the origin with the middle point and the angle θ with the x axis (step S213). An example of a state in which a line segment L is drawn on a virtual xy plane is shown in FIG.

  Next, referring back to FIG. 15, the motion blur filter generation unit 611 sets a grid whose size in the vertical and horizontal directions is 1 around a point whose coordinate value on the xy plane is an integer (step S214). The smallest rectangle surrounded by a broken line in FIG. 16 is a grid. An example of the grid is shown as G.

  Next, referring back to FIG. 15, the motion blur filter generation unit 611 calculates the length w (m, n) where each grid (center (m, n)) and the line segment L overlap (step S215).

Then, the motion blur filter generation unit 611 sets the motion blur filter B (m, n) to B (m, n) = w (m, n) / W.
Are generated and output (step S216).

  FIG. 17 is a diagram showing an example of calculation of the overlapping length of grids and line segments, and m-0.5 to m + 0.5 for x and n- for y for each grid specified by m and n. W (m, n) is calculated by performing numerical calculation using a function δ that has a value range of 0.5 to n + 0.5, and is 1 on the line segment L and 0 otherwise.

  Next, FIG. 18 is a view showing another example of generation of a motion blur filter. In FIG. 18, the motion blur filter generation unit 611 inputs a motion blur parameter (x-direction blur amount p, y-direction blur amount q) from the motion blur amount estimation unit 610 (step S221).

  Next, the motion blur filter generation unit 611 calculates the length W corresponding to the size of the motion blur and the angle θ 1 corresponding to the generation direction from the motion blur parameter (step S222). Each equation is as illustrated.

  Next, the motion blur filter generation unit 611 sets a straight line on the virtual xy plane such that the line segment L of length W overlaps the origin with the middle point and the angle θ with the x axis (step S223). An example of a state in which the line segment L is drawn on the virtual xy plane (except for the white cross and white circle on the line) is shown in FIG.

Next, returning to FIG. 18, the motion blur filter generation unit 611 sets points (including one end point of the line segment L) obtained by dividing the line segment L at equal intervals as l _i (i = 0,..., N). (Step S224). The coordinate value of l _i is ((i · W / N−W / 2) cos θ, (i · W / N−W / 2) sin θ)). In FIG. 19 (a), l _i in the case of seven equal divisions (N = 6) is indicated by a white cross.

Then, returning to FIG. 18, the motion blur filter generation unit 611 sets a value obtained by rounding off the coordinate values of _{l i} the _{(m i, n} i) _(step S225). The coordinate position indicated by the rounded value is indicated by an open circle in FIG. 19 (a). Note that the coordinate values to be rounded off may not be coordinate values of one end point and each divided point, but may be coordinate values of the middle point of each divided line segment.

Next, referring back to FIG. 18, the motion blur filter generation unit 611 counts the number of m = m _i and n = n _i and sets the result to C (m, n) (step S226). FIG. 19 (b) shows an example of C (m, n) (corresponding to FIG. 19 (a)).

  Next, returning to FIG. 18, the motion blur filter generation unit 611 generates a motion blur filter by dividing C (m, n) by the number of divisions (N + 1) and outputs the motion blur filter (step S227).

  Next, FIG. 20 is a diagram showing still another example of generation of a motion blur filter. In FIG. 20, the motion blur filter generation unit 611 inputs a motion blur parameter (x-direction blur amount p, y-direction blur amount q) from the motion blur amount estimation unit 610 (step S231).

  Next, the motion blur filter generation unit 611 calculates the length W corresponding to the size of the motion blur and the angle θ 1 corresponding to the generation direction from the motion blur parameter (step S232). Each equation is as illustrated.

  Next, the motion blur filter generation unit 611 sets a straight line having a length W and a width H at a virtual line on the virtual xy plane, the middle point overlapping the origin, and the inclination with the x axis being the angle θ (Step S233). The width H is preset to be slightly larger than the diagonal length of the 1 × 1 size grid. An example of a state in which a line segment L is drawn on a virtual xy plane is shown in FIG.

  Next, referring back to FIG. 20, the motion blur filter generation unit 611 sets a grid whose size in the vertical and horizontal directions is 1 around a point whose coordinate value on the xy plane is an integer (step S234).

  Next, the motion blur filter generation unit 611 calculates an area S (m, n) where each grid (center (m, n)) and the line segment L overlap (step S235).

Then, the motion blur filter generation unit 611 sets the motion blur filter B (m, n) to B (m, n) = S (m, n) / WH.
Are generated and output (step S236).

  FIG. 22 is a diagram showing an example of a motion blur filter generated with W = 7 and θ = 45 °, and elements on the diagonal from top right to bottom left are 1/7, and other elements are 0 . The length W has a diagonal length of the basic grid of 1 for convenience of explanation.

  Next, referring back to FIG. 7, the temporary dictionary generation unit 612 generates a temporary dictionary by applying the motion blur filter input from the motion blur filter generation unit 611 to the dictionary image of the dictionary 613 for lower case (convolution operation etc.) And output (step S112). FIG. 23 is a diagram showing an example of generation of a temporary dictionary. A state in which a temporary dictionary image is generated by applying a motion blur filter to the dictionary image of the original image is shown.

  Next, referring back to FIG. 7, the lower case recognition unit 614 recognizes and recognizes lower case letters from lower case parts excluding the serial number part from the plate candidate image 606 using the temporary dictionary generated by the temporary dictionary generation unit 612. And a lower case recognition result indicating that the upper case has not been recognized (step S113). FIG. 24 is a diagram showing an example of matching with a temporary dictionary image to which motion blurring has been added, and in recognizing characters surrounded by white frames on a captured image (plate image) by matching with a temporary dictionary, both are the same Since blurring has occurred, it is determined that the temporary dictionary image “A” is clearly matched and “B” is not clearly matched.

  Next, referring back to FIG. 7, the recognition result output unit 615 outputs a recognition result including the serial number recognition result of the serial number recognition unit 607 and the small letter recognition result of the small letter recognition unit 614 (step S114). It returns to (step S105).

  Next, FIG. 25 is a flow chart showing an example of processing in the recognition device 6 having the configuration shown in FIG.

  In FIG. 25, when the vehicle detection unit 601 receives a captured image from the imaging device 5 (step S301), the vehicle detection unit 601 determines whether the vehicle is detected based on whether the captured image includes an object that is determined to be a vehicle. (Step S302). When it is determined that the vehicle is not detected (No in step S302), the process returns to the input of the captured image (step S301). When it is determined that a vehicle is detected (Yes in step S302), at least two still images (still images of continuous frames or still images of frames picked up every few frames) accompanied by imaging time information are taken as the imaged image 602 It temporarily stores (step S303). Then, the vehicle detection result indicating that the vehicle has been detected (the steering wheel may be provided to the captured image temporarily stored together) is output (step S304), and the process returns to the input of the captured image (step S301).

  On the other hand, the plate detection unit 603 determines whether or not the vehicle is detected based on the vehicle detection result of the vehicle detection unit 601 (step S305), and when the vehicle is not detected (No in step S305), stands by. When a vehicle is detected (Yes in step S305), the plate candidate extraction unit 604 extracts an image portion of an outline of a predetermined shape (for example, a rectangle or parallelogram) which may be a plate from the captured image 602. It is extracted as a candidate, and the coordinate position on the image (for example, the coordinate position of the central portion of the lower part of the plate) is output as a plate candidate position (step S306).

  Next, the image conversion unit 605 cuts out a plate candidate image from the captured image 602 based on the plate candidate position output from the plate candidate extraction unit 604. Further, the image conversion unit 605 converts the image into a front image by affine conversion or the like based on the inclination of the contour, aspect ratio, etc. and temporarily holds the image as a plate candidate image 606, and indicates that the plate candidate image is generated. The result is output (step S307).

  Next, the motion blur amount estimation unit 610 estimates the occurrence direction and the size of the motion blur present on the captured image after conversion to the front image. This estimation is performed based on the plate candidate position (plate position) output from the plate candidate extraction unit 604 and the parameter (affine transformation matrix value, etc.) used for image conversion of the image conversion unit 605. In addition, the motion blur amount estimation unit 610 outputs, as a motion blur parameter, the generation direction and the size of the motion blur which is the estimation result (step S308). If the magnitude of motion blur can be given as a default value from the representative speed of the target vehicle, the motion blur amount estimation unit 610 may estimate only the direction of motion blur occurrence. .

  Next, the motion blur filter generation unit 611 generates and outputs a motion blur filter based on a convolution matrix or the like based on the motion blur parameter output from the motion blur amount estimation unit 610 (step S309).

  Next, the temporary dictionary generation unit 612-1 generates a temporary dictionary by applying the motion blur filter output from the motion blur filter generation unit 611 to the dictionary image of the dictionary 608 for serial numbers (convolution operation etc.) It outputs (step S310).

  Next, the serial number recognition unit 607 recognizes the serial number from the plate candidate image 606 using the temporary dictionary generated by the temporary dictionary generation unit 612-1, and displays a recognized serial number or a series indicating that the recognition could not be made. The number recognition result is output (step S311).

  Next, based on the serial number recognition result of the serial number recognition unit 607, the plate judgment unit 609 judges whether the plate is a plate according to the format of the serial number (for example, a 4-digit numeral string etc.) on the plate. The plate determination result shown is output, and when it is not a plate (No in step S312), the process returns to the vehicle detection standby (step S305).

  When it is determined to be a plate (Yes in step S312), the temporary dictionary generation unit 612-2 applies the motion blur filter input from the motion blur filter generation unit 611 to the dictionary image of the dictionary 613 for small letters (convolution operation Etc.) to generate and output a temporary dictionary (step S313).

  Next, the lower case recognition unit 614 recognizes a lower case character from the lower case part excluding the serial number part from the plate candidate image 606 using the temporary dictionary generated by the temporary dictionary generation unit 612-2, and recognizes the lower case The lower case recognition result indicating that the recognition could not be made is output (step S314).

  Then, the recognition result output unit 615 outputs a recognition result including the serial number recognition result of the serial number recognition unit 607 and the small letter recognition result of the small letter recognition unit 614 (step S315), and waits for vehicle detection (step S305). Return.

<Summary>
As described above, according to the present embodiment, when characters and the like are recognized from an image captured by an imaging device, it is possible to suppress a decrease in recognition accuracy of characters and the like even in a state in which motion blur occurs. . In the method of applying the inverse filter of the motion blur filter to the captured image, the size of the inverse filter becomes large, and the data processing load becomes high.

  Hereinabove, the preferred embodiments have been described. Although specific embodiments have been shown and described herein, it is apparent that various modifications and changes can be made to these embodiments without departing from the broad spirit and scope as defined in the claims. . That is, it should not be construed as being limited by the details of the specific example and the attached drawings.

  The motion blur amount estimation unit 610 is an example of an estimation unit. The temporary dictionary generation unit 612 is an example of a generation unit. The small letter recognition unit 614 is an example of a recognition unit.

The following sections will be further disclosed regarding the above description.
(Supplementary Note 1) Estimated by calculating the moving direction on the image of the recognition target from a plurality of images obtained by imaging the recognition target at different imaging times, and estimating the moving direction as the occurrence direction of motion blur occurring on the image Department,
A generation unit configured to generate a dictionary image obtained by adding a blur of a predetermined size to the generation direction to the dictionary image;
A recognition unit that collates any one of the plurality of images with the dictionary image generated by the generation unit, and recognizes the recognition target;
A recognition device characterized by comprising.
(Supplementary Note 2) In the recognition device described in Supplementary Note 1,
The estimation unit calculates a movement direction and a movement amount on the image to be recognized from the plurality of images, and estimates a generation direction and a size of motion blur using an exposure time of imaging.
The generation unit generates a dictionary image obtained by adding a blur of the magnitude of the motion blur with respect to the generation direction to the dictionary image.
A recognition device characterized by
(Supplementary Note 3) In the recognition device according to any one of supplementary notes 1 or 2,
The generation unit generates a convolution matrix corresponding to the generation direction and the size, and generates a dictionary image by performing a convolution operation using the convolution matrix on the dictionary image.
A recognition device characterized by
(Supplementary Note 4) In the recognition device described in Supplementary Note 3,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and the line segment within a lattice centered on a coordinate value which is an integer The convolution matrix is generated by setting the overlap length divided by the length of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition device characterized by
(Supplementary Note 5) In the recognition device described in Supplementary Note 3,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and divides the line segment at equal intervals and one end point of the line segment Integer coordinate value of the center of the lattice corresponding to the center of the corresponding lattice, rounding off the coordinate value of or the coordinate value of the midpoint of each dividing line segment dividing the line segment at equal intervals, and rounding off the number of points of the rounded off coordinate value The value obtained by counting each value and dividing the line segment by the total value of the number of points obtained by dividing the line segment at equal intervals and the number of one end points of the line segment, or the number obtained by dividing the line segment at equal intervals The convolution matrix is generated by setting the matrix position corresponding to each lattice coordinate value.
A recognition device characterized by
(Supplementary Note 6) In the recognition device described in Supplementary Note 3,
The generation unit sets a line segment of a predetermined width at an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and in a grid centered on coordinate values that are integers The convolution matrix is generated by setting the value obtained by dividing the area overlapping the line segment by the area of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition device characterized by
(Supplementary Note 7) In the recognition device according to any one of supplementary notes 1 to 6,
The recognition object is a character on a plate of a vehicle,
The plate number of the plate is recognized by a dictionary image of the original image,
Recognize the lower case after being recognized as a plate using the dictionary image generated by the generator
A recognition device characterized by
(Supplementary Note 8) In the recognition device according to any one of supplementary notes 1 to 6,
The recognition object is a character on a plate of a vehicle,
Recognize the serial number and the lower case of the plate with the dictionary image generated by the generator
A recognition device characterized by
(Supplementary Note 9) An imaging device, a lighting device, and a character recognition device are provided.
The character recognition device
An estimation unit that calculates a moving direction on the image of the recognition target from a plurality of images at different times of imaging, which are obtained by imaging the recognition target, and estimating the movement direction as the occurrence direction of motion blur occurring on the image;
A generation unit configured to generate a dictionary image obtained by adding a blur of a predetermined size to the generation direction to the dictionary image;
A recognition system comprising: a recognition unit that collates any one of the plurality of images with a dictionary image generated by the generation unit and performs recognition of the recognition target.
(Supplementary Note 10) In the recognition system described in Supplementary Note 9,
The estimation unit calculates a movement direction and a movement amount on the image to be recognized from the plurality of images, and estimates a generation direction and a size of motion blur using an exposure time of imaging.
The generation unit generates a dictionary image obtained by adding a blur of the magnitude of the motion blur with respect to the generation direction to the dictionary image.
A recognition system characterized by
(Supplementary Note 11) In the recognition system according to any one of supplementary notes 9 or 10,
The generation unit generates a convolution matrix corresponding to the generation direction and the size, and generates a dictionary image by performing a convolution operation using the convolution matrix on the dictionary image.
A recognition system characterized by
(Supplementary Note 12) In the recognition system described in supplementary note 11,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and the line segment within a lattice centered on a coordinate value which is an integer The convolution matrix is generated by setting the overlap length divided by the length of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition system characterized by
(Supplementary Note 13) In the recognition system described in supplementary note 11,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and divides the line segment at equal intervals and one end point of the line segment Integer coordinate value of the center of the lattice corresponding to the center of the corresponding lattice, rounding off the coordinate value of or the coordinate value of the midpoint of each dividing line segment dividing the line segment at equal intervals, and rounding off the number of points of the rounded off coordinate value The value obtained by counting each value and dividing the line segment by the total value of the number of points obtained by dividing the line segment at equal intervals and the number of one end points of the line segment, or the number obtained by dividing the line segment at equal intervals The convolution matrix is generated by setting the matrix position corresponding to each lattice coordinate value.
A recognition system characterized by
(Supplementary Note 14) In the recognition system described in supplementary note 11,
The generation unit sets a line segment of a predetermined width at an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and in a grid centered on coordinate values that are integers The convolution matrix is generated by setting the value obtained by dividing the area overlapping the line segment by the area of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition system characterized by
(Supplementary Note 15) In the recognition system according to any one of supplementary notes 9 to 14,
The recognition object is a character on a plate of a vehicle,
The plate number of the plate is recognized by a dictionary image of the original image,
Recognize the lower case after being recognized as a plate using the dictionary image generated by the generator
A recognition system characterized by
(Supplementary Note 16) In the recognition system according to any one of supplementary notes 9 to 14,
The recognition object is a character on a plate of a vehicle,
Recognize the serial number and the lower case of the plate with the dictionary image generated by the generator
A recognition system characterized by
(Supplementary note 17) A movement direction on the image of the recognition target is calculated from a plurality of images of the recognition target imaged at different imaging times, and the movement direction is estimated to be a generation direction of motion blur occurring on the image
Generating a dictionary image obtained by adding a blur of a predetermined size to the direction of occurrence to the dictionary image;
Collating any of the plurality of images with the generated dictionary image to recognize the recognition target;
A recognition program that causes a computer to execute a process.
(Supplementary Note 18) In the recognition program described in supplementary note 17,
The estimation process calculates the movement direction and movement amount on the image to be recognized from the plurality of images, and estimates the generation direction and size of motion blur using the exposure time of imaging,
The generating process generates a dictionary image in which a blur of the magnitude of the motion blur with respect to the generation direction is added to the dictionary image.
Recognition program.
(Supplementary Note 19) In the recognition program according to any one of supplementary notes 17 or 18,
The generation processing generates a convolution matrix corresponding to the generation direction and the size, and generates a dictionary image by performing a convolution operation using the convolution matrix on the dictionary image.
Recognition program.
(Supplementary Note 20) In the recognition program described in supplementary note 19,
The generation process sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and the line segment within a lattice centered on a coordinate value which is an integer The convolution matrix is generated by setting the value obtained by dividing the overlapping length by the length of the line segment at a matrix position corresponding to the coordinate value of each grid.
Recognition program.
(Supplementary Note 21) In the recognition program described in supplementary note 19,
In the processing to be generated, a line segment of an angle and a length corresponding to the generation direction and the size is set on a virtual xy plane, and a point obtained by dividing the line segment at equal intervals and a piece of the line segment Round off the coordinate value of the end point or the coordinate value of the midpoint of each dividing line segment dividing the line segment at equal intervals, and the number of points of the rounded coordinate value is an integral number of grid coordinates that is the center of the corresponding grid A value obtained by counting for each value and dividing the counted value by the total value of the number of points obtained by dividing the line segment at equal intervals and the number of one end points of the line segment, or by the number obtained by dividing the line segment at equal intervals To generate the convolution matrix by setting the matrix positions to matrix positions corresponding to respective lattice coordinate values,
Recognition program.
(Supplementary Note 22) In the recognition program described in supplementary note 19,
The processing for generating sets a line segment of a predetermined width at an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and in a grid centered on coordinate values that are integers The convolution matrix is generated by setting the value obtained by dividing the area overlapping the line segment by the area of the line segment at a matrix position corresponding to the coordinate value of each lattice.
Recognition program.
(Supplementary Note 23) In the recognition program according to any one of supplementary notes 17 to 22,
The recognition object is a character on a plate of a vehicle,
The plate number of the plate is recognized by a dictionary image of the original image,
Recognize the lower case after being recognized as a plate with the generated dictionary image
Recognition program.
(Supplementary Note 24) In the recognition program according to any one of supplementary notes 17 to 22,
The recognition object is a character on a plate of a vehicle,
Recognize the sequence number and lower case of the plate with the generated dictionary image
Recognition program.
(Supplementary Note 25)
The moving direction on the image of the recognition target is calculated from a plurality of images of the recognition target imaged at different imaging times, and the movement direction is estimated as the occurrence direction of motion blur occurring on the image,
Generating a dictionary image obtained by adding a blur of a predetermined size to the direction of occurrence to the dictionary image;
Collating any of the plurality of images with the generated dictionary image to recognize the recognition target;
A recognition method characterized by
(Supplementary Note 26) In the recognition method described in Supplementary Note 25,
The estimation process calculates the movement direction and movement amount on the image to be recognized from the plurality of images, and estimates the generation direction and size of motion blur using the exposure time of imaging,
The generating process generates a dictionary image in which a blur of the magnitude of the motion blur with respect to the generation direction is added to the dictionary image.
A recognition method characterized by
(Supplementary Note 27) In the recognition method according to any one of supplementary notes 25 or 26,
The generation processing generates a convolution matrix corresponding to the generation direction and the size, and generates a dictionary image by performing a convolution operation using the convolution matrix on the dictionary image.
A recognition method characterized by
(Supplementary Note 28) In the recognition method described in supplementary note 27,
The generation process sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and the line segment within a lattice centered on a coordinate value which is an integer The convolution matrix is generated by setting the value obtained by dividing the overlapping length by the length of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition method characterized by
(Supplementary Note 29) In the recognition method according to supplementary note 27,
In the processing to be generated, a line segment of an angle and a length corresponding to the generation direction and the size is set on a virtual xy plane, and a point obtained by dividing the line segment at equal intervals and a piece of the line segment Round off the coordinate value of the end point or the coordinate value of the midpoint of each dividing line segment dividing the line segment at equal intervals, and the number of points of the rounded coordinate value is an integral number of grid coordinates that is the center of the corresponding grid A value obtained by counting for each value and dividing the counted value by the total value of the number of points obtained by dividing the line segment at equal intervals and the number of one end points of the line segment, or by the number obtained by dividing the line segment at equal intervals To generate the convolution matrix by setting the matrix positions to matrix positions corresponding to respective lattice coordinate values,
A recognition method characterized by
(Supplementary Note 30) In the recognition method according to supplementary note 27,
The processing for generating sets a line segment of a predetermined width at an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and in a grid centered on coordinate values that are integers The convolution matrix is generated by setting the value obtained by dividing the area overlapping the line segment by the area of the line segment at a matrix position corresponding to the coordinate value of each lattice.
A recognition method characterized by
(Supplementary Note 31) In the recognition method according to any one of supplementary notes 25 to 30,
The recognition object is a character on a plate of a vehicle,
The plate number of the plate is recognized by a dictionary image of the original image,
Recognize the lower case after being recognized as a plate with the generated dictionary image
A recognition method characterized by
(Supplementary Note 32) In the recognition method according to any one of supplementary notes 25 to 30,
The recognition object is a character on a plate of a vehicle,
Recognize the sequence number and lower case of the plate with the generated dictionary image
A recognition method characterized by

Reference Signs List 1 road 2 vehicle 3 plate 4 illumination device 5 imaging device 6 recognition device 601 vehicle detection unit 602 captured image 603 plate detection unit 604 plate candidate extraction unit 605 image conversion unit 606 plate candidate image 607 serial number recognition unit 608 dictionary 609 plate determination unit 610 motion blur amount estimation unit 611 motion blur filter generation unit 612, 612-1, 612-2 temporary dictionary generation unit 613 dictionary 614 lower case recognition unit 615 recognition result output unit

Claims (11)

An estimation unit that calculates a moving direction on the image of the recognition target from a plurality of images at different times of imaging, which are obtained by imaging the recognition target, and estimating the movement direction as the occurrence direction of motion blur occurring on the image;
A generation unit configured to generate a dictionary image obtained by adding a blur of a predetermined size to the generation direction to the dictionary image;
A recognition unit that collates any one of the plurality of images with the dictionary image generated by the generation unit, and recognizes the recognition target;
A recognition device characterized by comprising. In the recognition device according to claim 1,
The estimation unit calculates a movement direction and a movement amount on the image to be recognized from the plurality of images, and estimates a generation direction and a size of motion blur using an exposure time of imaging.
The generation unit generates a dictionary image obtained by adding a blur of the magnitude of the motion blur with respect to the generation direction to the dictionary image.
A recognition device characterized by The recognition device according to any one of claims 1 or 2.
The generation unit generates a convolution matrix corresponding to the generation direction and the size, and generates a dictionary image by performing a convolution operation using the convolution matrix on the dictionary image.
A recognition device characterized by In the recognition device according to claim 3,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and the line segment within a lattice centered on a coordinate value which is an integer The convolution matrix is generated by setting the overlap length divided by the length of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition device characterized by In the recognition device according to claim 3,
The generation unit sets a line segment of an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and divides the line segment at equal intervals and one end point of the line segment Integer coordinate value of the center of the lattice corresponding to the center of the corresponding lattice, rounding off the coordinate value of or the coordinate value of the midpoint of each dividing line segment dividing the line segment at equal intervals, and rounding off the number of points of the rounded off coordinate value The value obtained by counting each value and dividing the line segment by the total value of the number of points obtained by dividing the line segment at equal intervals and the number of one end points of the line segment, or the number obtained by dividing the line segment at equal intervals The convolution matrix is generated by setting the matrix position corresponding to each lattice coordinate value.
A recognition device characterized by In the recognition device according to claim 3,
The generation unit sets a line segment of a predetermined width at an angle and a length corresponding to the generation direction and the size on a virtual xy plane, and in a grid centered on coordinate values that are integers The convolution matrix is generated by setting the value obtained by dividing the area overlapping the line segment by the area of the line segment at a matrix position corresponding to the coordinate value of each grid.
A recognition device characterized by The recognition apparatus according to any one of claims 1 to 6.
The recognition object is a character on a plate of a vehicle,
The plate number of the plate is recognized by a dictionary image of the original image,
Recognize the lower case after being recognized as a plate using the dictionary image generated by the generator
A recognition device characterized by The recognition apparatus according to any one of claims 1 to 6.
The recognition object is a character on a plate of a vehicle,
Recognize the serial number and the lower case of the plate with the dictionary image generated by the generator
A recognition device characterized by An imaging device, a lighting device, and a character recognition device;
The character recognition device
An estimation unit that calculates a moving direction on the image of the recognition target from a plurality of images at different times of imaging, which are obtained by imaging the recognition target, and estimating the movement direction as the occurrence direction of motion blur occurring on the image;
A generation unit configured to generate a dictionary image obtained by adding a blur of a predetermined size to the generation direction to the dictionary image;
A recognition system comprising: a recognition unit that collates any one of the plurality of images with a dictionary image generated by the generation unit and performs recognition of the recognition target. The moving direction on the image of the recognition target is calculated from a plurality of images of the recognition target imaged at different imaging times, and the movement direction is estimated as the occurrence direction of motion blur occurring on the image,
Generating a dictionary image obtained by adding a blur of a predetermined size to the direction of occurrence to the dictionary image;
Collating any of the plurality of images with the generated dictionary image to recognize the recognition target;
A recognition program that causes a computer to execute a process. The recognition device
The moving direction on the image of the recognition target is calculated from a plurality of images of the recognition target imaged at different imaging times, and the movement direction is estimated as the occurrence direction of motion blur occurring on the image,
Generating a dictionary image obtained by adding a blur of a predetermined size to the direction of occurrence to the dictionary image;
Collating any of the plurality of images with the generated dictionary image to recognize the recognition target;
A recognition method characterized by