JP4944720B2 - Image processing apparatus and method, and storage medium - Google Patents

Description

  The present invention relates to an image processing apparatus and method for recognizing license plate information displayed on a license plate of a vehicle such as an automobile and presenting a recognition result, and a storage medium.

  Conventionally, Patent Document 1 and Patent Document 2 are methods for presenting a recognition result of a character string on a license plate to a user. The method of Patent Document 1 performs image processing on a captured image of a license plate, performs pattern matching between the image data and character data stored in advance, and the similarity of the pattern matching is equal to or less than a predetermined value. The image is presented to the user for correction. The method of Patent Document 2 recognizes a character string from a license plate image, performs a data search of a customer database using the recognized character string data as a key, and outputs the search result (customer data).

  As a method for selecting one image from a plurality of images including the same license plate, there is a method disclosed in Patent Document 3. In the method of Patent Document 3, a license plate is detected from an input image, an evaluation value of the input image is obtained from the detection result, and an image to be recognized is determined based on an evaluation value representing recognition ease.

JP-A-8-249415 JP 2006-163446 A JP 2005-322146 A JP 2004-334461 A “Modern Camera and Lens Technology, p111-p114, Photo Industry Publishing Co., Ltd.”

  However, the methods described in Patent Literature 1 and Patent Literature 2 do not consider processing when a plurality of images include the same number. In addition, when presenting an image, information around the license plate is not presented. In the method described in Patent Document 3, a large evaluation value is given to a license plate whose horizontal size is large, so a license plate image with low visibility is selected when the image is taken from directly above the license plate. Will be.

  An object of the present invention is to provide an image processing apparatus and method capable of presenting an image more suitable for presentation of a license plate in a form having better visibility when presenting a user with license plate detection results of a plurality of images. It is to provide a storage medium.

  An aspect of the present invention for achieving the above object is to provide an image processing apparatus that inputs a plurality of images including a license plate and displays a representative image with good visibility of characters on the license plate among the images. In the input plurality of images, the license plate area, which is the area where the license plate is represented, is detected, and a means for obtaining a focus point, and the license plate area is taken as a main subject. A unit that selects an estimated image from the plurality of images based on the position of the license plate region and the focus point, and a unit that displays the selected image as the representative image. .

  In the aspect of the image processing apparatus, the display unit may include a unit that corrects the selected image so that the selected image is displayed in a predetermined size at a predetermined position in a face-to-face state.

  In the aspect of the image processing apparatus, the display unit may further include a unit that cuts out an image widened by a predetermined offset from the license plate region with respect to the image corrected by the correcting unit.

  In the aspect of the image processing apparatus, the selecting unit obtains the position of the center of gravity of the license plate area for the plurality of images, calculates the distance between the position and the focus point, and the distance among the images It is also possible to select the one with the shortest.

  In the aspect of the image processing apparatus, the image processing apparatus may further include means for performing character recognition on the detected license plate area, and the display means may further display the recognition result together with the selected image.

  Another aspect of the present invention for achieving the above object is to input a plurality of images including a license plate and display a representative image having good visibility of characters on the license plate among the images. A method of detecting a license plate area, which is an area in which a license plate is represented, for a plurality of input images, and obtaining a focus point; and taking the license plate area as a main subject. Selecting a presumed image from the plurality of images based on the position of the license plate region and the focus point, and displaying the selected image as the representative image. To do.

  Still another aspect of the present invention for achieving the above object is a storage medium in which a program is stored so as to be readable by a computer, and the program read by the computer includes a license plate in the computer. A step of inputting a plurality of images, a step of detecting a license plate region, which is a region in which a license plate is represented, for the plurality of inputted images, and obtaining a focus point; Selecting an image presumed to be captured as a subject from the plurality of images based on the position of the license plate region and the focus point, and displaying the selected image as the representative image. By executing the image processing method, among the plurality of images, the And wherein the displaying a good representative image visibility for the character in the over plate.

  In each aspect of the image processing method and the storage medium, the displaying step includes a step of correcting the selected image so that the selected image is displayed in a predetermined size at a predetermined position in a face-to-face state. It may be.

  In each aspect of the image processing method and the storage medium, the displaying step further includes a step of cutting out an image widened by a predetermined offset from the license plate area for the image corrected in the correcting step. It may be.

  In each aspect of the image processing method and the storage medium, in the selecting step, a position of the center of gravity of the license plate region is obtained for the plurality of images, a distance between the position and the focus point is calculated, and the image Of these, the one with the shortest distance can be selected.

  In each aspect of the image processing method and the storage medium, the display further includes a step of performing character recognition on the detected license plate area after the obtaining step and before the selecting step. In this step, the recognition result can be displayed together with the selected image.

  Each aspect of the image processing method and the storage medium may further include a step of hiding the display by the displaying step.

  According to the present invention of each aspect described above, when presenting a license plate detection result of a plurality of images to a user, an image more suitable for presentation of a license plate can be presented in a form with good visibility. In addition, when there is information around the license plate, it can be presented simultaneously with the license plate.

(First embodiment)
Hereinafter, the present invention relating to an image processing apparatus that classifies a plurality of image groups including license plates and presents them to a user will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus that realizes an embodiment of the present invention.

  The CPU 101 controls the entire apparatus according to a control program stored in the ROM 102. The ROM 102 stores various programs and various parameter data including a control program such as processing shown in a flowchart described later executed by the CPU 101.

  The RAM 103 stores images and various data. The RAM 103 functions as a data work area and a temporary save area. The external storage device 104 stores various data including a database for managing images. The external storage device 104 is configured by, for example, a hard disk drive, a CD-ROM drive, or the like. The display 105 is composed of, for example, an LCD or a CRT. The input device 106 is composed of, for example, a mouse or a pen tablet and receives an instruction from the user.

  A network interface (NIC) 107 communicates with an external device (for example, a server, an external storage device, etc.) connected on a network (not shown), and reads and writes programs and data. The network is typically one of the Internet, LAN, WAN, telephone line, dedicated digital line, ATM, frame relay line, communication satellite line, cable TV line, data broadcasting wireless line, etc. If possible. Alternatively, the network is a so-called communication network realized by a combination of these as long as data can be transmitted and received.

  The present invention can also be implemented by a general-purpose computer. In this case, the control program stored in the storage medium may be stored in the external storage device 104 and executed by the CPU 101 according to an operator's instruction or the like.

  The interface (I / F) 108 connects an imaging device (not shown) such as a digital camera that acquires an electrical signal to be processed with a user terminal. The components 101 to 108 are connected to each other via a system bus 109.

  Such an image processing apparatus is realized, for example, in the system shown in FIG. FIG. 2 is a diagram illustrating a configuration example of a computer system that can be employed in the present embodiment. A user terminal 201 and a server 202 are connected to each other via a LAN (Local Area Network) 204 so that they can communicate with each other. The user terminal 201 is an information processing apparatus such as a personal computer or a workstation, and realizes processing according to the embodiment of the present invention.

  Note that the processing according to the embodiment of the present invention is executed by the user terminal 201 and can be performed using image data such as BMP, TIFF, JPEG, and PDF. You may perform the process which concerns on embodiment of this invention using the image data memorize | stored in the user terminal 201. FIG. The processing according to the embodiment of the present invention may be performed by acquiring and using image data read by an imaging device such as the digital camera 203 connected to the user terminal 201. The processing according to the embodiment of the present invention may be performed by acquiring and using image data stored in the server 202 via the LAN 204. Further, the processing according to the embodiment of the present invention may be executed by the server 202 and the processing result may be confirmed using the user terminal 201.

  FIG. 3 is a flowchart showing the overall operation of the system according to the embodiment of the present invention.

  In step S301, when an image including a license plate is input, a license plate region (hereinafter referred to as an NP region) is detected from the input image. There are many detection methods for detecting a specific image area, such as an object in which a format related to a certain degree of character arrangement is determined in advance, such as a license plate. For example, a character image having a different size can be found by dividing the image into various strips and searching for a character image in the division (Non-Patent Document 1).

  5 is detected from the input image of FIG. 4, the NP region 701 of FIG. 7 is detected from the input image of FIG. 6, and the NP region 901 and the NP region of FIG. 9 are detected from the input image of FIG. 902 is detected. Each NP region has rectangular information of upper left, lower left, upper right, and lower right coordinates with respect to the image origin.

  In step S302, a character is cut out from the detected image of the NP region by a known method to perform character recognition. The character string “8888” is recognized from the NP area 501 in FIG. 5, “8888” from the NP area 701 in FIG. 7, “8888” from the NP area 901 in FIG. 9, and “2222” from the rectangular area 902. The

  In step S303, the focus point of the input image is calculated. In general, focus point information can be extracted from the inside of JPEG EXIF. However, for an input image without focus point information, focus point detection is performed by the following method. The calculation of the focus point is roughly classified into a correlation method and a contrast detection method. In the correlation method, two sets of light receiving elements corresponding to the left and right eyes are used, and the photoelectric outputs of the left and right corresponding light receiving elements are compared to find the most similar place. The contrast detection method uses the fact that the contrast is maximized when the focus is best, and the contrast decreases when the focus is out of focus, and searches for a place where the contrast is maximized. These focus point detection techniques are described in Non-Patent Document 1. Using such a known detection method, the focus point 502 in FIG. 5 from the input image in FIG. 4, the focus point 702 in FIG. 7 from the input image in FIG. 6, and the input image in FIG. The focus point 903 is detected. Each focus point is position information with respect to the image origin.

  In step S304, the center of gravity of the NP region detected in step S301 is obtained, and the distance between this center of gravity and the focus point detected in step S303 is calculated. Details of distance calculation will be described later.

  In step S305, it is determined whether all the input images have been processed in steps S301 to S304. If yes, the process proceeds to step S306. If not, the next input image is selected and the process proceeds to step S301. move on.

  In step S306, the distance between the center of gravity of the NP region and the focus point in all input images is referred to, and the input image with the shortest distance is selected as the representative image. The reason for selecting such an input image as a representative image is that such an input image is an image that tries to capture the license plate near the center of the image and is an image that has the license plate as the main subject. This is because it can be estimated that the visibility will be good. The representative image shown in FIG. 6 having the shortest distance is selected from the input images shown in FIGS.

  In step S307, distortion correction of the entire image is performed based on the rectangular information of the NP area of the representative image selected in step S306. A known technique can be used as a method for correcting image distortion based on the rectangular information. The corrected image shown in FIG. 10 is generated from the representative image shown in FIG. A dotted line in the corrected image in FIG. 10 indicates an outer frame of the original image.

  In step S308, an output image around the NP region is cut out from the corrected image generated in step S307. Note that the area to be cut out is an area widened by a predetermined offset from the NP area, and includes the NP area without leaking. This offset value may be specified by the user, or may be dynamically calculated from the size of the NP area and the image size. The output image of FIG. 11 is cut out from the corrected image of FIG.

  In step 309, the output image cut out in step S308 is output. When outputting, for example, the UI is output as shown in FIG.

  In FIG. 12, an input image group 1201 including the same license plate is displayed at the top of the UI. In the display, it is preferable to display the representative image selected in step S306 in a form that the user can easily recognize. In the input image group 1201, the visibility is improved by displaying the representative image on the leftmost side. An output image 1202 cut out from the corrected image of the representative image is displayed at a predetermined magnification at the bottom of the input image group 1201 and the center of the UI. In other words, the license plate is displayed in a predetermined size at a predetermined location with the license plate facing up. Note that the magnification may be designated by the user, or may be dynamically calculated from the size of the NP region and the input image size.

  In the upper part of the UI, a character recognition result 1203 of the NP area in the representative image is displayed. When the visibility of the displayed output image 1202 is not good, the user presses an arbitrary key, so that the input image group 1201 is different from the displayed image in the NP region of the input image (for example, the middle). Steps S307 to S308 may be performed and output again as shown in FIG. Note that the processing in steps S307 to S308 is performed on all input images in advance, the corrected and cutout output images are stored in the RAM 103, and output when there is a re-output command from the user as described above. The image may be fetched from the RAM 103.

  Here, details of step S304 relating to the calculation of the distance between the center of gravity of the NP region and the focus point will be described with reference to FIGS.

  FIG. 14 shows a detailed flowchart of step 304.

  First, in step S1401, the NP region having the largest size is selected from all the NP regions in the input image and set as the NP region to be processed. In the input image of FIG. 8, the NP area 901 out of the NP area 901 and the NP area 902 is a processing target. In addition, the processing target may be determined in consideration of the similarity of characters in the NP region. For example, among all NP regions, a method of selecting an NP region that includes characters having a similarity equal to or greater than a predetermined threshold and whose size is equal to or greater than a predetermined threshold can be used.

  In step S1402, the center of gravity of the NP region selected in step 1401 is calculated by a known method. For example, the center of gravity G (gx, gy) of the NP region 1501 in FIG.

It can be calculated by

  In step S1403, the distance between the focus point and the center of gravity is calculated. Note that the distance calculated here is not a mere distance between two points, but is calculated taking into account the vertical width and horizontal width of the circumscribed rectangle of the NP region. For example, in FIG. 15, when the width of the circumscribed rectangle 1502 of the NP region 1501 is W and the height is H, the distance between the focus point F (fx, fy) and the center of gravity G (gx, gy) is calculated by the following equation. To do.

  According to the present embodiment, when presenting the license plate detection results of a plurality of images to the user, an image more suitable for presentation of the license plate is selected based on the center of gravity of the license plate area and the position of the focus point. Can be presented in the best form. Further, by cutting out from the correction image an area that is widened by a predetermined offset from the NP area, when information exists around the license plate, it can be presented simultaneously with the license plate.

(Second Embodiment)
In this embodiment, the output image as shown in FIG. 16 in which the information in the NP area is hidden is generated by applying a process such as mosaic or filling to the NP area of the output image of FIG. 11 cut out in step S308. An image processing apparatus and method can be implemented. In the present embodiment, image processing such as mosaic or painting is described as an example. However, the processing is not particularly limited as long as it is processing for hiding information in the NP area.

(Third embodiment)
In this embodiment, in order to estimate an image having a license plate as a main subject from an image group, a focus point is obtained from the image, whether or not the license plate area includes the focus point, and the license plate area is verified. This is to test the aspect ratio.

(Fourth embodiment)
In this embodiment, in order to estimate an image having a license plate as a main subject from an image group, a focus point is obtained from the image, and a distance with respect to the image size from the license plate to the distance measuring point used for distance measurement is obtained. It is.

(Fifth embodiment)
In this embodiment, when a selected image is displayed as a representative image, a distance measuring point and a distance measuring point actually used for distance measurement are overwritten and displayed on the image.

(Sixth embodiment)
In this embodiment, in order to estimate an image having a license plate as a main subject from an image group, the distortion of the license plate is corrected and enlarged or reduced to a specified size. Then, a sum of second-order differential values of brightness per unit area on the corrected license plate is obtained, and an image having the license plate as a main subject is estimated based on the sum.

<Other embodiments>
As described above, the present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.) but also to an apparatus composed of a single device (for example, a copying machine, a facsimile machine). May be.

  In addition, a program code of software for realizing the functions of the embodiment is provided in an apparatus or a computer in the system connected to the various devices so as to operate the various devices so as to realize the functions of the above-described embodiments. What is implemented by operating the various devices in accordance with a program stored in a computer (CPU or MPU) of the system or apparatus supplied is also included in the scope of the present invention.

  Further, in this case, the program code of the software itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, a storage storing the program code The medium constitutes the present invention.

  As a storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) in which the program code is running on the computer, or other application software, etc. It goes without saying that the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

  Further, after the supplied program code is stored in a memory provided in a function expansion board of a computer or a function expansion unit connected to the computer, a CPU provided in the function expansion board or function storage unit based on an instruction of the program code However, it is needless to say that the present invention also includes a case where the function of the above-described embodiment is realized by performing part or all of the actual processing.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to the present invention. FIG. 2 is a system diagram illustrating a configuration of a computer system that is the image processing apparatus of FIG. 1. It is a flowchart which shows the process outline | summary of the computer system which is an image processing apparatus of FIG. It is a figure showing an example of an input image. It is explanatory drawing which shows the pattern which extracted the NP area | region (number plate area | region) and the focus point from the input image example of FIG. It is a figure showing another example of an input image. It is explanatory drawing which shows the pattern which extracted NP area | region (number plate area | region) and the focus point from the input image example of FIG. It is a figure showing another example of the example of an input image. It is explanatory drawing which shows the pattern which extracted NP area | region (number plate area | region) and the focus point from the input image example of FIG. It is a figure which shows the pattern which carried out distortion correction of the input image example of FIG. It is a figure which shows the pattern which cut out the image around NP area | region (number plate area | region) from the image of FIG. It is a figure showing an example of an output user interface. It is a figure showing another example of an output user interface. It is a flowchart which shows the detail of step S304 of the flowchart of FIG. It is explanatory drawing explaining the distance of the gravity center of NP area | region (number plate area | region), and a focus point. It is a figure which shows the image which concealed the information in NP area | region (number plate area | region) of an output image.

Explanation of symbols

101: CPU
102: ROM
103: RAM
104: External storage device 105: Display 106: Input device 107: NIC
108: I / F
109: LAN
201: Computer device 202: Server device 203: Digital camera device 204: LAN
501: NP area 502 detected from the input image in FIG. 4: Focus point 701 of the input image in FIG. 4: NP area 702 detected from the input image in FIG. 6: Focus point 901 in the input image in FIG. 6: Input in FIG. NP area 902 detected from the image: NP area 903 detected from the input image in FIG. 4: Focus point 1201 of the input image in FIG. 4: User interface 1202 for displaying the input image group: User interface 1203 for displaying the output image: Recognition User interface 1501: NP area (number plate area) for displaying results
1502: circumscribed rectangle of NP area

Claims (14)

An image processing apparatus for inputting a plurality of images including a license plate area, and displaying a representative image with good visibility of characters in the license plate area among the images,
Calculating means for detecting the license plate region and obtaining a focus point of the image for the plurality of input images;
Selecting means for selecting, as the representative image , the one having the shortest distance between the position of the license plate region and the position of the focus point from the plurality of images;
An image processing apparatus comprising: display means for displaying the selected image. The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the display unit includes a correction unit that corrects the selected image so that the selected image is displayed in a predetermined size at a predetermined position in a face-to-face state. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the display unit further includes a cutout unit that cuts out an image widened by a predetermined offset from the license plate area for the image corrected by the correction unit. In the image processing device according to any one of claims 1 to 3,
The selection means obtains the position of the center of gravity of the license plate area for each of the plurality of images, calculates the distance between the position and the focus point for each image, and the distance is the shortest among the images. An image processing apparatus that selects an object. In the image processing device according to any one of claims 1 to 4,
Further comprising character recognition means for performing character recognition on the detected license plate region,
The display means further displays the recognition result together with the selected image. In the image processing device according to any one of claims 1 to 4,
An image processing apparatus, further comprising means for hiding the display by the display means. An image processing method for inputting a plurality of images including a license plate area, and displaying a representative image with good visibility of characters in the license plate area among the images,
For the plurality of input images, a calculation unit detects a license plate region and calculates a focus point of the image; and
A selection step in which the selection unit selects, as the representative image , the one having the shortest distance between the position of the license plate region and the position of the focus point from the plurality of images;
A display step of displaying the selected image by a display means. The image processing method according to claim 7.
The display step further includes a correction step of correcting the selected image so that the selected image is displayed in a predetermined size at a predetermined location in a face-to-face state. The image processing method according to claim 8.
The display step further includes a cutout step of cutting out an image widened by a predetermined offset from the license plate area for the image corrected in the correction step. The image processing method according to any one of claims 7 to 9,
In the selection step, the position of the center of gravity of the license plate region is obtained for each of the plurality of images, the distance between the position and the focus point is calculated for each image, and the distance is the shortest among the images. An image processing method characterized by selecting one. In the image processing method according to any one of claims 7 to 10,
After the calculation step and before the selection step, further includes a character recognition step in which character recognition means performs character recognition for the detected license plate region,
In the display step, the recognition result is further displayed together with the selected image. The image processing method according to any one of claims 7 to 11,
The image processing method further comprising the step of hiding the display in the display step.   The program for making a computer perform each step of the image processing method described in Claim 7.   A computer-readable storage medium storing the program according to claim 13.

Images