JP7823722B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to a vehicle information acquisition system for acquiring vehicle information, a vehicle information acquisition method, a vehicle information acquisition program, and a storage medium for storing a vehicle information acquisition program.

This section describes an example of a system that reads the number on a vehicle's license plate and uses the characters that are read.

Patent Document 1 describes a system that extracts characters from a photographed image of a vehicle's license plate and searches a database (DB). The system then displays the data hits on a display device as a result of the search. Patent Document 2 describes a system that reads and temporarily stores vehicle numbers and other information. When the system receives an instruction to end continuous reading, it then sequentially searches the vehicle DB for the temporarily stored numbers. The system then displays the search results on a display.

Japanese Patent Application Publication No. 11-296785 JP 2000-322421 A

However, while the above-mentioned system can display the search results for license plates, it has the problem of being unable to display information that allows for comparison between the photographed vehicle and the registered vehicle. As a result, users of the above-mentioned system are unable to know if the license plate is counterfeit, etc. Therefore, the object of the present invention is to provide a vehicle information acquisition system that solves the problem of being unable to display information that allows for comparison between the photographed vehicle and the registered vehicle.

The first information processing device of the present invention comprises an acquisition unit that acquires an image of a vehicle photographed by a terminal, an extraction unit that extracts a character string from the area of the license plate included in the image, a vehicle information acquisition unit that acquires information about a registered vehicle associated with the character string, and an output unit that outputs information about the registered vehicle if the character string extracted from the image is any of a plurality of pre-stored registered information, and outputs information indicating that the character string extracted from the image is not any of the pre-stored registered information if the character string extracted from the image is not any of the plurality of pre-stored registered information.
In a second information processing method of the present invention, a computer includes a vehicle information acquisition unit that acquires an image of a vehicle taken by a terminal, extracts a character string from the area of the license plate included in the image, and acquires information about the registered vehicle associated with the character string; and if the character string extracted from the image is any of a plurality of pre-stored registered information, outputs information about the registered vehicle, and if the character string extracted from the image is not any of the plurality of pre-stored registered information, outputs information indicating that the character string extracted from the image is not the pre-stored registered information.
The third information processing program of the present invention causes a computer to perform the following processes: acquiring an image of a vehicle photographed by a terminal; extracting a character string from the area of the license plate included in the image; acquiring information about the registered vehicle associated with the character string; if the character string extracted from the image is any of a plurality of pre-stored registration information, outputting information about the registered vehicle based on the character string; and if the character string extracted from the image is not any of the plurality of pre-stored registration information, outputting information indicating that the character string extracted from the image is not the pre-stored registration information.

This invention provides a vehicle information acquisition system that can display information that allows comparison between photographed vehicles and registered vehicles.

FIG. 1 is a block diagram illustrating a configuration example of a first embodiment. FIG. 10 is an explanatory diagram illustrating photographing a license plate. 10 is an example of information stored in the storage unit 23. 10 is an example of information stored in the storage unit 23. 10 is an example of a display by the display unit 13. 4 is a flowchart illustrating an operation example of the first embodiment. FIG. 10 is a block diagram showing a configuration example of a second embodiment. FIG. 2 is an explanatory diagram for explaining photographing of a vehicle. 10 is an example of information stored in the storage unit 25. 10 is an example of a display by the display unit 13. 10 is a flowchart illustrating an operation example of the second embodiment. FIG. 10 is a block diagram showing a configuration example of a third embodiment. 10 is an example of a display by a display unit. 1 is an example of a hardware configuration. 1 is an example of a hardware configuration.

Embodiments of the present invention will be described in detail below with reference to the drawings. However, the components described in the following embodiments are merely examples and are not intended to limit the technical scope of the present invention.

(First embodiment)

(Configuration of the first embodiment)
A configuration example of the first embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of a vehicle information acquisition system 1000. In the first embodiment, it is assumed that the present invention is used in a special area where vehicle entry is restricted when an event or the like is held. Entry into the special area is prohibited for vehicles other than those of event-related personnel and local residents. Therefore, a person guarding the special area patrols the special area and confirms that vehicles parked within the special area are vehicles of pre-registered related personnel (hereinafter referred to as registered vehicles). Alternatively, the person guarding the special area checks vehicles at a checkpoint set up near the border between the special area and other areas. At this time, the person guarding the special area checks whether the characters displayed on the license plate attached to the vehicle are those of a registered vehicle.

Number plates are also called license plates. The characters displayed on a number plate are also called vehicle registration numbers or vehicle numbers. The characters include numbers and symbols.

Here, a license plate refers to a vehicle registration plate, vehicle number plate, or sign that is required to be attached to vehicles under traffic laws and other regulations. License plates typically display characters that are unique to each vehicle. Therefore, if the characters on the license plate match pre-registered characters, the vehicle is authorized to enter the special area. However, someone attempting to illegally enter a special area may forge a license plate. In this case, those guarding the special area may mistake a vehicle that is not authorized to enter for a registered vehicle.

The vehicle information acquisition system 1000 of the first embodiment therefore adopts the following configuration. Below, the user of the vehicle information acquisition system 1000 and the user of the mobile terminal 100 will be described as guarding a special area.

The vehicle information acquisition system 1000 includes a mobile terminal 100 and a server 200. The mobile terminal 100 is communicatively connected to the server 200 via a wireless or wired network. The mobile terminal 100 includes an acquisition unit 10, a character extraction unit 11, a display control unit 12, a transmission unit 14, a reception unit 15, and a display unit 13. The server 200 includes a reception unit 20, a vehicle information acquisition unit 21, a transmission unit 22, and a memory unit 23.

The acquisition unit 10 within the mobile terminal 100 acquires an image that captures at least a portion of the license plate attached to the vehicle. The image may be captured by a camera unit (not shown) included in the mobile terminal 100. The camera unit (not shown) may be a camera having, for example, a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, or the like. The camera unit (not shown) generates image data by converting the received light into image data.

Figure 2 is an explanatory diagram illustrating the capturing of a license plate by the imaging unit within the mobile terminal 100. The user of the mobile terminal 100 operates the mobile terminal 100 while pointing the imaging unit (not shown) towards the vehicle 1 being photographed, to capture a picture of the license plate attached to the vehicle 1 being photographed. The acquisition unit 10 acquires the image generated by the imaging unit (not shown). The image acquired by the acquisition unit 10 needs to show at least a portion of the license plate. Images 2 and 3 are examples of images acquired by the acquisition unit 10. Image 2 shows the entire license plate 5 attached to the vehicle 1 being photographed. Meanwhile, image 3 shows only a portion of the license plate 5.

The character extraction unit 11 extracts characters from the license plate portion of the image acquired by the acquisition unit 10. The character extraction unit 11 may identify the license plate portion of the image using pattern recognition or the like. The character extraction unit 11 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 10 to extract characters.

For example, when image 2 in Figure 2 is acquired, the character extraction unit 11 extracts the characters "Shinagawa," "300," "sa," and "12-34." The character extraction unit also determines which item the extracted characters correspond to based on the position data of the extracted characters and the position data of pre-stored items. The item position data may be stored in the storage unit 23. The position indicated by the position data may be an absolute position in the image, or the relative position of each character or item. An item indicates, for example, the content of information required to be displayed on a vehicle by traffic laws. For example, items include "Transport Bureau, etc.", "Type," "Business Use," and "Designated Number." The character extraction unit 11 does not have to determine the item corresponding to the characters.

The transmitting unit 14 in the mobile device 100 transmits the characters extracted by the character extraction unit 11 and the items corresponding to the characters to the server 200. The receiving unit 20 in the server 200 receives the characters and items transmitted by the transmitting unit 14.

The memory unit 23 in the server 200 stores the vehicle information of registered vehicles in association with the characters.

Here, vehicle information may be any information related to a vehicle. For example, vehicle information may be information recorded on the vehicle inspection certificate, such as the chassis number, model, body shape, and owner information; information displayed on the license plate attached to the vehicle; images of the vehicle; and features of the images. An example of information stored in the memory unit 23 is shown in Figure 3. The characters are divided into the following categories: "Transport Bureau, etc.", "Type," "Business Use," and "Designated Number." The characters contained in these categories are required to be displayed on license plates under traffic laws. Vehicle information also includes "Model," "Name," "Manufacturer," "Year," and "Color." Note that the information stored in the memory unit 23 is not limited to that shown in Figure 3.

From Figure 3, the characters "Shinagawa," "300," "Sa," and "12-34" correspond to the vehicle information "Standard passenger car," "Purlis," "Toyoda," "H23," and "White." Furthermore, the characters "Shinagawa," "350," "Ha," and "34-56" correspond to the vehicle information "Standard passenger car," "Ellie," "H25," and "Red." The characters "Kashiwa," "550," "Mi," and "56-78" correspond to the vehicle information "Compact passenger car," "Aldo," "Susuki," "H20," and "Black."

The memory unit 23 also stores vehicle information and comparison images in association with each other. An example of the information stored in the memory unit 23 is shown in Figure 4. In Figure 4, the vehicle information "Standard passenger car," "Purlis," "Toyoda," "H23," and "White" corresponds to the comparison image "aaa.jpg." The vehicle information "Standard passenger car," "Purlis," "Toyoda," "H23," and "Black" corresponds to the comparison image "bbb.jpg." The vehicle information "Standard passenger car," "Purlis," "Toyoda," "H23," and "Gray" corresponds to the comparison image "ccc.jpg."

Note that the image file format is not limited to JPEG (Joint Photographic Experts Group). For example, the image file format may be GIF (Graphics Interchange Format), PNG (Portable Network Graphics), BMP (Microsoft Windows (registered trademark) Bitmap Image), etc.

The comparison image includes an image of a vehicle that meets the vehicle information. In other words, the comparison image includes an image of a vehicle whose vehicle information, such as model, name, manufacturer, year, and color, matches that of a registered vehicle. For example, the comparison image "aaa.jpg" includes an image of a vehicle whose model is a passenger car, whose name is Purlis, whose manufacturer is Toyoda, whose year of manufacture is H23, and whose color is white. The comparison image may also be an image of a vehicle featured in a catalog, product introduction site, etc. The comparison image may also be an image of a registered vehicle.

The vehicle information acquisition unit 21 receives the characters from the receiving unit 20. Then, the vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 23. That is, the vehicle information acquisition unit 21 acquires vehicle information corresponding to characters that match the extracted characters from the storage unit 23. For example, assume that the extracted characters are "Shinagawa," "300," "sa," and "12-34," and the storage unit 23 stores the information shown in FIG. 3. In this case, the vehicle information acquisition unit 21 references the storage unit 23 and acquires the vehicle information "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White" that corresponds to the characters "Shinagawa," "300," "sa," and "12-34."

At this time, the vehicle information acquisition unit 21 may check for each item whether the extracted characters match the characters stored in the memory unit 23. For example, the vehicle information acquisition unit 21 determines whether the extracted characters corresponding to the item "Transport Bureau, etc." match the characters stored in the memory unit 23 that correspond to the item "Transport Bureau, etc." Similarly, the vehicle information acquisition unit 21 also determines whether the characters associated with the items "Type," "Business Use," and "Designation Number" match for each item. Then, if the characters match for all items, the vehicle information corresponding to the extracted characters may be acquired from the memory unit 23.

The vehicle information acquisition unit 21 also acquires items corresponding to vehicle information from the storage unit 23. For example, the vehicle information acquisition unit 21 associates "vehicle model" with "standard passenger car," "vehicle name" with "Purlis," "manufacturer" with "Toyoda," "year of manufacture" with "H23," and "color" with "white," and acquires these. Note that the vehicle information acquisition unit 21 does not have to acquire items from the storage unit 23.

Furthermore, the vehicle information acquisition unit 21 acquires, along with the vehicle information, a comparison image corresponding to the vehicle information from the storage unit 23. That is, the vehicle information acquisition unit 21 acquires, from the storage unit 23, a comparison image corresponding to the vehicle information acquired by the storage unit 23. For example, from FIG. 4, if the vehicle information acquisition unit 21 acquires "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White" as vehicle information, it acquires the comparison image "aaa.jpg" from the storage unit 23. The transmission unit 22 in the server 200 transmits the vehicle information, items, and comparison image acquired by the vehicle information acquisition unit 21 to the mobile terminal 100.

The receiving unit 15 in the mobile terminal 100 receives the vehicle information, items, and comparison images transmitted from the server 200. The display control unit 12 causes the display unit 13 to display the vehicle information and items acquired by the vehicle information acquisition unit 21. At this time, the display control unit 12 causes the display unit 13 to display the comparison images along with the vehicle information. Note that the display control unit 12 does not have to display the items on the display unit 13.

The display unit 13 displays vehicle information, items, and a comparison image under the control of the display control unit 12. An example of a display by the display unit 13 is shown in Figure 5. In Figure 5, the display unit 13 of the mobile terminal 100 displays the comparison image "aaa.jpg", vehicle information, and items. As described above, the vehicle shown in the comparison image "aaa.jpg" is a standard passenger car, with a model name of "Purulis", a manufacturer of "Toyoda", a model year of 2011, and a color of white. Furthermore, the vehicle information displayed by the display unit 13 in Figure 5 is "Standard passenger car", "Purulis", "Toyoda", "H23", and "White". Furthermore, the items displayed by the display unit 13 are "Model", "Model name", "Manufacturer", "Model year", and "Color".

Note that the display by the display unit 13 is not limited to that shown in FIG. 4. For example, the display unit 13 does not need to display any items. Furthermore, the display unit 13 may display the image acquired by the acquisition unit 10 (an image of the photographed vehicle 1) along with the above display. The display unit 13 may also display the characters extracted by the character extraction unit 11, i.e., the characters displayed on the license plate of the photographed vehicle 1.

(Operation of the first embodiment)
Next, a description will be given of the operation of the vehicle information acquisition system 1000. FIG.

First, the acquisition unit 10 in the mobile terminal 100 acquires an image that shows at least a portion of the license plate attached to the vehicle (S1). The image may be captured by a camera unit (not shown) included in the mobile terminal 100. The camera unit (not shown) may be a camera having, for example, a CCD sensor, a CMOS sensor, an image processor, etc. The camera unit (not shown) generates an image by converting the received light into data.

For example, the acquisition unit 10 may acquire an image after the user of the mobile terminal 100 launches a specific application. For example, when the user launches the application, a photographing unit (not shown) captures an image of the license plate. The acquisition unit 10 then acquires the image from the photographing unit (not shown).

Next, the character extraction unit 11 extracts characters from the license plate portion of the image acquired by the acquisition unit 10 (S2). The character extraction unit 11 performs OCR processing on the image acquired from the acquisition unit 10 to extract characters. The character extraction unit 11 also determines the items corresponding to the extracted characters. The transmission unit 14 then transmits the characters and items extracted by the character extraction unit 11 to the server 200.

The receiving unit 20 in the server 200 receives the characters and items sent by the transmitting unit 14. The vehicle information acquiring unit 21 receives the characters and items from the receiving unit 20. The vehicle information acquiring unit 21 then acquires vehicle information corresponding to the extracted characters from the storage unit 23 (S3). At this time, the vehicle information acquiring unit 21 acquires the items corresponding to the vehicle information. Furthermore, the vehicle information acquiring unit 21 acquires a comparison image corresponding to the acquired vehicle information from the storage unit 23 (S4). Note that the vehicle information acquiring unit 21 does not have to acquire the items from the storage unit 23.

The transmitting unit 22 in the server 200 transmits the vehicle information, items, and comparison images acquired by the vehicle information acquisition unit 21 to the mobile terminal 100. The receiving unit 15 in the mobile terminal 100 then receives the vehicle information, items, and comparison images transmitted from the server 200.

The display control unit 12 causes the display unit 13 to display the comparison image along with the vehicle information and items received by the receiving unit 15. Note that the display control unit 12 does not have to display the items on the display unit 13.

As described above, the vehicle information acquisition system 1000 of this embodiment can display vehicle information for registered vehicles whose license plate characters match those of the photographed vehicle. Furthermore, the vehicle information acquisition system 1000 can display an image of a vehicle that meets the vehicle information. This allows a user of the mobile device 100 to easily grasp the appearance of a vehicle with the name "Purlis," for example, by looking at the displayed image, even if they do not know what that vehicle looks like. Therefore, by comparing the image displayed by the mobile device 100 with the appearance, etc., of the photographed vehicle, the user can efficiently determine whether the photographed vehicle matches the registered vehicle.

The server 200 may also have a character extraction unit 11. In this case, the transmission unit 14 in the mobile terminal 100 transmits the image acquired by the acquisition unit 10 to the server 200. The character extraction unit 11 in the server 200 then extracts characters from the license plate portion of the image received from the mobile terminal 100 and passes the extracted characters to the vehicle information acquisition unit 21.

In addition, while the memory unit 23 previously had separate data associating text with vehicle information and data associating vehicle information with comparative images, it may instead have one data set associating text, vehicle information, and comparative images. In this case, the vehicle information acquisition unit 21 can acquire the vehicle information and comparative images corresponding to the text at the same time, thereby improving the processing speed of the vehicle information acquisition system 1000.

Furthermore, the vehicle information acquisition unit 21 may calculate the similarity between the extracted characters and the characters stored in the memory unit 23, and acquire the vehicle information corresponding to the character with the highest similarity. Alternatively, the vehicle information acquisition unit 21 may acquire all vehicle information corresponding to characters with a similarity equal to or greater than a threshold. In this case, the display control unit 12 may display all vehicle information acquired by the vehicle information acquisition unit 21 on the display unit 13. The display control unit 12 may also display the vehicle information on the display unit 13 in order of similarity. This allows the user to compare the registered vehicle with the photographed vehicle even when the characters displayed on the license plate in the image are unclear or when some of the characters are not captured in the image.

Furthermore, if the memory unit 23 does not store any characters whose similarity to the extracted characters is equal to or greater than the threshold, or any characters identical to the extracted characters, the vehicle information acquisition unit 21 may transmit a message to that effect to the mobile device 100 via the transmission unit 22. The display control unit 12 may then display a message on the display unit 13 indicating that the photographed vehicle is not registered. This allows the user to know that the photographed vehicle is not a registered vehicle. Alternatively, the display control unit 12 may display a message on the display unit 13 encouraging the user to take another photograph. This allows the user to take another photograph of the same vehicle and check whether it is registered. These messages may be stored in advance in a memory unit (not shown) within the mobile device 100, or may be transmitted to the mobile device 100 by the server 200.

Furthermore, the memory unit 23 may store information about the owner of the registered vehicle in association with the characters. For example, the memory unit 23 may register a facial image of the owner of the registered vehicle in association with the characters. In this case, the vehicle information acquisition unit 21 may acquire the facial image along with the vehicle information. The display control unit 12 may also display the facial image on the display unit 13 along with the vehicle information. This allows the user to determine whether the photographed vehicle matches the registered vehicle by checking the owner of the photographed vehicle, even if the vehicle with a forged license plate attached has the same model, name, color, etc. as the registered vehicle.

In addition, while in this embodiment the server 200 has a memory unit 23, the memory unit in the present invention may be realized by multiple hardware configurations. For example, a memory device within the server 200 may store data associating characters with vehicle information, and a memory device within another server that can communicate with the server 200 may store data associating vehicle information with comparison images.

Note that while the information displayed on license plates is regulated by the Road Transport Vehicle Act in Japan, it may differ in other countries. As a result, the information displayed on license plates may also differ from country to country. Taking these points into consideration, the content displayed on license plates is not limited to that described in this embodiment.

(Second embodiment)

(Configuration of the second embodiment)
An example of the configuration of the second embodiment will be described. Fig. 7 is a diagram showing an example of the configuration of a vehicle information acquisition system 1001. The vehicle information acquisition system 1001 differs from the vehicle information acquisition system 1000 in that it includes a vehicle information extraction unit 16, a matching unit 24, and a storage unit 25. The remaining configuration is the same as in the first embodiment, and therefore description thereof will be omitted where appropriate.

The acquisition unit 10 acquires an image that shows at least a portion of the vehicle in addition to an image that shows at least a portion of the license plate. The image that shows at least a portion of the vehicle may be captured by a capture unit (not shown) included in the mobile terminal 100.

Figure 8 is an explanatory diagram illustrating the capturing of a photographed vehicle 1 by a photographing unit within a mobile terminal 101. The user of the mobile terminal 101 operates the mobile terminal 101 while pointing the photographing unit (not shown) towards the photographed vehicle 1 to capture the photographed vehicle 1. The acquisition unit 10 then acquires the image generated by the photographing unit (not shown). The image acquired by the acquisition unit 10 need only capture at least a portion of the photographed vehicle 1. However, it is preferable that the image capture parts of the vehicle that have many irregularities or parts that vary depending on the vehicle model. For example, it is preferable that the image capture the vehicle's headlights, grille, bumper, emblem, ornament, wheels, etc.

Images 4 and 6 are examples of images acquired by the acquisition unit 10. Image 4 shows the entire photographed vehicle 1. On the other hand, image 6 shows only a portion of the photographed vehicle 1.

The vehicle information extraction unit 16 extracts vehicle information from an image that shows at least a portion of the vehicle. In this embodiment, the vehicle information is, for example, image features. The vehicle information extraction unit 16 extracts features from the vehicle portion of the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion of the image using pattern recognition or the like. Feature extraction may be achieved using techniques such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Here, for example, it is assumed that the vehicle information extraction unit 16 extracts feature data X from the vehicle portion of the image.

The transmission unit 14 transmits the characters "Shinagawa," "300," "sa," and "12-34" extracted by the character extraction unit 11 and the feature data X extracted by the vehicle information extraction unit 16 to the server 201.

The memory unit 25 in the server 201 stores vehicle information for registered vehicles in association with the characters displayed on the license plate of the registered vehicle. In addition to the information stored in the memory unit 23, the memory unit 25 stores image features as vehicle information. The image is an actual image of the registered vehicle. Note that the image may not be an image of the registered vehicle, but may be an image of a vehicle with the same model, name, color, etc. as the registered vehicle. For example, the image may be an image of a vehicle published in a catalog, product introduction site, etc.

An example of data stored in the memory unit 25 is shown in Figure 9. As can be seen from Figure 9, the characters "Shinagawa", "300", "sa", and "12-34" correspond to feature data X. Furthermore, the characters "Shinagawa", "350", "ha", and "34-56" correspond to feature data Y. The characters "Kashiwa", "550", "mi", and "56-78" correspond to feature data Z. Vehicle information other than the features is the same as in Figure 3, so a description will be omitted.

The vehicle information acquisition unit 21 acquires from the storage unit 25 vehicle features corresponding to the characters "Shinagawa," "300," "sa," and "12-34" received by the receiving unit 20. As can be seen from FIG. 9, the vehicle feature corresponding to the characters "Shinagawa," "300," "sa," and "12-34" is feature amount data X, so the vehicle information acquisition unit 21 acquires feature amount data X, which is the vehicle feature. Furthermore, the vehicle information acquisition unit 21 acquires the vehicle information corresponding to the characters "Shinagawa," "300," "sa," and "12-34," which is "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White."

The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 from the receiving unit 20. The matching unit 24 also acquires the vehicle information acquired by the vehicle information acquisition unit 21. The matching unit 24 then compares the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit 21.

The matching unit 24 then transmits the matching result to the mobile terminal 101 via the transmission unit 22. The matching result is, for example, information such as whether the matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not perfectly match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

For example, in the example described above, the matching unit 24 acquires feature amount data X of an image of the photographed vehicle 1. Furthermore, the matching unit 24 acquires feature amount data X of an image of a registered vehicle. The matching unit 24 then compares these acquired feature amounts and transmits the matching result to the transmission unit 22.

The transmission unit 22 transmits the vehicle information acquired by the vehicle information acquisition unit 21 and the matching result by the matching unit 24 to the mobile terminal 100. For example, the transmission unit 22 transmits the characters "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White" acquired by the vehicle information acquisition unit 21, along with the matching result. The transmission unit 22 may also transmit the feature values acquired by the vehicle information acquisition unit 21.

The display control unit 12 causes the display unit 13 to display the matching result by the matching unit 24 along with the vehicle information received by the receiving unit 15 from the server 201. An example of the display by the display unit 13 is shown in Figure 10. As shown in Figure 10, the display unit 13 displays the matching result "Matching successful" along with the words "Standard passenger car," "Purlis," "Toyoda," "H23," and "White."

(Operation of the second embodiment)
Next, a description will be given of the operation of the vehicle information acquisition system 1001. Fig. 11 is a sequence diagram illustrating the operation of the vehicle information acquisition system 1001. Descriptions of operations similar to those of the vehicle information acquisition system 1000 shown in Fig. 10 will be omitted where appropriate.

After characters are extracted from an image showing at least a portion of the license plate (S2), the acquisition unit 10 in the mobile terminal 100 acquires an image showing at least a portion of the vehicle. This image may be captured by a camera unit (not shown) included in the mobile terminal 100, similar to the image captured in S1. Note that if at least a portion of the vehicle is captured in the image captured in S1, the acquisition unit 10 does not need to acquire the image again. Furthermore, the acquisition unit 10 may acquire an image showing at least a portion of the vehicle before acquiring an image showing at least a portion of the license plate (S1).

Then, the vehicle information extraction unit 16 extracts vehicle information of the vehicle from the image that shows at least a portion of the vehicle (S6). In this embodiment, the vehicle information is, for example, image features. The vehicle information extraction unit 16 extracts features from the vehicle portion of the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion of the image using pattern recognition or the like. Extraction of features may be achieved using techniques such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Note that the order of S1, S2, and S6 is not limited to this.

The transmission unit 14 transmits the characters extracted by the character extraction unit 11 and the vehicle information extracted by the vehicle information extraction unit 16 to the server 201. The reception unit 20 in the server 200 receives the characters and vehicle information transmitted by the transmission unit 14.

The vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 25 (S3). The vehicle information acquired by the vehicle information acquisition unit 21 is, for example, image feature amounts.

The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 and the vehicle information acquired by the vehicle information acquisition unit 21. The matching unit 24 then compares the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit (S7). Thereafter, the matching unit 24 transmits the matching result to the mobile terminal 101 via the transmission unit 22.

The matching result is information such as whether the matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not perfectly match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

The display control unit 12 causes the display unit 13 to display the matching result by the matching unit 24 along with the vehicle information received by the receiving unit 15 from the server 201 (S8).

As described above, the vehicle information acquisition system 1001 of this embodiment can compare the vehicle information of the photographed vehicle with the vehicle information of the registered vehicle and display the comparison results. This allows the user of the mobile terminal 100 to know whether the registered vehicle and the photographed vehicle match in terms other than the license plate, without having to compare them themselves. In other words, the user can more efficiently determine whether the registered vehicle and the photographed vehicle match.

In this embodiment, the vehicle information extraction unit 16 acquires feature amounts from images of the photographed vehicle 1, but the vehicle information acquired from the images is not limited to feature amounts. For example, the vehicle information may be the ratio of the lengths of the vehicle's feature parts. The vehicle's feature parts may be, for example, the vehicle's wheelbase, vehicle height, overall length, etc. The lengths of the vehicle's feature parts may be extracted by the vehicle information extraction unit 16 through image recognition processing. Alternatively, the vehicle information extraction unit 16 may extract the lengths of the vehicle's feature parts from information input by the user via an input unit (not shown). The input unit (not shown) may be a mouse, touch panel, keyboard, etc.

At this time, the vehicle information extraction unit 16 extracts the length ratio of the characteristic parts by dividing the lengths of the two or more extracted characteristic parts. The mobile terminal 101 transmits the length ratio of the extracted characteristic parts to the server 201. The memory unit 25 in the server 201 stores the length ratio of the vehicle's characteristic parts as vehicle information. The vehicle information acquisition unit 21 then acquires the length ratio of the characteristic parts corresponding to the extracted characters from the memory unit 25. The comparison unit 24 compares the length ratio of the characteristic parts extracted by the vehicle information extraction unit 16 with the length ratio of the characteristic parts acquired by the vehicle information acquisition unit 21.

The vehicle information extraction unit 16 may also extract vehicle information item by item. For example, the vehicle information extraction unit 16 may extract the overall vehicle feature amount and the length of the vehicle's feature portion. In this case, the matching unit 24 may match the vehicle features item by item. The matching unit 24 may then calculate an overall degree of match based on the degree of match for each item and output the matching result.

(Third embodiment)

(Configuration of the third embodiment)
An example of the configuration of the third embodiment will be described. Fig. 12 is a diagram showing an example of the configuration of a vehicle information acquisition system 1002. The vehicle information acquisition system 1002 includes an acquisition unit 300, a character extraction unit 400, a vehicle information acquisition unit 500, and a display control unit 600.

The acquisition unit 300 acquires an image that shows at least a portion of the license plate attached to the vehicle. The image may be captured by a camera unit (not shown). The camera unit (not shown) may be, for example, a camera having a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, etc. The camera unit (not shown) generates image data by converting received light into image data. Note that the camera unit (not shown) may be located within the vehicle information acquisition system 1002 or in an external system.

The character extraction unit 400 extracts characters from the license plate portion of the image acquired by the acquisition unit 300. The character extraction unit 400 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 300 to extract characters.

The vehicle information acquisition unit 500 acquires vehicle information corresponding to the extracted characters from a storage unit (not shown) that stores vehicle information of registered vehicles in association with characters. Note that the storage unit (not shown) may exist within the vehicle information acquisition system 1002 or in an external system. The vehicle information may be any information related to the vehicle. For example, the vehicle information may be information displayed on the license plate attached to the vehicle (chassis number), information listed on the vehicle's inspection certificate, an image of the vehicle, features of the image, information about the owner of the vehicle, etc.

The display control unit 600 displays the vehicle information acquired by the vehicle information acquisition unit 500 on a display unit (not shown). Note that the display unit (not shown) may be located within the vehicle information acquisition system 1002 or in an external system.

An example of the display by the display unit is shown in Figure 13. In Figure 13, the display unit 700 displays the vehicle type "passenger car" and the vehicle color "white" as vehicle information for a registered vehicle. However, the vehicle information displayed by the display unit is not limited to the vehicle type and color.

The vehicle information acquisition system 1002 of this embodiment can display vehicle information for registered vehicles determined based on the license plate of the photographed vehicle. In other words, a user of the vehicle information acquisition system 1002 can compare the photographed vehicle with the registered vehicle by comparing the appearance, etc., of the photographed vehicle with the displayed vehicle information of the registered vehicle. In other words, this embodiment can provide a vehicle information acquisition system that can display information that allows comparison between the photographed vehicle and the registered vehicle.

(Hardware configuration)
Next, an example of the hardware configuration of the present invention will be described. Figures 14 and 15 are diagrams showing an example of the hardware configuration of vehicle information acquisition systems 1000 to 1002. A computer 1003 shown in Figure 14 and a mobile terminal shown in Figure 15 are connected to each other wirelessly or via a wire so as to be able to communicate with each other.

Computer 1003 may be any general-purpose computer. For example, computer 1003 may be a notebook PC, desktop PC, server, workstation, blade server, etc. Computer 1003 may also be communicatively connected to a server or other computer, etc.

As shown in FIG. 14, the computer 1003 includes a processor 1402, memory 1404, storage 1406, a northbridge 1408 connected to the memory 1404, and expansion slots 1410. The computer 1003 also includes a southbridge 1412 connected to a communication interface 1414 and storage 1406. These components are connected to each other by a bus and fixed to the motherboard.

Processor 1402 reads and executes instructions stored in memory 1404, storage 1406, etc. Processor 1402 may also execute instructions to display images on a display device (not shown).

The processor 1402 reads and executes programs stored in the memory 1404 and storage 1406. The processor 1402 may be a circuit such as a CPU (Central Processing Unit) or ASICs (Application Specific Integrated Circuits). The processor 1402 also realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24. That is, the processor 1402 realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24 by reading the programs stored in the memory 1404 and storage 1406. The processor 1402 may also realize the functions of the vehicle information acquisition unit 21.

Memory 1404 is a computer-readable storage medium that stores information. Memory 1404 may be a volatile storage medium such as RAM, or a non-volatile storage medium such as ROM. Memory 1404 may also be an optical disk, magnetic disk, etc. Storage 1406 may be a large-capacity computer-readable storage medium. Storage 1406 may be, for example, a floppy disk, hard disk drive, optical disk, flash memory, etc. The computer program stored in the computer-readable storage medium includes instructions that, when executed, realize the above-described embodiments. Memory 1404 also realizes memory unit 26. Memory unit 26 may be memory unit 23 of the first embodiment or memory unit 25 of the second embodiment.

The northbridge 1408 connects the brains of the computer, such as the CPU and memory, and controls the timing and speed of data transfer. The northbridge 1408 mainly mediates data between high-speed devices. On the other hand, the southbridge 1412 mediates data between relatively slower devices. For example, the northbridge 1408 may be connected to the memory 1404, the processor 1402, and an expansion slot 1410 that can accept an expansion card (not shown). The southbridge 1412 may also be connected to the storage 1406 and a communication interface 1414.

The communication interface 1414 may include various communication ports such as USB, Bluetooth (registered trademark), Ethernet (registered trademark), and wireless Ethernet. These communication ports may also be connected to one or more input/output devices, such as communication equipment such as a keyboard, pointing device, scanner, wireless LAN (Local Area Network) adapter, or router. In other words, the computer 1003 may be connected to the mobile terminal 1004 via the communication interface 1414. The communication interface 1414 may also include a wireless LAN adapter. The communication interface 1414 implements the functions of the receiving unit 20 and the transmitting unit 22.

Next, we will explain the mobile terminal 1004 shown in Figure 15. The mobile terminal 1004 may be a mobile terminal such as a mobile phone, smartphone, or PDA (personal digital assistant).

The mobile terminal 1004 may include a display 1454, a transceiver 1468, an audio codec 1460, a control interface 1458, and a display interface 1456. The mobile terminal 1004 may also include a communication interface 1466, a processor 1452, a GPS (Global Positioning System) receiving module 1470, additional memory 1474, memory 1464, and an external interface 1462.

The display 1454 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display), an OLED (Organic Light Emitting Diode) display, or other display. The display may also be connected to the control interface 1458 or the display interface 1456. The processor 1452 may exchange information with the user of the mobile terminal 1004 via the control interface 1458 or the display interface 1456. The display 1454 implements the functions of the display unit 13. The display 1454 may also implement the functions of the display unit 700.

The display interface 1456 may be configured with a circuit. The display interface 1456 controls a display to display images and other information to a user. The display interface 1456 implements the functions of the display control unit 600. The display interface 1456 may also implement the functions of the display control unit 12.

The control interface 1458 receives commands from the user. The control interface 1458 may then convert the received commands for passing to the processor 1452.

The mobile terminal 1004 may also have an external interface 1462 that enables communication between an external device and the processor 1402. The external interface 1462 may provide wired or wireless communication to the mobile terminal 1004.

Expansion memory 1474 connects to mobile terminal 1004 via expansion interface 1472. Expansion memory 1474 may also include, for example, a SIMM (Single In Line Memory Module) card interface. Expansion memory 1474 may also provide additional storage space for mobile terminal 1004 and store applications and other information. Expansion memory 1474 may also store security-related information. Expansion memory 1474 may function as a security management device for mobile terminal 1004. For example, expansion memory 1474 is programmed with instructions that ensure safe use of mobile terminal 1004.

The mobile terminal 1004 performs wireless communication via the communication interface 1466. In other words, the mobile terminal 1004 may communicate with the computer 1003 via the communication interface 1466. The communication interface 1466 realizes the functions of the transmitter 14 and receiver 15.

The communication interface 1466 may include a digital signal processing circuit. The communication interface 1466 may realize communication under modes and protocols such as GMS (Global System for Mobile Communication), SMS (Short Message Service), and EMS (Enhanced Messaging Service). Alternatively, the communication interface 1466 may realize communication under modes and protocols such as MMS (Multimedia Messaging Service), CDMA (Code Division Multiple Access), and TDMA (Time Division Multiple Access).

Furthermore, the communication interface 1466 may realize communication under modes and protocols such as PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, and GPRS (General packet radio service). The above-mentioned communication occurs, for example, via the transceiver 1468. Short-range communication, such as communication using Bluetooth or Wi-Fi (registered trademark), may also occur via the transceiver 1468.

The GPS receiver module 1470 may provide data for use by applications running on the mobile device 1004. This data may be, for example, location- or navigation-related data.

The mobile terminal 1004 may also perform voice communication via the audio codec 1460. The audio codec 1460 acquires voice information from the user and converts it into usable digital information. Furthermore, the audio codec 1460 generates audio that the user can hear. The information handled by the audio codec 1460 may include voice calls, recorded sounds (voice messages, music files, etc.), sounds generated by applications on the mobile terminal 1004, etc.

The processor 1452 and memory 1464 are similar to those of the computer 1003, so a detailed description will be omitted. The processor 1452 implements the functions of the acquisition unit 800, character extraction unit 400, and vehicle information extraction unit 16.

The vehicle information acquisition systems 1000 to 1002 may be realized by either the computer 1003 or the mobile terminal 1004. In this case, for example, the processor 1402 in the computer 1003 may also realize the functions of the acquisition unit 800, character extraction unit 400, vehicle information extraction unit 16, and display control unit 600. Furthermore, for example, the processor 1452 may also realize the functions of the vehicle information acquisition unit 21 and the matching unit 24.

The above-described embodiments are merely examples of the present invention, and various modifications may be made within the scope of the spirit of the present invention as set forth in the claims.

REFERENCE SIGNS LIST 1 Vehicle 2 Image 3 Image 4 Image 5 License plate 6 Image 10 Acquisition unit 11 Character extraction unit 12 Display control unit 13 Display unit 14 Transmission unit 15 Reception unit 16 Vehicle information extraction unit 20 Reception unit 21 Vehicle information acquisition unit 22 Transmission unit 23 Memory unit 24 Collation unit 25 Memory unit 26 Memory unit 100 Mobile terminal 101 Mobile terminal 200 Server 201 Server 300 Acquisition unit 400 Character extraction unit 500 Vehicle information acquisition unit 600 Display control unit 700 Display unit 1000 Vehicle information acquisition system 1001 Vehicle information acquisition system 1002 Vehicle information acquisition system 1003 Computer 1004 Mobile terminal 1402 Processor 1404 Memory 1406 Storage 1408 North bridge 1410 Expansion slot 1412 South bridge 1414 Communication interface 1452 Processor 1454 Display 1456 Display interface 1458 Control interface 1460 Audio codec 1466 Communication interface 1468 Transceiver 1462 External interface 1464 Memory 1470 GPS receiver module 1472 Expansion interface 1474 Expansion memory

Claims (9)

an acquisition unit that acquires an image of a vehicle taken by a terminal;
an extraction unit that extracts a character string from an area of the license plate included in the image;
a vehicle information acquisition unit that acquires information on a registered vehicle associated with the character string;
an output unit that outputs information about the registered vehicle when the character string extracted from the image is any one of a plurality of pieces of pre-stored registration information, and outputs information indicating that the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information when the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information;
Equipped with
the vehicle information acquisition unit acquires, from a storage unit, information about the registered vehicle and a comparison image corresponding to the information about the registered vehicle;
The output unit outputs the comparison image together with the information on the registered vehicle to a display device.
Information processing device. the output unit outputs the character string extracted from the image together with the image to a display device.
The information processing device according to claim 1 . When the character string extracted from the image is not a pre-stored character string, the output unit outputs, to the terminal, information indicating that the character string extracted from the image is not a pre-stored character string.
3. The information processing device according to claim 1. the output unit outputs, to a display device, information about the registered vehicle that differs depending on the character string extracted from the image.
The information processing device according to claim 1 . a storage unit that stores information about the registered vehicle in association with a character string on the license plate;
the vehicle information acquisition unit acquires information about the registered vehicle from the storage unit based on the character string extracted from the area of the license plate;
The information processing device according to claim 1 . The character string extracted from the image includes characters indicating the transport bureau, type, business type, and designated number.
The information processing device according to claim 1 . a vehicle information extraction unit that extracts feature amounts of a portion related to the vehicle from the image;
a matching unit that matches the extracted feature amount with a feature amount of an image related to the registered vehicle stored in a storage unit,
The output unit further outputs the collation result by the collation unit to a display device.
The information processing device according to claim 1 . The computer
The terminal acquires an image of the vehicle,
Extracting a character string from an area of the license plate included in the image;
Acquires information about a registered vehicle associated with the character string and a comparison image corresponding to the information about the registered vehicle from a storage unit ;
If the character string extracted from the image is any one of a plurality of pieces of pre-stored registration information, information relating to the registered vehicle and the comparison image are output to a display device , and if the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information, information indicating that the character string extracted from the image is not the pre-stored registration information is output.
Information processing methods. On the computer,
A process of acquiring an image of the vehicle by the terminal;
A process of extracting a character string from an area of the license plate included in the image;
A process of acquiring, from a storage unit , information on a registered vehicle associated with the character string and a comparison image corresponding to the information on the registered vehicle ;
a process of outputting information about the registered vehicle and the comparison image to a display device based on the character string when the character string extracted from the image is one of a plurality of pieces of pre-stored registration information, and outputting information indicating that the character string extracted from the image is not one of the plurality of pieces of pre-stored registration information when the character string extracted from the image is not one of the plurality of pieces of pre-stored registration information;
An information processing program that executes the above.