JP7613617B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to a vehicle information acquisition system for acquiring vehicle information, a vehicle information acquisition method, a vehicle information acquisition program, and a storage medium for storing a vehicle information acquisition program.

This article describes an example of a system that reads the number on a vehicle's license plate and uses the characters that are read.

Patent Document 1 describes a system that extracts characters from an image of a vehicle's license plate and searches a database (DB). The system displays the data that is found as a result of the search on a display device. Patent Document 2 describes a system that reads in vehicle numbers and the like and temporarily stores them. Furthermore, when the system receives an instruction to end the continuous reading, it sequentially searches the vehicle DB for the temporarily stored numbers. The system then displays the search results on a display.

Japanese Patent Application Publication No. 11-296785 JP 2000-322421 A

However, although the above-mentioned system can display the search results for license plates, it has the problem that it cannot display information that allows a comparison between the vehicle photographed and the registered vehicle. Therefore, a user who uses the above-mentioned system cannot know if the license plate is counterfeit or the like. Therefore, the object of the present invention is to provide a vehicle information acquisition system that solves the problem of not being able to display information that allows a comparison between the vehicle photographed and the registered vehicle.

The first system of the present invention includes an acquisition unit that acquires an image showing at least a portion of a license plate attached to a vehicle, a character extraction unit that extracts characters from the license plate portion of the image, a vehicle information acquisition unit that acquires vehicle information corresponding to the extracted characters from a storage unit that stores vehicle information of registered vehicles in association with characters, and a display control unit that displays the acquired vehicle information on a display unit.

The present invention provides a vehicle information acquisition system that can display information that allows comparison between a photographed vehicle and a registered vehicle.

FIG. 1 is a block diagram showing a configuration example of a first embodiment. FIG. 1 is an explanatory diagram for explaining photographing of a license plate. 11 is an example of information stored in the storage unit 23. 11 is an example of information stored in the storage unit 23. 13 is an example of a display by the display unit 13. 4 is a flowchart showing an operation example of the first embodiment. FIG. 13 is a block diagram showing a configuration example of a second embodiment. FIG. 2 is an explanatory diagram for explaining photographing of a vehicle. 11 is an example of information stored in the storage unit 25. 13 is an example of a display by the display unit 13. 10 is a flowchart showing an operation example of the second embodiment. FIG. 13 is a block diagram showing a configuration example of a third embodiment. 13 is an example of a display by a display unit. 2 is an example of a hardware configuration. 2 is an example of a hardware configuration.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples and are not intended to limit the technical scope of the present invention.
First Embodiment
(Configuration of the First Embodiment)
A configuration example of the first embodiment will be described. FIG. 1 is a diagram showing a configuration example of a vehicle information acquisition system 1000. In the first embodiment, it is assumed that the present invention is used in a special area where entry of vehicles is restricted when an event or the like is held. Vehicles other than those of event related parties and local residents are prohibited from entering the special area. Therefore, a person guarding the special area patrols the special area and confirms that the vehicles parked in the special area are vehicles of related parties registered in advance (hereinafter referred to as registered vehicles). Alternatively, a person guarding the special area checks the vehicles at a checkpoint set up near the border between the special area and other areas. At this time, a person guarding the special area checks whether the characters displayed on the license plate attached to the vehicle are those of a registered vehicle.

Number plates are also called license plates. The characters displayed on number plates are also called vehicle registration numbers or vehicle numbers. The characters include numbers and symbols.

Here, a license plate is an automobile registration plate, vehicle number plate, or sign that is required to be attached to vehicles under traffic laws and other regulations. Normally, license plates display characters that are unique to each vehicle. Therefore, if the characters on the license plate are the same as pre-registered characters, the vehicle is authorized to enter the special area. However, someone attempting to illegally enter a special area may forge a license plate. In this case, the person guarding the special area may determine that a vehicle that is not authorized to enter is a registered vehicle.

The vehicle information acquisition system 1000 of the first embodiment therefore adopts the following configuration. In the following, the user of the vehicle information acquisition system 1000 and the user of the mobile terminal 100 will be described as guarding a special area.

The vehicle information acquisition system 1000 includes a mobile terminal 100 and a server 200. The mobile terminal 100 is communicatively connected to the server 200 via a wireless or wired network. The mobile terminal 100 includes an acquisition unit 10, a character extraction unit 11, a display control unit 12, a transmission unit 14, a reception unit 15, and a display unit 13. The server 200 includes a reception unit 20, a vehicle information acquisition unit 21, a transmission unit 22, and a storage unit 23.

The acquisition unit 10 in the mobile terminal 100 acquires an image that shows at least a portion of the license plate attached to the vehicle. The image may be captured by a capture unit (not shown) included in the mobile terminal 100. The capture unit (not shown) may be, for example, a camera having a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, or the like. The capture unit (not shown) generates image data by converting the received light into image data.

Figure 2 is an explanatory diagram explaining the photographing of a license plate by a photographing unit in a mobile terminal 100. The user of the mobile terminal 100 operates the mobile terminal 100 with the photographing unit (not shown) facing the photographing vehicle 1 to photograph the license plate attached to the photographing vehicle 1. The acquisition unit 10 acquires the image generated by the photographing unit (not shown). It is sufficient that the image acquired by the acquisition unit 10 shows at least a part of the license plate. Images 2 and 3 are examples of images acquired by the acquisition unit 10. Image 2 shows the entire license plate 5 attached to the photographing vehicle 1. Meanwhile, image 3 shows only a part of the license plate 5.

The character extraction unit 11 extracts characters from the license plate portion in the image acquired by the acquisition unit 10. The character extraction unit 11 may identify the license plate portion in the image by pattern recognition or the like. The character extraction unit 11 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 10 to extract characters.

For example, when the character extraction unit 11 acquires image 2 in FIG. 2, it extracts the characters "Shinagawa," "300," "sa," and "12-34." The character extraction unit also determines which item the extracted characters correspond to based on the position data of the extracted characters and the position data of the items stored in advance. The item position data may be stored in the storage unit 23. The position indicated by the position data may be an absolute position in the image, or may be the relative position of each character or item. An item indicates, for example, the content of information that is required to be displayed on a vehicle by traffic laws. For example, items include "Transportation Bureau, etc.", "Type," "Business Use," and "Designated Number." The character extraction unit 11 does not have to determine the item that corresponds to the characters.

The transmitting unit 14 in the mobile terminal 100 transmits the characters extracted by the character extracting unit 11 and the items corresponding to the characters to the server 200. The receiving unit 20 in the server 200 receives the characters and items transmitted by the transmitting unit 14.

The memory unit 23 in the server 200 stores the vehicle information of the registered vehicle in association with the characters.

Here, vehicle information may be any information related to a vehicle. For example, vehicle information may be information written on the vehicle inspection certificate, such as the chassis number, model, body shape, and owner information, information displayed on the license plate attached to the vehicle, an image of the vehicle, and features of the image. An example of information stored in the memory unit 23 is shown in FIG. 3. The characters are divided into the categories of "Transportation Bureau, etc.", "Type," "Business Use," and "Designated Number." The characters that make up the contents of these categories are required to be displayed on license plates under traffic laws. In addition, the vehicle information includes "Model," "Vehicle Name," "Manufacturer," "Year," and "Color." Note that the information stored in the memory unit 23 is not limited to that shown in FIG. 3.

From Figure 3, the characters "Shinagawa", "300", "sa", and "12-34" correspond to the vehicle information "Standard passenger car", "Purulis", "Toyoda", "H23", and "White". In addition, the characters "Shinagawa", "350", "ha", and "34-56" correspond to the vehicle information "Standard passenger car", "Ellie", "H25", and "Red". The characters "Kashiwa", "550", "mi", and "56-78" correspond to the vehicle information "Compact passenger car", "Aldo", "Susuki", "H20", and "Black".

Furthermore, the memory unit 23 stores the vehicle information in association with the comparison image. An example of the information stored in the memory unit 23 is shown in FIG. 4. In FIG. 4, the vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "White" corresponds to the comparison image "aaa.jpg". Furthermore, the vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "Black" corresponds to the comparison image "bbb.jpg". The vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "Gray" corresponds to the comparison image "ccc.jpg".

The image file format is not limited to JPEG (Joint Photographic Experts Group). For example, the image file format may be GIF (Graphics Interchange Format), PNG (Portable Network Graphics), BMP (Microsoft Windows (registered trademark) Bitmap Image), etc.

The comparison image includes an image of a vehicle that satisfies the vehicle information. In other words, the comparison image includes an image of a vehicle whose vehicle information, such as model, name, manufacturer, year, and color, matches that of a registered vehicle. For example, the comparison image "aaa.jpg" includes an image of a vehicle whose model is a passenger car, whose name is Purlis, whose manufacturer is Toyoda, whose year is H23, and whose color is white. The comparison image may be an image of a vehicle published in a catalog, a product introduction site, or the like. The comparison image may also be an image of a registered vehicle.

The vehicle information acquisition unit 21 receives the characters from the receiving unit 200. Then, the vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 23. That is, the vehicle information acquisition unit 21 acquires vehicle information corresponding to characters that match the extracted characters from the storage unit 23. For example, assume that the extracted characters are "Shinagawa", "300", "sa", and "12-34", and the storage unit 23 stores the information shown in FIG. 3. In this case, the vehicle information acquisition unit 21 refers to the storage unit 23 and acquires the vehicle information "Standard passenger car", "Purulis", "Toyoda", "H23", and "White" that corresponds to the characters "Shinagawa", "300", "sa", and "12-34".

At this time, the vehicle information acquisition unit 21 may check for each item whether the extracted characters match the characters stored in the memory unit 23. For example, the vehicle information acquisition unit 21 determines whether the extracted characters corresponding to the item "Transportation Bureau, etc." match the characters stored in the memory unit 23 corresponding to the item "Transportation Bureau, etc." Similarly, the vehicle information acquisition unit 21 determines whether the characters associated with the items "Type," "Business use," and "Designated number" match for each item. Then, if the characters match for all items, the vehicle information corresponding to the extracted characters may be acquired from the memory unit 23.

The vehicle information acquisition unit 21 also acquires items corresponding to the vehicle information from the storage unit 23. For example, the vehicle information acquisition unit 21 associates "vehicle model" with "normal passenger car," "vehicle name" with "Purlis," "manufacturer" with "Toyoda," "year" with "H23," and "color" with "white," and acquires these. Note that the vehicle information acquisition unit 21 does not have to acquire items from the storage unit 23.

Furthermore, the vehicle information acquisition unit 21 acquires, together with the vehicle information, a comparison image corresponding to the vehicle information from the storage unit 23. That is, the vehicle information acquisition unit 21 acquires, from the storage unit 23, a comparison image corresponding to the vehicle information acquired by the storage unit 23. For example, in FIG. 4, when the vehicle information acquisition unit 21 acquires "Standard passenger car", "Purlis", "Toyoda", "H23", and "White" as the vehicle information, it acquires the comparison image "aaa.jpg" from the storage unit 23. The transmission unit 22 in the server 200 transmits the vehicle information, items, and comparison image acquired by the vehicle information acquisition unit 21 to the mobile terminal 100.

The receiving unit 15 in the mobile terminal 100 receives the vehicle information, items, and comparison images transmitted from the server 200. The display control unit 12 causes the display unit 13 to display the vehicle information and items acquired by the vehicle information acquisition unit 21. At this time, the display control unit 12 causes the display unit 13 to display the comparison images together with the vehicle information. Note that the display control unit 12 does not have to display the items on the display unit 13.

The display unit 13 displays vehicle information, items, and a comparison image under the control of the display control unit 12. An example of the display by the display unit 13 is shown in FIG. 5. In FIG. 5, the display unit 13 of the mobile terminal 100 displays the comparison image "aaa.jpg", vehicle information, and items. As described above, the vehicle shown in the comparison image "aaa.jpg" is a standard passenger car by type, a Purlis by name, a Toyoda by manufacturer, a H23 model year, and a white color. The vehicle information displayed by the display unit 13 in FIG. 5 is "Standard passenger car", "Purlis", "Toyoda", "H23", and "White". The items displayed by the display unit 13 are "Model", "Vehicle name", "Manufacturer", "Model year", and "Color".

The display by the display unit 13 is not limited to that shown in Fig. 4. For example, the display unit 13 may not display items. The display unit 13 may also display the image acquired by the acquisition unit 10 (an image of the photographed vehicle 1) together with the above display. The display unit 13 may also display the characters extracted by the character extraction unit 11, i.e., the characters displayed on the license plate of the photographed vehicle 1.
(Operation of the First Embodiment)
Next, a description will be given of the operation of the vehicle information acquisition system 1000. FIG.

First, the acquisition unit 10 in the mobile terminal 100 acquires an image that shows at least a part of the license plate attached to the vehicle (S1). The image may be captured by a photographing unit (not shown) included in the mobile terminal 100. The photographing unit (not shown) may be, for example, a camera having a CCD sensor, a CMOS sensor, an image processor, etc. The photographing unit (not shown) generates an image by converting the received light into data.

For example, the acquisition unit 10 may acquire an image after the user of the mobile terminal 100 starts a specific application. For example, when the user starts the application, a photographing unit (not shown) photographs the license plate. Then, the acquisition unit 10 acquires the image from the photographing unit (not shown).

Next, the character extraction unit 11 extracts characters from the license plate portion in the image acquired by the acquisition unit 10 (S2). The character extraction unit 11 performs OCR processing on the image acquired from the acquisition unit 10 to extract characters. The character extraction unit 11 also determines the item that corresponds to the extracted characters. The transmission unit 14 then transmits the characters and items extracted by the character extraction unit 11 to the server 200.

The receiving unit 20 in the server 200 receives the characters and items transmitted by the transmitting unit 14. The vehicle information acquiring unit 21 receives the characters and items from the receiving unit 20. Then, the vehicle information acquiring unit 21 acquires vehicle information corresponding to the extracted characters from the memory unit 23 (S3). At this time, the vehicle information acquiring unit 21 acquires items corresponding to the vehicle information. Furthermore, the vehicle information acquiring unit 21 acquires a comparison image corresponding to the acquired vehicle information from the memory unit 23 (S4). Note that the vehicle information acquiring unit 21 does not have to acquire items from the memory unit 23.

The transmitting unit 22 in the server 200 transmits the vehicle information, items, and comparison images acquired by the vehicle information acquiring unit 21 to the mobile terminal 100. The receiving unit 15 in the mobile terminal 100 then receives the vehicle information, items, and comparison images transmitted from the server 200.

The display control unit 12 causes the display unit 13 to display the comparison image together with the vehicle information and items received by the receiving unit 15. Note that the display control unit 12 does not have to cause the display unit 13 to display the items.

As described above, the vehicle information acquisition system 1000 of this embodiment can display vehicle information for registered vehicles whose license plate characters match those of the photographed vehicle. Furthermore, the vehicle information acquisition system 100 can display an image of a vehicle that meets the vehicle information. This allows a user of the mobile terminal 100 to easily grasp the appearance of a vehicle named, for example, "Purulis" by looking at the displayed image, even if the user does not know what the vehicle looks like. Therefore, by comparing the image displayed by the mobile terminal 100 with the appearance, etc., of the photographed vehicle, the user can efficiently determine whether the photographed vehicle matches the registered vehicle.

The server 200 may also have a character extraction unit 11. In this case, the transmission unit 14 in the mobile terminal 100 transmits the image acquired by the acquisition unit 10 to the server 200. The character extraction unit 11 in the server 200 then extracts characters from the license plate portion in the image received from the mobile terminal 100, and passes the extracted characters to the vehicle information acquisition unit 21.

In addition, the memory unit 23 has separate data associating characters with vehicle information and data associating vehicle information with a comparison image, but it may have one data associating characters, vehicle information, and a comparison image. In this case, the vehicle information acquisition unit 21 can acquire vehicle information corresponding to characters and a comparison image at the same time, thereby improving the processing speed of the vehicle information acquisition system 1000.

Furthermore, the vehicle information acquisition unit 21 may calculate the similarity between the extracted characters and the characters stored in the memory unit 23, and acquire the vehicle information corresponding to the character with the highest similarity. The vehicle information acquisition unit 21 may also acquire all vehicle information corresponding to characters whose similarity is equal to or greater than a threshold. In this case, the display control unit 12 may cause all vehicle information acquired by the vehicle information acquisition unit 21 to be displayed on the display unit 23. The display control unit 12 may cause the vehicle information to be displayed on the display unit 23 in order of similarity. This allows the user to compare the registered vehicle with the photographed vehicle even when the characters displayed on the license plate in the image are unclear or when some of the characters are not photographed.

In addition, when the storage unit 23 does not store characters whose similarity to the extracted characters is equal to or greater than the threshold value, or characters identical to the extracted characters, the vehicle information acquisition unit 21 may transmit that fact to the mobile terminal 100 via the transmission unit 22. Thereafter, the display control unit 12 may display a message on the display unit 13 indicating that the photographed vehicle is not registered. This allows the user to know that the photographed vehicle is not a registered vehicle. Alternatively, the display control unit 12 may display a message on the display unit 13 encouraging the user to photograph the vehicle again. This allows the user to photograph the same vehicle again and check whether it is registered. These messages may be stored in advance in a storage unit (not shown) in the mobile terminal 100, or may be transmitted to the mobile terminal 100 by the server 200.

Furthermore, the memory unit 23 may store information about the owner of the registered vehicle in association with the characters. For example, the memory unit 23 may register a facial image of the owner of the registered vehicle in association with the characters. In this case, the vehicle information acquisition unit 21 may acquire the facial image together with the vehicle information. Furthermore, the display control unit 12 may cause the display unit 13 to display the facial image together with the vehicle information. In this way, even if the vehicle with a forged license plate is the same model, name, color, etc. as the registered vehicle, the user can check the owner of the photographed vehicle and determine whether the photographed vehicle matches the registered vehicle.

In addition, in this embodiment, the server 200 has a memory unit 23, but the memory unit in the present invention may be realized by a plurality of hardware configurations. For example, a storage device in the server 200 may store data associating characters with vehicle information, and a storage device in another server that can communicate with the server 200 may store data associating vehicle information with a comparison image.

In Japan, the information displayed on the license plate is regulated by the Road Transport Vehicle Act, but may differ in other countries. Therefore, the items of information displayed on the license plate may also differ depending on the country. Taking these points into consideration, the contents displayed on the license plate are not limited to those described in this embodiment.
Second Embodiment
(Configuration of the second embodiment)
A configuration example of the second embodiment will be described. Fig. 7 is a diagram showing a configuration example of a vehicle information acquisition system 1001. The vehicle information acquisition system 1001 differs from the vehicle information acquisition system 1000 in that it includes a vehicle information extraction unit 16, a collation unit 24, and a storage unit 25. The rest of the configuration is the same as in the first embodiment, so the description will be omitted as appropriate.

The acquisition unit 10 acquires an image that includes at least a portion of the vehicle in addition to an image that includes at least a portion of the license plate. The image that includes at least a portion of the vehicle may be captured by a capture unit (not shown) included in the mobile terminal 100.

Figure 8 is an explanatory diagram for explaining the photographing of the photographed vehicle 1 by the photographing unit in the mobile terminal 101. The user of the mobile terminal 101 operates the mobile terminal 101 with the photographing unit (not shown) facing the photographed vehicle 1 to photograph the photographed vehicle 1. The acquisition unit 10 then acquires the image generated by the photographing unit (not shown). The image acquired by the acquisition unit 10 needs to show at least a part of the photographed vehicle 1. However, it is preferable that the image shows parts of the vehicle with many projections and recesses and parts that differ depending on the vehicle model. For example, it is preferable that the image shows the vehicle's headlights, grille, bumper, emblem, ornament, wheels, etc.

Images 4 and 6 are examples of images acquired by the acquisition unit 10. Image 4 shows the entire photographing vehicle 1. On the other hand, image 6 shows only a part of the photographing vehicle 1.

The vehicle information extraction unit 16 extracts vehicle information of the vehicle from an image in which at least a portion of the vehicle is shown. In this embodiment, the vehicle information is, for example, a feature of the image. The vehicle information extraction unit 16 extracts the feature from the vehicle portion in the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion in the image by pattern recognition or the like. The extraction of the feature may be realized by a method such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Here, for example, it is assumed that the vehicle information extraction unit 16 extracts feature data X from the vehicle portion in the image.

The transmission unit 14 transmits the characters "Shinagawa", "300", "sa", and "12-34" extracted by the character extraction unit 11 and the feature data X extracted by the vehicle information extraction unit 16 to the server 201.

The memory unit 25 in the server 201 stores vehicle information of the registered vehicle in association with characters displayed on the license plate of the registered vehicle. In addition to the information stored in the memory unit 23, the memory unit 25 stores image features as vehicle information. The image is an actual image of the registered vehicle. Note that the image may not be an image of the registered vehicle, but may be an image of a vehicle having the same model, name, color, etc. as the registered vehicle. For example, the image may be an image of a vehicle published in a catalog, a product introduction site, etc.

An example of data stored in the memory unit 25 is shown in FIG. 9. As can be seen from FIG. 9, the characters "Shinagawa", "300", "sa", and "12-34" correspond to feature amount data X. Furthermore, the characters "Shinagawa", "350", "ha", and "34-56" correspond to feature amount data Y. The characters "Kashiwa", "550", "mi", and "56-78" correspond to feature amount data Z. Vehicle information other than the features is the same as in FIG. 3, so a description thereof will be omitted.

The vehicle information acquisition unit 21 acquires from the memory unit 25 the vehicle features corresponding to the characters "Shinagawa", "300", "sa", and "12-34" received by the receiving unit 20. As can be seen from FIG. 9, the vehicle feature corresponding to the characters "Shinagawa", "300", "sa", and "12-34" is feature amount data X, so the vehicle information acquisition unit 21 acquires the feature amount data X, which is the vehicle feature. Furthermore, the vehicle information acquisition unit 21 acquires "Standard passenger car", "Purulis", "Toyoda", "H23", and "White", which are the vehicle information corresponding to the characters "Shinagawa", "300", "sa", and "12-34".

The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 from the receiving unit 20. The matching unit 24 also acquires the vehicle information acquired by the vehicle information acquisition unit 21. The matching unit 24 then compares the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit 21.

Then, the matching unit 24 transmits the matching result to the mobile terminal 101 via the transmission unit 22. The matching result is, for example, information on whether the matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not completely match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

For example, in the above example, the matching unit 24 acquires feature amount data X of an image of the photographed vehicle 1. Furthermore, the matching unit 24 acquires feature amount data X of an image of the registered vehicle. Then, the matching unit 24 compares these acquired feature amounts and transmits the matching result to the transmission unit 22.

The transmission unit 22 transmits the vehicle information acquired by the vehicle information acquisition unit 21 and the matching result by the matching unit 24 to the mobile terminal 100. For example, the transmission unit 22 transmits the characters "Standard passenger car", "Purulis", "Toyoda", "H23", and "White" acquired by the vehicle information acquisition unit 21 and the matching result. The transmission unit 22 may also transmit the feature amount acquired by the vehicle information acquisition unit 21.

The display control unit 12 causes the display unit 13 to display the matching result by the matching unit 24 together with the vehicle information received by the receiving unit 15 from the server 201. An example of a display by the display unit 13 is shown in Fig. 10. As shown in Fig. 10, the display unit 13 displays the matching result "Matching successful" together with the characters "Standard passenger car", "Purulis", "Toyoda", "H23", and "White".
(Operation of the second embodiment)
Next, a description will be given of the operation of the vehicle information acquisition system 1001. Fig. 11 is a sequence diagram for explaining the operation of the vehicle information acquisition system 1001. Descriptions of operations similar to those of the vehicle information acquisition system 1000 shown in Fig. 10 will be omitted as appropriate.

The acquisition unit 10 in the mobile terminal 100 acquires an image showing at least a part of a vehicle after characters are extracted from the image showing at least a part of the license plate (S2). The image may be taken by a photographing unit (not shown) of the mobile terminal 100, similar to the image acquired in S1. Note that if at least a part of a vehicle is shown in the image acquired in S1, the acquisition unit 10 does not need to acquire an image again. Furthermore, the acquisition unit 10 may acquire an image showing at least a part of a vehicle before acquiring an image showing at least a part of a license plate (S1).

Then, the vehicle information extraction unit 16 extracts vehicle information of the vehicle from the image in which at least a portion of the vehicle is shown (S6). The vehicle information in this embodiment is, for example, image features. The vehicle information extraction unit 16 extracts features from the vehicle portion in the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion in the image by pattern recognition or the like. The extraction of features may be achieved by a method such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Note that the order of S1, S2, and S6 is not limited to this.

The transmission unit 14 transmits the characters extracted by the character extraction unit 11 and the vehicle information extracted by the vehicle information extraction unit 16 to the server 201. The reception unit 200 in the server 200 receives the characters and vehicle information transmitted by the transmission unit 14.

The vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 25 (S3). The vehicle information acquired by the vehicle information acquisition unit 21 is, for example, image feature amounts.

The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 and the vehicle information acquired by the vehicle information acquisition unit 21. Then, the matching unit 24 compares the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit (S7). After that, the matching unit 24 transmits the matching result to the mobile terminal 101 via the transmission unit 22.

The matching result is, for example, information on whether matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not completely match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

The display control unit 12 causes the display unit 13 to display the matching result by the matching unit 24 together with the vehicle information received by the receiving unit 15 from the server 201 (S8).

As described above, the vehicle information acquisition system 1001 of this embodiment can compare the vehicle information of the photographed vehicle with the vehicle information of the registered vehicle and display the comparison result. This allows the user of the mobile terminal 100 to know whether the registered vehicle and the photographed vehicle match in terms other than the license plate without having to compare them by themselves. In other words, the user can more efficiently determine whether the registered vehicle and the photographed vehicle match.

In the present embodiment, the vehicle information extraction unit 16 acquires feature amounts from an image of the photographed vehicle 1, but the vehicle information acquired from the image is not limited to feature amounts. For example, the vehicle information may be the ratio of the length of a characteristic part of the vehicle. The characteristic part of the vehicle may be, for example, the wheelbase, vehicle height, or overall length of the vehicle. The length of the characteristic part of the vehicle may be extracted by the vehicle information extraction unit 16 through image recognition processing. Alternatively, the vehicle information extraction unit 16 may extract the length of the characteristic part of the vehicle from information input by a user via an input unit (not shown). The input unit (not shown) may be a mouse, a touch panel, a keyboard, or the like.

At this time, the vehicle information extraction unit 16 extracts the length ratio of the characteristic parts by dividing the lengths of the two or more extracted characteristic parts. The mobile terminal 101 transmits the length ratio of the extracted characteristic parts to the server 201. The memory unit 25 in the server 201 stores the length ratio of the characteristic parts of the vehicle as vehicle information. The vehicle information acquisition unit 21 then acquires the length ratio of the characteristic parts corresponding to the extracted characters from the memory unit 25. The matching unit 25 matches the length ratio of the characteristic parts extracted by the vehicle information extraction unit 16 with the length ratio of the characteristic parts acquired by the vehicle information acquisition unit 21.

Furthermore, the vehicle information extraction unit 16 may extract vehicle information for each item. For example, the vehicle information extraction unit 16 may extract the feature amount of the entire vehicle and the length of the characteristic part of the vehicle. In this case, the matching unit 24 may match the vehicle features for each item. Then, the matching unit 24 may calculate an overall degree of match based on the degree of match for each item, and output the matching result.
Third Embodiment
(Configuration of the third embodiment)
12 is a diagram showing an example of the configuration of a vehicle information acquisition system 1002. The vehicle information acquisition system 1002 includes an acquisition unit 300, a character extraction unit 400, a vehicle information acquisition unit 500, and a display control unit 600.

The acquisition unit 300 acquires an image that shows at least a portion of the license plate attached to the vehicle. The image may be captured by a capture unit (not shown). The capture unit (not shown) may be, for example, a camera having a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, etc. The capture unit (not shown) generates image data by converting the received light into image data. The capture unit (not shown) may be present within the vehicle information acquisition system 1002 or may be present in an external system.

The character extraction unit 400 extracts characters from the license plate portion of the image acquired by the acquisition unit 300. The character extraction unit 400 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 300 to extract characters.

The vehicle information acquisition unit 500 acquires vehicle information corresponding to the extracted characters from a storage unit (not shown) that stores vehicle information of registered vehicles in association with characters. The storage unit (not shown) may exist within the vehicle information acquisition system 1002 or may exist in an external system. The vehicle information may be any information related to the vehicle. For example, the vehicle information may be information displayed on the license plate attached to the vehicle (chassis number), information recorded on the vehicle inspection certificate of the vehicle, an image of the vehicle, features of the image, information related to the owner of the vehicle, etc.

The display control unit 600 displays the vehicle information acquired by the vehicle information acquisition unit 500 on a display unit (not shown). Note that the display unit (not shown) may be present within the vehicle information acquisition system 1002 or may be present in an external system.

An example of the display by the display unit is shown in FIG. 13. In FIG. 13, the display unit 700 displays the vehicle type "ordinary passenger car" and the vehicle color "white" as vehicle information of a registered vehicle. However, the vehicle information displayed by the display unit is not limited to the vehicle type and vehicle color.

The vehicle information acquisition system 1003 of this embodiment can display vehicle information for a registered vehicle determined based on the license plate of the photographed vehicle. That is, a user of the vehicle information acquisition system 1003 can compare the photographed vehicle with a registered vehicle by comparing the appearance, etc., of the photographed vehicle with the displayed vehicle information of the registered vehicle. In other words, this embodiment can provide a vehicle information acquisition system that can display information that allows a comparison between the photographed vehicle and the registered vehicle.
(Hardware configuration)
Next, an example of the hardware configuration of the present invention will be described. Fig. 14 and Fig. 15 are diagrams showing an example of the hardware configuration of the vehicle information acquisition systems 1000 to 1002. The computer 1003 shown in Fig. 14 and the mobile terminal shown in Fig. 15 are connected to each other wirelessly or via a wire so as to be able to communicate with each other.

The computer 1003 may be any general-purpose computer. For example, the computer 1003 may be a notebook PC, a desktop PC, a server, a workstation, a blade server, etc. The computer 1003 may also be communicatively connected to a server or other computers, etc.

As shown in FIG. 14, the computer 1003 includes a processor 1402, a memory 1404, a storage 1406, a north bridge 1408 connected to the memory 1404, and an expansion slot 1410. The computer 1003 further includes a south bridge 1412 connected to a communication interface 1414 and the storage 1406. These components are connected to each other by a bus and fixed to the motherboard.

The processor 1402 reads and executes instructions stored in the memory 1404, storage 1406, etc. The processor 1402 may execute instructions to display an image on a display device (not shown).

The processor 1402 reads and executes programs stored in the memory 1404 and storage 1406. The processor 1402 may be a circuit such as a CPU (Central Processing Unit) or ASICs (Application Specific Integrated Circuits). The processor 1402 also realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24. That is, the processor 1402 realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24 by reading out programs stored in the memory 1404 and storage 1406. The processor 1402 may also realize the functions of the vehicle information acquisition unit 21.

The memory 1404 is a computer-readable storage medium that stores information. The memory 1404 may be a volatile storage medium such as a RAM, or a non-volatile storage medium such as a ROM. The memory 1404 may also be an optical disk, a magnetic disk, or the like. The storage 1406 may be a large-capacity computer-readable storage medium. The storage 1406 may be, for example, a floppy disk, a hard disk drive, an optical disk, a flash memory, or the like. The computer program stored in the computer-readable storage medium includes instructions that are executed to realize the above-mentioned embodiment. The memory 1404 also realizes the storage unit 26. The storage unit 26 may be the storage unit 23 of the first embodiment or the storage unit 25 of the second embodiment.

The northbridge 1408 connects the brains of the computer, such as the CPU and memory, and controls the timing and speed of data transfer. The northbridge 1408 mainly mediates data between devices that operate at high speeds. On the other hand, the southbridge 1412 mediates data between devices that operate at relatively low speeds. For example, the northbridge 1408 may be connected to the memory 1404, the display device 14, and an expansion slot 1410 that can accept an expansion card (not shown). The southbridge 1412 may also be connected to the storage 1406 and a communication interface 1414.

The communication interface 1414 may include various communication ports such as USB, Bluetooth (registered trademark), Ethernet (registered trademark), and wireless Ethernet. These communication ports may also be connected to one or more input/output devices, such as communication devices such as a keyboard, a pointing device, a scanner, a wireless LAN (Local Area Network) adapter, and a router. That is, the computer 1003 may be connected to the mobile terminal 1004 via the communication interface 1414. The communication interface 1414 may also include a wireless LAN adapter. The communication interface 1414 realizes the functions of the receiving unit 20 and the transmitting unit 22.

Next, the mobile terminal 1004 shown in FIG. 15 will be described. The mobile terminal 1004 may be a mobile terminal such as a mobile phone, a smartphone, or a PDA (personal digital assistant).

The mobile terminal 1004 may include a display 1454, a transceiver 1468, an audio codec 1460, a control interface 1458, and a display interface 1456. The mobile terminal 1004 may further include a communication interface 1466, a processor 1452, a GPS (Global Positioning System) receiving module 1470, additional memory 1474, memory 1464, and an external interface 1462.

The display 1454 may be, for example, a TFT LCD (Thin-film-Transistor Liquid Crystal Display), an OLED (Organic Light Emitting Diode) display, or other display. The display may also be connected to the control interface 1458 or the display interface 1456. The processor 1452 may exchange information with the user of the mobile terminal 1004 via the control interface 1458 or the display interface 1456. The display 1454 realizes the function of the display unit 13. The display 1454 may also realize the function of the display unit 700.

The display interface 1456 may be configured with a circuit. The display interface 1456 controls a display to display images and other information to a user. The display interface 1456 realizes the functions of the display control unit 600. The display interface 1456 may also realize the functions of the display control unit 12.

The control interface 1458 receives commands from the user. The control interface 1458 may then convert the received commands for passing to the processor 1452.

The mobile terminal 1004 may also have an external interface 1462 that allows communication between an external device and the processor 1402. The external interface 1462 may provide wired or wireless communication to the mobile terminal 1004.

The additional memory 1474 is connected to the mobile terminal 1004 via the additional interface 1472. The additional memory 1474 may also include, for example, a SIMM (Single In Line Memory Module) card interface. The additional memory 1474 may also provide additional storage space for the mobile terminal 1004 and store applications and other information. The additional memory 1474 may also store information related to security. The additional memory 1474 may function as a security management device for the mobile terminal 1004. For example, the additional memory 1474 is programmed with instructions that realize safe use of the mobile terminal 1004.

The mobile terminal 1004 performs wireless communication via the communication interface 1466. In other words, the mobile terminal 1004 may communicate with the computer 1003 via the communication interface 1466. The communication interface 1466 realizes the functions of the transmitting unit 14 and the receiving unit 15.

The communication interface 1466 may include a digital signal processing circuit. The communication interface 1466 may realize communication under a mode or protocol such as Global System for Mobile Communication (GMS), Short Message Service (SMS), or Enhanced Messaging Service (EMS). Alternatively, the communication interface 1466 may realize communication under a mode or protocol such as Multimedia Messaging Service (MMS), Code Division Multiple Access (CDMA), or Time Division Multiple Access (TDMA).

Furthermore, the communication interface 1466 may realize communication under modes and protocols such as PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, and GPRS (General packet radio service). The above-mentioned communication occurs, for example, via the transceiver 1468. In addition, short-range communication, such as communication using Bluetooth, Wi-Fi (registered trademark), etc., may also occur via the transceiver 1468.

The GPS receiver module 1470 may provide data for use by applications running on the mobile device 1004. The data may be, for example, location or navigation related data.

The mobile terminal 1004 may also perform voice communication via the audio codec 1460. The audio codec 1460 acquires voice information from the user and converts it into usable digital information. Furthermore, the audio codec 1460 generates audio that the user can hear. The information handled by the audio codec 1460 may be voice calls, recorded sounds (voice messages, music files, etc.), sounds generated by applications on the mobile terminal 1004, etc.

The processor 1452 and memory 1464 are similar to those of the computer 1003, and therefore a description thereof will be omitted. The processor 1452 realizes the functions of the acquisition unit 800, the character extraction unit 400, and the vehicle information extraction unit 16.

The vehicle information acquisition systems 1000 to 1002 may be realized by either the computer 1003 or the mobile terminal 1004. In this case, for example, the processor 1402 in the computer 1003 may also realize the functions of the acquisition unit 800, the character extraction unit 400, the vehicle information extraction unit 16, and the display control unit 600. In addition, for example, the processor 1452 may also realize the functions of the vehicle information acquisition unit 21 and the matching unit 24.

The above-described embodiments are merely examples of the present invention, and various modifications can be made within the scope of the spirit of the present invention as described in the claims.

REFERENCE SIGNS LIST 1 vehicle 2 image 3 image 4 image 5 license plate 6 image 10 acquisition unit 11 character extraction unit 12 display control unit 13 display unit 14 transmission unit 15 reception unit 16 vehicle information extraction unit 20 reception unit 21 vehicle information acquisition unit 22 transmission unit 23 storage unit 24 matching unit 25 storage unit 26 storage unit 100 mobile terminal 101 mobile terminal 200 server 201 server 300 acquisition unit 400 character extraction unit 500 vehicle information acquisition unit 600 display control unit 700 display unit 1000 vehicle information acquisition system 1001 vehicle information acquisition system 1002 vehicle information acquisition system 1003 computer 1004 mobile terminal 1402 processor 1404 memory 1406 storage 1408 North bridge 1410 Expansion slot 1412 South bridge 1414 Communication interface 1452 Processor 1454 Display 1456 Display interface 1458 Control interface 1460 Audio codec 1466 Communication interface 1468 Transceiver 1462 External interface 1464 Memory 1470 GPS receiver module 1472 Expansion interface 1474 Expansion memory

Claims (15)

An acquisition unit that acquires an image of a vehicle;
An extraction unit that extracts a character string from an area of the license plate included in the image;
a vehicle information acquisition unit that acquires information on a registered vehicle associated with the character string;
an output unit that outputs information about the registered vehicle based on the character string;
Equipped with
the output unit outputs information indicating that the character string extracted from the image is not one of a plurality of pieces of registered information stored in advance, when the character string extracted from the image is not one of the pieces of registered information stored in advance.
Information processing device. The output unit outputs the character string extracted from the image together with the image to a display device.
The information processing device according to claim 1 . When the character string extracted from the image is not a pre-stored character string , the output unit outputs, to a mobile terminal , information indicating that the character string extracted from the image is not a pre-stored character string.
3. The information processing device according to claim 1 or 2 . The output unit outputs , to a display device, information about the registered vehicle that differs depending on the character string extracted from the image.
The information processing device according to claim 1 . A storage unit that stores information about the registered vehicle in association with a character string of the license plate,
the vehicle information acquisition unit acquires information of the registered vehicle from the storage unit based on the character string extracted from the area of the license plate;
The information processing device according to claim 1 . The character string extracted from the image includes characters indicating a transport bureau, a type, a business type, and a designated number.
The information processing device according to claim 1 . a vehicle information extraction unit that extracts a feature amount of a portion related to the vehicle from the image;
a matching unit that matches the extracted feature amount with a feature amount of an image related to the registered vehicle stored in a storage unit,
The output unit further outputs a collation result by the collation unit to a display device.
The information processing device according to claim 1 . The vehicle information acquisition unit acquires, from a storage unit, information about the registered vehicle and a comparison image corresponding to the information about the registered vehicle;
The output unit outputs the comparison image together with the information of the registered vehicle to a display device.
The information processing device according to claim 1 . The vehicle information acquisition unit acquires, from a storage unit, a facial image of an owner of the registered vehicle together with information of the registered vehicle;
The output unit outputs the facial image together with the information of the registered vehicle to a display device.
The information processing device according to claim 1 . Acquire an image of the vehicle ,
Extracting a character string from a license plate area included in the image;
Acquire information on a registered vehicle associated with the character string;
outputting information about the registered vehicle based on the character string;
outputting information indicating that the character string extracted from the image is not one of a plurality of pieces of registered information stored in advance, when the character string extracted from the image is not one of the pieces of registered information stored in advance ;
Information processing methods. and outputting the character string extracted from the image together with the image to a display device.
The information processing method according to claim 10. extracting a feature amount of a portion related to the vehicle from the image;
The extracted feature amount is compared with a feature amount of an image related to the registered vehicle stored in a storage unit; and
outputting the result of the matching to a display device;
12. The information processing method according to claim 10 or 11. On the computer,
Acquiring images of a vehicle;
A process of extracting a character string from an area of the license plate included in the image;
A process of acquiring information on a registered vehicle associated with the character string;
outputting information about the registered vehicle based on the character string;
a process of outputting information indicating that the character string extracted from the image is not one of a plurality of pieces of registered information stored in advance, when the character string extracted from the image is not one of the pieces of registered information stored in advance;
A program that executes the following. The computer includes:
outputting the character string extracted from the image together with the image to a display device;
Run
The program according to claim 13. The computer includes:
A process of extracting feature amounts of a portion related to the vehicle from the image;
A process of comparing the extracted feature amount with a feature amount of an image related to the registered vehicle stored in a storage unit;
outputting a result of the matching process to a display device;
Run
15. The program according to claim 13 or 14.

Images