JP7603542B2 - Identification device, identification method, and program - Google Patents

Description

The present disclosure relates to an identification device, an identification method, and a program.

Technology for reading vehicle license plates is used to determine the vehicle model and identify individual vehicles. For example, Patent Document 1 describes technology for determining the vehicle model using license plate information read from either the license plate on the front or rear of the vehicle.

JP 2019-61506 A

A system is being considered that identifies individual vehicles using license plate information obtained at the entrance and exit toll gates of a toll road, and charges each vehicle a toll according to the distance it has traveled (the distance from the entrance toll gate to the exit toll gate). However, systems currently used at toll gates only use license plate information from the front of the vehicle. This makes it difficult to identify that vehicles arriving at the entrance toll gate and the exit toll gate are the same vehicle if license plate information is misrecognized at the entrance toll gate or the exit toll gate. This may make it impossible to identify the section the vehicle was traveling on and charge the toll.

This disclosure has been made in consideration of these issues, and provides an identification device, an identification method, and a program that can prevent a decrease in the accuracy of identifying individual vehicles due to erroneous recognition of license plate information.

According to one aspect of the present disclosure, the identification device includes a first acquisition unit that acquires, from a first front-view camera and a first rear-view camera at an entrance toll booth, a first front image including the front license plate of a vehicle that has arrived at the entrance toll booth, and a first rear image including the rear license plate of the vehicle, respectively; a first image recognition unit that generates first license plate information by integrating character strings of the license plate read from each of the first front-view image and the first rear-view image; and a second image recognition unit that acquires, from a second front-view camera and a second rear-view camera at an exit toll booth, a first image recognition unit that generates first license plate information by integrating character strings of the license plate read from each of the first front-view image and the first rear-view image. The system includes a second acquisition unit that acquires a second front image including the front license plate and a second rear image including the rear license plate of the vehicle, a second image recognition unit that generates second license plate information by integrating license plate character strings read from the second front image and the second rear image, and a selection unit that selects, from the first license plate information, first license plate information related to the same vehicle as the vehicle that arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

According to one aspect of the present disclosure, the identification method includes the steps of acquiring a first front image including the front license plate of a vehicle arriving at the entrance toll booth and a first rear image including the rear license plate of the vehicle from a first front camera and a first rear camera at the entrance toll booth, respectively, generating first license plate information by integrating license plate character strings read from the first front image and the first rear image, acquiring a second front image including the front license plate of a vehicle arriving at the exit toll booth and a second rear image including the rear license plate of the vehicle from a second front camera and a second rear image at the exit toll booth, respectively, generating second license plate information by integrating license plate character strings read from the second front image and the second rear image, respectively, and selecting, from the first license plate information, the first license plate information related to the same vehicle as the vehicle arriving at the exit toll booth based on a comparison between the first license plate information and the second license plate information.

According to one aspect of the present disclosure, the program causes the computer of the identification device to execute the steps of acquiring a first front image including the front license plate of a vehicle arriving at the entrance toll booth and a first rear image including the rear license plate of the vehicle from each of the first front image and the first rear image at the entrance toll booth, generating first license plate information by integrating the license plate character strings read from each of the first front image and the first rear image, acquiring a second front image including the front license plate of a vehicle arriving at the exit toll booth and a second rear image including the rear license plate of the vehicle from each of the second front image and the second rear image at the exit toll booth, generating second license plate information by integrating the license plate character strings read from each of the second front image and the second rear image, and selecting, from the first license plate information, the first license plate information related to the same vehicle as the vehicle arriving at the exit toll booth based on a comparison between the first license plate information and the second license plate information.

The identification device, identification method, and program disclosed herein can prevent a decrease in the accuracy of individual vehicle identification due to erroneous recognition of license plate information.

1 is a diagram showing an overall configuration of an identification system according to a first embodiment of the present disclosure. 1 is a diagram illustrating a functional configuration of a recognition device according to a first embodiment of the present disclosure. 4 is a first flowchart showing an example of processing of the identification device according to the first embodiment of the present disclosure. FIG. 2 is a diagram showing an example of first license plate information according to the first embodiment of the present disclosure. 11 is a second flowchart showing an example of processing of the identification device according to the first embodiment of the present disclosure. FIG. 2 is a diagram for explaining a function of the identification device according to the first embodiment of the present disclosure. FIG. 11 is a diagram for explaining a function of an identification device according to a second embodiment of the present disclosure. FIG. 13 is a diagram for explaining a function of an identification device according to a third embodiment of the present disclosure. FIG. 13 is a diagram illustrating a functional configuration of an identification device according to a modified example of the present disclosure.

First Embodiment
Hereinafter, an identification system 1 and an identification device 10 according to a first embodiment of the present disclosure will be described with reference to FIGS. 1 to 6. FIG.

(Overall composition)
FIG. 1 is a diagram showing an overall configuration of an identification system according to a first embodiment of the present disclosure.
As shown in Figure 1, the identification system 1 comprises a first front-view camera 21 and a first rear-view camera 22 installed at the entrance toll gate EN, a second front-view camera 31 and a second rear-view camera 32 installed at the exit toll gate EX, and an identification device 10.

The first pre-photography camera 21 is installed on a gantry at a predetermined position on lane L1 at the entrance toll gate EN. The first pre-photography camera 21 photographs the front license plate NP1 of vehicle A traveling on lane L1 from above lane L1. The image captured by the first pre-photography camera 21 (first pre-photography image F1) is transmitted to the identification device 10 via a communication line.

The first rear camera 22 is installed on a gantry upstream (-X side) in the lane direction from the first front camera 21. The first rear camera 22 photographs the rear license plate NP2 of vehicle A traveling on lane L1 from above lane L1. The image captured by the first rear camera 22 (first rear image R1) is transmitted to the identification device 10 via a communication line.

The second pre-photography camera 31 is installed on a gantry at a predetermined position on lane L2 at the exit toll gate EX. The second pre-photography camera 31 photographs the front license plate NP1 of vehicle A traveling on lane L2 from above lane L2. The image captured by the second pre-photography camera 31 (second pre-photography image F2) is transmitted to the identification device 10 via a communication line.

The second rear camera 32 is installed on a gantry upstream (-X side) in the lane direction from the second front camera 31. The second rear camera 32 photographs the rear license plate NP2 of vehicle A traveling on lane L2 from above lane L2. The image captured by the second rear camera 32 (second rear image R2) is transmitted to the identification device 10 via a communication line.

For example, the first front camera 21 and the first rear camera 22 constantly capture images of the lane L1, and when the license plate of vehicle A is detected from the captured image by a predetermined image processing of vehicle A, the image is transmitted to the identification device 10. The first front camera 21 and the first rear camera 22 may also capture the first front image F1 and the first rear image R1, for example, when a vehicle detector (not shown) detects that the head or tail of vehicle A is in a predetermined position on lane L1. The same applies to the second front camera 31 and the second rear camera 32.

The identification device 10 collects a first front image F1 and a first rear image R1 from a first front camera 21 and a first rear camera 22 installed at each of a plurality of entrance toll gates EN via a communication line. The identification device 10 also collects a second front image F2 and a second rear image R2 from a second front camera 31 and a second rear camera 32 installed at each of a plurality of exit toll gates EX via a communication line. The identification device 10 identifies each vehicle that arrives at the entrance toll gate EN and the exit toll gate EX based on the license plate information read from the collected images.

(Functional configuration)
FIG. 2 is a diagram illustrating a functional configuration of the identification device according to the first embodiment of the present disclosure.
As shown in FIG. 2, the identification device 10 includes a CPU 11 , a storage medium 12 , an output device 13 , an input device 14 , and a communication interface (I/F) 15 .

The CPU 11 is a processor that controls the overall operation of the identification device 10. The CPU 11 operates according to a predetermined program to fulfill the functions of a first acquisition unit 110, a first image recognition unit 111, a second acquisition unit 112, a second image recognition unit 113, a selection unit 114, and a processing unit 115.

The first acquisition unit 110 acquires a first front image F1 including the front license plate NP1 of vehicle A that has arrived at the entrance toll gate EN, and a first rear image R1 including the rear license plate NP2 of vehicle A, from each of the first front camera 21 and the first rear camera 22 at the entrance toll gate EN.

The first image recognition unit 111 generates first license plate information (first NP information) D1 by integrating the license plate character strings read from each of the first pre-captured image F1 and the first post-captured image R1.

The second acquisition unit 112 acquires a second front image F2 including the front license plate NP1 of vehicle A that has arrived at the exit toll gate EX, and a second rear image R2 including the rear license plate NP2 of vehicle A, from each of the second front camera 31 and the second rear camera 32 at the exit toll gate EX.

The second image recognition unit 113 generates second license plate information (second NP information) D2 by integrating the license plate character strings read from each of the second pre-captured image F2 and the second post-captured image R2.

The selection unit 114 selects, from among the multiple pieces of first NP information D1, the first NP information D1 that is related to the same vehicle as the vehicle that arrived at the exit toll gate EX, based on a comparison between the first NP information D1 and the second NP information D2.

The processing unit 115 executes processing for the vehicle A related to the NP information based on the second NP information D2 generated by the second image recognition unit 113 and the first NP information D1 selected by the selection unit 114. For example, the processing unit 115 identifies from the first NP information D1 and the second NP information D2 which entrance toll gate EN the vehicle A entered the toll road from and which exit toll gate EX the vehicle A exited from. The processing unit 115 then transmits the identification information (toll gate ID) of the entrance toll gate EN and the first NP information D1, and the identification information (toll gate ID) of the exit toll gate EX and the second NP information D2 as information related to the vehicle A to the higher-level device (not shown). The higher-level device can perform a charging process to charge the vehicle A a toll according to the travel distance of the vehicle A (the distance from the entrance toll gate EN to the exit toll gate EX) based on the information received from the identification device 10.

The storage medium 12 is a so-called auxiliary storage device, and may be, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage medium 12 stores various data acquired and generated by each part of the CPU 11. For example, the storage medium 12 stores a first pre-shot image F1, a first post-shot image R1, a second pre-shot image F2, and a second post-shot image R2 collected by the first acquisition unit 110 and the second acquisition unit 112. The storage medium 12 also stores first NP information D1 and second NP information D2 generated by the first image recognition unit 111 and the second image recognition unit 113.

The output device 13 is a display device that presents various information to the operator of the identification device 10. The output device 13 may also be a printer, etc.

The input device 14 is an input device such as a keyboard or a mouse that accepts operations from an operator of the identification device 10. The output device 13 and the input device 14 may be integrated into a touch panel.

The communication I/F 15 transmits and receives data to and from a higher-level device via a wide area communication network such as the Internet.

(Processing flow)
FIG. 3 is a first flowchart illustrating an example of processing of the identification device according to the first embodiment of the present disclosure.
Hereinafter, with reference to FIG. 3, a process flow in which the identification device 10 collects the first NP information D1 of a vehicle that has arrived at the entrance toll gate EN will be described.

When vehicle A arrives at the entrance toll gate EN, the first pre-photography camera 21 captures an image of the front license plate NP1 of vehicle A. The first acquisition unit 110 of the identification device 10 acquires the first pre-photography image F1 captured by the first pre-photography camera 21 (step S100A).

Next, the first rear camera 22 captures an image of the rear side NP2 of the vehicle A. The first acquisition unit 110 then acquires the first rear image R1 captured by the first rear camera 22 (step S100B).

Next, the first image recognition unit 111 performs a predetermined image processing on the first pre-captured image F1 to read the NP information (text information) of the front license plate NP1 of vehicle A (step S101).

The first image recognition unit 111 also performs a predetermined image processing on the first rear-view image R1 to read the NP information (text information) of the rear license plate NP2 of vehicle A (step S102).

The first image recognition unit 111 generates first NP information D1 by integrating the NP information read from each of the first pre-shot image F1 and the first post-shot image R1 (step S103).

FIG. 4 is a diagram showing an example of first license plate information according to the first embodiment of the present disclosure.
As shown in FIG. 4, the NP information read from the first pre-photographed image F1 and the first post-photographed image R1 includes "place name (maximum 4 characters)", "classification number (maximum 3 characters)", "use code (maximum 1 character)", and "serial number (maximum 4 characters)". The first image recognition unit 111 generates first NP information D1 by combining the NP information of the first pre-photographed image F1 and the NP information of the first post-photographed image R1. In the example of FIG. 4, the first NP information D1 is information consisting of a character string of 24 characters. In addition, the first image recognition unit 111 associates the image acquisition date and time, information (toll gate ID) that can identify the entrance toll gate EN that is the source of the image, and the first pre-photographed image F1 and the first post-photographed image R1 with the generated first NP information D1, and stores and accumulates it in the storage medium 12.

The identification device 10 executes the series of processes shown in FIG. 3 each time it acquires the first pre-captured image F1 and the first post-captured image R1.

FIG. 5 is a second flowchart showing an example of processing of the identification device according to the first embodiment of the present disclosure.
Hereinafter, with reference to FIG. 5, a process flow in which the identification device 10 performs individual identification of a vehicle that has arrived at the exit toll gate EX will be described.

When vehicle A arrives at the exit toll gate EX, the second pre-photographing camera 31 captures an image of the front license plate NP1 of vehicle A. The second acquisition unit 112 of the identification device 10 acquires the second pre-photographed image F2 captured by the second pre-photographing camera 31 (step S110A).

Then, the rear license plate NP2 of vehicle A is photographed by the second rear camera 32. The second acquisition unit 112 then acquires the second rear image R2 captured by the second rear camera 32 (step S110B).

Next, the second image recognition unit 113 performs a predetermined image processing on the second pre-captured image F2 to read the NP information (text information) of the front license plate NP1 of vehicle A (step S111).

The second image recognition unit 113 also performs a predetermined image processing on the second rear-view image R2 to read the NP information (text information) of the rear license plate NP2 of vehicle A (step S112).

The second image recognition unit 113 generates second NP information D2 by integrating the NP information read from each of the second pre-shot image F2 and the second post-shot image R2 (step S113).

Next, the selection unit 114 compares the first NP information D1 with the second NP information D2 to identify, among the multiple vehicles that have passed through each entrance toll gate EN, a vehicle that is the same as vehicle A that has arrived at the exit toll gate EX. Specifically, the selection unit 114 first extracts candidates for the first NP information D1 that are related to the same vehicle A as the second NP information D2 from the storage medium 12, and determines the degree of match between each candidate (step S114).

FIG. 6 is a diagram for explaining the function of the identification device according to the first embodiment of the present disclosure.
In step S113, it is assumed that the second image recognition unit 113 generates the second NP information D2 as shown in Fig. 6. The selection unit 114 extracts the first NP information D1 having the same or similar serial number from the multiple pieces of first NP information D1 stored in the storage medium 12, for example, using the "serial number" of the pre-taken image included in the second NP information D2 as a search key. The selection unit 114 may use other items such as the "serial number" of the post-taken image, "place name", "classification number", etc. as a search key, or may use a combination of multiple items as a search key. The method by which the selection unit 114 searches for the same or similar information using one or more search keys can be performed using existing technology, so a description thereof will be omitted.

In the example of FIG. 6, it is assumed that the selection unit 114 extracts the first NP information D1_1 and the first NP information D1_2. The selection unit 114 determines the degree of match between the extracted first NP information D1_1 and D1_2. The degree of match is expressed, for example, as the number or percentage of matching characters among a string of 24 characters included in the first NP information D1 and the second NP information D2. In the example of FIG. 6, the degree of match (for example, the number of matching characters) of the first NP information D1_1 is "23 characters," and the degree of match of the first NP information D1_2 is "20 characters."

In the example of Figure 6, it is assumed that the "classification number" of the pre-captured image for the first NP information D1_1 was misrecognized. As a result, the first NP information D1_1 does not completely match the second NP information D2. In conventional toll road billing systems, it is common to identify individual vehicles based only on the NP information obtained from the pre-captured image. As a result, if different NP information is recorded at the entrance toll gate and the exit toll gate due to misrecognition, as in the example of Figure 6, it is difficult to identify these as NP information relating to the same vehicle.

Therefore, the selection unit 114 according to this embodiment compares the first NP information D1 and the second NP information D2, which are obtained by integrating the NP information of the pre-captured image and the NP information of the post-captured image into a single character string, and selects the first NP information D1 that matches the second NP information D2 most closely overall. Specifically, the selection unit 114 determines whether the extracted multiple pieces of first NP information D1 can be narrowed down to one that matches the second NP information D2 at a threshold or higher and is the highest (step S115). The threshold is set arbitrarily depending on the performance of each camera 21, 22, 31, 32, the recognition accuracy of the first image recognition unit 111 and the second image recognition unit 113, etc.

As described above, the first NP information D1 and the second NP information D2 are a combination of the NP information read from the front image and the rear image, respectively, so the amount of information is doubled (24 characters) compared to the conventional method of performing individual identification using only the NP information (12 characters) from one of the front image and the rear image. As a result, even if part of the NP information in the front image is misrecognized as in the example of Figure 6, as long as the NP information in the rear image matches, the overall degree of match will be high and it will be less susceptible to the effects of misrecognition.

In the example of FIG. 6, the degree of match (e.g., the number of matching characters) of the first NP information D1_1 is "23 characters," and the degree of match of the first NP information D1_2 is "20 characters." In this case, the selection unit 114 can narrow down the degree of match that is equal to or greater than the threshold and is the highest to only the first NP information D1_1 (step S115: YES). In this case, the selection unit 114 selects this first NP information D1_1 as the first NP information related to the same vehicle as the second NP information D2 generated in step S113 (step S116).

On the other hand, if there is no first NP information D1 with a degree of coincidence equal to or greater than the threshold, or if there are multiple pieces of first NP information D1 with a degree of coincidence equal to or greater than the threshold and the maximum degree of coincidence (step S115: NO), the selection unit 114 accepts the selection of the first NP information D1 by the operator of the identification device 10 (step S117). For example, the selection unit 114 displays, on the output device 13 of the identification device 10, the multiple extracted first NP information D1, the first pre-shot image F1 and the first post-shot image R1 linked to each of the first NP information D1, the second NP information D2, and the second pre-shot image F2 and the second post-shot image R2 linked to the second NP information D2. The operator compares each image and the NP information, and performs an operation to select the first NP information D1 corresponding to the second NP information D2 via the input device 14.

The processing unit 115 also transmits information including the generated second NP information D2 and the selected first NP information D1 to the higher-level device (step S118). The information transmitted to the higher-level device includes the acquisition date and time and the toll gate ID shown in FIG. 6. The higher-level device can charge a toll according to the distance traveled by the vehicle A based on the information received from the identification device 10. After transmitting the first NP information D1 to the higher-level device, the processing unit 115 may also delete the first NP information D1 from the storage medium 12.

The identification device 10 executes the series of processes in FIG. 5 each time it acquires the second pre-photographed image F2 and the second post-photographed image R2. Note that in another embodiment, the identification device 10 may store the acquired second pre-photographed image F2 and second post-photographed image R2 in the storage medium 12, and execute the series of processes in FIG. 5 for multiple sets of images stored in the storage medium 12 at a predetermined timing (e.g., every hour).

(Action and Effect)
As described above, the identification device 10 according to this embodiment includes a first acquisition unit 110 that acquires the first front-view image F1 and the first rear-view image R1 from the first front-view camera 21 and the first rear-view camera 22 at the entrance toll gate EN, a first image recognition unit 111 that generates first NP information D1 by integrating NP information read from the first front-view image F1 and the first rear-view image R1, and a first image recognition unit 112 that generates first NP information D1 by integrating NP information read from the second front-view camera 31 and the second rear-view camera 32 at the exit toll gate EX. The system is equipped with a second acquisition unit 112 that acquires a second pre-view image F2 and a second rear-view image R2 from each of the first and second images, a second image recognition unit 113 that generates second NP information D2 by integrating the NP information read from each of the second pre-view image F2 and the second rear-view image R2, and a selection unit 114 that selects, from the first NP information D1, first NP information D1 that relates to the same vehicle as vehicle A that has arrived at the exit toll gate EX based on a comparison between the first NP information D1 and the second NP information.

As described above, in conventional toll road billing systems, it is common to identify individual vehicles based only on NP information obtained from the front image. In that case, if different NP information is recorded at the entrance toll gate and the exit toll gate due to erroneous recognition, it is difficult to identify these as NP information related to the same vehicle. In contrast, the identification device 10 of this embodiment integrates the NP information read from the front image and the rear image at each of the entrance toll gate EN and the exit toll gate EX into a single character string and compares them. As a result, even if part of the NP information of either the front image or the rear image is erroneously recognized, as long as the NP information of the other image matches, the overall degree of match is high and the system is less susceptible to the influence of erroneous recognition. Therefore, the identification device 10 can suppress a decrease in the accuracy of individual vehicle identification due to erroneous recognition of NP information.

The selection unit 114 also determines the degree of match (the number or percentage of matching characters) between the first NP information D1 and the second NP information D2, and selects the first NP information D1 with the highest degree of match among the first NP information D1 whose degree of match is equal to or greater than a predetermined threshold.

By doing this, even if the first NP information D1 and the second NP information D2 contain misrecognized characters, the identification device 10 can appropriately select the first NP information D1 that is closest to the second NP information D2 from among the multiple pieces of first NP information D1.

The selection unit 114 also uses at least one item of the second NP information D2 as a search key to extract first NP information D1 that matches or is similar to the search key from multiple pieces of first NP information, and determines the degree of match of the extracted first NP information D1.

In this way, the identification device 10 can reduce the time required for the process of determining the degree of match by pre-extracting the first NP information D1 that is likely to have a high degree of match with the second NP information D2.

In addition, when there is no first NP information D1 whose degree of matching is equal to or greater than the threshold, or when there are multiple pieces of first NP information D1 whose degree of matching is equal to or greater than the threshold and is the highest, the selection unit 114 may present the extracted first NP information D1 to the operator as a candidate.

This allows the operator to select the appropriate first NP information D1 even when there are a large number of misrecognized characters.

Second Embodiment
Next, an identification system 1 and an identification device 10 according to a second embodiment of the present disclosure will be described with reference to FIG.
The components common to the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted. In this embodiment, the selection unit 114 uses weights for each item when determining the degree of coincidence of the first NP information D1.

The identification device 10 according to this embodiment is provided in advance with statistical information indicating the misrecognition rate for each item of NP information based on previously read NP information and its correct answer information (for example, NP information input by an operator based on a captured image). The selection unit 114 sets a weight according to the misrecognition rate for each item of NP information based on this statistical information.

For example, the selection unit 114 sets the weight for each item so that the degree of match when characters in an item with a low misrecognition rate (less likely to be misrecognized) do not match is lower than the degree of match when characters in an item with a high misrecognition rate (more likely to be misrecognized) do not match. Alternatively, the selection unit 114 sets the weight for each item so that the degree of match when characters in an item with a low misrecognition rate match is higher than the degree of match when characters in an item with a high misrecognition rate match.

FIG. 7 is a diagram for explaining the function of the identification device according to the second embodiment of the present disclosure.
FIG. 7 shows an example of the second NP information D2 generated by the second image recognition unit 113 and the first NP information D1 (D1_3, D1_4) extracted by the selection unit 114 in steps S113 and S114 in FIG.

For example, assume that statistical information indicates that the misrecognition rate for the "classification number" is equal to or greater than a predetermined value, and that for the "application code" is less than a predetermined value. In this case, the selection unit 114 sets the weights so that the degree of match when the "application code" does not match between the first NP information D1 and the second NP information D2 is lower than the degree of match when the "classification number" does not match. In other words, the selection unit 114 accepts mismatched characters for items with a high misrecognition rate, but sets the weights for the degree of match so that items with mismatched characters are not selected for items with a low misrecognition rate.

In the example of FIG. 7, the extracted first NP information D1_3 and D1_4 have the same number of characters as the second NP information D2, ie, 22 characters. In this case, if the determination is based only on the number of matching characters, it may not be possible to automatically select either of the first NP information D1_3 and D1_4, and confirmation and operation by the operator may be required.

In contrast, the selection unit 114 according to this embodiment further calculates the degree of match using weights for each item, making it possible to automatically select one of the first NP information D1. In the example of FIG. 7, the selection unit 114 calculates the degree of match by assigning weights to the "classification number," which has a high rate of misrecognition, and the "use code," which has a low rate of misrecognition. As a result, the degree of match is lower overall for the first NP information D1_4, in which the "use code" does not match, than for the first NP information D1_3, in which the "classification number" does not match. Therefore, the selection unit 114 can select the first NP information D1_3 as having the highest degree of match, above the threshold value.

(Action and Effect)
As described above, in the identification device 10 of this embodiment, the selection unit 114 determines the degree of similarity between the first NP information D1 and the second NP information D2 by weighting each of the multiple items contained in the first NP information D1 and the second NP information D2 according to the error recognition rate.

In this way, the identification device 10 can determine an appropriate degree of match so that mismatched characters are acceptable for items with a high rate of misrecognition, but items with a low rate of misrecognition are not selected. Furthermore, even if there are multiple pieces of first NP information D1 that have the same number of characters as the second NP information D2, the identification device 10 can vary the degree of match by weighting each item, thereby improving the possibility of automatically selecting one of the first NP information D1.

Third Embodiment
Next, an identification system 1 and an identification device 10 according to a third embodiment of the present disclosure will be described with reference to FIG.
The components common to the first and second embodiments are denoted by the same reference numerals and will not be described in detail. This embodiment is different from the first and second embodiments in that the selection unit 114 uses a score representing the recognition accuracy for each read character when determining the degree of match of the first NP information D1.

The first image recognition unit 111 according to this embodiment generates first NP information D1 in steps S101 to S103 of FIG. 3, in which the characters on the license plate read from the first pre-photographed image F1 and the first post-photographed image R1 are associated with a score indicating the reliability of the recognition of each character. Similarly, the second image recognition unit 113 generates second NP information D2 in steps S111 to S113 of FIG. 5, in which the characters on the license plate read from the second pre-photographed image F2 and the second post-photographed image R2 are associated with a score for each character.

For example, when the first image recognition unit 111 and the second image recognition unit 113 recognize each character by template matching, they use the similarity to the template as a score. Also, when the first image recognition unit 111 and the second image recognition unit 113 recognize each character by machine learning (including deep learning), they use the reliability of the recognition result as a score.

For the first NP information D1 and the second NP information D2 related to the same vehicle A, there is a high possibility that characters with high scores will match. On the other hand, there is a possibility that characters with low scores will not match due to misrecognition. For this reason, the selection unit 114 according to the present embodiment sets weights by score so that the degree of match when characters with high scores (low possibility of misrecognition) do not match is lower than the degree of match when characters with low scores (high possibility of misrecognition) do not match. Alternatively, the selection unit 114 sets weights by score so that the degree of match when characters with high scores match is higher than the degree of match when characters with low scores match. In other words, the selection unit 114 differentiates the weights between low-score characters and high-score characters so that mismatch is allowed for low-score characters that are likely to be misrecognized and not for high-score characters that are unlikely to be misrecognized.

FIG. 8 is a diagram for explaining the function of the identification device according to the third embodiment of the present disclosure.
Fig. 8 shows an example of the second NP information D2 generated by the second image recognition unit 113 and the first NP information D1 (D1_5, D1_6) extracted by the selection unit 114 in steps S113 to S114 in Fig. 5. As shown in Fig. 8, a score is assigned to each of the first NP information D1 and the second NP information D2 for each recognized character. The score is, for example, a value between 0 and 1, with 1 indicating the highest reliability and 0 indicating the lowest reliability.

For example, the selection unit 114 multiplies the scores of corresponding characters in the first NP information D1 and the second NP information D2 to obtain the weight. Specifically, if the recognition results match, the selection unit 114 obtains the value multiplied by the score as a positive weight, and if the recognition results do not match, the selection unit 114 obtains the value multiplied by the score as a negative weight. Below, the method of determining the score weighting will be described in detail with reference to the example in Figure 8.

(1) When both scores are high and the recognition results match The recognition results ("a") of the usage codes CH53 and CH23 of the pre-shot images of the first NP information D1_5 and the second NP information D2 match, and both scores are high ("1"). In this case, the weight of the score of the usage code of the first NP information D1_5 is "+1", which is a positive value obtained by multiplying the score of CH53 ("1") and the score of CH23 ("1").

(2) When both scores are high and the recognition results do not match The recognition result of the purpose code CH60 of the pre-shot image of the first NP information D1_6 ("O") does not match the recognition result of the purpose code CH23 of the second NP information D2 ("A"). In addition, both of these scores are high values ("1"). In this case, the weight of the score of the purpose code of the first NP information D1_6 is "-1", which is the negative value obtained by multiplying the score of CH63 ("1") and the score of CH23 ("1").

(3) When one score is low and the recognition results match Regarding the classification numbers of the pre-shot images of the first NP information D1_6 and the second NP information D2, the recognition results ("3") of the first characters CH60 and CH20 match. In addition, the score of the first NP information D1_6 (CH60) is "0.5", and the score of the second NP information D2 (CH20) is "1". In other words, one of the scores is higher and the other is lower. In this case, the weight of the score of the classification number (first character) of the first NP information D1_6 is "+0.5", which is a positive value obtained by multiplying the score of CH60 ("0.5") and the score of CH20 ("1").

(4) When one score is low and the recognition results do not match With regard to the classification number of the pre-shot image of the first NP information D1_5, the recognition result of the first character CH50 ("8") does not match the recognition result of the corresponding character CH20 of the first NP information NP ("3"). In addition, the score of the first NP information D1_5 (CH50) is "0.5", and the score of the second NP information D2 (CH20) is "1". In other words, one of the scores is higher and the other is lower. In this case, the weight of the score of the classification number (first character) of the first NP information D1_5 is "-0.5", which is a negative value obtained by multiplying the score of CH50 ("0.5") and the score of CH20 ("1").

(5) When both scores are low and the recognition results match With regard to the classification numbers of the pre-shot images of the first NP information D1_6 and the second NP information D2, the recognition results ("1") of the second characters CH61 and CH21 match. Furthermore, the scores of these characters CH61 and CH21 are both low values ("0.5"). In this case, the weight of the score of the classification number (second character) of the first NP information D1_6 is "+0.25", which is a positive value obtained by multiplying the score of CH61 ("0.5") and the score of CH21 ("0.5").

(6) When both scores are low and the recognition results do not match With regard to the classification number of the pre-shot image of the first NP information D1_5, the recognition result of the second character CH51 ("0") does not match the recognition result of the corresponding character CH21 of the first NP information NP ("1"). In addition, the scores of these characters CH51 and CH21 are both low values ("0.5"). In this case, the weight of the score of the classification number (second character) of the first NP information D1_5 is "-0.25", which is the negative value obtained by multiplying the score of CH51 ("0.5") and the score of CH21 ("0.5").

In this way, the selection unit 114 calculates the weight of each character based on whether the recognition results match for each corresponding character in the first NP information D1 and the second NP information D2 and the score of the recognition results, and calculates the sum of these as the degree of match. Taking the weight into consideration, the highest degree of match is "24" (all 12 characters of the NP information in the front image + all 12 characters of the NP information in the rear image match), and the lowest value is "-24" (all 12 characters of the NP information in the front image + all 12 characters of the NP information in the rear image do not match). As a result, the degree of match between characters with high recognition results scores is significantly reduced when the recognition results do not match. In addition, for characters with low recognition results scores, the impact on the degree of match when the recognition results do not match is small. In this way, it is possible to reduce the possibility that the NP information of the same vehicle will not be selected appropriately due to a decrease in recognition accuracy caused by dirt on the license plate, etc.

The selection unit 114 selects the first NP information D1 whose degree of matching is a predetermined threshold (e.g., 20 or more) and is the highest. In the example of FIG. 8, the degree of matching of the second NP information D2 with the first NP information D1_5 is "21.25", and the degree of matching with the first NP information D1_6 is "18.75". Therefore, the selection unit 114 selects the first NP information D1_5.

In the present embodiment, an example in which the weight is calculated for each character included in the first NP information D1 and the second NP information D2 has been described, but the present invention is not limited to this. In other embodiments, the weight may be calculated for each character string instead of for each character. For example, since a place name, such as "Kobe" or "Himeji", is a place name in which a plurality of characters (character strings) express one meaning (place name), it is more reasonable to match the entire character string rather than matching each character one by one. For this reason, the selection unit 114 treats the four characters included in the place name as one character string and calculates only one weight for the entire character string. At this time, the selection unit 114 sets the average value of the scores of each character in the place name as the score for the entire character string. The selection unit 114 calculates the weight of the character string indicating the place name based on whether the character strings indicating the place name in the first NP information D1 and the second NP information D2 match and the score of the character string. In this case, the degree of match of the NP information is in the range of "-18" to "+18".

In yet another embodiment, the selection unit 114 may take the minimum or maximum score of each character in the place name as the score for the entire character string representing the place name.

(Action and Effect)
As described above, in the identification device 10 according to this embodiment, the selection unit 114 determines the degree of similarity between the first NP information D1 and the second NP information D2 by weighting the characters contained in the first NP information D1 and the second NP information D2 according to their scores.

In this way, the identification device 10 can determine an appropriate degree of match so that low-score characters that are likely to be misrecognized do not match the corresponding characters in the second NP information D2, but high-score characters that are unlikely to be misrecognized are not selected that do not match the corresponding characters in the second NP information D2. Furthermore, even if there are multiple pieces of first NP information D1 that match the second NP information D2 with the same number of characters, the identification device 10 can vary the degree of match by weighting according to the score, thereby improving the possibility of automatically selecting one of the first NP information D1.

In this embodiment, the first image recognition unit 111 and the second image recognition unit 113 recognize the characters on the license plate one by one, and assign a score to each character, but the present invention is not limited to this. In other embodiments, the first image recognition unit 111 and the second image recognition unit 113 may recognize a specific item as a character string without dividing it into characters. For example, in the example of FIG. 8, the first image recognition unit 111 and the second image recognition unit 113 recognize the character string "KOBE" representing a place name by dividing it into two characters, "神" and "戸", and assign a score to each of these two characters. However, in other embodiments, "KOBE" may be recognized as a single character string without dividing it, and only one score may be assigned to the entire character string.

In each of the above-mentioned embodiments, the various processes of the identification device 10 are stored in the form of a program on a computer-readable recording medium, and the above-mentioned various processes are performed by the computer reading and executing this program. In addition, computer-readable recording media refers to magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc. In addition, this computer program may be distributed to a computer via a communication line, and the computer that receives this distribution may execute the program.

The above program may be for realizing some of the functions described above. Furthermore, it may be a so-called differential file (differential program) that can realize the above functions in combination with a program already recorded in the computer system.

As described above, several embodiments of the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

For example, in the identification system 1 according to each of the above-mentioned embodiments, the identification device 10 has the functions of the first acquisition unit 110, the first image recognition unit 111, the second acquisition unit 112, and the second image recognition unit 113, but this is not limited to the above. For example, as in the modified example described below, these functional units may be possessed by a device other than the identification device 10.

<Modification>
FIG. 9 is a diagram illustrating a functional configuration of an identification device according to a modified example of the present disclosure.
As shown in Figure 9, the identification system 1 of this modified example further includes a first number plate recognition device (first NP recognition device) 20 installed at the entrance toll gate EN, and a second number plate recognition device (second NP recognition device) 30 installed at the exit toll gate EX.

The first NP recognition device 20 is a device for generating first NP information D1 for vehicle A that has arrived at the entrance toll gate EN. The first NP recognition device 20 includes a first front camera 21 and a first rear camera 22, which are the same as those in the first embodiment, and a CPU 23.

The CPU 23 is a processor that controls the overall operation of the first NP recognition device 20. The CPU 23 performs the functions of the first acquisition unit 230 and the first image recognition unit 231 by operating according to a predetermined program. Note that the functions of the first acquisition unit 230 and the first image recognition unit 231 are similar to the functions of the first acquisition unit 110 and the first image recognition unit 111 according to the first to third embodiments, respectively.

The second NP recognition device 30 is a device for generating second NP information D2 of vehicle A that has arrived at the exit toll gate EX. The second NP recognition device 30 is equipped with the same second front camera 31 and second rear camera 32 as in the first embodiment, and a CPU 33.

The CPU 33 is a processor that controls the overall operation of the second NP recognition device 30. The CPU 33 performs the functions of the second acquisition unit 330 and the second image recognition unit 331 by operating according to a predetermined program. Note that the functions of the second acquisition unit 330 and the second image recognition unit 331 are similar to the functions of the second acquisition unit 112 and the second image recognition unit 113 according to the first to third embodiments, respectively.

The CPU 11 of the identification device 10 according to this embodiment further includes a license plate information acquisition unit (NP information acquisition unit) 116. The NP information acquisition unit 116 acquires the first NP information D1 from the first NP recognition device 20, and acquires the second NP information D2 from the second NP recognition device 30. The functions of the selection unit 114 and the processing unit 115 of the identification device 10 are the same as those of the above-mentioned embodiments.

Even if the identification system 1 has an equipment configuration like this modified example, it can achieve the same effects as the above-mentioned embodiments.

<Additional Notes>
The identification device, the identification method, and the program described in the above-described embodiments can be understood, for example, as follows.

According to a first aspect of the present disclosure, the identification device (10) includes a first acquisition unit (110, 230) that acquires a first front image including the front license plate of a vehicle that has arrived at the entrance toll gate and a first rear image including the rear license plate of the vehicle from a first front camera (21) and a first rear camera (22) at the entrance toll gate, respectively; a first image recognition unit (111, 231) that generates first license plate information by integrating character strings of the license plate read from each of the first front image and the first rear image; and a second image recognition unit (112, 233) that acquires the first front image including the front license plate of a vehicle that has arrived at the entrance toll gate and a first rear image including the rear license plate of the vehicle from each of a second front camera (31) and a second rear camera (32) at the exit toll gate, respectively. The toll gate includes a second acquisition unit (112, 330) that acquires a second front image including the front license plate of a vehicle arriving at the toll gate and a second rear image including the rear license plate of the vehicle, a second image recognition unit (113, 331) that generates second license plate information by integrating the license plate character strings read from the second front image and the second rear image, and a selection unit (114) that selects, from the first license plate information, the first license plate information related to the same vehicle as the vehicle that arrived at the toll gate, based on a comparison between the first license plate information and the second license plate information.

At each of the entrance toll gate EN and the exit toll gate EX, the identification device integrates the NP information read from the front and rear images into a single string and compares it. Therefore, even if part of the NP information in either the front or rear image is misrecognized, as long as the NP information in the other image matches, the overall degree of match will be high and the device will be less susceptible to the effects of misrecognition. Therefore, the identification device can suppress a decrease in the accuracy of individual vehicle identification due to misrecognition of NP information.

According to the second aspect of the present disclosure, in the identification device (10) according to the first aspect, the selection unit (114) determines the degree of match between the first license plate information and the second license plate information, and selects the first license plate information with the highest degree of match from among the first license plate information whose degree of match is equal to or greater than a predetermined threshold.

By doing this, the identification device can appropriately select from among multiple pieces of first license plate information the one that is closest to the second license plate information, even if the first license plate information and the second license plate information contain misrecognized characters.

According to a third aspect of the present disclosure, in the identification device (10) according to the second aspect, the selection unit (114) determines the degree of match by weighting the items included in the first license plate information and the second license plate information according to the error recognition rate.

In this way, the identification device can determine an appropriate degree of match so that mismatched characters are acceptable for items with a high rate of misrecognition, but items with a low rate of misrecognition are not selected with mismatched characters. Furthermore, even if there are multiple pieces of first license plate information that have the same number of characters as the second license plate information, the identification device can vary the degree of match by weighting each item, thereby improving the possibility of automatically selecting one of the first license plate information.

According to a fourth aspect of the present disclosure, in the identification device (10) according to the first aspect, the first license plate information and the second license plate information further include a score representing the reliability of recognition of each character of the read license plate, and the selection unit (114) determines the degree of match by applying a weight according to the score.

In this way, the identification device can determine an appropriate degree of match so that low-score characters that may be misrecognized do not match the corresponding characters in the second license plate information, but high-score characters that are less likely to be misrecognized do not match the corresponding characters in the second license plate information. In addition, even if there are multiple pieces of first license plate information that match the same number of characters as the second license plate information, the identification device can vary the degree of match by weighting according to the score, thereby improving the possibility of automatically selecting one of the first license plate information.

According to a fifth aspect of the present disclosure, the identification method includes the steps of acquiring a first front image including the front license plate of a vehicle arriving at the entrance toll gate and a first rear image including the rear license plate of the vehicle from a first front camera (21) and a first rear camera (22) at the entrance toll gate, respectively, generating first license plate information by integrating the license plate character strings read from the first front image and the first rear image, respectively, and acquiring the exit toll plate character string from a second front camera (31) and a second rear camera (32) at the exit toll gate, respectively. The method includes the steps of acquiring a second front image including the front license plate of a vehicle arriving at the exit toll gate and a second rear image including the rear license plate of the vehicle, generating second license plate information by integrating the license plate character strings read from the second front image and the second rear image, and selecting, from the first license plate information, the first license plate information related to the same vehicle as the vehicle arriving at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

According to a sixth aspect of the present disclosure, the program includes the steps of acquiring a first front image including the front license plate of a vehicle arriving at the entrance toll booth and a first rear image including the rear license plate of the vehicle from a first front camera (21) and a first rear camera (22) at the entrance toll booth, respectively, generating first license plate information by integrating the license plate character strings read from the first front image and the first rear image, respectively, and acquiring a first license plate information including the front license plate character strings read from the first front image and the first rear image, respectively, from a second front camera (31) and a second rear camera (32) at the exit toll booth, respectively. The computer of the identification device (10) executes the steps of acquiring a second front image including the front license plate and a second rear image including the rear license plate of the vehicle, generating second license plate information by integrating the license plate character strings read from the second front image and the second rear image, and selecting, from the first license plate information, the first license plate information related to the same vehicle as the vehicle that arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

1 Identification system 10 Identification device 11 CPU
110: First acquisition unit 111: First image recognition unit 112: Second acquisition unit 113: Second image recognition unit 114: Selection unit 115: Processing unit 116: License plate information acquisition unit (NP information acquisition unit)
12 Storage medium 13 Output device 14 Input device 20 First number plate recognition device (first NP recognition device)
21 First front camera 22 First rear camera 23 CPU
230 First acquisition unit 231 First image recognition unit 30 Second license plate recognition device (second NP recognition device)
31 Second front camera 32 Second rear camera 33 CPU
330 Second acquisition unit 331 Second image recognition unit

Claims (6)

a first acquisition unit that acquires a first front image including a front side license plate of a vehicle that has arrived at the entrance toll gate and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll gate, respectively;
A first image recognition unit that generates first license plate information , which is a character string having twice the number of characters of the license plate by concatenating character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image, and
a second acquisition unit that acquires, from a second front image including a front side license plate of a vehicle that has arrived at the exit toll gate and a second rear image including a rear side license plate of the vehicle, from a second front image camera and a second rear image camera of the exit toll gate, respectively;
A second image recognition unit that generates second license plate information , which is a character string having twice the number of characters of the license plate by concatenating character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image, and
a selection unit that selects, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate, based on a comparison between the first license plate information and the second license plate information;
An identification device comprising: The selection unit obtains a degree of match between the first license plate information and the second license plate information, and selects the first license plate information having the highest degree of match from among the first license plate information whose degree of match is equal to or greater than a predetermined threshold.
The identification device according to claim 1 . the selection unit determines the degree of match by weighting the items included in the first license plate information and the second license plate information according to a misrecognition rate of each of the items.
The identification device according to claim 2 . The first license plate information and the second license plate information further include a score representing a degree of confidence in the recognition of each character of the license plate read;
The selection unit determines the degree of match by weighting the scores.
The identification device according to claim 2 or 3. acquiring a first front image including a front side license plate of a vehicle arriving at the entrance toll booth and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll booth, respectively;
A step of generating first license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image;
acquiring a second front image including a front side license plate of a vehicle arriving at the exit toll booth and a second rear image including a rear side license plate of the vehicle from a second front image camera and a second rear image camera at the exit toll booth, respectively;
A step of generating second license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image;
selecting, from among the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information;
The identification method has the following features: acquiring a first front image including a front side license plate of a vehicle arriving at the entrance toll booth and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll booth, respectively;
A step of generating first license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image;
acquiring a second front image including a front side license plate of a vehicle arriving at the exit toll booth and a second rear image including a rear side license plate of the vehicle from a second front image camera and a second rear image camera at the exit toll booth, respectively;
A step of generating second license plate information that is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image;
selecting, from among the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information;
A program for causing a computer of an identification device to execute the above.

Images