JP7593651B2 - Camera installation support program, camera installation support method, and camera installation support system - Google Patents

Description

This disclosure relates to a camera installation support program, a camera installation support method, and a camera installation support system.

Patent Document 1 discloses an elevator system including an image processing device capable of detecting a user near the door of a car using an image captured by a camera that includes the inside of the car and the landing, and an administrator terminal that is connected to the image processing device and can be operated by an administrator. The elevator system obtains from the camera an image captured with a marker that can distinguish between the floor surface of the car and the floor surface of the landing, displays a recognition result in which the marker is recognized from the obtained image, and detects a deviation in the mounting position of the camera based on the position of the recognized marker based on the administrator's operation in response to the displayed recognition result.

Patent Application No. 2019-53670

The elevator system of Patent Document 1 uses a camera to capture images of the area near the elevator car door and detects users moving toward the elevator. Recently, cameras have been developed that capture and detect license plates of stopped or moving vehicles. However, when attempting to detect vehicle license plates with a camera using the configuration of Patent Document 1, there is a problem in that it is difficult to determine whether the camera is properly installed, because the way the license plate appears in the actual captured image varies depending on the vehicle's running position relative to the width of the road, the mounting position or mounting angle of the license plate, or the shape of the road (e.g., the inclination angle). When installing such cameras, workers want to know whether the camera is properly installed to detect license plates during actual operation.

The present disclosure has been devised in consideration of the above-mentioned conventional circumstances, and aims to provide a camera installation support program, a camera installation support method, and a camera installation support system that support camera installation and improve the recognition accuracy of recognition processing using captured images.

The present disclosure provides a camera installation assistance program for causing a terminal device, which is a computer communicatively connected to a camera that captures images of a vehicle, to perform the following steps: detect the license plate of the vehicle that appears in one or more predefined detection areas from each of a plurality of captured images captured by the camera; calculate the maximum and minimum luminance values of pixels in the area of the detected license plate; and, if it is determined that the luminance difference between the maximum and minimum luminance values is less than a predetermined value, generate and output advice information regarding the installation or parameter settings of the camera.

The present disclosure also provides a camera installation support method for a camera installation support system in which a camera and a terminal device are communicatively connected, the camera installation support method detecting a vehicle license plate appearing in one or more predefined detection areas from each of a plurality of captured images captured by the camera, calculating a maximum luminance value and a minimum luminance value of pixels in the area of each of the detected license plates, and generating and outputting advice information regarding the installation or parameter settings of the camera if it is determined that the luminance difference between the maximum luminance value and the minimum luminance value is less than a predetermined value.

The present disclosure also provides a camera installation support system including a camera and a terminal device communicatively connected to the camera, in which the camera transmits each of a plurality of captured images to the terminal device, and the terminal device detects vehicle license plates from one or more detection areas set in advance among the plurality of captured images, calculates maximum and minimum luminance values of pixels in the area of each of the detected license plates, and if it is determined that the luminance difference between the maximum and minimum luminance values is less than a predetermined value, generates and outputs advice information regarding the installation or parameter settings of the camera.

This disclosure can assist with camera installation and improve the recognition accuracy of recognition processing using captured images.

FIG. 1 is a block diagram showing an example of an internal configuration of a camera installation support system according to a first embodiment. A flowchart showing an example of an operation procedure of a terminal device according to the first embodiment. FIG. 1 is a diagram illustrating an example of a plate width, a plate inclination, and a detection area of a license plate. A table showing examples of the conditions for achieving and not achieving each indicator Table showing an example of advice information A diagram showing an example of a number recognition result screen. A diagram showing an example of a plate position confirmation screen before and after camera installation support. A diagram showing an example of a plate position confirmation screen before and after camera installation support. A diagram showing an example of a plate position confirmation screen before and after camera installation support. FIG. 13 is a diagram showing an example of a plate position confirmation screen when a plurality of detection areas are each set.

Below, with reference to the drawings as appropriate, each embodiment that specifically discloses the configuration and operation of the camera installation support program, camera installation support method, and camera installation support system according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to make it easier for those skilled in the art to understand. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
1 is a block diagram showing an example of the internal configuration of a camera installation support system 100 according to the first embodiment. The camera installation support system 100 is a system for supporting the installation of a camera C1 desired by a user, for example, when the camera C1 is installed in a predetermined location and environment. The camera installation support system 100 executes a recognition process for, for example, a vehicle license plate as an image capture target based on an image captured by the camera C1, and generates and outputs adjustment advice information for adjusting the installation position or settings of the camera based on the recognition result. The camera installation support system 100 includes a camera C1, a network NW, and a terminal device P1.

Note that the camera C1 according to the first embodiment is, for example, a surveillance camera, but the use and type of the camera are not limited to this. Also, the camera installation support system 100 according to the first embodiment shows an example of performing detection and recognition of a vehicle license plate, but is not limited to this. For example, the camera installation support system 100 may detect, for example, a vehicle, a motorcycle, a person, etc., and recognize their characteristics (for example, license plate, license number, vehicle type, color, gender, clothing, age, etc.).

Camera C1 is installed in a position where the user can capture an image of the target, detects the target from within a predefined detection area in the captured image, and executes recognition processing. Camera C1 associates the captured image with the recognition result and transmits it to terminal device P1 at predetermined intervals (e.g., 100 ms, 200 ms, etc.). Camera C1 includes a communication unit 10, a processor 11, a memory 12, and an image capture unit 13.

The communication unit 10 is configured using a communication interface circuit for transmitting and receiving data or information to and from the terminal device P1 via the network NW. The communication unit 10 associates the captured image captured by the imaging unit 13 with the recognition result generated by the processor 11 and transmits them to the terminal device P1 via a wireless communication network or a wired communication network.

The processor 11 is configured using, for example, a CPU (Central Processing unit) or an FPGA (Field Programmable Gate Array) and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 references the programs and data stored in the memory 12 and executes the programs to realize the functions of each part.

The processor 11 acquires each of the captured images captured by the imaging unit 13, and executes various processes for the object (the vehicle license plate NP in the example shown in the first embodiment) shown in the captured image. Each recognition process executed by the processor 11 will be described below with reference to FIG. 3. FIG. 3 is a diagram illustrating an example of the plate width ΔX, plate inclination θ, and detection area AR1 of the license plate NP.

The processor 11 generates an image with a detection area by superimposing a frame line indicating the area of the set detection area AR1 (for example, a solid frame line indicating the area of the detection area AR2 shown in FIG. 7) on each captured image taken at a predetermined interval (for example, 100 ms, 200 ms, etc.) and detects the license plate NP reflected in this detection area AR1. Note that here, the processor 11 may generate an image with a detection area by superimposing dashed lines indicating the areas of the first detection zone AR11 and the second detection zone AR12 in addition to the frame line indicating the area of the detection area AR1.

The processor 11 recognizes and acquires each piece of character information based on the number plate NP detected within the detection area AR1. The each piece of character information here is information about the vehicle shown on the number plate NP, such as local transport bureau information, vehicle type information, type information, serial number (vehicle number) information, etc. It goes without saying that the above-mentioned pieces of character information are merely examples and are not limited to these. Furthermore, the information recognized by the processor 11 may be set arbitrarily depending on the subject imaged by the camera C1, and for example, if the subject imaged is a person, the information may be height, clothing, clothing color, age, gender, etc.

Furthermore, based on each of these character information, processor 11 assigns a vehicle ID that enables the vehicle to be identified to number plates NP that are determined to be the same vehicle, and stores the vehicle ID in association with the captured image. Note that if multiple detection areas are set and multiple vehicle license plates are detected from one captured image, a vehicle ID is associated with each license plate detected from each detection area. Processor 11 also stores information on the detection area and detection zone (for example, whether it is the first detection zone AR11 or the second detection zone AR12 of the detection area AR1 shown in Figure 3) as the location where license plate NP was detected, in association with the captured image for each license plate.

The processor 11 also measures the coordinates of two opposing vertices PS1, PS2 of the four vertices of the detected license plate NP. Here, in the example shown in FIG. 2, the processor 11 measures the coordinates (X1, Y1) of the top left vertex PS1 of the license plate NP and the coordinates (X2, Y2) of the bottom right vertex PS2. Based on the measured coordinates of the two vertices PS1, PS2, the processor 11 calculates the coordinates (X3, Y3) of the midpoint PS3 and the coordinates (X4, Y4) of the midpoint PS4 in the short direction of the license plate NP.

The processor 11 determines whether the license plate NP is within the detection area AR1 set by the user based on the coordinates of the two calculated midpoints PS3 and PS4. Specifically, the processor 11 determines whether the coordinates of the two midpoints PS3 and PS4 are within the detection area AR1. Furthermore, the processor 11 determines whether the license plate NP is detected in the first detection zone AR11 or the second detection zone AR12 of the detection area AR1. The first detection zone AR11 here is an area of width ΔX2 centered on the midpoint of the detection area AR1 set by the user in the width direction (left-right direction of the paper), and is equal to the minimum value (e.g., 120 pixels) set as the achievement condition of the plate width ΔX as an index described later. The second detection zone AR12 is two areas obtained by subtracting the width ΔX2 of the first detection zone AR11 from the width ΔX1 of the detection area AR1. The first detection zone AR11 is adjacent to the two second detection zones AR12 and is an area sandwiched between the two second detection zones AR12.

The processor 11 calculates the plate width ΔX and the plate inclination θ of the license plate NP based on the coordinates of the two calculated midpoints PS3 and PS4. (Formula 1) is a formula for calculating the plate width ΔX. Note that the plate width ΔX indicates the number of pixels between the coordinates (X3, Y3) of the midpoint PS3 and the coordinates (X4, Y4) of the midpoint PS4. (Formula 2) is a formula for calculating the plate inclination θ.

Processor 11 also calculates the maximum luminance value (i.e., the luminance value of the pixels in the background portion other than the text portion of number plate NP) and the minimum luminance value (i.e., the luminance value of the pixels in the text portion of number plate NP) among the pixels included in the area of the detected number plate NP. Processor 11 may also calculate a luminance difference ΔL that indicates the difference between the maximum luminance value and the minimum luminance value.

The camera C1 stores (stores) the captured image with the detection area in association with each recognition result. Note that the recognition results referred to here are, for example, each character information, vehicle ID information, detection area information, detection zone information, the coordinates of each of the two midpoints PS3 and PS4, the plate width ΔX, the plate inclination θ, the maximum brightness value, or the minimum brightness value, but needless to say, are not limited to these.

Based on a control command sent from the terminal device P1 requesting the start of acquisition of captured images and recognition results, the processor 11 associates the accumulated (saved) captured images with the detection areas with each recognition result and sends them to the terminal device P1. Note that the captured images and each recognition result for a predetermined period of time may be stored in the memory 12, or may be stored in a storage medium (not shown) such as a HDD (Hard Disk Drive), or an external storage medium (not shown) such as an SD card (registered trademark) connected to the camera C1 so as to be capable of data communication.

The memory 12 has, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the processor 11, and a ROM (Read Only Memory) that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM has written therein programs that define the operation of the processor 11. The memory 12 stores product number information of the camera C1, information on various setting parameters (e.g., gain level, light area correction, dark area correction, etc.), information on the detection area, etc.

The imaging unit 13 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state imaging element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and converts the optical image formed on the imaging surface into an electrical signal. The imaging unit 13 outputs the captured image to the processor 11.

The lighting unit 14 is, for example, a light for irradiating the imaging area of the camera C1 with light, and is realized by, for example, an IR-LED (Light Emission Diode), an LED, etc. The lighting unit 14 is controlled by the processor 11 to perform on/off control and adjust the amount of light (brightness) that is irradiated.

The terminal device P1 has an interface (e.g., a keyboard, a mouse, a touch panel display, etc.) that can accept user input operations as an input unit 23. The terminal device P1 is communicatively connected to the camera C1 via the network NW, and transmits a signal generated based on the user input operations to the server S1 via the network NW. The terminal device P1 also displays a composite image received from the server S1 via the network NW on the monitor 24. The terminal device P1 is configured to include a communication unit 20, a processor 21, a memory 22, an input unit 23, a monitor 24, and an HDD 25.

The communication unit 20 is configured using a communication interface circuit for transmitting and receiving data or information to and from the camera C1 via the network NW. The communication unit 20 transmits a control signal based on a user's input operation generated by the processor 21 to the camera C1 via a wireless communication network or a wired communication network, and acquires the captured image transmitted from the camera C1.

The processor 21 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 references the programs and data stored in the memory 22 and executes the programs to realize the functions of each part.

The memory 22 has, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM for storing programs and data that define the operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has programs that define the operation of the processor 21 written therein. The memory 22 stores various data transmitted from the camera C1, information on the set detection area, etc. The various data referred to here are the start time of acquisition of the captured image, the end time of acquisition of the captured image, the IP address of the camera C1, product number information of the camera C1, the captured image with the set detection area superimposed, position (coordinate) information of the recognition target as the recognition result, the project name as the storage destination, etc.

The input unit 23 is, for example, a user interface that detects input operations by the user, and is configured using a mouse, a keyboard, a touch panel, or the like. Based on the user's input operations, the input unit 23 accepts settings related to the camera C1, settings related to the detection and recognition processing, etc., converts them into signals, and outputs them to the processor 21.

The monitor 24 is configured using a display such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence).

The HDD 25 records various data transmitted from the camera C1, the recognition results performed by the processor 21, and adjustment advice information regarding the installation and settings of the camera C1.

The network NW connects the terminal device P1 and the camera C1 so that they can communicate with each other via a wireless communication network or a wired communication network. The wireless communication network referred to here is provided in accordance with wireless communication standards such as wireless LAN (Local Area Network), wireless WAN (Wide Area Network), 4G (fourth generation mobile communication system), 5G (fifth generation mobile communication system), or Wi-Fi (registered trademark).

Next, the operation procedure of the terminal device P1 will be described with reference to FIG. 2. FIG. 2 is a flowchart showing an example of the operation procedure of the terminal device P1 according to the first embodiment.

Based on a user operation requesting the start of acquisition of captured images and recognition results by camera C1, terminal device P1 transmits a control signal to camera C1 requesting captured images for a predetermined period associated with the recognition results. Terminal device P1 associates and saves a start date and time indicating the timing at which the user operation requesting the start of acquisition of captured images and recognition results was accepted with the IP (Internet Protocol) address of camera C1 as the target IP (St1). Note that the information to be saved is not limited to this, and may be, for example, a product number that can identify camera C1, the name of a folder set as a save destination, a project name entered by the user, etc.

The terminal device P1 acquires each of the captured images (i.e., captured images with detection areas) for a predetermined period of time transmitted from the camera C1, associates them with various information such as the start date and time or the IP address of the camera C1, and stores them in the memory 12 (St2). The terminal device P1 also acquires each recognition result associated with each captured image, and stores them in the memory 12 (St3).

Based on a user operation requesting the end of acquisition of the captured images and recognition results transmitted from the camera C1, the terminal device P1 ends the acquisition and storage of each of the captured images for a specified period and the acquisition and storage of each of the recognition results associated with these captured images, and stores in memory 12 the end date and time indicating the timing at which the user operation requesting the end of acquisition of the captured images and recognition results was accepted, in association with the start date and time, IP address, etc. (St4).

Based on a user operation requesting the display of the recognition results, the terminal device P1 copies and stores the various data (specifically, the start date and time, IP address, end date and time, all captured images acquired between the start date and time and the end date and time, and each recognition result, etc.) stored in the memory 12 during the processing of steps St1 to St4 to the HDD 25 (St5).

The terminal device P1 executes a process of deleting temporally overlapping captured images, an image separation process, etc. from each of the multiple captured images copied to the HDD 25, and generates each of the multiple plate images (plate images NP1, NP2, NP3, NP4 included in the number recognition result screen SC0 shown in FIG. 6 ) cut out from the detected license plate NP. The terminal device P1 generates display recognition result data DT0 (see FIG. 6 ) based on each of the generated multiple plate images and each recognition result (St6).

The terminal device P1 also performs an analysis process of the installation or setting of the camera C1 based on each recognition result (St7). Specifically, the terminal device P1 performs an analysis process of each analysis item, such as the pixel count correctness rate, plate tilt correctness rate, brightness difference correctness rate, detection area inner frame rate, serial number matching rate, and number of recognition results, based on each recognition result. Note that each analysis item here is, for example, an item for determining whether or not the camera C1 that captures the license plate NP of the vehicle to be captured has been installed or set up, and is not limited to the above-mentioned items.

The terminal device P1 generates a number recognition result screen SC0 (see FIG. 6) based on the analysis results of the multiple analysis items (St8), and outputs and displays it on the monitor 24. The terminal device P1 also generates advice information based on the analysis results of the multiple analysis items for each set detection area, and outputs the advice information for each corresponding detection area to the analysis result display area DT17 (see FIG. 6) of the number recognition result screen SC0 (St9).

When the plate position confirmation button BT2 included in the number recognition result screen SC0 is pressed (selected) by a user operation, the terminal device P1 generates a plate position drawing image (see Figures 7 to 10) and outputs it to the monitor 24 for display (St10).

Now, referring to FIG. 4 and FIG. 5, the process of steps St7 to St9 shown in FIG. 2 (i.e., the analysis process of multiple analysis items and the generation process of advice information executed by the terminal device P1) will be described. FIG. 4 is a table TB1 showing an example of the achievement conditions and the unachieved conditions of each analysis item of the license plate NP. FIG. 5 is a table TB2 showing an example of advice information. Note that each index, each achievement condition, and each unachieved condition shown in table TB1 are examples, and it goes without saying that they are not limited to these. Similarly, it goes without saying that the example of advice information corresponding to each analysis item shown in table TB2 is an example, and it goes without saying that they are not limited to these. Also, with reference to table TB2, an example of advice information corresponding to each analysis item that is generated when it is determined that the achievement condition of each analysis item is unachieved will be described, but when it is determined that the achievement condition of each analysis item is achieved, notification information that notifies the user that the achievement condition of this analysis item is achieved may be generated.

(Analysis of pixel number appropriateness rate)
The terminal device P1 analyzes the pixel number appropriateness rate based on the plate width ΔX captured in the detection area of the camera C1. The terminal device P1 judges whether the plate width ΔX of all license plates NP detected from each detection area of all captured images satisfies the achievement condition of 480 pixels ≧ ΔX ≧ 120 pixels. If the terminal device P1 judges that the plate width ΔX satisfies the achievement condition, it judges that the width (i.e., the number of pixels) of the captured license plate NP is a number of pixels more suitable for recognizing each piece of character information, or that the mounting position and mounting angle of the camera C1 (i.e., the depression angle or swing angle of the camera C1) are mounting angles that allow each piece of character information of the license plate NP to be recognized. On the other hand, if the terminal device P1 determines that the plate width ΔX does not satisfy the achievement condition (i.e., satisfies the unachieved condition), it determines that the current zoom magnification setting of the camera C1, or the mounting position or mounting angle of the camera C1, needs to be adjusted, and generates advice information indicating that the number of angles of view of the license plate NP reflected in each detection area needs to be increased, such as "Adjust the camera zoom so that the plate width within the detection area is increased," or advice information indicating that the mounting position or mounting angle of the camera C1 needs to be adjusted, such as "Adjust the camera mounting position (distance/swing angle) so that the plate width within the detection area is increased."

Furthermore, the terminal device P1 calculates (analyzes) the percentage of all number plates NP assigned with the same vehicle ID that have a measured plate width ΔX that satisfies the achievement condition as the pixel number appropriateness rate for this vehicle ID. Furthermore, when the terminal device P1 receives a request for analyzing the pixel number appropriateness rate for all vehicle IDs through user operation, it calculates (analyzes) the percentage of all number plates NP with all vehicle IDs that have a measured plate width ΔX that satisfies the achievement condition as the pixel number appropriateness rate for all vehicle IDs. This allows the terminal device P1 to determine whether the zoom magnification, mounting position (i.e., imaging distance), or mounting angle of the camera C1 are more appropriate based on the pixel number appropriateness rate.

(Analysis of inclination appropriateness rate)
The terminal device P1 also analyzes the plate tilt appropriateness rate based on the plate tilt θ reflected in each detection area of the camera C1. The terminal device P1 judges whether the plate tilt θ of all number plates NP detected from each detection area of all captured images satisfies the achievement condition "θ<11°". If the terminal device P1 judges that the plate tilt θ satisfies the achievement condition, it judges that the mounting angle of the camera C1 (i.e., the depression angle or swing angle of the camera C1) is an installation angle at which each character information of the number plate NP can be recognized. On the other hand, if the terminal device P1 judges that the plate tilt θ does not satisfy the achievement condition (i.e., satisfies the unachieved condition "θ≧11°"), it judges that the mounting angle of the camera C1 (i.e., the depression angle or swing angle of the camera C1) is large, and generates advice information indicating that the depression angle or mounting angle of the camera C1 needs to be adjusted, such as "Adjust the depression angle/swing angle of the camera so that the plate angle in the detection area becomes smaller."

The terminal device P1 calculates (analyzes) the proportion of all number plates NP assigned with the same vehicle ID that have a measured plate tilt θ that satisfies the achievement condition as the plate tilt appropriateness rate for this vehicle ID. Furthermore, when the terminal device P1 receives a request for analyzing the plate tilt appropriateness rate of all vehicle IDs recognized in all captured images through user operation, it calculates (analyzes) the proportion of all number plates NP of all vehicle IDs that have a measured plate tilt θ that satisfies the achievement condition as the plate tilt appropriateness rate for all vehicle IDs. This allows the terminal device P1 to determine whether the mounting angle of the camera C1 is more appropriate based on the calculated plate tilt appropriateness rate.

(Analysis of luminance difference appropriate rate)
The terminal device P1 also analyzes the brightness difference appropriateness rate based on the brightness difference ΔL of the license plate NP reflected in each detection area of the camera C1. The terminal device P1 calculates the brightness difference ΔL, which is the difference between the maximum brightness value and the minimum brightness value of all license plates NP detected from each detection area in all captured images, and judges whether this brightness difference ΔL satisfies the achievement condition "ΔL≧60". If the terminal device P1 judges that the brightness difference ΔL satisfies the achievement condition, it judges that the gain level or brightness correction (i.e., dark area correction and bright area correction) settings of the camera C1, or the settings of the illumination unit 14 are set so that each character information of the license plate NP can be recognized. On the other hand, if the terminal device P1 judges that the brightness difference ΔL does not satisfy the achievement condition (i.e., satisfies the unachieved condition "ΔL<60"), it judges that the gain level or brightness correction settings of the camera C1, or the brightness (illuminance) of the illumination unit 14 need to be adjusted, and generates advice information.

Here, if the luminance difference ΔL does not satisfy the achievement condition, the terminal device P1 may generate more detailed advice information based on the maximum and minimum luminance values of the license plate NP, or the image capture time of the license plate NP. Below, an example of generating advice information based on the maximum and minimum luminance values of the license plate NP, or the image capture time of the license plate NP, is described.

First, if the luminance difference ΔL does not satisfy the achievement condition, the terminal device P1 determines whether the captured image has blackout or whiteout based on the maximum and minimum luminance values. In addition, the terminal device P1 determines whether the license plates NP were captured during the day or night based on the information on the capture times of the multiple captured images.

When the terminal device P1 determines that the captured image is blacked out and that the image was taken during the daytime, it generates advice information indicating that the camera C1 needs to be adjusted to increase the dark area correction, such as "Increase the dark area correction so that the difference between the maximum luminance and minimum luminance of the plate is large." When the terminal device P1 determines that the captured image is blacked out and that the image was taken during the nighttime, it generates advice information indicating that the camera C1 needs to be adjusted to increase the gain level, such as "Increase the gain level so that the difference between the maximum luminance and minimum luminance of the plate is large," or advice information indicating that the brightness of the imaged area captured by the camera C1 needs to be adjusted using the lighting unit 14 (IR-LED) or other lighting different from the lighting unit 14 provided in the camera C1, such as "Adjust the brightness using IR-LED or lighting as necessary."

On the other hand, when the terminal device P1 determines that the captured image is overexposed and that the image was taken during the daytime, it generates advice information indicating that the camera C1 needs to be adjusted to increase the correction for bright areas, such as "Increase the correction for bright areas so that the difference between the maximum and minimum luminance of the plate is large." When the terminal device P1 determines that the captured image is overexposed and that the image was taken during the nighttime, it generates advice information indicating that the camera C1 needs to be adjusted to decrease the gain level in its settings, such as "Decrease the gain level so that the difference between the maximum and minimum luminance of the plate is large," or advice information indicating that the brightness of the imaged area captured by the camera C1 needs to be adjusted using the lighting unit 14 (IR-LED) or other lighting different from the lighting unit 14 provided in the camera C1, such as "Adjust the brightness using the IR-LED or lighting as necessary."

The determination of whether the captured image is blacked out or overexposed may be performed by the user. Similarly, the determination of whether the time period during which the license plate NP was captured is daytime or nighttime may be performed by the user. In such a case, the terminal device P1 accepts, via the input unit 23, an input operation by the user as to whether the captured image is blacked out or overexposed, and an input operation by the user as to whether the time period during which the image was captured is daytime or nighttime. The terminal device P1 generates advice information based on the user input. The determination of whether the captured image is blacked out or overexposed is not essential and may be omitted.

The terminal device P1 calculates (analyzes) the proportion of all number plates NP assigned with the same vehicle ID that have a measured brightness difference ΔL that satisfies the achievement condition as the brightness difference appropriateness rate for this vehicle ID. Furthermore, when the terminal device P1 receives a request for analyzing the brightness difference appropriateness rate of all vehicle IDs recognized in all captured images through a user operation, it calculates (analyzes) the proportion of all number plates NP with vehicle IDs that have a measured brightness difference ΔL that satisfies the achievement condition as the brightness difference appropriateness rate for all vehicle IDs. This allows the terminal device P1 to determine whether the camera C1 is set to be able to more accurately acquire an image of the number plate NP that has a brightness difference sufficient to recognize each character information of the number plate NP, based on the calculated brightness difference appropriateness rate.

(Analysis of detection area inner frame rate)
The terminal device P1 also analyzes the detection area inner frame rate based on the number of license plates NP detected in the second detection zone among all license plates NP detected in the detection area of the camera C1. The terminal device P1 judges in which detection zone the license plate NP detected in each detection area of the camera C1 was detected. Specifically, the terminal device P1 judges in which detection zone of each detection area each of the two midpoints PS3 and PS4 of the license plate NP was detected. The terminal device P1 judges that the achievement condition is met when both of the two midpoints PS3 and PS4 are detected in the first detection zone. The terminal device P1 also judges that the achievement condition is not met when either of the two midpoints PS3 and PS4 is detected in the second detection zone.

The terminal device P1 calculates (analyzes) the detection area inner frame rate based on the number of number plates NP detected in the second detection zone among all number plates NP detected in each detection area of the camera C1 as a result of the detection zone determination process in which each of the number plates NP was detected, and determines whether the number plate NP can be more appropriately detected in each detection area. If the terminal device P1 determines that the detection area inner frame rate is equal to or greater than a predetermined value (e.g., 90%, 95%, etc.) set by the user, it determines that the number plate NP can be more appropriately detected in each currently set detection area. If the terminal device P1 determines that the detection area inner frame rate is not equal to or greater than the predetermined value set by the user, it generates advice information indicating that the number plate NP may not be more appropriately detected in the currently set detection area, such as "Adjust the detection area on the setting screen of the number recognition app so that it fits within the detection area inner frame (dotted line) so as not to be affected by the vehicle's driving route." This allows the terminal device P1 to determine, based on the calculated detection area inner frame rate, whether the detection area currently set for the camera C1 is set to a more appropriate range when recognizing each piece of character information on the license plate NP.

The terminal device P1 calculates (analyzes) the detection area inner frame rate for vehicles having the same vehicle ID and detected within the first detection zone AR11 among the license plates NP of vehicles assigned with this vehicle ID. Furthermore, when the terminal device P1 receives a request for analyzing the detection area inner frame rate for all vehicle IDs through user operation, it calculates (analyzes) the proportion of all detected license plates NP that satisfy the achievement condition (i.e., detection within the first detection zone) as the detection area inner frame rate for all vehicle IDs. This allows the terminal device P1 to determine whether an appropriate detection area has been set by detecting the license plate NP based on the calculated (analyzed) detection area inner frame rate.

(Analysis of serial number match rate)
The terminal device P1 also determines the most frequent serial number (hereinafter referred to as the "most frequent serial number") among all the serial numbers recognized on all the license plates NP of vehicles with the same vehicle ID detected within each detection area of the camera C1, and calculates (analyzes) the serial number match rate based on the number of the most frequent serial number among all the recognized serial numbers. If the terminal device P1 determines that the calculated serial number match rate is equal to or greater than a predetermined value (e.g., 95%, 98%, etc.), it determines that the recognized serial numbers are unlikely to have been erroneously recognized. On the other hand, if the terminal device P1 determines that the calculated serial number match rate is not equal to or greater than the predetermined value, it determines that the recognized serial numbers are likely to have been erroneously recognized, and generates advice information based on the analysis items that do not satisfy the achievement conditions among the analysis results of the other analysis items. For example, when the terminal device P1 determines that each of the serial number matching rate, the pixel number appropriate rate, and the plate inclination appropriate rate does not satisfy the achievement condition, it generates advice information that is generated when the achievement condition of each of the pixel number appropriate rate or the plate inclination appropriate rate is not satisfied, and when it determines that each of the serial number matching rate and the brightness difference appropriate rate does not satisfy the achievement condition, it generates advice information that is generated when the achievement condition of the brightness difference appropriate rate is not satisfied. In this way, the terminal device P1 can visualize the rate of erroneous recognition of the most frequent serial number recognition result performed by the camera C1 based on the calculated (analyzed) serial number matching rate, and can generate advice information regarding the installation or setting of the camera C1 based on the achievement status of each achievement condition of the other analysis items (i.e., the determination result of whether the achievement condition is satisfied or not).

(Analysis of the number of recognition results)
The terminal device P1 also calculates (analyzes) the total number of license plates NP for which recognition of each character information has been performed among the license plates NP of all vehicles captured by the camera C1 (hereinafter, referred to as the "number of recognition results"). The terminal device P1 judges whether the calculated (analyzed) number of recognition results is equal to or greater than a predetermined value (e.g., 9 or more). If the terminal device P1 judges that the number of recognition results is equal to or greater than the predetermined value, it judges that the number of samples for generating a best shot image of each vehicle is sufficient. On the other hand, if the terminal device P1 judges that the calculated (analyzed) number of recognition results is not equal to or greater than the predetermined value, it judges that the number of samples for generating a best shot image of each vehicle is not sufficient, and generates advice information indicating that the depression angle or mounting angle of the camera C1 needs to be adjusted, such as "Adjust the depression angle/swing angle of the camera so that the number of number recognition results is equal to or greater than a certain value." This allows the terminal device P1 to judge whether the depression angle or mounting angle of the camera C1 is more appropriately adjusted based on the calculated (analyzed) number of recognition results.

As a result of the above, the terminal device P1 generates the number recognition result screen SC0 and the analysis result display area DT17 included in the number recognition result screen SC0. Here, with reference to FIG. 6, the number recognition result screen SC0 generated by the processing of step St8 and the analysis result display area DT17 generated by the processing of step St9 will be described. FIG. 6 is a diagram showing an example of the number recognition result screen SC0.

The license plate recognition result screen SC0 is generated including display recognition result data DT0 including each of the recognition results sent from camera C1 and each of the generated plate images NP1 to NP4, analysis result display tabs ARA and ARB that display the analysis results and advice information for each set detection area, a setting screen button BT1, and a plate position confirmation button BT2 for displaying a plate position drawing image. Note that the license plate recognition result screen SC0 shown in FIG. 6 shows an example in which two detection areas ("detection area 1" and "detection area 2") have been set, but it goes without saying that the number of detection areas that can be set is not limited to this.

The display recognition result data DT0 is an area in which each of the generated plate images NP1 to NP4 and each recognition result based on the captured image from which each of these plate images NP1 to NP4 was generated are displayed. Note that while the number of plate images and recognition results included in the display recognition result data DT0 shown in FIG. 6 is an example in which the number is four, it goes without saying that the number may be three or less, or five or more.

The license plate image field DT1 displays each of the multiple plate images NP1 to NP4 that have been generated. The license plate image field DT1 may also display thumbnail images of each of the multiple plate images NP1 to NP4. For example, in the example shown in FIG. 6, the license plate image field DT1 displays the plate images NP1 and NP3 of the license plate "Tama XXX sa △△-XX" and the plate images NP2 and NP4 of the license plate "Setagaya △△△ a XX-XXX".

The Regional Land Transport Bureau column DT2 shows the recognition results performed by the camera C1, which recognizes the Regional Land Transport Bureau information from among the character information of the license plates shown in the captured images corresponding to each of the multiple license plate images NP1 to NP4. For example, in the example shown in FIG. 6, the Regional Land Transport Bureau column DT2 shows that the Regional Land Transport Bureau information for plate image NP1 is "Tama," the Regional Land Transport Bureau information for plate image NP2 is "Setagaya," the Regional Land Transport Bureau information for plate image NP3 is "Tama," and the Regional Land Transport Bureau information for plate image NP4 is "Setagaya."

The vehicle type column DT3 shows the results of the recognition performed by the camera C1, which recognizes the vehicle type information of the vehicle from among the character information of the license plates shown in the captured images corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in Figure 6, the vehicle type column DT3 shows that the vehicle type information of plate image NP1 is "XXX", the vehicle type information of plate image NP2 is "△△△", the vehicle type information of plate image NP3 is "XXX", and the vehicle type information of plate image NP4 is "△△△".

The type column DT4 shows the result of the recognition performed by the camera C1 to recognize the vehicle type information from among the character information of the license plates shown in the captured images corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in Figure 6, the type column DT4 shows that the type information of plate image NP1 is "sa", the type information of plate image NP2 is "a", the type information of plate image NP3 is "sa", and the type information of plate image NP4 is "a".

The serial number column DT5 shows the recognition result executed by the camera C1, which recognizes the serial number information from among the character information of the license plates shown in the captured images corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in FIG. 6, the serial number column DT5 shows that the serial number information of plate image NP1 is "△△××", the serial number information of plate image NP2 is "××○○", the serial number information of plate image NP3 is "△△××", and the serial number information of plate image NP4 is "××○○".

The vehicle identification ID column DT6 indicates the vehicle ID assigned to each vehicle to make each vehicle identifiable. The vehicle ID may be set in advance by the user, or may be assigned by the terminal device P1 based on each recognition result acquired by the camera C1. For example, in the example shown in FIG. 6, the vehicle identification ID column DT6 indicates that the vehicle ID assigned to the vehicle corresponding to plate image NP1 is "2A", the vehicle ID assigned to the vehicle corresponding to plate image NP2 is "38", the vehicle ID assigned to the vehicle corresponding to plate image NP3 is "2A", and the vehicle ID assigned to the vehicle corresponding to plate image NP4 is "38".

The time column DT7 indicates the imaging time when the image corresponding to each of the plate images NP1 to NP4 was captured. For example, in the example shown in FIG. 6, the time column DT7 indicates that the imaging date and time information when the image corresponding to plate image NP1 was captured is "12:11:12.34", the imaging date and time information when the image corresponding to plate image NP2 was captured is "12:11:12.66", the imaging date and time information when the image corresponding to plate image NP3 was captured is "12:11:12.34", and the imaging date and time information when the image corresponding to plate image NP4 was captured is "12:11:12.66".

The area column DT8 indicates the detection area in which each of the multiple plate images NP1 to NP4 was detected, out of the one or more detection areas that were set. For example, in the example shown in FIG. 6, the area column DT8 indicates that the detection area information in which the license plate corresponding to plate image NP1 was detected is "1", the detection area information in which the license plate corresponding to plate image NP2 was detected is "2", the detection area information in which the license plate corresponding to plate image NP3 was detected is "1", and the detection area information in which the license plate corresponding to plate image NP4 was detected is "2".

The width column DT9 indicates the plate width ΔX of each license plate detected in the captured image corresponding to each of the multiple plate images NP1 to NP4. The unit of plate width ΔX is "pixels (px)." For example, in the example shown in FIG. 6, the width column DT9 indicates that the plate width ΔX in plate image NP1 is "160," the plate width ΔX in plate image NP2 is "130," the plate width ΔX in plate image NP3 is "160," and the plate width ΔX in plate image NP4 is "130."

The tilt column DT10 indicates the plate tilt θ of each license plate detected in the captured images corresponding to each of the multiple plate images NP1 to NP4. The unit of plate tilt θ is degrees (°). For example, in the example shown in FIG. 6, the tilt column DT10 indicates that the plate tilt θ in plate image NP1 is "5", the plate tilt θ in plate image NP2 is "18", the plate tilt θ in plate image NP3 is "5", and the plate tilt θ in plate image NP4 is "18".

The brightness difference column DT11 shows the brightness difference, which is the difference between the maximum brightness value and the minimum brightness value of each license plate detected in the captured images corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in FIG. 6, the brightness difference column DT11 shows that the brightness difference ΔL of the license plate captured in plate image NP1 is "80", the brightness difference ΔL of the license plate captured in plate image NP2 is "30", the brightness difference ΔL of the license plate captured in plate image NP3 is "80", and the brightness difference ΔL of the license plate captured in plate image NP4 is "30".

The in-area position column DT12 indicates in which detection zone in each detection area each license plate detected in the captured image corresponding to each of the multiple plate images NP1 to NP4 was detected. Specifically, the terminal device P1 determines that the detection zone is the second detection zone (an example of "zone B") when at least one of the two midpoints PS3 and PS4 of the detected license plate is detected in the second detection zone, and determines that the detection zone is the first detection zone (an example of "zone A") when both of the two midpoints PS3 and PS4 of the detected license plate are detected in the first detection zone. For example, in the example shown in FIG. 6, the in-area position column DT12 indicates that the plate image NP1 was detected in the detection zone indicated by "zone A", the plate image NP2 was detected in the detection zone indicated by "zone B", the plate image NP3 was detected in the detection zone indicated by "zone A", and the plate image NP4 was detected in the detection zone indicated by "zone B".

The left center coordinate column DT13 indicates the coordinate of the midpoint PS3 of the two midpoints PS3, PS4 of each license plate detected in the captured image corresponding to each of the multiple license plate images NP1 to NP4. For example, in the example shown in FIG. 6, the left center coordinate column DT13 indicates that the left center coordinate of the license plate captured in license plate image NP1 is "1550, 150", the left center coordinate of the license plate captured in license plate image NP2 is "1551, 151", the left center coordinate of the license plate captured in license plate image NP3 is "1551, 152", and the left center coordinate of the license plate captured in license plate image NP4 is "1550, 153".

The right center coordinate column DT14 indicates the coordinate of the midpoint PS4 of the two midpoints PS3, PS4 of each license plate detected in the captured image corresponding to each of the multiple license plate images NP1 to NP4. For example, in the example shown in FIG. 6, the right center coordinate column DT14 indicates that the right center coordinate of the license plate captured in license plate image NP1 is "2150,550", the right center coordinate of the license plate captured in license plate image NP2 is "2150,550", the right center coordinate of the license plate captured in license plate image NP3 is "2150,550", and the right center coordinate of the license plate captured in license plate image NP4 is "2150,550".

The low brightness value column DT15 indicates the minimum brightness value of each license plate detected in the captured image corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in FIG. 6, the low brightness value column DT15 indicates that the minimum brightness value of plate image NP1 is "97", the minimum brightness value of plate image NP2 is "70", the minimum brightness value of plate image NP3 is "97", and the minimum brightness value of plate image NP4 is "70".

The high brightness value column DT16 indicates the maximum brightness value of each license plate detected in the captured image corresponding to each of the multiple plate images NP1 to NP4. For example, in the example shown in FIG. 6, the high brightness value column DT16 indicates that the maximum brightness value of plate image NP1 is "177", the maximum brightness value of plate image NP2 is "100", the maximum brightness value of plate image NP3 is "177", and the maximum brightness value of plate image NP4 is "100".

The analysis result display tabs ARA and ARB are tabs for displaying analysis results and advice information corresponding to each detection area. When the analysis result display tab ARA is selected (pressed) by a user operation, it displays the analysis results and advice information regarding the license plate of a vehicle detected within a detection area whose name has been set as "detection area 1". When the analysis result display tab ARB is selected (pressed) by a user operation, it displays the analysis results and advice information regarding the license plate of a vehicle detected within a detection area whose name has been set as "detection area 2".

The analysis result display area DT17 shown in FIG. 6 displays the analysis results and advice information related to the vehicle license plate when the analysis result display tab ARA is selected. When the vehicle list field LT1 is selected (pressed) by a user operation, it displays in a pull-down menu the vehicle IDs detected in "detection area 1" that are the targets for which the analysis results and advice information are to be displayed. For example, when the vehicle list field LT1 is selected (pressed) by a user operation, it displays in a pull-down menu "All vehicle IDs," "2A," "38," etc. The terminal device P1 generates analysis results and advice information based on each recognition result of the vehicle ID selected by the user operation in the vehicle list field LT1, and displays them in the analysis result display area DT17.

The terminal device P1 generates an analysis result display area DT17 that includes the analysis results for each analysis item and advice information based on the analysis results, and generates a number recognition result screen SC0 that includes this analysis result display area DT17 and displays it on the monitor 24.

For example, in the example shown in FIG. 6 , the analysis result display area DT17 includes, as the analysis results and advice information for each analysis item, an analysis result for the plate width appropriateness rate "Plate width appropriateness rate 95%" and a message "The detection area, distance, depression angle, and swing angle are appropriate", an analysis result for the plate inclination appropriateness rate "Plate inclination appropriateness rate 90%" and advice information "Please adjust the depression angle and swing angle", an analysis result for the brightness difference appropriateness rate "Brightness difference appropriateness rate 60%" and advice information "Please adjust the brightness", an analysis result for the detection area within rate "Detection area within rate 95%" and a message "The vehicle passing position is appropriate", an analysis result for the number of recognition results "Number of recognition results 5" and advice information "Please adjust if the vehicle speed is too fast", and an analysis result for the recognition result match rate as an example of the serial number match rate "Recognition result match rate 90%" and advice information "Some serial numbers do not match. Within the detection area...". Note that the advice information regarding the recognition result matching rate is an example in which only a portion is shown, and the entire text can be confirmed by a user operation (e.g., a scroll operation, etc.). The message shown in Fig. 6 is notification information that notifies the user that the analysis result is appropriate.

The setting screen button BT1 is a button that, when selected (pressed) by a user operation, displays a setting screen (not shown) for setting various parameters of the camera C1 (e.g., parameters such as gain level, brightness compensation, and darkness compensation).

When the plate position confirmation button BT2 is selected (pressed) by a user operation, the display transitions to a plate position confirmation screen (e.g., plate position confirmation screens SC1, SC2, SC3, SC4, SC5, SC6, SC7 shown in Figures 7 to 10) including a plate position drawing image (e.g., plate position drawing images CI1, CI2, CI3, CI4, CI5, CI6, CI7 shown in Figures 7 to 10). When the plate position confirmation button BT2 is selected (pressed) by a user operation, the terminal device P1 generates a plate position drawing image, generates a plate position confirmation screen including this plate position drawing image, and displays it on the monitor 24.

Next, examples of plate position confirmation screens will be described with reference to Figs. 7 to 10. Fig. 7 is a diagram showing examples of plate position confirmation screens SC1 and SC2 before and after camera installation assistance. Fig. 8 is a diagram showing examples of plate position confirmation screens SC3 and SC4 before and after camera installation assistance. Fig. 9 is a diagram showing examples of plate position confirmation screens SC5 and SC6 before and after camera installation assistance. Fig. 10 is a diagram showing an example of a plate position confirmation screen SC7 when multiple detection areas AR6 and AR7 have each been set.

In each of the plate position confirmation screens SC1 to SC7, the two points and lines indicating the plate positions of each of the multiple license plates detected within each detection area are marked with symbols only in a portion of the plate position drawing image CI1 shown in Figure 7, and are omitted from the other plate position drawing images CI2 to CI7.

First, each of the plate position confirmation screens SC1 to SC7 will be described. Each of the plate position confirmation screens SC1 to SC7 is generated to include a message MS stating, "Check to display data that does not meet the conditions with a red dotted line," an indicator list LT2 that allows the user to select one or more indicators (plate width ΔX, plate tilt θ, brightness difference ΔL, and detection zone within the detection area) for which the user wishes to determine whether or not the achievement condition is met, a save button BT3 that saves the plate position drawing image displayed on the plate position confirmation screen when selected (pressed) by the user, a vehicle list LT3 as the target for checking the plate position in each detection area, and a plate position drawing image.

Next, each of the plate position drawing images CI1 to CI7 will be described. Each of the plate position drawing images CI1 to CI7 is displayed by superimposing two midpoints PS3 and PS4 (e.g., points PP1 and PP2 shown in FIG. 7) indicating the positions of each of the multiple license plates NP detected from each captured image on an image with a detection area generated by the terminal device P1, and a straight line connecting these two midpoints PS3 and PS4 (e.g., straight line SL1 connecting points PP1 and PP2 shown in FIG. 7). Here, the terminal device P1 visualizes whether each of the multiple license plates NP indicated by the two points and the straight line meets the achievement conditions of the indicators specified (selected) by the user by changing the color of the superimposed points and the line type of the straight line based on the determination result of whether or not the achievement conditions of one or more indicators specified (selected) by the user from the indicator list LT2 are met.

Here, we will explain the drawing process of the plate position (i.e., the position of the license plate NP) executed by the terminal device P1.

When generating the plate position drawing image, the terminal device P1 judges whether each of the detected number plates NP satisfies the achievement condition based on each index specified in the index list LT2 (for example, in the example of the plate position confirmation screen SC1 shown in FIG. 7, each of the plate width ΔX, plate inclination θ, and brightness difference ΔL of the number plate NP). Furthermore, based on the judgment result, the terminal device P1 judges whether the position of two points indicating the position of each number plate NP is located within the first detection zone. As a result of these judgments, the terminal device P1 superimposes in red on the captured image with the detection area the two points indicating the position of the number plate NP that does not satisfy the achievement condition. Furthermore, if at least one of the two points is superimposed in red, the terminal device P1 superimposes the straight line connecting these two points as a red dashed line.

Below, we will explain a specific example of drawing the plate position (i.e., the position of the license plate NP) with reference to the plate position drawing image CI1 shown in Figure 7.

When generating the plate position drawing image CI1 shown in FIG. 7, for example, the terminal device P1 judges whether each of the detected number plates NP satisfies the achievement conditions based on the respective indicators of the plate width ΔX, plate inclination θ, and brightness difference ΔL of the number plate NP specified in the indicator list LT2. Furthermore, the terminal device P1 judges whether the positions of the two points indicating the position of each number plate NP are located within the first detection zone AR21 based on the judgment results. As a result of these judgments, the terminal device P1 superimposes in red on the captured image with the detection area the two points indicating the positions of the number plates NP that do not satisfy the achievement conditions and the points that are determined to be not located within the first detection zone AR21. Furthermore, when at least one of the two points is superimposed in red, the terminal device P1 superimposes the straight line connecting these two points as a red and dashed line.

For example, each of the two points PP1 and PP2 is a point that has been determined not to satisfy any of the achievement conditions for each index, and is superimposed in red on the captured image with the detection area. The straight line SL1 connecting points PP1 and PP2 is a point that has been determined not to satisfy any of the achievement conditions for each index, and is superimposed in red dashed line on the captured image with the detection area.

For example, each of the two points PP3 and PP4 is determined to have met the achievement conditions for all of the indices, and is superimposed in blue on the captured image with the detection area. The straight line SL2 connecting the points PP3 and PP4 is superimposed in blue solid line on the captured image with the detection area, because both of the two points PP3 and PP4 are determined to have met the achievement conditions for all of the indices.

For example, point PP5 indicating the midpoint of the left short side of the license plate NP (i.e., the left central coordinate) is a point that is determined not to satisfy any of the achievement conditions of each of the indicators, and is superimposed in red on the captured image with the detection area. The midpoint of the right short side of the license plate NP (i.e., the right central coordinate) is a point that is determined to satisfy the achievement conditions of all of the indicators, and is superimposed in blue on the captured image with the detection area. The straight line SL3 connecting points PP5 and PP6 is superimposed as a red dashed line on the captured image with the detection area, because point PP5 is a point that is determined not to satisfy any of the achievement conditions of each of the indicators.

As a result, the user can check at a glance the positions of both ends of each license plate NP (specifically, the midpoints of the license plate NP in the short side direction) based on the positions of the two points, and the plate inclination of each license plate NP based on the inclination of the straight line, and can confirm whether the camera C1 has been installed and set up in a way that is suitable for the recognition of the license plate NP by the camera C1. Furthermore, based on the plate position drawing images CI1 to CI7, the user can easily imagine where and at what angle the vehicle's license plate NP will be captured in the captured image when the currently installed camera C1 is actually operated. Therefore, the camera installation support system 100 can support the installation and setting of the camera C1 as desired by the user.

The plate position confirmation screen SC1 shown in FIG. 7 is displayed including a plate position drawing image CI1. The plate position drawing image CI1 shows the plate positions of each of the 13 license plates NP in a captured image with detection areas, in which a frame line indicating the detection area AR2 and dashed lines indicating the first detection zone AR21 and two second detection zones AR22 within the detection area AR2 are superimposed.

This allows the user to intuitively confirm at a glance that the achievement conditions are not met for any of the multiple indicators specified in the indicator list LT2 (i.e., plate width ΔX, plate tilt θ, or brightness difference ΔL) based on the colors of the dots and the colors and line types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI1. In other words, the camera installation support system 100 can efficiently support the user in installing and setting up the desired camera C1 using the plate position drawing image CI1.

The plate position confirmation screen SC2 shown in FIG. 7 is a screen generated by the terminal device P1 based on an image captured again by the camera C1 after the user sets the mounting position, mounting angle, depression angle or swing angle of the camera C1, or the lighting unit 14 of the camera C1 or various parameters (e.g., gain level adjustment, dark area correction adjustment, light area correction adjustment, etc.), and the recognition results based on the captured image. The plate position confirmation screen SC2 is displayed including a plate position drawing image CI2. The plate position drawing image CI2 shows the plate positions of each of the eight license plates NP in an image with detection areas, in which a frame line indicating the detection area AR3 and dashed lines indicating the first detection zone AR21 and two second detection zones AR22 of the detection area AR2 are superimposed.

This allows the user to intuitively confirm at a glance that the detection area has been set so that license plates NP are more accurately detected within the set detection area AR2, based on the colors of the dots and the colors and types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI2. In other words, the camera installation support system 100 can efficiently support the user in installing and setting up the desired camera C1 by generating a plate position drawing image CI1 that visualizes the determination result of whether or not each of the detected license plates NP satisfies the achievement conditions of the indicators specified by the user.

The plate position confirmation screen SC3 shown in FIG. 8 is displayed including a plate position drawing image CI3. In generating the plate position drawing image CI3 shown in FIG. 8, the terminal device P1 determines whether each of the detected number plates NP satisfies the achievement condition based on the detection area, which is one index specified by the user in the index list LT2. The plate position drawing image CI3 shows the plate positions of the 12 number plates NP in a captured image with a detection area, in which a frame line indicating the detection area AR3 and dashed lines indicating the first detection zone AR31 and two second detection zones AR32 of the detection area AR3 are superimposed.

This allows the user to efficiently confirm at a glance that the range of the set detection area AR3 is not suitable for detecting the vehicle license plate NP, based on the colors of the dots and the colors and types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI3.

The plate position confirmation screen SC4 shown in FIG. 8 is a screen generated by the terminal device P1 based on the captured image captured by the camera C1 again after the user changes the detection area from AR3 to AR4, and on the recognition results based on the captured image. The plate position drawing image CI4 shows the plate positions of the 12 license plates NP in an image with a detection area, in which a frame line indicating the detection area AR4 and dashed lines indicating the first detection zone AR41 and two second detection zones AR42 of the detection area AR4 are superimposed.

This allows the user to intuitively confirm at a glance that the detection area has been set so that license plates NP are more accurately detected within the set detection area AR4, based on the colors of the dots and the colors and types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI4. In other words, the camera installation support system 100 can efficiently support the user in installing and setting up the desired camera C1 by generating a plate position drawing image CI3 that visualizes the determination result of whether or not each of the detected license plates NP satisfies the achievement conditions of the indicators specified by the user.

The plate position confirmation screen SC5 shown in FIG. 9 is displayed including a plate position drawing image CI5. In generating the plate position drawing image CI5 shown in FIG. 9, the terminal device P1 determines whether each of the detected number plates NP satisfies the achievement condition based on the brightness difference, which is one index specified by the user in the index list LT2. In the example shown in FIG. 9, the region HL in the detection area AR5 indicates the area that is brightened by strong light from the lighting unit 14 or other lighting. The plate position drawing image CI5 shows the plate positions of the 12 number plates NP in a captured image with a detection area, in which a frame line indicating the detection area AR5 and dashed lines indicating the first detection zone AR51 and two second detection zones AR52 of the detection area AR5 are superimposed.

This allows the user to efficiently confirm at a glance that the brightness difference of each license plate NP detected in area HL does not satisfy the achievement condition based on the colors of the dots and the colors and types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI5, and that the brightness is not more suitable for the camera C1 to detect the license plate NP and perform the recognition process of each piece of character information.

The plate position confirmation screen SC6 shown in FIG. 9 is a screen generated by the terminal device P1 based on an image captured by the camera C1 again after the user adjusts the settings of the illumination unit 14 of the camera C1 or the settings of various parameters of the camera C1 (e.g., gain level adjustment, dark area compensation adjustment, light area compensation adjustment, etc.), and on the recognition results based on the captured image. The plate position confirmation screen SC6 is displayed including a plate position drawing image CI6. The plate position drawing image CI6 shows the plate positions of the 12 license plates NP in an image with a detection area, in which a frame line indicating the detection area AR5 and dashed lines indicating the first detection zone AR51 and two second detection zones AR52 of the detection area AR5 are superimposed.

This allows the user to intuitively confirm at a glance that the lighting unit 14 of the camera C1 or various parameters of the camera C1 have been set so that the license plate NP can be more accurately detected within the set detection area AR5, based on the colors of the dots and the colors and types of the lines indicating each plate position drawn (superimposed) on the plate position drawing image CI4. In other words, the camera installation support system 100 can efficiently support the user in installing and setting up the camera C1 as desired by generating a plate position drawing image CI5 that visualizes the determination result of whether or not each of the detected license plates NP satisfies the achievement conditions of the indicators specified by the user.

The plate position confirmation screen SC7 shown in Figure 10 is an example of a plate position confirmation screen generated when two detection areas AR6, AR7 are set. The plate position confirmation screen SC7 is displayed including a plate position drawing image CI7. In generating the plate position drawing image CI7 shown in Figure 10, the terminal device P1 determines whether each of the multiple license plates NP detected in each of the two detection areas AR6, AR7 satisfies the achievement conditions based on all the indicators (i.e., plate width ΔX, plate inclination θ, brightness difference ΔL, and detection area) specified by the user in the indicator list LT2. The plate position drawing image CI7 shows the plate positions of multiple license plates NP in an image with detection areas superimposed with frame lines indicating each of the two detection areas AR6 and AR7, dashed lines indicating the first detection zone AR61 and the two second detection zones AR62 of the detection area AR6, and dashed lines indicating the first detection zone AR71 and the two second detection zones AR72 of the detection area AR7.

This allows the user to intuitively check at a glance whether the current installation and settings of camera C1 are suitable for the vehicle license plate detection and recognition processing in each of detection areas AR6 and AR7 performed by camera C1, based on the colors of the dots and the colors and types of the lines indicating the plate positions in each of detection areas AR6 and AR7 drawn (superimposed) on plate position drawing image CI7. In other words, camera installation support system 100 can efficiently support the user in installing and setting up camera C1 as desired by generating plate position drawing image CI7 that visualizes the determination results of whether each of the multiple license plates NP detected in each of the two detection areas AR6 and AR7 meets the achievement conditions of the indicators specified by the user.

As described above, the camera installation assistance system 100 according to the first embodiment includes a terminal device P1, which is a computer connected to a camera C1 that captures images of a vehicle, and detects the vehicle's license plate NP reflected in one or more detection areas AR1 that are set in advance for the angle of view of the camera C1 from each of a plurality of captured images captured by the camera C1, determines whether the detection results satisfy a plurality of imaging conditions (i.e., satisfy the achievement conditions), and generates and outputs advice information regarding the installation or parameter settings of the camera C1 based on the determination result of whether each of the plurality of imaging conditions is satisfied.

As a result, the camera installation assistance system 100 according to embodiment 1 can generate advice information regarding the installation or parameter settings of the camera C1 based on the determination result of whether or not each of the license plates NP of vehicles detected within the detection area AR1 set by the user satisfies each imaging condition (each achievement condition) for recognizing each piece of character information on the license plate NP, thereby assisting in the installation or setting of the camera C1. In addition, the user can adjust the current installation or setting of the camera C1 based on the generated advice information.

Furthermore, as described above, the camera installation support system 100 according to the first embodiment calculates the coordinates of the two midpoints (e.g., each of the two midpoints PS3 and PS4 shown in FIG. 3) in a pair of short-side directions for each of the detected license plates NP, and generates and outputs a plate position drawing image (e.g., plate position drawing images CI1 to CI7 shown in FIGS. 7 to 10) in which the two midpoints for each license plate are plotted on the background image of the imaging area captured by the camera C1. Note that the background image referred to here may be an imaging image with a detection area. In this way, the camera installation support system 100 according to the first embodiment can visualize to the user an image of the plate width and plate inclination of the license plate NP captured by the camera C1 during actual operation.

In addition, as described above, the camera installation support system 100 according to the first embodiment calculates the width (plate width ΔX) of each of the multiple license plates NP based on the coordinates of the two midpoints (for example, each of the two midpoints PS3 and PS4 shown in FIG. 3), and outputs advice information if it is determined that the width (plate width ΔX) is equal to or less than a predetermined value (for example, does not satisfy the achievement condition for the plate width ΔX shown in FIG. 4). As a result, the camera installation support system 100 according to the first embodiment can determine whether the license plate NP shown in the captured image is a size suitable for the recognition processing of each character information, and output advice information regarding the installation or setting of the camera C1 based on the determination result. Therefore, the user can check whether the current installation or setting of the camera C1 is suitable for the recognition processing of each character information of the license plate NP, and adjust the installation or setting of the current camera C1 based on the advice information.

As described above, the camera installation support system 100 according to the first embodiment calculates the width (plate width ΔX) of each of the multiple license plates NP based on the coordinates of the two midpoints (for example, each of the two midpoints PS3 and PS4 shown in FIG. 3), and outputs advice information when it is determined that the detection ratio of license plates NP in which a part of the license plate width (plate width ΔX) is detected outside the detection area among all the detected license plates NP is equal to or greater than a predetermined ratio. This allows the camera installation support system 100 according to the first embodiment to output advice information regarding the current detection area setting and the installation or setting of the camera C1. Therefore, the user can check whether the current detection area setting and the installation or setting of the camera C1 are suitable for the recognition processing of each character information of the license plate NP, and can adjust the current detection area setting and the installation or setting of the camera C1 based on the advice information.

As described above, the camera installation support system 100 according to the first embodiment calculates the width (plate width ΔX) of each of the multiple license plates NP based on the coordinates of the two midpoints (for example, each of the two midpoints PS3 and PS4 shown in FIG. 3), sets multiple second detection zones (for example, the second detection zone AR12 shown in FIG. 3, etc.) that are smaller than the detection area within the detection area, and outputs advice information when it is determined that a part of the width (plate width ΔX) of each of the multiple detected license plates NP is detected within the multiple second detection zones. As a result, the camera installation support system 100 according to the first embodiment can determine whether each of the multiple detected license plates NP is located in the center of the detection area. Therefore, the user can adjust the current detection area setting, the installation or setting of the camera C1 based on the advice information.

Furthermore, as described above, the camera installation support system 100 according to the first embodiment outputs advice information when it determines that the detection ratio of license plates NP in which a part of the width (plate width ΔX) of the license plate NP is detected within a plurality of second detection zones among all detected license plates NP is equal to or greater than a predetermined ratio. As a result, the camera installation support system 100 according to the first embodiment can output advice information based on the ratio at which each of the detected license plates NP is located in the central part of the detection area. Therefore, the user can adjust the current detection area settings and the installation or settings of the camera C1 based on the advice information.

As described above, the camera installation support system 100 according to the first embodiment calculates the longitudinal tilt (plate tilt θ) of each of the detected number plates NP, and outputs advice information if it is determined that the calculated tilt (plate tilt θ) is equal to or greater than a set predetermined angle. This allows the camera installation support system 100 according to the first embodiment to determine whether the number plate NP shown in the captured image is captured at an angle that is more suitable for the recognition processing of each piece of character information, and to output advice information regarding the installation or setting of the camera C1 based on the determination result. Therefore, the user can check whether the current installation or setting of the camera C1 is more suitable for the recognition processing of each piece of character information of the number plate NP, and can adjust the installation or setting of the current camera C1 based on the advice information.

In addition, as described above, the camera installation support system 100 according to the first embodiment calculates the maximum and minimum luminance values of the pixels in each area of the detected license plate NP, and generates advice information when it is determined that the luminance difference ΔL between the maximum and minimum luminance values is less than a predetermined value. As a result, the camera installation support system 100 according to the first embodiment can determine whether the license plate NP shown in the captured image is captured with a brightness suitable for the recognition processing of each piece of character information, and output advice information regarding the installation or setting of the camera C1 based on the determination result. Therefore, the user can check whether the current installation or setting of the camera C1 is suitable for the recognition processing of each piece of character information of the license plate NP, and can adjust the current installation or setting of the camera C1, or the arrangement, brightness, etc. of the lighting installed around the camera C1 based on the advice information.

Furthermore, as described above, when the camera installation support system 100 according to the first embodiment determines that the brightness difference ΔL is less than a predetermined value, it determines advice information based on the capture time of the captured image. As a result, the camera installation support system 100 according to the first embodiment can output advice information for capturing the license plate NP in the captured image at a brightness more suitable for the recognition processing of each piece of character information, based on the surrounding environment in which the camera C1 is installed (i.e., whether it is daytime or nighttime). Therefore, the user can check whether the current installation or setting of the camera C1 is suitable for the recognition processing of each piece of character information of the license plate NP, and can adjust the current installation or setting of the camera C1, or the arrangement, brightness, etc. of the lighting installed around the camera C1, based on the advice information.

As described above, the camera installation support system 100 according to the first embodiment recognizes the vehicle numbers (i.e., the vehicle's serial number, serial designation number, etc.) based on each detected license plate, calculates the percentage of matches for all recognized vehicle numbers, and generates advice information if it is determined that the percentage is equal to or less than a predetermined value. This allows the camera installation support system 100 according to the first embodiment to determine the percentage of recognized vehicle serial numbers that are incorrectly recognized.

As a result of the above, the camera installation support system 100 according to the first embodiment generates advice information when it is determined that the number of recognized serial numbers (vehicle numbers, serial designation numbers, etc.) among all detected license plates NP is equal to or less than a predetermined number. This allows the camera installation support system 100 according to the first embodiment to determine whether or not it is possible to generate a best shot image of the vehicle generated by the camera C1 during actual operation.

As a result, the plate position drawing image generated by the camera installation support system 100 according to embodiment 1 depicts plate positions that satisfy multiple imaging conditions (i.e., satisfy the achievement conditions) and plate positions that do not satisfy multiple imaging conditions (i.e., do not satisfy the achievement conditions) in different colors based on the judgment results. This allows the camera installation support system 100 according to embodiment 1 to generate a plate position drawing image that visualizes whether or not the achievement conditions are satisfied. Therefore, the user can efficiently check at a glance whether or not the achievement conditions are satisfied.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, amendments, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components in the various embodiments described above may be combined in any manner as long as it does not deviate from the spirit of the invention.

The present disclosure is useful as a camera installation support program, a camera installation support method, and a camera installation support system that support camera installation and improve the recognition accuracy of recognition processing using captured images.

10, 20 Communication unit 11, 21 Processor 12, 22 Memory 13 Imaging unit 14 Illumination unit 23 Input unit 24 Monitor 25 HDD
AR1, AR2, AR3, AR4, AR5, AR6, AR7 Detection areas AR11, AR21, AR31, AR41, AR51, AR61, AR71 First detection zones AR12, AR22, AR32, AR42, AR52, AR62, AR72 Second detection zones C1 Cameras CI1, CI2, CI3, CI4, CI5, CI6, CI7 Plate position drawing image P1 Terminal devices SC1, SC2, SC3, SC4, SC5, SC6, SC7 Plate position confirmation screen NW Network ΔX Plate width θ Plate inclination

Claims (13)

A terminal device that is a computer connected to a camera that captures images of a vehicle so as to be capable of communicating with the camera,
A step of detecting a license plate of the vehicle reflected in one or more detection areas set in advance from each of a plurality of captured images captured by the camera;
calculating maximum and minimum luminance values of pixels in the detected area of the license plate;
generating and outputting advice information regarding installation or parameter setting of the camera when it is determined that the luminance difference between the maximum luminance value and the minimum luminance value is less than a predetermined value;
Camera installation support program. when it is determined that the luminance difference is less than a predetermined value, determining the advice information based on an image capturing time of the captured image,
The camera installation support program according to claim 1 . Calculating coordinates of two midpoints in a pair of short sides of each of the detected number plates;
generating and outputting a plate position drawing image in which the two midpoints for each license plate are plotted on a background image of an imaging area captured by the camera;
The camera installation support program according to claim 1 . Calculating a width of each of the plurality of license plates based on the coordinates of the two midpoints;
and outputting the advice information when it is determined that the calculated width is equal to or smaller than a predetermined value.
The camera installation support program according to claim 3 . Calculating a width of each of the plurality of license plates based on the coordinates of the two midpoints;
outputting the advice information when it is determined that a detection ratio of license plates in which a part of the width of the license plate is detected outside the detection area among all the detected license plates is a predetermined ratio or more,
The camera installation support program according to claim 3 . Calculating a width of each of the plurality of license plates based on the coordinates of the two midpoints;
a step of setting a plurality of detection zones smaller than the detection area within the detection area, and outputting the advice information when it is determined that a part of the width of each of the detected number plates is detected within the plurality of detection zones, in order to further realize the step of
The camera installation support program according to claim 3 . and outputting the advice information when it is determined that a detection ratio of the number plates in which a part of the width of the number plates is detected within the plurality of detection zones is equal to or greater than a predetermined ratio, in order to further realize the step of:
The camera installation support program according to claim 6. Calculating a longitudinal tilt of each of the detected number plates;
outputting the advice information when it is determined that the calculated inclination is equal to or greater than a set predetermined angle,
The camera installation support program according to claim 3 . recognizing a vehicle number of the vehicle based on each of the detected license plates;
calculating a matching ratio of each of all the recognized vehicle numbers, and generating the advice information when it is determined that the ratio is equal to or smaller than a predetermined value,
The camera installation support program according to claim 1 . generating the advice information when it is determined that the number of vehicle numbers recognized from among all the detected license plates is equal to or less than a predetermined number;
The camera installation support program according to claim 9. In the plate position drawing image, the plate position of the license plate whose luminance difference is not less than the predetermined value and the plate position of the license plate whose luminance difference is less than the predetermined value are drawn in different colors.
The camera installation support program according to claim 3 . A camera installation support method for a camera installation support system in which a camera and a terminal device are communicatively connected, comprising:
Detecting a license plate of a vehicle appearing in one or more detection areas set in advance from each of a plurality of captured images captured by the camera;
Calculating maximum and minimum luminance values of pixels in each of the detected license plate regions;
generating and outputting advice information regarding installation or parameter setting of the camera when it is determined that the luminance difference between the maximum luminance value and the minimum luminance value is less than a predetermined value;
Camera installation assistance methods. Camera and
A camera installation support system comprising: a terminal device communicably connected to the camera,
The camera includes:
Transmitting each of the captured images to the terminal device;
The terminal device
Detecting a license plate of a vehicle from one or more detection areas set in advance among the plurality of captured images;
Calculating maximum and minimum luminance values of pixels in each of the detected license plate regions;
generating and outputting advice information regarding installation or parameter setting of the camera when it is determined that the luminance difference between the maximum luminance value and the minimum luminance value is less than a predetermined value;
Camera installation support system.

Images