JP7654326B2 - Systems and methods - Google Patents

Description

The present invention relates to a system and method for counting and classifying moving objects such as vehicles.

The applicant has filed an invention relating to a system that detects the movement of moving objects (e.g., vehicles) based on vibration data detected by a vibration sensor and counts the number of detected moving objects (Patent Application No. 2020-117429). When classifying the detected moving vehicles into, for example, large vehicles, small vehicles, etc. (vehicle type discrimination), it is necessary to provide features and correct answer data corresponding to the features in advance so that the vehicles can be classified from the features. In this case, as correct answer data, it is necessary to photograph the moving vehicles and create labeling data from the captured video indicating when and what type of vehicle passed. However, if labeling data is created by humans, it takes time to create it. For example, when a traffic volume survey (including vehicle type discrimination) is conducted at many points at the same time, it becomes difficult to respond to the survey quickly.

Patent Document 1 discloses that machine learning is performed based on measurement data obtained by a sensor and correct answer data (teacher data) corresponding to the measurement data to create a learning model.

JP 2020-026164 A

As mentioned above, in order to count vehicles and identify vehicle types, labeling data including vehicle types (large, small, etc.) and vehicle passing times are required as correct answer data. However, if the labeling data is created by humans, it takes time to create and there are problems with labor costs.

The objective of the present invention is to quickly and inexpensively generate the correct answer (labeling) data required to create a learning model for acquiring the number and type of passing vehicles and other moving objects.

The system of the first invention is characterized in that, based on video data of a moving object, the type of the moving object and the time at which the moving object passes through a reference position are obtained, and correct answer data including the obtained type of the moving object and the time at which the moving object passes through the reference position are created.

In the present invention, the system obtains the type of moving object and the time at which the moving object passes through a reference position based on video data of the moving object, and creates correct answer data including the obtained type of moving object and the time at which the moving object passes through the reference position. This eliminates the need for humans to create correct answer data, making it possible to create correct answer data quickly and at low cost.

The system of the second invention is characterized in that, in the system of the first invention, a learning model is created based on the generated correct answer data and the measurement data of the moving object to obtain the type of the moving object and the number of times the moving object passes through the reference position.

The system of the third invention is characterized in that, in the system of the second invention, at least one of the type of the moving object and the number of times the moving object passes the reference position is obtained based on the created learning model and the measurement data.

In the present invention, the number of moving objects (number of times passing the reference position), type, etc. can be obtained by using a learning model based on the created correct answer data.

The system of the fourth invention is characterized in that, in the system of any one of the first to third inventions, the video data is image-recognized, and the type of the moving object and the time when the moving object passed through the reference position are obtained.

The system of the fifth invention is characterized in that in any one of the systems of the first to fourth inventions, the moving body is a vehicle.

The sixth invention system is the system of any one of the first to fifth inventions, characterized in that the type of the moving object is the size of the vehicle.

The seventh invention system is characterized in that in the system of any one of the first to sixth inventions, the moving body is a vehicle, the license plate of the vehicle is recognized from video data captured of the moving body, and the size of the vehicle is obtained as the type of the moving body based on the recognized license plate of the vehicle.

The system of the eighth invention is characterized in that, in the system of any one of the first to seventh inventions, the time when the moving body passes the reference position is obtained by acquiring the time when the front of the moving body passes the reference position, the time when the center of the moving body passes the reference position, and the time when the rear of the moving body passes the reference position.

The system of the ninth invention is characterized in that in any one of the systems of the first to eighth inventions, the measurement data is vibration data.

The method of the tenth invention is characterized in that, based on video data in which a moving object is photographed, the type of the moving object and the time at which the moving object passes through a reference position are obtained, and correct answer data including the obtained type of the moving object and the time at which the moving object passes through the reference position are created.

The present invention makes it possible to create correct answer data quickly and at low cost.

1 is a diagram illustrating an example of the configuration of a system according to an embodiment of the present invention. FIG. 2 is a diagram showing a scene of a video based on video data captured by a video camera. FIG. 13 is a diagram showing an example of correct answer (labeling) data. FIG. 1 is a diagram showing an example of a license plate. 4 is a flowchart showing a processing operation executed by a PC.

The following describes an embodiment of the present invention. FIG. 1 is a diagram showing an example of the configuration of a system 1 according to an embodiment of the present invention. The system 1 is composed of, for example, a personal computer (hereinafter referred to as "PC") 2, a vibration sensor 3, a video camera 4, and the like. The PC 2 creates correct answer (labeling) data, creates a learning model, and creates a file showing the results of a traffic volume survey including vehicle counting and vehicle classification. The processing performed by the PC 2 will be described later. Note that FIG. 1 shows an example in which information output from the vibration sensor 3 and information output from the video camera 4 are processed by one PC 2. It goes without saying that the present invention is not limited to this, and the information output from the vibration sensor 3 and the information output from the video camera 4 may be processed by different PCs.

The vibration sensor 3 is installed, for example, on a road or the like. The vibration sensor 3 detects vibrations transmitted to the road from vehicles passing in front of the vibration sensor 3, and outputs vibration data. The video camera 4 is installed, for example, near a road or the like. The video camera 4 captures images of vehicles traveling on the road, and outputs the video data. FIG. 2 is a diagram showing a scene of an image based on the video data captured by the video camera 4. As shown in FIG. 2, the image shows the state of a vehicle traveling on the road. In addition, the time is displayed in the lower right corner of FIG. 2. Furthermore, a reference line, which will be described later, is displayed on the left side of FIG. 2.

Figure 3 is a diagram showing an example of correct answer (labeling) data created by PC2. The labeling data includes information such as ID, LANE1_NUMBER, LANE1, LANE1_TYPE, CTR_IN_MIN, CTR_IN_SEC, and CTR_IN_FRAME. Here, it is assumed that the correct answer data has been created from video data captured on an oncoming one-lane road as shown in Figure 2. The lane on the near side in Figure 2 is called LANE1, and the lane on the far side is called LANE2. ID indicates an identification number (1600, 1700, ...) for identifying each record. LANE1 indicates information on whether or not a vehicle has passed through LANE1. If a vehicle has passed through LANE1, a flag of "1" is set for LANE1.

Figure 4 is a diagram showing an example of a license plate. On the left side of the upper row of the license plate, characters indicating the local transport bureau or its motor vehicle inspection and registration office (Shinagawa in Figure 4) are written. On the right side of the upper row of the license plate (indicated by the dashed arrow) is the classification number according to the type of motor vehicle (500 in Figure 4).

Returning to FIG. 3, LANE1_NUMBER indicates the number information of the vehicle that passed through LANE1. Specifically, LANE1_NUMBER contains the first character of the classification number according to the type of vehicle. For example, "5" indicates that the first character of the classification number is "5". LANE1_TYPE indicates the type of vehicle that passed through LANE1 (large vehicle, small vehicle). "TYPE_STANDARD" indicates that the vehicle that passed through LANE1 was a small vehicle. "TYPE_LARGE" indicates that the vehicle that passed through LANE1 was a large vehicle. If the first character of the classification number of the recognized license plate is "1, 2, 8, 9, 0", PC2 stores "TYPE_LARGE", which indicates that it is a large vehicle. If the first character of the classification number of the recognized license plate is "3, 4, 5, 7, 9", PC2 stores "TYPE_STANDARD", which indicates that it is a small vehicle.

CTR_IN_MIN, CTR_IN_SEC, and CTR_IN_FRAME indicate the time when the front of the vehicle passed the reference line shown in Figure 2. Note that in Figure 3, the information contained in the correct data includes CTR_IN_MIN, CTR_IN_SEC, and CTR_IN_FRAME, which indicate the time when the front of the vehicle passed the reference line, but also includes information indicating the time when the center of the vehicle passed the reference line and information indicating the time when the rear of the vehicle passed the reference line. Also, while Figure 3 only shows information about vehicles that passed LANE 1, information about vehicles that passed LANE 2 is also included.

Based on the video data of the vehicle (mobile body) captured by the video camera 3, the PC2 acquires the type of vehicle (mobile body) (large vehicle, small vehicle) and the passing time of the vehicle (mobile body) across the reference line (reference position). The PC2 then creates correct answer data including the acquired vehicle type and the passing time of the vehicle across the reference line. The PC2 acquires the vehicle type and the passing time of the vehicle across the reference line by performing image recognition on the video data.

PC2 judges whether the vehicle has passed through LANE1 or LANE2 based on the video data. If PC2 judges that the vehicle has passed through LANE1, it sets a flag of "LANE1" to "1". PC2 performs image recognition of the vehicle's license plate to obtain the vehicle type. If PC2 recognizes "500" as the classification number shown in FIG. 4 as a result of the license plate image recognition, and judges that the vehicle has passed through LANE1, it stores the information "5" as "LANE1_NUMBER". Furthermore, if PC2 recognizes "500" as the classification number, it obtains "compact car (TYPE_STANDARD)" as the vehicle type. Then, PC2 stores the information of "TYPE_STANDARD" as "LANE1_TYPE".

Furthermore, PC2 obtains the time when the front of the vehicle passes the reference line, the time when the center of the vehicle passes the reference line, and the time when the rear of the vehicle passes the reference line based on the video data. PC2 then stores the time when the front of the vehicle passes the reference line as "CTR_IN_MIN", "CTR_IN_SEC", and "CTR_IN_FRAME". PC2 also stores the time when the center of the vehicle passes the reference line and the time when the rear of the vehicle passes the reference line as correct answer data.

Figure 5 is a flowchart showing the processing operations executed by PC2. First, PC2 synchronizes the vibration data output from vibration sensor 3 with the video data output from video camera 4 (S1). Next, PC2 creates correct answer data from the video data output from video camera 4 (S2). Next, PC2 performs pre-learning of the correct answer data and vibration data (S3). That is, PC2 creates a learning model for acquiring the vehicle type and the number of times the vehicle has passed the reference line (reference position) based on the created correct answer data and the vibration (measurement) data of the vehicle (moving body).

Next, PC2 counts the vehicles and determines the vehicle type (S4). That is, PC2 acquires the vehicle (moving object) type and the number of times the vehicle has passed the reference line (reference position) based on the created learning model and the vibration (measurement) data. Then, PC2 creates a file of traffic volume observation results that includes the acquired vehicle type and the number of times the vehicle has passed the reference line (S5).

As described above, in this embodiment, system 1 acquires the type of vehicle (mobile body) and the passing time of the vehicle (mobile body) across the reference line (reference position) based on video data of the vehicle (mobile body), and creates correct answer data including the acquired type of vehicle (mobile body) and the passing time of the vehicle (mobile body) across the reference line (reference position). This eliminates the need to create correct answer data manually, so that correct answer data can be created quickly and at low cost. In addition, the number of vehicles (mobile bodies) (number of passes across the reference line (reference position)), type, etc. can be acquired by a learning model based on the created correct answer data.

Although the embodiments of the present invention have been described above, the forms to which the present invention can be applied are not limited to the above-mentioned embodiments, and as exemplified below, appropriate modifications can be made without departing from the spirit of the present invention.

In the above embodiment, a vehicle is given as an example of a moving body, but the moving body is not limited to a vehicle. Also, while vibration data detected by the vibration sensor 3 is given as an example of measurement data, the measurement data is not limited to vibration data.

The present invention can be suitably used in systems and methods for counting and classifying moving objects such as vehicles.

1 System 2 PC
3. Vibration sensor 4. Video camera

Claims (9)

Based on video data of a moving object, a type of the moving object and a passing time of the moving object at a reference position are acquired, and correct answer data including the acquired type of the moving object and the passing time of the moving object at the reference position are generated;
A system characterized by creating a learning model for obtaining the type of moving object and the number of times the moving object passes through the reference position based on the created correct answer data and measurement data of the moving object . The system described in claim 1, characterized in that at least one of the type of the moving object and the number of times the moving object has passed the reference position is obtained based on the created learning model and the measurement data . 3. The system according to claim 1, further comprising: performing image recognition on the video data to acquire the type of the moving object and the time when the moving object passed a reference position. The system according to any one of claims 1 to 3 , wherein the moving body is a vehicle. The system according to any one of claims 1 to 4 , characterized in that the type of the moving object is a size of the vehicle. the moving object is a vehicle,
Recognizing a license plate of the vehicle from image data captured of the vehicle, which is the moving body;
The system according to any one of claims 1 to 5 , characterized in that the size of the vehicle is obtained as the type of the moving object based on the recognized license plate of the vehicle. Based on video data of a moving object, a type of the moving object and a passing time of the moving object at a reference position are acquired, and correct answer data including the acquired type of the moving object and the passing time of the moving object at the reference position are generated;
A system characterized by obtaining the time when the moving body passes through a reference position, the time when the front of the moving body passes through the reference position, the time when the center of the moving body passes through the reference position, and the time when the rear of the moving body passes through the reference position, as the passing times of the moving body . The system according to any one of claims 1 to 6 , characterized in that the measurement data is vibration data. Based on video data of a moving object, a type of the moving object and a passing time of the moving object at a reference position are acquired, and correct answer data including the acquired type of the moving object and the passing time of the moving object at the reference position are generated;
A method characterized by creating a learning model for obtaining the type of moving object and the number of times the moving object passes through the reference position based on the created correct answer data and measurement data of the moving object .

Images