JP7845700B2 - Intersection edge terminal, method, and computer program - Google Patents

Description

This disclosure relates to intersection edge terminals, methods, and computer programs.

Patent Document 1 describes a technology in which an object detection device placed at an intersection generates movement information and transmits it to an object detection device placed near an adjacent intersection.

In recent years, the smart application of IT and AI has spread to all industries, including transportation, which is a vital social infrastructure. In particular, the advancements in AI since Transformer technology have been remarkable, with the emergence of large-scale, highly accurate AI engines such as GPT, and their application in a wide range of fields is expected in the future.

Japanese Patent Publication No. 2022-037998

In the aforementioned applications for transportation infrastructure, reliable real-time processing is a crucial factor. However, cloud-based AI applications from remote locations, such as GPT, present various challenges regarding processing time (including delays due to communication networks) and response requirements.

Applications of smart intersections utilizing images from cameras deployed at intersection edge terminals include intersection state recognition and monitoring, efficient and safe signal control and information transmission to intersection users using recognition information, and identification and response to issues related to intersection safety, efficiency, and convenience. For these applications, the images collected by the cameras must be highly detailed, accurate, and reliable in recognizing vehicles, pedestrians, pets, and backgrounds, and the results must be utilized for various purposes. To meet this demand, methods such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI processing model can be considered, but all of these significantly increase the computational load on the intersection edge terminals. Furthermore, for vehicles (high-speed moving objects) at intersections, the aforementioned massive information processing must be performed within an extremely limited time, requiring even more complex and large-scale processing in terms of computational load per unit of time. To handle this massive computational load challenge using only edge terminals installed at intersections, it is necessary to equip them with a large amount of high-performance hardware resources such as CPUs, GPUs, and memory, leading to larger equipment sizes and high power consumption.

One example of the purpose of this disclosure is to provide an intersection edge terminal, method, and computer program that can solve the aforementioned problems.

An intersection edge terminal according to one aspect of this disclosure comprises a communication unit that receives an image captured at its own intersection, and a processing unit that detects whether the image includes a high-speed moving object and performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

A method relating to one aspect of this disclosure includes: a communication unit of an intersection edge terminal receiving an image captured at its own intersection; a processing unit of the intersection edge terminal detecting that the image contains a high-speed moving object; and the processing unit performing a high-speed moving object recognition process using information associated with the high-speed moving object acquired at another intersection.

A computer program according to one aspect of this disclosure causes an intersection edge terminal to execute communication means for receiving an image captured at its own intersection, and processing means for detecting whether the image contains a high-speed moving object and for performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

According to the above embodiment, the computational complexity of object recognition processing at intersection edge terminals can be significantly reduced, and efficient object recognition processing can be achieved.

This is a schematic diagram showing the external configuration of the system for recognizing the state of an intersection related to this disclosure. This figure shows the functional configuration of an intersection edge terminal for recognizing the state of an intersection related to this disclosure. This is a flowchart of the method for recognizing the state of an intersection related to this disclosure. This figure shows an example of detailed information related to this disclosure. This is a flowchart of the abstraction classification process related to this disclosure. This diagram shows the functional configuration of the system for recognizing the state of an intersection related to this disclosure. This is a flowchart of the method for recognizing the state of an intersection related to this disclosure. This figure shows an example of detailed information related to this disclosure. This figure shows an example of the arrangement of linked cameras related to this disclosure. This figure shows another example of the configuration of the intersection edge terminal related to this disclosure. This figure shows another example of a method for recognizing the state of an intersection as described herein. This figure shows an example of the computer configuration related to this disclosure.

The following describes each embodiment with reference to the drawings. In all drawings, identical or equivalent components are denoted by the same reference numerals, and common descriptions are omitted.

(Overall structure)
Figure 1 is a schematic diagram showing the external configuration of a system 1 for recognizing the state of an intersection according to this disclosure. Figure 1 shows an intersection having a traffic light 11.

System 1 comprises at least one camera 2 and an intersection edge terminal 3. Camera 2 acquires images of the intersection. Camera 2 is positioned to monitor fixed objects such as roads, crosswalks, and guardrails at the intersection, as well as moving objects passing through the intersection, such as cars 13, motorcycles or bicycles 14, pedestrians 15, and pets. Camera 2 may be installed on traffic lights 11 or poles 12 at the intersection. The intersection edge terminal 3 may also be installed on traffic lights 11 or poles 12 at the intersection. The intersection edge terminal 3 acquires image information about the state of the intersection, using the camera 2 mounted on the traffic light 11 and/or pole 12 as its input source.

The image may include multiple objects, including a car 13, a motorcycle or bicycle 14, and a pedestrian 15. These multiple objects may also include any other objects, such as houses or animals like pets. The intersection edge terminal 3 can communicate with other intersection edge terminals installed at the neighboring intersection 5. The intersection edge terminal 3 can also communicate with the neighboring road camera 6 installed on the neighboring road. The above-described communication is performed wirelessly or via wired communication through the network 4.

<First Embodiment>
Hereinafter, one embodiment relating to this disclosure will be described with reference to Figures 2 to 4.

(Functional configuration)
Figure 2 shows the functional configuration of the intersection edge terminal 3 for recognizing the state of the intersection according to this disclosure.
The intersection edge terminal 3 includes a processing unit 31 and a communication unit 32. The processing unit 31 may be a processor such as a CPU or GPU. The communication unit 32 can communicate with any device other than the intersection edge terminal 3 provided in the communication unit 32. For example, the communication unit 32 can communicate with the camera 2 in system 1 or other intersection edge terminals at the neighboring intersection 5.

(Processing flow)
Figure 3 is a flowchart showing the operation of an intersection edge terminal 3 according to one embodiment of the present disclosure.
First, the communication unit 32 acquires information about high-speed moving objects obtained by other intersection edge terminals at other nearby intersections 5 (step S301). For example, this information may be detailed information indicated by object ID 1 and object ID 2 in Figure 4. For example, this detailed information may include the type of vehicle, the occupants and their facial expressions, etc.
Camera 2 acquires an image of the intersection (step S302). Camera 2 transmits the acquired image to the communication unit 32.

Next, the processing unit 31 determines whether or not the acquired image contains a high-speed moving object (step S303). For example, the processing unit 31 may perform AI object detection processing to extract objects from the image and determine whether or not a high-speed moving object is included by recognizing whether the extracted object is a background, pedestrian, vehicle, or other high-speed moving object. The above-described AI object detection processing may be performed based on a trained neural network acquired by deep learning.

If it is determined that the image does not contain a high-speed moving object (step S303: No), the processing unit 31 returns to executing step S302. If it is determined that the image contains a high-speed moving object (step S303: Yes), it is determined whether the information about the high-speed moving object acquired by the other intersection edge terminals includes information corresponding to the high-speed moving object acquired by the current intersection edge terminal 3 (step S304). For example, the intersection edge terminal 3 may detect the vehicle type, body color, license plate, etc., of the high-speed moving object and perform the determination in step S304 by comparing it with the information from the other intersection edge terminals. If it is determined that the information from the other intersection edge terminals does not include information corresponding to the high-speed moving object acquired by the current intersection edge terminal 3 (step S304: No), the processing unit 31 returns to executing step S302.

If it is determined that the corresponding information is included in the information from the other intersection edge terminals (step S304: yes), the information associated with the high-speed mobile object is obtained from the information from the other intersection edge terminals (step S305). The processing unit 31 uses the information associated with the high-speed mobile object to perform recognition processing of the high-speed mobile object (step S306).

The processing unit 31 acquires detailed information about the high-speed moving object through the recognition process in step S305 (step S307). For example, the detailed information acquired by the intersection edge terminal 3 may be additional information to the detailed information acquired by other intersection edge terminals in step S301. For example, the additional information may include a characteristic image of the vehicle, the vehicle type, the occupants, and their facial expressions.

Furthermore, the additional information may be used to fill in the missing portion of the license plate indicated by "??" in object ID 2 of Figure 4. The missing information to be filled in is not limited to license plate information. This enables the complete processing results of detailed and unique object information, and its utilization, for challenges where computation and image collection cannot be completed at a single intersection, by passing through multiple intersections. This function is also expected to have the effect of enabling data completion in response to challenges such as sudden information collection or processing delays at individual intersections, thereby improving system redundancy and reliability.

The processing unit 31 performs synthesis processing based on the detailed information obtained in step S307 (step S308).
The processing unit 31 may perform synthesis processing according to various applications. For example, the processing unit 31 may perform synthesis processing based only on the necessary information from the detailed information, according to applications such as traffic accident analysis or traffic signal control. Furthermore, the synthesized information may be used for various applications such as signal display switching timing control, providing information to intersection users before they enter, monitoring accidents and violations, accident analysis, traffic problem identification, traffic problem countermeasures, image verification, and search. It may also be stored for these applications.

(Effect, Action)
As described above, the intersection edge terminal 3 includes a communication unit 32 that receives images captured at its own intersection, and a processing unit 31 that detects whether the image contains a high-speed moving object and performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at other intersections.

In other words, because the results of processing already performed at nearby intersections 5 become available at the intersection edge terminal 3 before a high-speed moving object entering the intersection enters the intersection, the computational load for object recognition processing at the intersection edge terminal 3 can be significantly reduced, and efficient object recognition processing can be achieved. This eliminates the need for larger devices and high power consumption, and enables high-load calculations such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI processing model.

<First modified example of the first embodiment>
A modified example of one embodiment of the present disclosure will be described below with reference to Figure 5.
Figure 5 is a flowchart of the abstraction classification process related to this disclosure.

First, camera 2 acquires an image containing an object (step S501). Camera 2 then transmits the acquired image to the communication unit 32.

The processing unit 31 detects objects in the image received via the communication unit 32 (step S502). For example, the processing unit 31 performs AI object detection processing to extract objects from the image. The processing unit 31 then performs abstraction classification processing to abstract the extracted objects and classify them into categories such as background, pedestrians, vehicles, and other high-speed moving objects.

Next, the intersection edge terminal 3 determines whether or not a request for detailed recognition processing has been received in the communication unit 32 (step S503).
For example, a sound sensor linked to the processing unit 31 detects the occurrence of an emergency, such as a traffic accident, based on the detected volume. At this time, the sound sensor transmits a request for detailed recognition processing to the communication unit 32. The above emergency may also be detected by various sensors such as light, vibration, distance, laser, and ultrasound.

If a request for detailed recognition processing is received (step S503: Yes), the intersection edge terminal 3 executes detailed recognition processing (steps S302 to S307).
Furthermore, the processing unit 31 may perform detailed recognition processing (steps S302 to S307) based on the information about the object acquired in step S502. That is, if a high-speed moving object is detected in step S502, the processing unit 31 may perform the detailed recognition processing in steps S304 to S307.
Alternatively, detailed recognition processing may be performed by a simple process that overlays detailed recognition processing onto the information obtained in the object extraction process in the abstraction classification process.

If no request for detailed recognition processing is received (step S503: No), the processing unit 31 continues the abstraction classification processing (steps S504 to S506).

The processing unit 31 recognizes the relative relationship between the objects (step S504).
For example, the processing unit 31 measures the distance between multiple pedestrians on the road (background), or the distance between multiple cars.

The processing unit 31 performs traffic condition recognition processing (step S505).
For example, the processing unit 31 detects that there is a high concentration of pedestrians at the intersection where the intersection edge terminal 3 is located, based on the small distances between multiple pedestrians measured in step S504. The processing unit 31 also detects that there is a high probability of traffic congestion occurring, or that traffic congestion has already occurred, at the intersection where the intersection edge terminal 3 is located, based on the small distances between multiple vehicles.

The processing unit 31 executes a process to make effective use of the acquired traffic condition information (step S506).
For example, if the processing unit 31 detects that pedestrians are densely packed at an intersection where the intersection edge terminal 3 is located, it may extend the duration of the green light for pedestrian signals to control the signals and alleviate the congestion. Alternatively, if the processing unit 31 detects that there is a high probability of traffic congestion occurring, or that traffic congestion is already occurring, it may extend the duration of the green light for vehicle signals to control the signals and avoid traffic congestion.

(Effect, Action)
As described above, the processing unit 31 of the intersection edge terminal 3 further detects multiple objects, performs an abstraction classification process to abstract and classify the multiple objects, and, in response to the detection of a predetermined event at the intersection that requires detailed recognition processing, performs detailed recognition processing on the multiple objects detected in the abstraction classification process.

This system separates two distinct functions: a computationally intensive abstraction classification process and a detailed recognition process. By offering two processing configurations—one where each function operates independently and another where they are combined for result processing—it's possible to optimize the computational load according to the application's requirements. In other words, it allows for reducing the load on the intersection edge terminal 3 and optimizing the computational load depending on traffic conditions and the application.

<Second modified example of the first embodiment>
Information from other intersection edge terminals at neighboring intersection 5 may include the recording time and recorded vehicle speed at the other intersection edge terminals. The processing unit 31 may predict at least one of the order and time of vehicle entry based on the distance from its own intersection to neighboring intersection 5.
The processing unit 31 may prepare and execute recognition processing based on at least one of the predicted order and time of entry of vehicles, etc. For example, the processing unit 31 may prepare relevant information in the memory of the intersection edge terminal 3 so that it can read and update information on the vehicle that is predicted to enter the intersection first.

As described above, the processing unit 31 predicts at least one of the order and time at which high-speed moving objects will enter its own intersection, based on information associated with high-speed moving objects acquired at other intersections, and performs recognition processing at its own intersection based on at least one of the predicted order and time.

This allows for predictions of the order in which a vehicle will enter an intersection in its intended direction, as well as the approximate time of entry. By having a method to anticipate the vehicle's entry before it actually enters, the linking process becomes simpler and more reliable, and the results of the image unique detail recognition process can be prepared in a timely manner.

<Third modified example of the first embodiment>
After the processing unit 31 obtains the detailed information in step S307, it may transmit the detailed information to other intersection edge terminals at nearby intersections.
Here, the intersection edge terminal 3 may acquire the direction of travel of the high-speed moving object and transmit the detailed information only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.

In other words, the intersection edge terminal 3 has the function of sending data only in the direction in which individual vehicles are leaving the intersection. With this function, the data will not be sent to unnecessary nearby intersections 5, or the intersection edge terminals of nearby intersections that deem the data unnecessary may discard the data without receiving it.
This significantly reduces the amount of data transmitted through the communication network between neighboring intersections, and also reduces the hardware resources required for transmission and reception processing at the intersection edge terminals 3 between neighboring intersections.

<Second Embodiment>
Hereinafter, one embodiment relating to this disclosure will be described with reference to Figures 6 to 9.

(Functional configuration)
Figure 6 is a diagram showing the functional configuration of System 1 according to one embodiment of the present disclosure. Since the functional configuration common to Figure 2 is the same as that of Figure 2, the differences will be explained below.

The intersection edge terminal 3 includes a memory 33 and a cooling unit 34.
As a result, the cooling unit 34 can prevent failures due to heat, even in harsh environments such as outdoor ambient temperatures.

The memory 33 may be any storage medium, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or NVMe (Non-Voltage Memory Express). The cooling unit 34 may be any cooling device.

The intersection edge terminal 3 communicates with at least one of the neighboring intersections 5 and neighboring road cameras 6 via the communication unit 32 and the network 4. This communication is performed by wireless or wired communication.
The nearby intersection 5 may refer to an adjacent intersection or an intersection that is not adjacent but is located nearby. The nearby road camera 6 is a camera placed on a traffic light 11 or pole 12 on a road that is not an intersection.

System 1 further includes at least one of the following: a linked camera 7, a linked wireless terminal 8, a linked vehicle-mounted terminal 9, and a linked satellite terminal 10.

The linked camera 7 is positioned at a different location from the camera 2, which corresponds to the intersection edge terminal 3, at the intersection where the camera 2 is located. The linked camera 7 is linked to the camera 2. As shown in Figure 9, the linked camera 7 can capture images of the subject simultaneously or nearly simultaneously from a different angle than the linked camera 2.
For example, one method for realizing the linking processing function is to use image recognition of vehicle license plates. If the camera 2 in the intersection is simply installed from above on the traffic light 11 or the intersection support pole 12, when multiple vehicles enter in a line, the vehicle in front will be in a blind spot, and the vehicle license plate can only be confirmed by camera 2 after the vehicle in front just before entering the intersection has passed, resulting in a problem where the time available for processing after linking is very short. As a solution to this problem, the linking camera 7 for vehicle linking can be configured to be positioned and oriented so that the vehicle in front of the target vehicle passes by, and the vehicle license plate is not in a blind spot due to the vehicle in front.
The attached camera 7 may be positioned below the road surface before the intersection, with its camera surface angled from upward to away from the intersection. Alternatively, it may be positioned on the side of the road before the intersection, with its camera surface angled from facing the vehicle to away from the intersection.

As described above, the image is acquired by camera 2 at the intersection, and the communication unit 32 further acquires a linked image from linked camera 7 which is linked to camera 2, and linked camera 7 is located at a different position from camera 2. The processing unit 31 performs recognition processing based on the image and the linked image.
This increases the probability of acquiring vehicle information, even when the vehicle in front has just passed the intersection, resulting in a very short processing time after linking.

The linked wireless terminal 8 is a short-range wireless terminal mounted on a vehicle.
For example, the linked wireless terminal 8 is a vehicle-mounted short-range wireless terminal with strong directivity and high-speed processing capabilities, similar to those used in ETC systems on toll roads. System 1 and the vehicle are each equipped with wireless equipment for short-range communication. The intersection edge terminal 3 recognizes the unique ID for linking each vehicle, links it with the image-processed vehicle information, and sends this linked set information to other nearby intersections 5. At the next nearby intersection 5 where the vehicle has moved, the vehicle can be identified from the unique ID for linking and the sent vehicle information can be utilized.

The linked in-vehicle terminal 9 is a drive recorder or vehicle-mounted camera of a car 13 entering the intersection, and is linked to the intersection edge terminal 3.
For example, the linked in-vehicle terminal 9 receives information from in-vehicle cameras such as a drive recorder and autonomous driving cameras, and before entering an intersection, it wirelessly transmits information such as the license plates of the vehicles in front of and behind a single vehicle, and whether the vehicles have been swapped, to the intersection edge terminal 3 from that vehicle.
This simplifies and improves the accuracy of the linking process, and enables the timely preparation of the results of the image unique detail recognition process.

The linked satellite terminal 10 is a satellite terminal linked to the intersection edge terminal 3. The linked satellite terminal 10 transmits satellite images to the intersection edge terminal 3. The intersection edge terminal 3 can trace the movement status of individual vehicles from the satellite images and collect image unique detail recognition processing information collected at other nearby intersections 5 before each vehicle enters each intersection, and perform linking processing.
This simplifies and improves the accuracy of the linking process, and enables the timely preparation of the results of the image unique detail recognition process.

System 1 may further include other sensors, such as sound, light, vibration, distance, laser, and ultrasonic sensors, to monitor the intersection conditions.

(Processing flow)
Figure 7 is a flowchart showing the operation of an intersection edge terminal 3 according to one embodiment of the present disclosure.
First, camera 2 acquires an image containing an object that was captured at the intersection (step S701). Camera 2 transmits the acquired image to the communication unit 32.

Next, the processing unit 31 detects objects contained in the received image (step S702). For example, the processing unit 31 performs AI object detection processing to extract objects from the image, recognize whether the extracted objects belong to the background, vehicles, or pedestrians, and uniquely identify each. In this embodiment, it is assumed that the processing unit 31 has recognized four objects. The processing unit 31 assigns a unique identifier (object ID 1 to object ID 4) to each of the four recognized objects. The above-described AI object detection processing may be performed based on a trained neural network obtained by deep learning.

Next, the processing unit 31 calculates the amount of time variation for each detected object (step S703).
For example, the processing unit 31 calculates the difference in the positions of each object included in the first image and the second image, which is taken a certain time after the first image was taken, and calculates the amount of time variation.
The amount of time variation may be calculated by averaging the amount of time variation in a series of three or more images. For objects recognized as background in step S702, the calculation of the amount of time variation may be omitted in order to reduce the computational load.

The processing unit 31 classifies each object into a predetermined category based on the amount of time variation (step S704).
For example, the predetermined categories are four: fixed background, temporarily stationary object, slow-moving object, and fast-moving object. The number of predetermined categories may be two or more. The predetermined categories may also be classified into categories with different content than the four mentioned above.

For example, when comparing a second image taken after a predetermined period (e.g., 5 seconds) has elapsed since the first image was taken, if the time variation value of the first object corresponding to object ID 1 during that predetermined period is 0m, the first object is classified into a first category indicating a fixed background.
If the second object corresponding to object ID 2 has a speed of 0 m/s during a predetermined period, and then has a speed greater than 0 m/s and less than 1.3 m/s during a certain period after the predetermined period has elapsed, the second object is classified into a second category indicating a temporarily stationary object.
When the third object corresponding to object ID 3 travels at a speed greater than 1.3 m/s and less than 5.0 m/s during a predetermined period, the third object is classified into a third category indicating a slow-moving object.
If the fourth object corresponding to object ID 4 travels at a speed greater than 5.0 m/s during a predetermined period, the fourth object is classified into the fourth category, indicating a high-speed moving object.

The processing unit 31 recognizes each of the multiple objects according to a predetermined recognition method corresponding to the classified category (steps S705-1 to S705-4).
For example, the processing unit 31 recognizes a first object classified into a first category according to a method for recognizing a fixed background. For example, the processing unit 31 recognizes the first object at low resolution and over a long period of time.
Furthermore, the processing unit 31 recognizes the second object classified into the second category according to a method for recognizing a temporarily stationary object. For example, the method for recognizing a temporarily stationary object may specify recognition with a higher resolution and shorter period than the method for recognizing the fixed background described above.
Furthermore, the processing unit 31 recognizes the third object classified into the third category according to a method for recognizing slow-moving objects. For example, the method for recognizing slow-moving objects may specify recognition with a higher resolution and shorter period than the method for recognizing temporarily stationary objects described above. Unlike the method for recognizing a fixed background and the method for recognizing temporarily stationary objects, the method for recognizing slow-moving objects may acquire color information of the third object.
Furthermore, the processing unit 31 recognizes the fourth object, which has been classified into the fourth category, according to a method for recognizing high-speed moving objects. For example, the method for recognizing high-speed moving objects may specify recognition with a higher resolution and shorter period than the method for recognizing low-speed moving objects described above. In the method for recognizing high-speed moving objects, unlike the method for recognizing a fixed background and the method for recognizing a temporarily stationary object, the color information of the fourth object may be acquired. Also, in the method for recognizing high-speed moving objects, unlike the method for recognizing a fixed background and the method for recognizing a low-speed moving object, information on the vehicle's license plate or images of the vehicle from multiple directions may be acquired.
The differences in settings for each recognition method should not be limited to the resolution and period described above; arbitrary settings may be used. Furthermore, the detailed settings for different recognition methods do not necessarily have to differ. For example, the resolution settings for a method recognizing a fixed background may be the same as those for a method recognizing a temporary, stationary object. Each recognition method may be performed in parallel.

The processing unit 31 obtains detailed information by performing recognition processing corresponding to each recognition method in steps S705-1 to S705-4 (step S706).
For example, the information may be detailed as shown in Figure 8, but it is not limited to this form of information. It may also include information such as the type of vehicle, the occupants, and their facial expressions.

The processing unit 31 performs synthesis processing based on the detailed information obtained in step S706 (step S707).
The processing unit 31 may perform synthesis processing according to various applications. For example, if the application is limited to pedestrian information at an intersection, the synthesis processing may be limited to detailed information of objects in the second category. If the application is for traffic accident analysis, the synthesis processing may be performed based on detailed information of the third to fourth categories. If the application is for traffic signal control, the synthesis processing may be limited to detailed information of objects in the second to fourth categories. Furthermore, the synthesized information may be used for various applications such as signal display switching timing control, providing information to intersection users before they enter, monitoring accidents and violations, accident analysis, traffic problem extraction, traffic problem countermeasures, image verification, and search. The information may also be stored for these applications.

(Effect, Action)
As described above, the communication unit 32 receives multiple images captured at the intersection, the processing unit 31 detects objects included in the multiple images, calculates the amount of time variation of the objects based on the multiple images, classifies the objects into predetermined categories based on the amount of time variation, and performs object recognition processing according to a predetermined recognition method corresponding to the category.

Specifically, the intersection edge terminal 3 has a function to classify objects from the image data collected from the camera based on the amount of time variation of each object per unit of time. This classification includes high-speed moving objects such as cars 13 and motorcycles, low-speed moving objects such as bicycles 14, pedestrians 15, and pets, temporary stationary objects such as garbage and fallen objects within the intersection, and fixed background information such as road surfaces, signs, utility poles, signal poles, and guardrails that are always present. Furthermore, different object recognition processing cycles and recognition methods are used for each classified category, and recognition processing is performed at different cycles to match the changes in the state and position of each object.

This allows for recognition processing using the optimal recognition method for each object, thereby reducing the computational load on the recognition processing at intersection edge terminals and achieving efficient object recognition in images.

<Modified form of the second embodiment>
The intersection edge terminal at the nearby intersection 5 or the terminal at the nearby road camera 6 may communicate with a vehicle-mounted terminal (linked wireless terminal 8) and receive a vehicle identifier (for example, a vehicle-specific identifier such as a serial number) from the vehicle-mounted terminal. At this time, the intersection edge terminal at the nearby intersection 5 or the terminal at the nearby road camera 6 links the received vehicle identifier with the detailed information acquired at the terminal. The intersection edge terminal at the nearby intersection 5 or the terminal at the nearby road camera 6 transmits the linked detailed information to the intersection edge terminal 3 at the nearby intersection. The intersection edge terminal 3 receives the linked detailed information and uses the linked detailed information to perform detailed recognition processing.

The above description explains the linking process by which the intersection edge terminal of the nearby intersection 5 or the terminal in the nearby road camera 6 communicates with the vehicle-mounted terminal. However, the intersection edge terminal 3 shown in Figure 1, which has a similar configuration to the intersection edge terminal of the nearby intersection 5, may also perform the same process.
In other words, the communication unit 32 of the intersection edge terminal 3 receives a vehicle identifier from the vehicle-mounted terminal, the processing unit 31 associates the vehicle identifier with information generated by the recognition process, and transmits the associated information to other intersection edge terminals.

This ensures the unique identification of each vehicle and improves the reliability of the acquired information.

<Third Embodiment>
Figure 10 shows another example of the configuration of an intersection edge terminal according to the present disclosure. In the configuration shown in Figure 10, the intersection edge terminal 100 comprises a processing unit 101 and a communication unit 102.

In this configuration, the communication unit 102 receives an image captured at its own intersection. The processing unit 101 detects that the image contains a high-speed moving object and performs recognition processing of the high-speed moving object using information associated with high-speed moving objects acquired at other intersections.

According to the intersection edge terminal 100, the results of processing already performed at nearby intersections 5 become available to the intersection edge terminal 3 before a high-speed moving object entering the intersection enters the intersection. This significantly reduces the computational load for object recognition processing at the intersection edge terminal 3, enabling efficient object recognition. As a result, it eliminates the need for larger devices and high power consumption, while also enabling computationally intensive tasks such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI processing model.

<Fourth Embodiment>
Figure 11 shows another example of a method for recognizing the state of an intersection according to this disclosure.
The method shown in Figure 11 includes the communication unit 102 of the intersection edge terminal receiving an image captured at its own intersection (step S1101). The method includes the processing unit 101 of the intersection edge terminal detecting that the image contains a high-speed moving object (step S1102). The method includes the processing unit 101 performing a high-speed moving object recognition process using information associated with the high-speed moving object acquired at another intersection (step S1103).

According to the method shown in Figure 11, the results of processing already performed at nearby intersections 5 become available at the intersection edge terminal 3 before a high-speed moving object entering the intersection enters the intersection. This significantly reduces the computational load for object recognition processing at the intersection edge terminal 3 and enables efficient object recognition. As a result, it is possible to achieve high-load computations such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI processing model, without requiring larger devices or high power consumption.

Figure 12 shows an example of the computer configuration related to this disclosure.
As shown in Figure 12, the computer 1200 comprises a CPU 1201, a main memory 1202, an auxiliary memory 1203, an interface 1204, and a non-volatile recording medium 1205.

One or more, or parts thereof, of the above-mentioned intersection edge terminals 3 and 100 may be implemented in the computer 1200. In that case, the operation of each processing unit described above is stored in the auxiliary storage device 1203 in the form of a program. The CPU 1201 reads the program from the auxiliary storage device 1203, expands it in the main memory 1202, and executes the above processing according to the program. The CPU 1201 also allocates memory areas in the main memory 1202 corresponding to each of the above-mentioned storage units according to the program. Communication between each device and other devices is performed by the interface 1204, which has a communication function and communicates according to the control of the CPU 1201. The interface 1204 also has a port for the non-volatile recording medium 1205, and performs reading information from and writing information to the non-volatile recording medium 1205.

When the intersection edge terminal 3 is implemented in the computer 1200, the operation of the processing unit 31 and its various components is stored in auxiliary storage device 1203 in program form. The CPU 1201 reads the program from auxiliary storage device 1203, expands it into main memory 1202, and executes the above processing according to the program.

Furthermore, the CPU 1201 allocates memory in the main memory 1202 for processing by the intersection edge terminal 3 according to the program. Communication with other devices via the communication unit 32 is performed by the interface 1204, which has communication functions and operates under the control of the CPU 1201. Interaction between the intersection edge terminal 3 and the user is performed by the interface 1204, which has input and output devices, presenting information to the user via the output device and accepting user input via the input device, under the control of the CPU 1201.

When the intersection edge terminal 100 is implemented in the computer 1200, the operation of the processing unit 101 is stored in auxiliary storage device 1203 in program form. The CPU 1201 reads the program from auxiliary storage device 1203, expands it in main memory 1202, and executes the above processing according to the program.

Furthermore, the CPU 1201 allocates memory in the main memory 1202 for processing by the intersection edge terminal 100, according to the program. Communication between the intersection edge terminal 100 and other devices is performed by the interface 1204, which has communication functions and operates under the control of the CPU 1201. Interaction between the intersection edge terminal 100 and the user is performed by the interface 1204, which has input and output devices, presenting information to the user via the output device and accepting user input via the input device, under the control of the CPU 1201.

One or more of the programs described above may be recorded on the non-volatile recording medium 1205. In this case, the interface 1204 may be configured to read the program from the non-volatile recording medium 1205. The CPU 1201 may then either directly execute the program read by the interface 1204, or temporarily save it in the main memory 1202 or auxiliary memory 1203 before execution.

Alternatively, a program for executing all or part of the processing performed by the intersection edge terminal 3 and the intersection edge terminal 100 may be recorded on a computer-readable recording medium, and the processing of each part may be performed by having the computer system read and execute the program recorded on this recording medium. The term "computer system" here includes hardware such as the OS (Operating System) and peripheral devices.
Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROM (Read Only Memory), CD-ROM (Compact Disc Read Only Memory), and storage devices such as hard disks built into computer systems. The above-mentioned program may be intended to implement only a part of the functions described above, and may also be able to implement the above-mentioned functions in combination with programs already recorded in the computer system.

While the present disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, as can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

Some or all of the above embodiments may also be described as follows, but are not limited to the following:

(Note 1)
A communication unit that receives images captured at the intersection,
The image is detected to contain a high-speed moving object.
An intersection edge terminal comprising a processing unit that performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

(Note 2)
The aforementioned processing unit further,
The image is detected to contain an object.
The aforementioned objects are abstracted and classified,
In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object.
The intersection edge terminal described in Appendix 1.

(Note 3)
The aforementioned communications unit further,
Multiple images captured at the aforementioned intersection are received.
The aforementioned processing unit further,
The object contained in the aforementioned multiple images is detected,
Based on the aforementioned multiple images, the amount of time variation of the object is calculated,
Based on the aforementioned time variation, the objects are classified into a predetermined category.
The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category.
Intersection edge terminal as described in Appendix 1 or 2.

(Note 4)
Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that
If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions.
An intersection edge terminal as described in any one of the appendices 1 to 3.

(Note 5)
The aforementioned processing unit,
Based on the information associated with the high-speed moving object obtained at other intersections, at least one of the order and time in which the high-speed moving object enters the current intersection is predicted.
Based on at least one of the predicted order and time, the recognition process at the intersection is performed.
An intersection edge terminal as described in any one of the appendices 1 to 4.

(Note 6)
The aforementioned processing unit,
The direction of travel of the high-speed moving object is obtained,
The aforementioned communications unit is
The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.
An intersection edge terminal as described in any one of the appendices 1 to 5.

(Note 7)
The aforementioned image was acquired by a camera at the aforementioned intersection.
The communication unit further acquires linked images from a linked camera linked to the camera, and the linked camera is positioned differently from the camera.
The processing unit performs the recognition process based on the image and the associated image.
An intersection edge terminal as described in any one of the appendices 1 to 6.

(Note 8)
The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal.
The processing unit associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to other intersection edge terminals.
An intersection edge terminal as described in any one of the notes 1 through 7.

(Note 9)
The communication unit of the intersection edge terminal receives images captured at its own intersection,
The processing unit of the aforementioned intersection edge terminal detects that the image contains a high-speed moving object,
A method comprising the processing unit performing recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

(Note 10)
The aforementioned processing unit further,
The image is detected to contain an object.
The aforementioned objects are abstracted and classified,
In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object.
The method described in Appendix 9.

(Note 11)
The aforementioned communications unit further,
Multiple images captured at the aforementioned intersection are received.
The aforementioned processing unit further,
The object contained in the aforementioned multiple images is detected,
Based on the aforementioned multiple images, the amount of time variation of the object is calculated,
Based on the aforementioned time variation, the objects are classified into a predetermined category.
The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category.
The method described in Appendix 9 or 10.

(Note 12)
Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that
If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions.
The method described in any one of the appendices 9 to 11.

(Note 13)
The aforementioned processing unit,
Based on the information associated with the high-speed moving object obtained at other intersections, at least one of the order and time in which the high-speed moving object enters the current intersection is predicted.
Based on at least one of the predicted order and time, the recognition process at the intersection is performed.
The method described in any one of the appendices 9 to 12.

(Note 14)
The aforementioned processing unit,
The direction of travel of the high-speed moving object is obtained,
The aforementioned communications unit is
The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.
The method described in any one of the appendices 9 to 13.

(Note 15)
The aforementioned image was acquired by a camera at the aforementioned intersection.
The communication unit further acquires linked images from a linked camera linked to the camera, and the linked camera is positioned differently from the camera.
The processing unit performs the recognition process based on the image and the associated image.
The method described in any one of the appendices 9 to 14.

(Note 16)
The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal.
The processing unit associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to other intersection edge terminals.
The method described in any one of the appendices 9 to 15.

(Note 17)
At the intersection edge terminal,
A communication means for receiving images captured at the intersection,
The image is detected to contain a high-speed moving object.
A processing means that performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at other intersections, and a computer program that causes the program to perform this process.
(Note 18)
The processing means further,
The image is detected to contain an object.
The aforementioned objects are abstracted and classified,
In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object.
The computer program described in Appendix 17.

(Note 19)
The aforementioned communication means further,
Multiple images captured at the aforementioned intersection are received.
The processing means further,
The object contained in the aforementioned multiple images is detected,
Based on the aforementioned multiple images, the amount of time variation of the object is calculated,
Based on the aforementioned time variation, the objects are classified into a predetermined category.
The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category.
The computer program described in Appendix 17 or 18.

(Note 20)
Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that
If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions.
A computer program described in any one of the appendices 17 to 19.

(Note 21)
The processing means is
Based on the information associated with the high-speed moving body acquired at other intersections, at least one of the order and time in which the high-speed moving body enters the current intersection is predicted.
Based on at least one of the predicted order and time, the recognition process at the intersection is performed.
A computer program described in any one of the appendices 17 to 20.

(Note 22)
The processing means is
The direction of travel of the high-speed moving object is obtained,
The aforementioned communication means is
The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.
A computer program as described in any one of the appendices 17 to 21.

(Note 23)
The aforementioned image was acquired by a camera at the aforementioned intersection.
The communication means further acquires linked images from a linked camera linked to the camera, and the linked camera is located at a different position from the camera.
The processing means performs the recognition process based on the image and the associated image.
A computer program described in any one of the appendices 17 to 22.

(Note 24)
The aforementioned communication means receives a vehicle identifier from a vehicle-mounted terminal.
The processing means associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to another intersection edge terminal.
A computer program described in any one of the appendices 17 to 23.

(Note 25)
It is a system,
A camera that acquires images at the intersection,
A system comprising an intersection edge terminal as described in any one of the appendices 1 to 8.

1 System 2 Camera 3 Intersection Edge Terminal 4 Network 5 Nearby Intersection 6 Nearby Road Camera 7 Linked Camera 8 Linked Wireless Terminal 9 Linked Vehicle Terminal 10 Linked Satellite Terminal 11 Traffic Light 12 Pole 13 Car 14 Bicycle 15 Pedestrian 31 Processing Unit 32 Communication Unit 33 Memory 34 Cooling Unit 100 Intersection Edge Terminal 101 Processing Unit 102 Communication Unit 1200 Computer 1202 Main Memory 1203 Auxiliary Memory 1204 Interface 1205 Non-Volatile Recording Medium

Claims (9)

A communication unit that receives information regarding a first image taken at its own intersection and a second image taken at another intersection ,
Information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers obtained from the second image taken at the other intersection ,
A processing unit that uses feature information based on the second image associated with the vehicle received from the other intersection and the first image to perform recognition processing for a vehicle that has newly passed through the other intersection and the current intersection, and performs recognition processing for additional information indicating the vehicle's features, including at least one of the vehicle type, body color, information contained in the license plate, and passengers, as indicated by the feature information , and transmits the information obtained from the first image through this recognition processing to an intersection identified based on the direction of travel of the vehicle, in addition to the information associated with the vehicle based on the second image acquired at the other intersection .
An intersection edge terminal equipped with this feature. The aforementioned communications unit further,
Multiple images of the first image taken at the aforementioned intersection are received.
The aforementioned processing unit further,
The object including the vehicle included in the plurality of first images is detected,
Based on the plurality of first images, the amount of time variation of the object is calculated,
Based on the aforementioned time variation, the objects are classified into a predetermined category.
Recognition processing of information indicating the characteristics of the object is performed according to a predetermined recognition method corresponding to the aforementioned category.
The intersection edge terminal according to claim 1. If the recognition of the characters on the license plate in the feature information associated with the vehicle acquired at the aforementioned other intersection includes a portion that is poorly recognized, the recognition process includes supplementing the missing portion by synthesizing the feature information calculated from the first image captured at the aforementioned intersection.
The intersection edge terminal according to claim 1. The aforementioned processing unit,
Based on the characteristic information associated with the vehicle acquired at another intersection, which includes at least one of the recording time and vehicle speed, and the distance between the current intersection and the other intersection, the system predicts at least one of the order and time at which the vehicle enters the current intersection.
Based on at least one of the predicted order and time, the recognition process is performed, which includes identifying feature information about the vehicle included in the first image captured at the intersection, and generating information by combining the feature information associated with the vehicle acquired at other intersections with the feature information calculated from the first image captured at the intersection.
The intersection edge terminal according to claim 1. The aforementioned processing unit,
The direction of travel of the aforementioned vehicle is obtained,
The aforementioned communications unit is
The characteristic information generated by the vehicle recognition process is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.
The intersection edge terminal according to claim 1. The aforementioned first image is an image acquired by a plurality of different cameras positioned at different locations in the intersection.
The processing unit links the characteristic information of the vehicle obtained as a result of the recognition process using the first images acquired by the multiple different cameras.
The intersection edge terminal according to claim 1. The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal.
The processing unit associates the vehicle identifier with the feature information generated by the recognition process, and transmits the associated feature information to another intersection edge terminal.
The intersection edge terminal according to claim 1. The communication unit of the intersection edge terminal receives information regarding the first image captured at its own intersection and the second image captured at another intersection ,
The processing unit of the intersection edge terminal acquires information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers, based on the second image taken at the other intersection.
Using the feature information based on the second image associated with the vehicle received from the other intersection and the first image , a recognition process is performed for a vehicle that has newly passed through the other intersection and the current intersection, and the recognition process is performed for additional information indicating the vehicle's features, including the vehicle type, the color of the vehicle body, the information contained in the license plate, and at least one of the passengers, as indicated by the feature information , and the information obtained from the first image through this recognition process is added to the information associated with the vehicle based on the second image acquired at the other intersection and transmitted to an intersection identified based on the direction of travel of the vehicle .
Methods that include... At the intersection edge terminal,
A communication means for receiving information regarding a first image captured at the intersection and a second image captured at another intersection ,
Information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers obtained from the second image taken at the other intersection ,
Processing means that uses feature information based on the second image associated with the vehicle received from the other intersection and the first image to perform recognition processing of a vehicle that has newly passed through the other intersection and the current intersection, and performs recognition processing of additional vehicle features indicating information including at least one of the vehicle type, body color, information contained in the license plate , and passengers indicated by the feature information , and transmits the information obtained from the first image by the recognition processing to an intersection identified based on the direction of travel of the vehicle, in addition to the information associated with the vehicle based on the second image acquired at the other intersection ,
A computer program that executes an action.