JP7148299B2 - Server device, vehicle-mounted device, and speed monitoring method - Google Patents

Description

TECHNICAL FIELD The present invention relates to a technology of a server device, an in-vehicle device, and a speed monitoring method.

"Image acquisition means for acquiring an image outside the vehicle, candidate area extraction means for extracting a candidate area of an object representing the start or end of a section on the road from the image by image processing, and the candidate area, each of which is the object. an object recognition means for recognizing the candidate area as the object based on a result of matching with a plurality of matching targets corresponding to any of them; an object storage means for storing the object recognized by the object recognition means; matching target limiting means for selecting a predetermined part of matching targets according to the type of the object stored in the object storage means from among a plurality of matching targets and using the matching targets for the matching; A device for identifying an object that assumes the following."

JP 2012-160116 A

Although the above technology avoids erroneous detection of traffic signs on the road, the processing load in the object identification device is high, so relatively expensive hardware is required to achieve both responsiveness and responsiveness. . On the other hand, there is a demand to implement speed monitoring using a drive recorder or the like, which is cheaper than a navigation device or the like. However, relatively expensive hardware is required to realize monitoring with a single drive recorder. Requires relatively expensive hardware in the device.

SUMMARY OF THE INVENTION An object of the present invention is to provide a server device, an on-vehicle device, and a speed monitoring method that perform speed monitoring with a simple configuration without requiring relatively expensive hardware.

The present invention includes a plurality of means for solving at least part of the above problems, and examples thereof are as follows. In order to solve the above problems, a server device according to the present invention includes a recognition target storage unit that stores a general road recognition image including image information of the appearance of a predetermined target that can be recognized as traveling on a general road; a communication unit that communicates with an on-vehicle device mounted on a vehicle, identification information of the on-board device, and an image of the surroundings of the vehicle associated with the identification information of the on-board device an image recognition unit for determining whether or not the general road recognition image is included in the image of the surroundings of the vehicle when received via the communication unit from the vehicle; The judging section is characterized in that, when the image recognizing section judges that the general road recognition image is included in the image taken around the vehicle, the judging section identifies the overspeeding of the vehicle.

According to the present invention, it is possible to provide a server device, a vehicle-mounted device, and a speed monitoring method that perform speed monitoring with a simple configuration without requiring relatively expensive hardware. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

It is a figure which shows the structure of the speed monitoring system to which embodiment which concerns on this invention is applied. It is a figure which shows the data structure example of a threshold-value memory|storage part. FIG. 4 is a diagram showing an example data structure of a received image storage unit; It is a figure which shows the data structure example of a speed reference storage part. It is a figure which shows the data structure example of a recognition object memory|storage part. It is a figure which shows the hardware structural example of a vehicle equipment. It is a figure which shows the hardware structural example of a server apparatus. FIG. 5 is a diagram showing the flow of autonomous monitoring processing and excessive speed determination processing; It is a figure which shows the flow of a continuous monitoring process. It is a figure which shows the example of a speed change and a transmission image. FIG. 10 is a diagram showing another example of speed change and transmission image;

Hereinafter, a speed monitoring system including an in-vehicle device and a server device according to the present invention will be described with reference to the drawings. Note that FIGS. 1 to 11 do not show the entire configuration of the speed monitoring system 1, and some of the configurations are omitted for ease of understanding. In principle, the same members are denoted by the same reference numerals throughout the drawings for describing the embodiments, and repeated descriptions thereof may be omitted. In addition, in the following embodiments, the constituent elements (including element steps, etc.) are not necessarily essential, unless otherwise specified or clearly considered essential in principle. Needless to say. In addition, when saying "consisting of A", "consisting of A", "having A", or "including A", other elements are excluded unless it is explicitly stated that only those elements are included. It goes without saying that it is not something to do. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., unless otherwise specified or in principle clearly considered otherwise, the shape is substantially the same. It shall include things that are similar or similar to, etc.

Generally, in the management of commercial vehicles, it is necessary to know whether the relevant employees are following the speed limit and driving in urban areas. In order to do so, it is necessary to determine whether the vehicle is located on a pedestrian-prohibited motorway, expressway, or other general road (including urban areas). However, when performing this discrimination processing on individual vehicle-mounted devices, it is necessary to install a module for a GPS (Global Positioning System) system, or to constantly analyze driving images by image processing to determine the type of road (general road, automobile, etc.). However, there is a limit to reducing the hardware cost of the in-vehicle device. Further, when performing the determination process using a server device, the communication cost for transmitting a large amount of image information to the server device at any time increases.

In particular, when using a drive recorder as an in-vehicle device that acquires optical images at a predetermined viewing angle for the front and rear of the vehicle, there are often significant differences in performance depending on the model of the drive recorder. , there is a risk that it will be difficult to implement the management of business vehicles.

FIG. 1 shows the structure of a speed monitoring system to which an embodiment of the invention is applied. In the speed monitoring system 1 to which the embodiment of the present invention is applied, control by application software is partly included in the control of the vehicle-mounted device 100 and the server device 300 serving as a management server. In addition, in the in-vehicle device 100, the functions of the application software are realized by controlling the hardware by an application software program that operates under the control of the OS (Operating System) of the in-vehicle device 100. It is not limited, and may be realized by a hardware circuit such as ASIC (Application Specific Integrated Circuit). On the other hand, in the server device 300 , application software functions are realized by controlling hardware by an application software program that operates under the control of the OS of the server device 300 or the like.

However, the in-vehicle device 100 is not limited to a dedicated device used in-vehicle, and may be various independently operable devices such as a personal computer, a smart phone, and a tablet terminal.

The in-vehicle device 100 includes a storage unit 110 , a control unit 120 , a communication unit 130 , an imaging unit 140 and a vehicle information acquisition unit 150 . The storage unit 110 includes a captured image storage unit 111 and a threshold storage unit 112 . The control unit 120 includes an image acquisition unit 121 , a vehicle speed identification unit 122 , an autonomous monitoring processing unit 123 and a continuous monitoring processing unit 124 .

The captured image storage unit 111 is a storage unit in which the image acquisition unit 121 acquires and stores an image captured by the imaging unit 140 of the vehicle-mounted device 100 . Specifically, the captured image storage unit 111 stores information specifying the date and time when the moving image acquired by the imaging unit 140 was captured, and an image associated with the identification information of the vehicle-mounted device 100 capturing the surroundings of the vehicle. , is stored.

FIG. 2 is a diagram illustrating an example data structure of a threshold storage unit. Information on various thresholds is stored in the threshold storage unit 112 . Specifically, the threshold storage unit 112 stores a report start speed 112a and a detailed report speed 112b. The report start speed 112a is a travel speed threshold that triggers the autonomous monitoring processing unit 123, which will be described later, to start the autonomous monitoring process. The detailed report speed 112b is a running speed threshold that triggers the continuous monitoring processing unit 124, which will be described later, to transmit an image to the server device 300 in continuous monitoring processing.

In addition, the report start speed 112a is a threshold value based on the speed limit mainly on general roads (roads and urban areas that do not include motorways and national highways), and driving at a speed above that speed is extremely dangerous. It is sufficient if the speed threshold is high for In this embodiment, the report start speed 112a is assumed to be 60 km/h (kilometers per hour).

In addition, the detailed report speed 112b is a threshold value based on the traveling speed of the degree of stopping or slowing down. I wish I had. In this embodiment, the detailed report speed 112b is assumed to be 20 km/h (kilometers per hour).

Returning to the description of FIG. The image acquisition unit 121 acquires the captured image by controlling the imaging unit 140, and stores the image in the captured image storage unit 111 in association with the time.

The vehicle speed specifying unit 122 specifies the vehicle speed by specifying the information about the vehicle speed acquired from the vehicle information acquiring unit 150, for example, the vehicle speed value obtained from the integration of the vehicle speed pulse and the acceleration.

The autonomous monitoring processing unit 123 performs autonomous monitoring processing of the vehicle by the in-vehicle device 100 . Specifically, the autonomous monitoring processing unit 123 uses the vehicle speed information specified by the vehicle speed specifying unit 122 to determine whether or not the report start speed 112a has been exceeded (whether the speed has become faster than the report start speed 112a). ) is determined. Then, when the time is exceeded, the autonomous monitoring processing unit 123 stores images of a predetermined number of seconds (for example, 10 seconds before and after) based on that time (the time when the excess is determined) in the captured image storage unit 111 . is obtained by sampling at intervals of 1 second and transmitted to the server device 300 .

Upon receiving a continuous monitoring request from the server device 300, the continuous monitoring processing unit 124 starts continuous monitoring processing, which will be described later. More specifically, the continuous monitoring processing unit 124 uses the vehicle speed information specified by the vehicle speed specifying unit 122 to determine whether or not the vehicle speed is less than the detailed report speed 112b. Then, if it is less than the detailed report speed 112b, the continuous monitoring processing unit 124 displays images of a predetermined number of seconds (for example, 180 seconds before) based on that time (when it is determined that the detailed report speed is less than 112b). are obtained by sampling from the captured image storage unit 111 at intervals of 5 seconds, and transmitted to the server device 300 .

The communication unit 130 communicates with the server device 300 via a mobile phone network (LTE: Long Term Evolution), a wireless LAN (WiFi: registered trademark), or the like, for example, the Internet.

The image capturing unit 140 controls an optical camera or the like provided to capture an image in a predetermined direction of the vehicle in which the vehicle-mounted device 100 is mounted, captures an image, and transfers the captured image to the image acquisition unit 121 .

The vehicle information acquisition unit 150 acquires information related to vehicle management in which the vehicle-mounted device 100 is installed. For example, the vehicle information acquisition unit 150 receives information from an in-vehicle network (not shown) and acquires vehicle speed pulse or acceleration information.

The server device 300 is communicably connected to the vehicle-mounted device 100 via a network such as the Internet. Server device 300 is a so-called computer server device, and includes storage unit 310 , control unit 320 , and communication unit 330 .

Storage unit 310 stores received image storage unit 311 , speed reference storage unit 312 , and recognition target storage unit 313 . The control unit 320 includes an image recognition unit 321 , an excess speed criteria identification unit 322 , an excess speed determination unit 323 , a continuous monitoring request unit 324 , and a determination recording processing unit 325 .

FIG. 3 is a diagram showing an example data structure of a received image storage unit. The received image storage unit 311 includes an in-vehicle device identifier 311a, a driver identifier 311b, a date and time 311c, a received image 311d, a vehicle speed 311e, an overspeed determination result 311f, and a continuous monitoring identifier 311g.

The vehicle-mounted device identifier 311 a is information for identifying the vehicle-mounted device 100 . The driver identifier 311b is information for identifying the driver who drives the vehicle equipped with the vehicle-mounted device identifier 311a. The date and time 311c is information specifying the date and time of communication from the vehicle-mounted device 100 mounted on the vehicle.

Received image 311d is a single image or a series of multiple images received in a single report. The vehicle speed 311e is information specifying the speed of the vehicle. The speeding determination result 311f is information specifying the result of determination as to whether or not there is speeding. For example, the speeding determination result 311f stores information indicating one of "speeding", "no speeding", and "unknown".

The continuous monitoring identifier 311g is an identifier for storing the image received during continuous monitoring in association with the image received during autonomous monitoring. For example, when it is determined that continuous monitoring is required in autonomous monitoring, by sharing a common identifier with images received in continuous monitoring related to the determination result, it is possible to associate autonomous monitoring and continuous monitoring. can take

FIG. 4 is a diagram showing an example data structure of a speed reference storage unit. The speed reference storage unit 312 includes a road type 312a and a reference speed 312b.

The road type 312a is information for identifying the type of road. For example, the types of roads include types classified according to predetermined rules, such as general roads, urban areas, expressways, and highways. However, the type is not limited to this, and may be classified from the viewpoint of the speed limit. The reference speed 312b is information specifying the standard maximum speed for each type of road specified by the road type 312a.

FIG. 5 is a diagram showing an example data structure of a recognition target storage unit. The recognition target storage unit 313 includes target objects 313a, positions 313b, and possible road types 313c. The target object 313a is information specifying a predetermined object (recognition target) included in the image received by the image recognition unit 321 . For example, the object 313a includes a general road recognition image including image information of the external appearance of a predetermined object such as a road sign, a traffic light, a pedestrian, a bicycle, a traffic light, a crosswalk, etc., which can be recognized as traveling on a general road. , a green sign, a gate of a toll gate, a median strip, and a motorway recognition image including image information of the external appearance of a predetermined object that can be recognized as driving on a motorway. The position 313b is information specifying in which region of the predetermined regions on the image the recognition target specified by the target object 313a should be included. For example, it is a predetermined area such as the upper half of the image or the left half of the image. The possible road type 313c is information specifying the estimated road type of the position 313b when there is an image in which the object 313a is included in the position 313b. For example, if an object "P" (for example, a traffic light) is reflected in the "upper half of the image" area, it is possible to specify that the position is likely to be on a "general road". be.

The image recognition unit 321 determines whether or not an image recognized as a general road and an image recognized as an automobile-only road are included in the captured image of the surroundings of the vehicle. Specifically, the image recognition unit 321 receives one or a series of multiple images transmitted from the vehicle-mounted device 100 and stores them in the received image storage unit 311 . Then, the image recognition unit 321 identifies the existence of the object stored in the object 313a and the position on the image for each of the images. Then, if an object exists and its position on the image is included in the area specified by the position 313b, the type of road at the position where the image was captured is specified by the road type 313c. presumed to be of the type

Note that if the series of multiple images are all of the same road type, the image recognition unit 321 specifies the road type at that position. ”. In addition, when a plurality of road types are included, the image recognition unit 321 identifies the road type according to a predetermined standard (for example, identifies the most identified road types).

The speeding reference identification unit 322 identifies a threshold that serves as a reference for speeding. Specifically, the speeding reference identification unit 322 refers to the speed reference storage unit 312 according to the road type identified by the image recognition unit 321, and identifies the reference speed 312b.

The speeding determination unit 323 determines whether the speed of the vehicle exceeds the speed standard specified by the speeding standard specifying unit 322 .

The continuous monitoring request unit 324 transmits a continuous monitoring request to the vehicle-mounted device 100 for further images when the image recognition unit 321 cannot identify the road type from the image received during the autonomous monitoring. More specifically, when the continuous monitoring request unit 324 receives the image information obtained by the autonomous monitoring process, the image recognition unit 321 determines that the general road recognition image is not included in the image captured around the vehicle. , a continuous speed monitoring request for requesting the vehicle-mounted device 100 to continuously monitor the speed is transmitted.

The determination record processing unit 325 records the determination record. Specifically, the determination record processing unit 325, as a result of the determination by the speeding determination unit 323, indicates "excessive speed" if the speed is not exceeded, "not overspeeded" if the speed is not exceeded, and if it is unknown , is associated with the corresponding image in the received image storage unit 311 and stored in the excess speed determination result 311f as the determination result.

The communication unit 330 communicates with the in-vehicle device 100 via a mobile phone network (LTE), a wireless LAN (WiFi: registered trademark), or the like.

FIG. 6 is a diagram illustrating an example of the hardware structure of the vehicle-mounted device. The in-vehicle device 100 includes a camera 101, an arithmetic device 102, a communication device 103, a memory 104, an external storage device 105, an external device connection device 106, and a bus 109 connecting them.

The camera 101 is a device that acquires an optical image of visible light with a predetermined angle of view. The computing device 102 is, for example, a computing device such as a CPU (Central Processing Unit).

The communication device 103 is a device such as a network module that establishes a communication path with another device and transmits/receives information via a mobile phone network, a wireless network, or the like.

The memory 104 is a main memory such as RAM (Random Access Memory).

The external storage device 105 is, for example, a hard disk, an SSD (Solid State Drive), or a device that writes information to various nonvolatile storage media, or reads predetermined information from the nonvolatile storage media.

The external device connection device 106 is, for example, a device that connects with other devices, and for example, connects with other devices via an in-vehicle network to acquire vehicle speed pulse information or acceleration information.

Each functional unit of the control unit 120 described above, that is, the image acquisition unit 121, the vehicle speed identification unit 122, the autonomous monitoring processing unit 123, and the continuous monitoring processing unit 124 is constructed by the arithmetic unit 102 reading and executing a predetermined program. . Therefore, the memory 104 stores a program for realizing the processing of each functional unit.

Further, the storage unit 110 described above uses the memory 104 or the external storage device 105, the imaging unit 140 uses the camera 101, the vehicle information acquisition unit 150 uses the external device connection device 106, and the communication unit 130 uses the communication device 103. , respectively.

It should be noted that each of the components described above is a classification of the configuration of the vehicle-mounted device 100 according to the main processing content, in order to facilitate understanding. Therefore, the present invention is not limited by the method of classifying the constituent elements or their names. The configuration of the in-vehicle device 100 can also be classified into more components according to the content of processing. Also, one component can be grouped to perform more processing.

Moreover, each functional unit of the control unit 120 may be constructed by hardware (ASIC, GPU (Graphics Processing Unit), etc.) without being limited to the CPU. Also, the processing of each functional unit may be executed by one piece of hardware, or may be executed by a plurality of pieces of hardware.

FIG. 7 is a diagram showing an example hardware structure of the server device 300. As shown in FIG. The server device 300 includes an arithmetic device 302, a communication device 303, a memory 304, an external storage device 305, and a bus 309 connecting them.

The computing device 302 is, for example, a computing device such as a CPU. The communication device 303 is a device such as a network module that establishes a communication path with another device and transmits/receives information via a mobile phone network, a wireless network, or the like.

The memory 304 is, for example, a memory device such as a RAM.

The external storage device 305 is a device that writes information to, for example, a hard disk, SSD, or other various nonvolatile storage media, or reads predetermined information from the nonvolatile storage media.

Each functional unit of the control unit 320, that is, the image recognition unit 321, the speeding reference specifying unit 322, the speeding determination unit 323, the continuous monitoring request unit 324, and the determination recording processing unit 325, the arithmetic unit 302 executes a predetermined program. Constructed by reading and executing. Therefore, the memory 304 stores a program for realizing the processing of each functional unit.

The storage unit 310 described above is realized by the memory 304 or the external storage device 305, and the communication unit 330 is realized by the communication device 303, respectively.

It should be noted that each component described above is a classification of the configuration of the server device 300 according to the main processing content, in order to facilitate understanding. Therefore, the present invention is not limited by the method of classifying the constituent elements or their names. The configuration of the server device 300 can also be classified into more components according to the processing content. Also, one component can be grouped to perform more processing.

Moreover, each functional unit of the control unit 320 may be constructed by hardware such as an ASIC instead of the CPU. Also, the processing of each functional unit may be executed by one piece of hardware, or may be executed by a plurality of pieces of hardware.

[Description of Operation] Next, the operation related to processing during autonomous monitoring will be described. FIG. 8 is a diagram showing the flow of autonomous monitoring processing and excessive speed determination processing. The autonomous monitoring process is started when the vehicle speed exceeds the speed threshold of the report start speed 112a after the onboard device 100 is activated.

First, the autonomous monitoring processing unit 123 acquires an image by sampling at equal intervals when the threshold value is exceeded and 10 seconds before and after the threshold value is exceeded (step S001). Specifically, the autonomous monitoring processing unit 123 identifies the time when the vehicle speed exceeds the reporting start speed, and images captured at that time and at predetermined times before and after that time (for example, 10 seconds before and 10 seconds after) is specified from the photographed image storage unit 111, and a total of 21 images are obtained by sampling at intervals of 1 second. Note that the sampling interval is not limited to 1 second, and the range of images to be acquired is not limited to 10 seconds before and after the vehicle speed exceeds the reporting start speed. For example, when the image size to be acquired is large and the communication time is expected to be longer than a predetermined amount (for example, transmission takes 20 seconds or more), the autonomous monitoring processing unit 123 determines when the vehicle speed exceeds the report start speed. The amount of data to be transmitted may be reduced, for example, by limiting to the previous 10 seconds, or by setting the sampling interval to 2 seconds.

Then, the autonomous monitoring processing unit 123 associates the in-vehicle device identifier, the driver identifier, and the vehicle speed, and transmits the image to the server device 300 (step S002). Specifically, the autonomous monitoring processing unit 123 reads out an on-vehicle device identifier and a driver identifier pre-stored in an area (not shown) of the storage unit 110 of the on-vehicle device 100 and acquires the vehicle speed from the vehicle speed identification unit 122 . And it transmits to the server apparatus 300 with the image acquired in step S001.

The above is the flow of the autonomous monitoring process. According to the autonomous monitoring process, when it is detected that the vehicle equipped with the vehicle-mounted device 100 travels at a speed faster than a predetermined threshold, video information that serves as a clue for identifying the position is transmitted to the server device 300. becomes possible.

The flow of excessive speed determination processing will be described. Upon receiving the in-vehicle identifier, the driver identifier, the speed and the image transmitted in step S002 of the autonomous monitoring process, the image recognition unit 321 performs the process of recognizing the excess time image (step S003). Specifically, the image recognition unit 321 stores the received vehicle-mounted identifier, driver identifier, speed and image in the received image storage unit 311 . Then, it specifies whether or not the target object 313a of the recognition target storage unit 313 is included in the position in the image specified by the position 313b in any of the received images, and if it is included, It also identifies the target object.

Then, the speeding reference specifying unit 322 determines the speeding reference (step S004). Specifically, the speeding reference identification unit 322 identifies a possible road type 313c in the recognition target storage unit 313 using the object 313a recognized in step S003. Then, it is determined whether or not the possible road type is "general road". In this process, if even one "general road" is included in the possible road types, it may be determined to be a "general road", or it may be mixed with other possible road types. If it does, it may be determined that it is a "general road" according to the degree (for example, if both "general road" and "motorized road" are included as possible road types , it may be determined that the image is not a general road unless more than half of the images are determined to be a "general road").

If it is determined to be a "general road" ("general road" in step S004), the speeding determination unit 323 determines that there is overspeeding, and the determination recording processing unit 325 is recorded (step S005). Specifically, the determination recording processing unit 325 records “excessive speed” in the excessive speed determination result 311 f of the received image storage unit 311 .

Then, the determination record processing unit 325 notifies the vehicle-mounted device 100 of the excessive speed (step S006). Specifically, the determination record processing unit 325 transmits a predetermined message to the in-vehicle device 100 notifying that the speed limit has been exceeded.

If it is determined to be other than "general road" ("non-general road" in step S004), the continuous monitoring request unit 324 instructs the in-vehicle device 100 to perform continuous monitoring processing to continuously monitor the speed. request (step S007). Specifically, the continuous monitoring request unit 324 transmits a continuous monitoring request to the vehicle-mounted device 100, issues unique key information, and records it in the continuous monitoring identifier 311g.

Then, the image recognition unit 321 performs recognition processing of the pre-deceleration image (step S008). Specifically, the image recognition unit 321 stores the vehicle-mounted identifier, the driver identifier, the speed, the image, and the continuous-monitoring identifier received from the vehicle-mounted device 100 after the continuous-monitoring request in the received-image storage unit 311 . Then, it specifies whether or not the target object 313a of the recognition target storage unit 313 is included in the position in the image specified by the position 313b in any of the received images, and if it is included, It also identifies the target object.

Then, the speeding reference identifying unit 322 determines whether or not there is an object to be recognized on the motorway (step S009). Specifically, the speeding reference identification unit 322 identifies a possible road type 313c in the recognition target storage unit 313 using the object 313a recognized in step S008. Then, it is determined whether or not the possible road type is "motorway".

In this process, if even one "motorized road" is included in the possible road types, it may be determined to be a "motorized road", or other possible road types may be determined. If it is mixed with , it may be determined that it is an "automobile exclusive road" according to the degree. For example, when both "general road" and "motorized road" are included as possible road types, the image determined as "general road" is more likely than the image determined as "motorized road". If it is acquired on an old date and time, it may be determined that it is not a "motorway".

Since it is unclear on which type of road the overspeeding detected in the preceding autonomous monitoring process occurred, this process is intended to retroactively identify it. This is because if an image captured by the camera with an older date and time suggests that the vehicle is traveling on a general road, it should be assumed that the excessive speeding of the autonomous monitoring process was performed on the general road.

If there is an object to be recognized on the motorway (“yes” in step S009), the speeding reference specifying unit 322 changes to the speed reference for the motorway (step S010). Specifically, the speeding reference identification unit 322 refers to the speed reference storage unit 312, identifies the reference speed 312b according to the road type 312a, and changes the speed reference. That is, for example, when it is determined that the general road recognition image is not included in the image of the surroundings of the vehicle and the motor vehicle road recognition image is included in the image of the surroundings of the vehicle, it is determined that the vehicle is overspeeding. Change the threshold to motorway.

Then, the speeding determination unit 323 determines whether or not there is speeding (step S011). Specifically, the overspeed determination unit 323 determines whether or not the speed during the autonomous monitoring process was overspeed using the speed reference changed in step S010.

If there is overspeeding ("Yes" in step S011), determination record processing unit 325 records the overspeeding (step S005). Specifically, the determination recording processing unit 325 identifies the overtime images having the same continuous monitoring identifier, and records “excessive speed” in the overspeed determination result 311f of the received image storage unit 311 . Then, the determination record processing unit 325 shifts the control to step S006.

If there is no speeding ("No" in step S011), the determination record processing unit 325 records that there is no speeding (step S012). Specifically, the determination recording processing unit 325 identifies the overtime images having the same continued monitoring identifier, and records “no overspeed” in the overspeed determination result 311f of the received image storage unit 311 . This is because the overspeeding detected in the autonomous monitoring process is presumed to have occurred on a motorway, and as a result of changing the speed standard to that of the motorway, it is presumed that it was not overspeeding. It is for

If there is no motorway recognition target object ("None" in step S009), the speeding reference identification unit 322 determines the speed reference (step S013). Specifically, the speeding reference identification unit 322 identifies a possible road type 313c in the recognition target storage unit 313 using the object 313a recognized in step S008. Then, it is determined whether or not the possible road type is "general road". In this process, if even one "general road" is included in the possible road types, it may be determined to be a "general road", or it may be mixed with other possible road types. If it does, it may be determined that it is a "general road" according to the degree (for example, if both "general road" and "motorized road" are included as possible road types , it may be determined that the image is not a general road unless more than half of the images are determined to be a "general road").

Further, the criteria for determining whether or not the road is a general road in step S013 may be the same as in step S004, or may be different. For example, in the process of step S013, the image is determined by continuous monitoring after it has already been once determined to be "not a general road" in step S004, which is the determination of the image by autonomous monitoring. It is also conceivable that the criteria may be such that it is difficult to judge that the For example, when both “general road” and “motorized road” are included as possible road types, the number of images determined as “general road” does not exceed a predetermined percentage (for example, 70%). It may be determined that it is not a general road unless it exceeds

If it is determined to be a "general road" ("general road" in step S013), the overspeed determination unit 323 advances the control to step S005. Specifically, the determination recording processing unit 325 identifies the overtime images having the same continuous monitoring identifier, and records “excessive speed” in the overspeed determination result 311f of the received image storage unit 311 . Then, the determination record processing unit 325 shifts the control to step S006.

If it is determined to be other than "general road" ("other than general road" in step S013), the continuous monitoring request unit 324 causes the image transmission from the vehicle-mounted device 100 related to the autonomous monitoring process to be repeated a predetermined number of times. It is determined whether or not it is less than the number of times (step S014). The number of repetitions has a predetermined threshold value (for example, "3"). If the number of image transmissions from the vehicle-mounted device 100 related to the autonomous monitoring process is less than the predetermined number of repetitions ("Yes" in step S014), the continuous monitoring request unit 324 advances the control to step S007. However, the continuous monitoring request unit 324 only transmits the continuous monitoring request to the in-vehicle device 100 in step S007 after proceeding with the control, and reuses the key information.

If the number of image transmissions from the vehicle-mounted device 100 related to the autonomous monitoring process is equal to or greater than the predetermined number of repetitions (“No” in step S014), the determination record processing unit 325 records determination impossibility (step S015). Specifically, the determination record processing unit 325 identifies the overtime images having the same continued monitoring identifier, and records “determination impossible” in the overspeed determination result 311f of the received image storage unit 311 . This is because it is not possible to determine whether the overspeeding detected in the autonomous monitoring process was on a motorway or on a general road, so it is presumed that it may or may not have been overspeeding. This is because

The above is the flow of the excessive speed determination process. According to the speeding determination process, if it is clear that the speeding reported as a result of autonomous monitoring was done on a general road, it is determined as overspeeding and recorded, and if it is not clear that it is a general road, continuous monitoring is performed on the vehicle request to machine 100. If the image reported in the process of continuous monitoring includes a motorway, it is determined that the speeding of autonomous monitoring occurred on a motorway, and the threshold for overspeeding is set again according to the road type. to judge excessive speed. If the image reported during continuous monitoring does not include a motorway and is not considered to be a general road, record it as undecidable after receiving multiple continuous monitoring reports. .

Therefore, according to the speeding determination process, speeding can be determined by communication of sampling images for a certain period of time that are transmitted only when the vehicle-mounted device 100 is in a predetermined running state (mainly vehicle speed). Processing load can be suppressed because image recognition processing is not involved. In addition, the amount of communication can be greatly reduced compared to constant image transmission.

FIG. 9 is a diagram showing the flow of continuous monitoring processing. The continuous monitoring process is started when the vehicle-mounted device 100 receives the continuous monitoring request transmitted from the server device 300 in step S007 of the excessive speed determination process.

First, the continuous monitoring processing unit 124 acquires the vehicle speed (step S101). Specifically, the continuous monitoring processing unit 124 acquires vehicle speed information from the vehicle speed specifying unit 122 .

Then, the continuous monitoring processing unit 124 determines whether or not the vehicle speed is less than the detailed report speed (step S102). Specifically, the continuous monitoring processing unit 124 determines whether or not the vehicle speed acquired in step S101 is less than the detailed report speed 112b of the threshold storage unit 112 . If the vehicle speed is not less than the detailed report speed ("No" in step S102), the continuous monitoring processing unit 124 returns the control to step S101.

When the vehicle speed is less than the detailed report speed ("Yes" in step S102), the continuous monitoring processing unit 124 samples images at equal intervals for the last three minutes (step S103). Specifically, the continuous monitoring processing unit 124 identifies an image captured at a predetermined time (for example, 3 minutes) before the time from the captured image storage unit 111, and samples the image at intervals of 5 seconds. get a copy. Note that the sampling interval is not limited to 5 seconds, and the range of images to be acquired is not limited to the three minutes immediately before the vehicle speed below the detailed report speed is detected. For example, when the image size to be acquired is large and the communication time is expected to be longer than a predetermined amount (for example, transmission takes 20 seconds or more), the continuous monitoring processing unit 124 sets the sampling interval to 10 seconds, The amount of data to be transmitted may be reduced, for example, the range may be two minutes immediately before the vehicle speed less than the detailed report speed is detected.

Then, the continuous monitoring processing unit 124 excludes images before the transmitted image (step S104). Specifically, the continuous monitoring processing unit 124 excludes, from the images acquired in step S103, the images in the period in which the images already transmitted to the server apparatus 300 in the autonomous monitoring process or the continuous monitoring process are included. This is because even if images from overlapping periods are sent again, there will be no big difference in the images that are captured, and there is a high possibility that only the same processing results will be obtained. to achieve.

Then, the continuous monitoring processing unit 124 associates the vehicle-mounted device identifier, the driver identifier, and the continuous monitoring identifier, and transmits the image to the server device 300 (step S105). Specifically, the continuous monitoring processing unit 124 reads the vehicle-mounted device identifier and the driver identifier stored in advance in a region (not shown) of the storage unit 110 of the vehicle-mounted device 100 . It is transmitted to the server device 300 together with the continuous monitoring identifier included in the continuous monitoring request and the image acquired in step S104.

The above is the flow of continuous monitoring processing. According to the continuous monitoring process, when it is detected that the vehicle equipped with the in-vehicle device 100 has traveled or stopped at a speed slower than a predetermined threshold, video information (pre-deceleration image) for a predetermined period immediately before that is sent to the server device 300. can be sent.

FIG. 10 is a diagram showing an example of speed change and transmission image. FIG. 10 shows a graph 400 plotting the vehicle speed 403 with the speed axis on the vertical axis 401 and the time axis on the horizontal axis 402, and a series of moving images (at least 1 FPS (frames per second)) acquired parallel to the time axis. ), one or more still images 431 cut out from the image 430 in the autonomous monitoring process, and one or more still pictures 432 cut out from the image 430 in the continuous monitoring process.

In the example of FIG. 10, the vehicle speed 403 exceeds the reporting start speed 410 at time t1, and then the continuous monitoring request is accepted at time t2. Then, at time t3 after that (180 seconds or more after t2), the vehicle speed 403 is less than the detailed report speed 420. FIG. In this case, a total of 21 images to be transmitted in the autonomous monitoring process are cut out from the images 430 in the period of 10 seconds before and after the time t1 (from the time t1s to the time t1e) with a sampling interval of 1 second.

As for the images to be transmitted in the continuous monitoring process, a total of 37 images to be transmitted in the continuous monitoring process are cut out from the image 430 during the period from time t3 to time t4, which is 180 seconds before time t3, at a sampling interval of 5 seconds.

FIG. 11 is a diagram showing another example of speed change and transmission image. FIG. 11 shows a graph 400' plotting the vehicle speed 403' with the speed axis on the vertical axis 401 and the time axis on the horizontal axis 402, and a series of moving images (at least 1 FPS (frame every second)), one or more still images 431 cropped from the image 430 in the autonomous monitoring process, and one or more still images 432′ cropped from the image 430 in the continuous monitoring process. there is

In the example of FIG. 11, the vehicle speed 403' exceeds the reporting start speed 410 at time t1, and the continuous monitoring request is accepted at time t2. Then, at time t3' after that (within 180 seconds from t2), the vehicle speed 403' is less than the detailed report speed 420. In this case, a total of 21 images to be transmitted in the autonomous monitoring process are extracted from the images 430 in the period of 10 seconds before and after the time t1 (from the time t1s to the time t1e) with a sampling interval of 1 second. is the same as the example in

On the other hand, as for the images to be transmitted in the continuous monitoring process, a total of 37 images to be transmitted in the continuous monitoring process are cut out from the image 430 during the period from time t3′ to time t4′, which is 180 seconds before time t3′, at a sampling interval of 5 seconds. However, the image from time t4' to time t1e included in the period from time t1s to t1e is excluded from transmission because it falls within the overlapping period. That is, still images 432' to be transmitted are cut out from the images 430 during the period from t1e to t3', for example, a total of 34 images to be transmitted in the continuous monitoring process at sampling intervals of 5 seconds.

The above is the speed monitoring system 1 to which the embodiment according to the present invention is applied. According to the speed monitoring system 1, it is possible to provide a server device, an in-vehicle device, and a speed monitoring method that perform speed monitoring with a simple configuration without requiring relatively expensive hardware.

However, the invention is not limited to the above embodiments. Various modifications of the above embodiment are possible within the scope of the technical idea of the present invention. For example, in the above-described embodiment, the server device 300 requests the vehicle-mounted device 100 to perform continuous monitoring when receiving a report of autonomous monitoring and cannot determine whether the road is a general road, but the present invention is not limited to this. For example, the in-vehicle device 100 may start continuous monitoring immediately after reporting autonomous monitoring. In that case, for example, the continuous monitoring process is executed when image transmission in the continuous monitoring is performed a predetermined number of times, when a predetermined time (for example, 10 minutes) elapses, or when a request to end the continuous monitoring is received from the server device 300. It may be terminated. In the server device 300, when the presence or absence of speeding is specified, the speeding determination unit 323 may request the vehicle-mounted device 100 to end continuous monitoring.

REFERENCE SIGNS LIST 1 speed monitoring system 100 in-vehicle device 110 storage unit 111 photographed image storage unit 112 threshold storage unit 120 control unit 121 Image acquisition unit 122 Vehicle speed identification unit 123 Autonomous monitoring processing unit 124 Continuous monitoring processing unit 130 Communication unit 140 Imaging unit 150 Vehicle information Acquisition unit 300 Server device 310 Storage unit 311 Received image storage unit 312 Speed reference storage unit 313 Recognition target storage unit 320 Control unit , 321... Image recognition unit, 322... Speeding reference identification unit, 323... Speeding determination unit, 324... Continuous monitoring request unit, 325... Judgment record processing unit, 330... Communication department.

Claims (11)

a recognition target storage unit for storing a general road recognition image including image information of the external appearance of a predetermined target that can be recognized as traveling on a general road;
a communication unit that communicates with an in-vehicle device mounted on a predetermined vehicle;
receiving the identification information of the vehicle-mounted device, the image of the surroundings of the vehicle associated with the identification information of the vehicle-mounted device, and the predetermined vehicle speed from the vehicle-mounted device via the communication unit; an image recognition unit for determining whether or not the general road recognition image is included in an image captured around the vehicle;
and an excess speed determination unit,
The speeding determination unit compares the vehicle speed of the predetermined vehicle with a predetermined threshold when the image recognition unit determines that the general road recognition image is included in the image of the surroundings of the vehicle. identifying overspeeding of the vehicle;
A server device characterized by: The server device according to claim 1,
The recognition object storage unit stores an automobile road recognition image including image information of the appearance of a predetermined object that can be recognized as traveling on the automobile road,
The image recognition unit
When the identification information of the vehicle-mounted device and the image of the surroundings of the vehicle associated with the identification information of the vehicle-mounted device are received from the vehicle-mounted device via the communication unit, the vehicle-only road recognition image is generated. Determining whether it is included in the image taken around the vehicle,
The speed excess determination unit
When the image recognition unit determines that the general road recognition image is not included in the image of the surroundings of the vehicle and that the motor vehicle road recognition image is included in the image of the surroundings of the vehicle; changing the threshold for overspeeding of the vehicle to the motorway and comparing it with the predetermined vehicle speed to determine overspeeding;
A server device characterized by: The server device according to claim 1,
a received image storage unit that stores an image of the surroundings of the vehicle received via the communication unit;
The image recognition unit stores the received image of the surroundings of the vehicle in the received image storage unit.
A server device characterized by: The server device according to claim 1,
a received image storage unit for storing an image of the surroundings of the vehicle received via the communication unit; and a judgment recording processing unit,
The image recognition unit stores the received image of the surroundings of the vehicle in the received image storage unit,
When the speeding determination unit identifies the speeding of the vehicle, the determination recording processing unit stores information indicating the speeding in association with an image of the surroundings of the vehicle in the received image storage unit.
A server device characterized by: The server device according to claim 1,
A continuous speed monitoring request for requesting the in-vehicle device to continuously monitor the speed when the image recognition unit determines that the general road recognition image is not included in the image captured around the vehicle. comprising a continuous monitoring request unit that transmits
A server device characterized by: The server device according to claim 2,
A continuous speed monitoring request for requesting the in-vehicle device to continuously monitor the speed when the image recognition unit determines that the general road recognition image is not included in the image captured around the vehicle. a continuous monitoring request unit that transmits
The image recognition unit receives from the vehicle-mounted device to which the continuous speed monitoring request is transmitted, the identification information of the vehicle-mounted device, an image of the surroundings of the vehicle associated with the identification information of the vehicle-mounted device, and the When the vehicle speed of the vehicle is received via the communication unit, determining whether or not the image for recognizing the motorway is included in the image captured around the vehicle;
A server device characterized by: The server device according to claim 6 ,
When the image recognition unit determines that the general road recognition image is included in the image of the surroundings of the vehicle, the speeding determination unit identifies the speeding of the vehicle and continues to the in-vehicle device. request to end monitoring,
A server device characterized by: an image acquisition unit that acquires an image of the surroundings of the mounted vehicle;
a vehicle speed identification unit that identifies the speed at which the vehicle travels;
a communication unit that communicates with another device;
an autonomous monitoring processing unit that transmits a part of the image to a predetermined device via the communication unit when the vehicle travels faster than a predetermined speed;
a continuous monitoring processing unit that performs continuous monitoring processing when receiving a continuous speed monitoring request requesting continuous speed monitoring from the predetermined device after transmitting a part of the image to the predetermined device,
When the vehicle travels or stops at a speed slower than an assumed stop speed, which is a predetermined speed different from the predetermined speed, the continuous monitoring processing unit updates the image obtained by the image obtaining unit via the communication unit. send a portion to a given device,
An in-vehicle device characterized by: The in-vehicle device according to claim 8 ,
The part of the image transmitted by the continuous monitoring processing unit includes an image acquired in the past a predetermined time from when the vehicle traveled or stopped at the deemed stopping speed or less.
An in-vehicle device characterized by: The in-vehicle device according to claim 9 ,
The continuous monitoring processing unit, when a part of the image to be transmitted includes an overlapping image acquired in a period overlapping with the period in which the image transmitted by the autonomous monitoring processing unit is included, selects one of the images Exclude the duplicate image from the part and send it,
An in-vehicle device characterized by: A speed monitoring method by a server device,
The server device has
a recognition target storage unit for storing a general road recognition image including image information of the external appearance of a predetermined target that can be recognized as traveling on a general road;
A communication unit that communicates with an in-vehicle device mounted on a predetermined vehicle, and a control unit,
The control unit
receiving the identification information of the vehicle-mounted device, the image of the surroundings of the vehicle associated with the identification information of the vehicle-mounted device, and the predetermined vehicle speed from the vehicle-mounted device via the communication unit; an image recognition step of determining whether or not the general road recognition image is included in an image of the surroundings of the vehicle;
Carry out an excess speed determination step and
In the speeding determination step, when it is determined in the image recognition step that the general road recognition image is included in the image taken around the vehicle, the vehicle speed of the predetermined vehicle is compared with a predetermined threshold value. identifying overspeeding of the vehicle;
A speed monitoring method characterized by:

Images