JP7123530B2 - Communication device, communication system, and communication method - Google Patents

Description

The present invention relates to technology for uploading image data.

Various systems have been developed in which a sensor server collects information about road conditions transmitted from a communication device in a vehicle. An example of such a system is disclosed in US Pat.

JP 2006-318356 A (paragraph 0018)

In the road information distribution system disclosed in Patent Literature 1, a vehicle-side device constantly uploads images captured by an in-vehicle camera that captures the surroundings of a vehicle to a road information distribution server. In other words, in the road information distribution system disclosed in Patent Literature 1, images are continuously transmitted from vehicle-side devices to the road information distribution server without restriction. As a result, the amount of communication between the vehicle-side device and the road information distribution server increases, and at least part of the bandwidth used for communication between the vehicle-side device and the road information distribution server is always consumed by video uploads. occupied.

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide a technique for uploading image data while saving the amount of communication and the band used for communication.

A communication device according to the present invention includes an acquisition unit that acquires an image captured by a camera that captures the surroundings of a vehicle, and a first transmission unit that transmits identification information for identifying a section in which the vehicle is traveling. a receiving unit for receiving presence/absence information indicating whether or not an obstacle exists in the section in which the vehicle is traveling; and detecting an obstacle present in the section in which the vehicle is traveling from the image. a detection unit, a match determination unit for determining whether or not the detection result of the detection unit and the presence/absence information match, and a second image transmission unit configured to transmit the image when the match determination unit determines that the detection result does not match the presence/absence information. and a transmission unit (first configuration).

Further, in the communication device having the first configuration, the second transmission unit may be configured to transmit the detection result of the detection unit (second configuration).

In addition, the communication device having the first or second configuration above further includes a measuring unit that measures the passage time of the section in which the vehicle is traveling, and the receiving section is set to the section in which the vehicle is traveling. The second transmission unit receives the information of the passed reference time, and the second transmission unit determines that the detection result of the detection unit is not zero and is measured by the measurement unit even when the match determination unit determines that there is no match. A configuration (a third configuration) may be adopted in which the image is not transmitted if the determined passing time is less than the passing reference time.

The communication device having any one of the first to third configurations further includes an avoidance determination unit that determines whether or not the vehicle has traveled while avoiding the obstacle, wherein the second transmission unit includes: A configuration (fourth configuration) in which the image is not transmitted if the detection result of the detection unit is not zero and the avoidance determination unit determines non-avoidance even if the match determination unit determines non-match. There may be.

Further, in the communication device having any one of the first to fourth configurations, the receiving unit receives attribute information of an obstacle existing in a section in which the vehicle is traveling, and the coincidence determining unit detects the A configuration (fifth configuration) may be employed in which it is determined whether or not the part detection result matches the presence/absence information and the attribute information.

A communication system according to the present invention includes a communication device having any one of the first to fifth configurations, transmits the presence/absence information to the communication device, and receives the specific information and the image transmitted from the communication device. and a center server (sixth configuration).

A communication method according to the present invention includes an acquisition step of acquiring an image captured by a camera that captures the surroundings of a vehicle, and a first transmission step of transmitting identification information for identifying a section in which the vehicle is traveling. a receiving step of receiving presence/absence information indicating whether or not an obstacle exists in the section in which the vehicle is traveling; and detecting an obstacle present in the section in which the vehicle is traveling from the image. a detection step, a match determination step of determining whether or not the detection result of the detection step and the presence/absence information match, and a second step of transmitting the image when the match determination step determines that there is no match. and a transmission step (seventh configuration).

According to the present invention, image data useful for updating a database generated by a center server is uploaded, so that image data can be uploaded while saving the amount of communication and the band used for communication.

A diagram showing a configuration example of a communication system Diagram for explaining the outline of the drive recorder Diagram showing one configuration example of a drive recorder Flowchart showing one operation example of the center server and drive recorder Flowchart showing another operation example of the center server and drive recorder Flowchart showing yet another operation example of the center server and drive recorder

Exemplary embodiments of the invention are described in detail below with reference to the drawings.

<1. Configuration example of communication system>
FIG. 1 is a diagram showing a configuration example of a communication system. The communication system shown in FIG. 1 includes communication device 1 and center server 2 . Although only one communication device 1 is shown in FIG. 1 , the center server 2 communicates with a plurality of communication devices 1 .

<2. Configuration example of the center server>
As shown in FIG. 1 , the center server 2 includes a communication section 21, a control section 22, and a storage section 23.

The communication unit 21 performs data communication with a plurality of communication devices 1 via a network (not shown).

The storage unit 23 nonvolatilely stores various kinds of information. For example, a hard disk drive or the like can be used as the storage unit 23 . In this embodiment, the storage unit 23 stores a dynamic map database 23a and a program 23b.

The dynamic map database 23a accumulates data forming a dynamic map. A dynamic map is a digital map in which static information such as road surface information, lane information, and three-dimensional structures is combined with quasi-static information, quasi-dynamic information, dynamic information, and the like. Quasi-static information includes, for example, traffic regulation information, road construction information, wide-area weather information, and the like. Semi-dynamic information includes accident information, traffic congestion information, narrow-area weather information, and the like. The dynamic information includes surrounding vehicle information, pedestrian information, signal information, and the like.

The form of use of the dynamic map is not particularly limited, but it may be used for automatic driving of the vehicle 4 shown in FIG. 2, for example. Specifically, an automatic driving control ECU may be provided in the vehicle 4, and the automatic driving control ECU may select the driving lane and driving route based on the dynamic map.

The control unit 22 is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), which are not shown. The control unit 22 is connected to the storage unit 23 and the like, processes and transmits/receives information based on the program 23b stored in the storage unit 23, and controls the center server 2 as a whole.

<3. Configuration example of drive recorder>
For example, a drive recorder can be used as the communication device 1 . FIG. 2 is a diagram for explaining the outline of the drive recorder. A drive recorder 3 is mounted on a vehicle 4 . The drive recorder 3 is arranged at an appropriate location on the vehicle 4 .

The drive recorder 3 acquires an image around the vehicle captured by a camera 5 mounted on the vehicle 4 . The recording method of the drive recorder 3 may be, for example, a constant recording method in which information about the driving situation, including images captured by the camera 5, is periodically recorded on a recording medium (not shown). lane departure), sudden braking, collision, etc., the image captured by the camera 5 and information related to the driving situation other than the image including the event content are recorded on a recording medium. may

FIG. 3 is a diagram showing a configuration example of the drive recorder 3. As shown in FIG. The drive recorder 3 has a microcomputer as a control unit that controls the entire device. Specifically, the drive recorder 3 includes a CPU 31 that implements various control functions by performing arithmetic processing, a RAM 32 that serves as a work area for arithmetic processing, and a non-volatile memory 33 that stores various data. The nonvolatile memory 33 is composed of, for example, a hard disk or flash memory. The nonvolatile memory 33 stores programs as firmware, setting parameters, and the like. The function of controlling each part of the drive recorder 3 is realized by the CPU 31 executing arithmetic processing according to a program pre-stored in the non-volatile memory 33 .

A camera 5 and a microphone 6 are connected to the drive recorder 3 . Camera 5 and microphone 6 may be included in drive recorder 3 . The camera 5 has a lens and an imaging element, and can acquire image data electronically. In this embodiment, the camera 5 is arranged near the top of the windshield with its optical axis directed forward of the vehicle 4 . The number and arrangement of cameras 5 may be changed as appropriate. For example, a front camera that captures the front and a back camera that captures the rear may be provided. The microphone 6 is arranged, for example, in the main body of the drive recorder 3 or at an appropriate location on the vehicle 4 . A microphone 6 collects sounds in the vicinity of the vehicle to obtain audio data. The number and arrangement of the microphones 6 may be changed as appropriate. For example, a vehicle exterior sound microphone that mainly collects sounds generated outside the vehicle 4 and a vehicle interior sound microphone that mainly collects sounds generated by the driver may be provided.

The drive recorder 3 includes an image processing section 34 that processes image data captured by the camera 5 . In this embodiment, the image processor 34 is a hardware circuit. The image processing unit 34 performs predetermined image processing on the image data signal input from the camera 5 to generate digital image data in a predetermined format. Predetermined image processing includes, for example, A/D conversion, brightness correction, contrast correction, and the like. The predetermined format may be, for example, JPEG format. Image data processed by the image processing unit 34 is stored in the RAM 32 .

A part of the storage area of the RAM 32 is used as a ring buffer. Image data processed by the image processing unit 34 and audio data acquired by the microphone 6 are always stored in this ring buffer. In the ring buffer, when data is stored up to the last area, it returns to the first area and new data is stored. That is, in the ring buffer, the oldest data is sequentially overwritten with new data. Therefore, the RAM 32 is always in a state of storing image data and audio data for a certain period of time in the past.

The drive recorder 3 is connected to an operation unit 7 that receives instructions from the driver or the like. The operation unit 7 may be included in the drive recorder 3 . The operation unit 7 is arranged at an appropriate location of the vehicle 4, such as near a steering wheel (not shown), so that it can be easily operated by a driver or the like. The drive recorder 3 also includes a card slot 35 , a clock circuit 36 , an acceleration sensor 37 , a GPS (Global Positioning System) receiver 38 , and a communication section 39 .

The card slot 35 is configured so that the memory card 8 can be inserted/removed. Instead of the memory card 8, other recording media such as a hard disk drive, an optical disk, a magneto-optical disk, a flexible disk, etc. may be used.

The image data, audio data, etc. stored in the ring buffer of the RAM 32 are periodically recorded in the memory card 8 loaded in the card slot 35 according to instructions from the CPU 31 .

The clock circuit 36 generates a signal corresponding to the current time and outputs it to the CPU 31 .

The acceleration sensor 37 detects acceleration indicating the magnitude of the impact applied to the vehicle 4 in units of G of gravitational acceleration. The detected acceleration is an instantaneous value of acceleration at the current time, and has magnitudes in directions corresponding to, for example, three axes or two axes orthogonal to each other. The acceleration sensor 37 outputs a signal corresponding to such acceleration to the CPU 31 .

The GPS receiver 38 receives signals from a plurality of GPS satellites and acquires the vehicle position, which is the position of the vehicle 4 at the current time. The GPS receiver 38 acquires the position of the vehicle as position information represented by longitude and latitude on the earth, and outputs it to the CPU 31 . The GPS receiver 38 can detect accurate time based on signals from GPS satellites. For this purpose, a GPS receiver 38 may be used instead of the timer circuit 36 . Also, the GPS receiver 38 may be used to correct the time of the timer circuit 36 .

The communication unit 39 transmits and receives data to and from the communication unit 21 of the center server 2 via the network.

In addition, vehicle running information including data from a vehicle speed sensor is input to the CPU 31 via a CAN (Controller Area Network) bus (not shown). The vehicle travel information may include steering information, brake information, accelerator information, and the like. Furthermore, the CPU 31 may be connected to a navigation device and a vehicle control ECU (Electronic Control Unit) via a CAN (not shown).

<4. Operation example of center server and drive recorder>
FIG. 4 is a flow chart showing an operation example of the center server 2 and the drive recorder 3. As shown in FIG. FIG. 4 and the flowcharts of FIGS. 5 and 6, which will be described later, show operations related to one section in which the vehicle 4 is traveling. Although the length of each section in which the vehicle 4 is traveling is not particularly limited, it may be, for example, several tens of meters.

The CPU 31 of the drive recorder 3 uses the communication unit 39 to transmit, to the center server 2, specific information for specifying a section (driving section) in which the vehicle 4 is traveling (step S110). The identification information for identifying the section during travel may be identification information for the section during travel generated by the CPU 31 based on the vehicle position acquired by the GPS receiver 38, for example. It may be the own vehicle position itself acquired by 38 .

In the center server 2, the control unit 22 uses the communication unit 21 to receive the identification information for identifying the running section (step S210). Then, the control unit 22 of the center server 2 extracts from the dynamic map database 23a the obstacle presence/absence information corresponding to the running section identified by the received identification information (step S220). The types of obstacles are not particularly limited, but examples include scattered objects scattered on the road surface due to strong winds, fallen objects from vehicles, vehicles that have broken down and stopped, construction sites, fallen rocks, and the like.

In step S<b>230 following step S<b>220 , the control unit 22 of the center server 2 uses the communication unit 21 to transmit the extracted obstacle presence/absence information to the drive recorder 3 .

In the drive recorder 3, in step S120 following step S110, the CPU 31 detects obstacles present in the section during travel from the digital image data generated by the image processing section . Thereafter, CPU 31 receives obstacle presence/absence information using communication unit 39 (step S130).

In step S140 following step S130, the CPU 31 of the drive recorder 3 determines whether or not the detection result detected in step S120 matches the obstacle presence/absence information received in step S130. For example, when the detection result is zero (when no obstacle is detected), if the content of the obstacle presence/absence information is "obstacle present", it is determined that there is no match, and the obstacle presence/absence information is "no obstacle", it is determined as a match. Conversely, when the detection result is not zero (when at least one obstacle is detected), if the content of the obstacle presence/absence information is "obstacle present", it is determined to be a match, and the obstacle presence/absence information is "no obstacle", it is determined that they do not match.

If the detection result detected in step S120 matches the obstacle presence/absence information received in step S130 (YES in step S140), the drive recorder 3 converts the digital image data generated by the image processing unit 34 into The flow operation ends without transmitting to the center server 2. - 特許庁

On the other hand, if the detection result detected in step S120 does not match the obstacle presence/absence information received in step S130 (NO in step S140), the CPU 31 of the drive recorder 3 uses the communication unit 39 to transmit the image The digital image data generated by the processing unit 34 and the detection result detected in step S120 are transmitted to the center server 2 (step S150), and then the flow operation is terminated.

In the center server 2, the control unit 22 of the center server 2 determines whether or not the digital image data and the detection result have been received within a predetermined period of time after the processing of step S230 is completed (step S240). . If the digital image data and the detection result have not been received (NO in step S240), the center server 2 terminates the flow operation without updating the dynamic map database 23a.

On the other hand, if the digital image data and the detection result have been received (YES in step S240), the center server 2 updates the dynamic map database 23a (step S250), and then ends the flow operation.

According to the flow operation of the drive recorder 3 shown in FIG. 4, image data useful for updating the dynamic map database 23a generated by the center server 2 is uploaded, thus saving the amount of communication and the band used for communication. You can upload image data using Thereby, the communication cost required for uploading image data can be reduced. Further, it becomes easy to use the communication between the drive recorder 3 and the center server 2 for services other than updating the dynamic map (for example, automatic driving of the vehicle 4 based on the dynamic map described above).

Further, according to the flow operation of the drive recorder 3 shown in FIG. 4, when transmitting the digital image data to the center server 2, the detection result is also transmitted. This eliminates the need for the center server 2 to detect obstacles from the digital image data sent from the drive recorder 3 . That is, it is possible to eliminate the waste of duplicate detection processing being executed by both the drive recorder 3 and the center server 2 .

FIG. 5 is a flow chart showing another operation example of the center server 2 and the drive recorder 3. As shown in FIG. The flowchart shown in FIG. 5 is obtained by adding steps S131 and S141 on the drive recorder 3 side and adding steps S221 and S231 on the center server 2 side to the flowchart shown in FIG. Therefore, description of the same parts as in FIG. 4 will be omitted.

When executing the flow operation shown in FIG. 5, the CPU 31 of the drive recorder 3 uses the clock circuit 36 and the GPS receiver 38 to measure the time required for passing through each section during travel (passage time).

Further, when the flow operation shown in FIG. 5 is executed, the dynamic map database 23a contains information on the passing reference time set for each running section.

In step S221 following step S220, the control unit 22 of the center server 2 extracts from the dynamic map database 23a information on the passage reference time corresponding to the section during travel identified by the received specific information. Note that steps S220 and S221 may be integrated into one process.

In step S<b>231 following step S<b>230 , the control unit 22 of the center server 2 uses the communication unit 21 to transmit the extracted passage reference time information to the drive recorder 3 . Note that steps S230 and S231 may be integrated into one process.

Step S141 is provided between step S140 and step S150. In step S141, the CPU 31 of the drive recorder 3 determines whether the detection result is not zero and the passing time is less than the passing reference time.

If the detection result is not zero and the passing time is less than the passing reference time (YES in step S141), the drive recorder 3 does not transmit the digital image data generated by the image processing section 34 to the center server 2. End the flow operation.

On the other hand, if the detection result is not zero and the passage time is not less than the passage reference time (NO in step S141), the process proceeds to step S150.

As a result, if the passage time is less than the passage reference time, the image is not uploaded even if a new obstacle is detected. That is, uploading is performed with a focus on obstacles (obstacles in a narrow sense) that affect the passage time of the running section, so that the amount of communication and the band used for communication can be further reduced.

FIG. 6 is a flow chart showing still another operation example of the center server 2 and the drive recorder 3. As shown in FIG. The flowchart shown in FIG. 6 is obtained by adding step S142 on the drive recorder 3 side to the flowchart shown in FIG. Therefore, description of the same parts as in FIG. 4 will be omitted.

Step S142 is provided between step S140 and step S150. In step S142, the CPU 31 of the drive recorder 3 determines whether or not the detection result is not zero and the vehicle 4 has traveled without avoiding obstacles. Whether or not the vehicle 4 has traveled without avoiding obstacles can be determined, for example, by analyzing vehicle travel information and digital image data generated by the image processing unit 34 .

If the detection result is not zero and the vehicle 4 runs without avoiding the obstacle (YES in step S142), the drive recorder 3 sends the digital image data generated by the image processing section 34 to the center server 2. End the flow operation without sending.

On the other hand, if the detection result is not zero and the vehicle 4 has traveled without avoiding the obstacle (NO in step S142), the process proceeds to step S150.

As a result, when the vehicle 4 travels without avoiding obstacles, image uploading is not performed even if an obstacle is newly detected. In other words, since the upload is limited to obstacles (obstacles in a narrow sense) that need to be avoided, the amount of communication and the band used for communication can be further reduced. Obstacles that do not require avoidance running include, for example, obstacles that are sufficiently lower than the minimum ground clearance of the vehicle 4 .

<5. Variation>
The above embodiments should be considered illustrative in all respects and not restrictive, and the technical scope of the present invention is indicated by the scope of claims rather than the description of the above embodiments. It should be understood that all modifications that fall within the meaning and range of equivalents of the claims are included.

For example, in the above-described embodiment, the drive recorder 3, which is an example of the communication device 1, has been described. good.

Further, for example, the flow operation shown in FIG. 5 and the flow operation shown in FIG. 6 may be combined and executed.

Further, for example, in the above-described embodiment, it is determined whether or not the detection result and the obstacle presence/absence information match. may be determined. In this case, the dynamic map database 23a includes obstacle attribute information. Then, the attribute information of the obstacle is transmitted from the center server 2 to the communication device 1 such as the drive recorder 3 together with the obstacle presence/absence information. When detecting an obstacle, the communication device 1 such as the drive recorder 3 also detects the attributes of the obstacle. Although the attributes of the obstacle are not particularly limited, for example, the type of obstacle, the size of the obstacle, the position of the obstacle, and the like can be considered. By taking into account the attribute information of obstacles, images can be uploaded even when, for example, one obstacle exists in one running section to two obstacles. becomes possible. This improves the obstacle accuracy of the dynamic map database.

1 Communication Device 2 Center Server 3 Drive Recorder 4 Vehicle 5 Camera

Claims (5)

an acquisition unit that acquires an image captured by a camera that captures the surroundings of the vehicle;
a first transmission unit that transmits identification information for identifying a section in which the vehicle is traveling;
a receiving unit that receives presence/absence information indicating whether or not there is an obstacle on the road surface in the section in which the vehicle is traveling;
a detection unit that detects an obstacle on the road surface present in the section where the vehicle is traveling from the image;
a match determination unit that determines whether the detection result of the detection unit and the presence/absence information match;
a second transmission unit that transmits the image when the match determination unit determines that the image does not match;
a measuring unit that measures the passage time of the section in which the vehicle is traveling;
with
The receiving unit receives information on a passage reference time set for a section in which the vehicle is traveling,
The second transmission unit detects at least one obstacle on the road surface by the detection unit and the passage time measured by the measurement unit even when the match determination unit determines that the match determination unit does not match. A communication device that does not transmit the image if it is less than the passage reference time. an acquisition unit that acquires an image captured by a camera that captures the surroundings of the vehicle;
a first transmission unit that transmits identification information for identifying a section in which the vehicle is traveling;
a receiving unit that receives presence/absence information indicating whether or not there is an obstacle on the road surface in the section in which the vehicle is traveling;
a detection unit that detects an obstacle on the road surface present in the section where the vehicle is traveling from the image;
a match determination unit that determines whether the detection result of the detection unit and the presence/absence information match;
a second transmission unit that transmits the image when the match determination unit determines that the image does not match;
with
The receiving unit receives attribute information of obstacles on the road surface present in the section on which the vehicle is traveling,
The communication device, wherein the match determination unit determines whether or not the detection result of the detection unit matches the presence/absence information and the attribute information. A communication device according to claim 1 or claim 2 ,
a center server that transmits the presence/absence information to the communication device and receives the specific information and the image transmitted from the communication device;
communication system. an acquisition step of acquiring an image captured by a camera capturing the surroundings of the vehicle;
a first transmission step of transmitting identification information for identifying a section in which the vehicle is traveling;
a receiving step of receiving presence/absence information indicating whether or not there is an obstacle on the road surface in the section in which the vehicle is traveling;
a detection step of detecting an obstacle on the road surface present in the section on which the vehicle is traveling from the image;
a match determination step of determining whether or not the detection result of the detection step and the presence/absence information match;
a second transmission step of transmitting the image when the match determination step determines that the images do not match;
a measuring step of measuring the passage time of the section in which the vehicle is traveling;
with
In the receiving step, information on a passage reference time set for a section in which the vehicle is traveling is received;
In the second transmission step, at least one obstacle on the road surface is detected in the detection step and the passage measured in the measurement step is detected in the detection step even if the match determination step determines that there is no match. A method of communication, wherein the image is not transmitted if the time is less than the passage reference time. an acquisition step of acquiring an image captured by a camera capturing the surroundings of the vehicle;
a first transmission step of transmitting identification information for identifying a section in which the vehicle is traveling;
a receiving step of receiving presence/absence information indicating whether or not there is an obstacle on the road surface in the section in which the vehicle is traveling;
a detection step of detecting an obstacle on the road surface present in the section on which the vehicle is traveling from the image;
a match determination step of determining whether or not the detection result of the detection step and the presence/absence information match;
a second transmission step of transmitting the image when the match determination step determines that the images do not match;
with
In the receiving step, attribute information of obstacles on the road surface present in the section on which the vehicle is traveling is received;
The communication method, wherein, in the match determination step, it is determined whether or not the detection result of the detection step matches the presence/absence information and the attribute information.

Images