CN110366107A - Vehicle communication method and device using same - Google Patents

Abstract

The present invention provides a vehicle communication method and a device using the method. The method includes the following steps: S1: Collect live information in a predetermined area of a vehicle in real time, wherein the live information includes vehicles in the predetermined area and the image data of the surrounding environment of the vehicle and the voice data broadcast by the driver; S2: data processing the live information to obtain traffic anomaly data, wherein the data processing includes using a pre-trained artificial intelligence model to collect images Perform anomaly detection on the data and perform text conversion and word slot filling on the collected voice data; S3: wirelessly transmit the traffic anomaly data to the cloud server, and broadcast to the target vehicles in the predetermined area through the cloud server. The present invention acquires traffic abnormality data through automatic vehicle detection and voice active reporting, and sends the traffic abnormality data to target vehicles through a cloud server in a targeted manner, thereby realizing real-time and accurate sharing of information.

Description

Vehicle communication method and device using same

technical field

The present invention relates to the technical field of information communication between vehicles, and more specifically, relates to a vehicle communication method and a device using the method.

Background technique

With the rapid economic development in the new era and the rapid concentration of urban population, the number of household cars in cities is showing a trend of rapid increase. Due to the continuous increase of road vehicles, the problem of urban road congestion and automobile safety issues has gradually become prominent, and traffic safety issues are closely related to the broad masses of the people. At present, the transmission of vehicle information mainly relies on the way of honking the horn and interacting with lights, as shown in Figure 1, and there are only a limited number of transformation forms of vehicle lights, and it is difficult to cover many situations encountered by vehicles or car owners. This kind of traffic situation makes it difficult for surrounding vehicles to obtain the most authentic and effective traffic information. In addition, the traffic information that can be expressed by honking is limited, and many road sections in the city also prohibit cars from honking, which further limits the transmission between vehicles. way of information.

There are different types of voice communication methods established by mobile terminals in the prior art, voice communication methods using car WIFI, methods based on vehicles sharing their own information, and image processing technology to detect the tire information of the vehicle in front. And the way of wireless communication, etc., but part of the way of information transmission requires the driver to actively send or read information, and the vehicle cannot automatically obtain abnormal conditions, the way is relatively simple, and the other part of the way is only to share the information of the vehicle itself, and notify The target is all surrounding vehicles, the scope of shared information is not accurate, and it is impossible to send notifications to the target vehicles that will be affected.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a vehicle communication method and a device using the method. The traffic abnormal data is obtained through the automatic detection of the vehicle camera and the active reporting of the driver's voice, and the traffic abnormal data is targeted through the cloud server. broadcast to the target vehicle.

According to one aspect of the present invention, a vehicle communication method is provided, the method includes the following steps: S1: Collect real-time information in a predetermined area of the vehicle in real time, wherein the live information includes vehicles and vehicles in the predetermined area The image data of the surrounding environment of the vehicle and the voice data broadcast by the driver; S2: data processing the live information to obtain traffic anomaly data, wherein the data processing includes using a pre-trained artificial intelligence model to collect image data Perform anomaly detection and perform text conversion and word slot filling on the collected speech data; S3: wirelessly transmit the traffic anomaly data to a cloud server, and broadcast to target vehicles in a predetermined area through the cloud server.

Preferably, the pre-trained artificial intelligence model includes a convolutional neural network model, and the step S2 includes: classifying the collected image data according to the vehicle abnormality type according to the convolutional neural network model; textualizing the collected voice data Convert to obtain the corresponding text information, and match the text information with the predefined instruction template, and fill the necessary word slots for the successfully matched text information.

Preferably, the abnormality type of the vehicle includes at least one of brake failure of the vehicle, loss of control of the steering of the vehicle, abnormal tire pressure of the vehicle in front, shedding of goods in the vehicle in front, obstacles in front, door opening of the vehicle in front, fire at the rear of the vehicle in front, and an accident in front .

Preferably, the instruction template is a collection of abnormal traffic data reported by the driver's voice that needs to be processed, and the necessary word slot includes any one of address word slot, time word slot and vehicle word slot, wherein the address word slot Including road and latitude and longitude, vehicle word slot includes vehicle color and license plate number.

Preferably, the step S2 further includes: extracting the license plate image data in the image data, and obtaining the license plate number through license plate positioning, character segmentation and character recognition, wherein the license plate number is the vehicle information in the cloud server Stored unique key value.

Preferably, the step S3 includes: performing standard formatting processing on the abnormal traffic data to construct event information; wirelessly transmitting the constructed event information to a cloud server, and the cloud server analyzes the constructed event information to obtain the target vehicle information, and send traffic abnormality data to the target vehicle; the target vehicle converts the received traffic abnormality data into voice data and transmits the information to the driver in the form of voice broadcast.

Preferably, the step S3 further includes: analyzing the constructed event information through the cloud server and sending the abnormal traffic data to the intelligent terminal device bound to the target vehicle.

Preferably, the method further includes: sending the location information of the vehicles in the area to the cloud server at a fixed frequency.

According to another aspect of the present invention, there is provided a device using the above-mentioned vehicle communication method, the device comprising: a data collection module configured to collect live information in a predetermined area of the vehicle in real time, wherein the live information includes The image data of the vehicle and the surrounding environment of the vehicle in the predetermined area and the voice data broadcast by the driver; the data processing module is configured to perform data processing on the live information to obtain traffic abnormal data, wherein the data processing Including using a pre-trained artificial intelligence model to perform anomaly detection on the collected image data and performing text conversion and word slot filling on the collected voice data; the wireless communication module is configured to wirelessly transmit the traffic anomaly data to a cloud server, and Broadcast to target vehicles in the predetermined area through the cloud server.

Preferably, the pre-trained artificial intelligence model includes a convolutional neural network model, and the data processing module is configured as: an image processing unit, which classifies the collected image data according to the vehicle abnormality type; The voice processing unit performs text conversion on the collected voice data to obtain corresponding text information, matches the text information with a predefined instruction template, and fills necessary word slots for the successfully matched text information.

Preferably, the abnormality type of the vehicle includes at least one of brake failure of the vehicle, loss of control of the steering of the vehicle, abnormal tire pressure of the vehicle in front, shedding of goods in the vehicle in front, obstacles in front, door opening of the vehicle in front, fire at the rear of the vehicle in front, and an accident in front .

Preferably, the instruction template is a collection of abnormal traffic data reported by the driver's voice that needs to be processed, and the necessary word slot includes any one of address word slot, time word slot and vehicle word slot, wherein the address word slot Including road and latitude and longitude, vehicle word slot includes vehicle color and license plate number.

Preferably, the image processing unit further includes: extracting the license plate image data in the image data, and obtaining the license plate number through license plate positioning, character segmentation and character recognition, wherein the license plate number is the vehicle information in the cloud server Unique key value for storage.

Preferably, the wireless communication module is configured to: perform standard format processing on the abnormal traffic data to construct event information; wirelessly transmit the constructed event information to a cloud server, and the cloud server analyzes the constructed event information to obtain The information of the target vehicle is obtained, and the abnormal traffic data is sent to the target vehicle; the target vehicle converts the received abnormal traffic data into voice data and transmits the information to the driver in the form of voice broadcast.

Preferably, the wireless communication module is further configured to: analyze the constructed event information through the cloud server and send traffic anomaly data to the intelligent terminal device bound to the target vehicle.

Preferably, the device further includes: sending the location information of the vehicles in the area to the cloud server at a fixed frequency.

Description of drawings

The detailed description of the present invention will be carried out below in conjunction with accompanying drawing, and above-mentioned feature of the present invention and other object, characteristic and advantage will become clearer, wherein:

Fig. 1 shows a schematic diagram of transmitting information through light interaction and whistle in the prior art;

FIG. 2 shows a flowchart of a vehicle communication method according to an embodiment of the present invention;

3 shows a schematic diagram of anomaly classification of a convolutional neural network according to an exemplary embodiment of the present invention;

Fig. 4 shows the flowchart of the license plate recognition process according to an exemplary embodiment of the present invention;

Fig. 5 shows the flowchart of the voice reporting process according to an exemplary embodiment of the present invention;

Fig. 6 shows a schematic diagram of automatic vehicle detection abnormality processing according to an exemplary embodiment of the present invention;

Fig. 7 shows a schematic diagram of a driver actively reporting an abnormality process according to an exemplary embodiment of the present invention;

FIG. 8 shows a block diagram of a vehicle communication device according to an embodiment of the present invention;

Fig. 9 shows a block diagram of a data processing module according to an exemplary embodiment of the present invention.

Throughout the drawings, like reference numerals will be understood to refer to like elements, features and/or structures.

Detailed ways

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Wherein, the same reference numerals always represent the same components.

FIG. 2 is a flowchart illustrating a method of vehicle communication according to an embodiment of the present invention.

As shown in FIG. 2 , in step S1 , live information in a predetermined area of the vehicle is collected in real time. Wherein, the real-time information includes the image data of the vehicle and the surrounding environment of the vehicle in the predetermined area, and the voice data broadcast by the driver. Specifically, for example, the image data of the vehicle itself and the image data of the vehicle's surrounding environment in a predetermined area can be collected in real time through the camera installed in the vehicle and the driving computer, and the driver's broadcast information can be obtained through the microphone device installed in the vehicle. voice data.

Next, in step S2, data processing is performed on the live information to obtain traffic anomaly data. According to an embodiment of the present invention, specifically, a pre-trained artificial intelligence model may be used to perform anomaly detection on the collected image data and to perform text conversion and word slot filling on the collected voice data. Among them, the artificial intelligence model includes a pre-trained convolutional neural network model, and the convolutional neural network model is used to identify and process the collected image data to realize the classification of vehicle abnormal types, thereby completing the abnormal detection of image data. The text conversion and word slot filling of the collected voice data refers to converting the collected voice data into corresponding text information, and matching the converted text information with the predefined instruction template, and then, matching the successfully matched text information to fill in the necessary word slots. It should be understood that the convolutional neural network model is just an example, and other image processing and voice recognition schemes can also be used to perform vehicle abnormal type classification and voice conversion according to embodiments of the present invention. Here, the abnormal type of the vehicle includes at least one of brake failure of the vehicle, out-of-control steering of the vehicle, abnormal tire pressure of the vehicle in front, shedding of goods in the vehicle in front, obstacles in front, door opening of the vehicle in front, fire at the rear of the vehicle in front, and an accident in front. For example, according to an embodiment of the present invention, it is assumed that there is an abnormal situation of the vehicle in front in the image data of the vehicle in front collected by the camera installed in the vehicle in real time, such as the trunk of the vehicle in front is opened, the cargo in the vehicle in front falls off, the rear of the vehicle in front catches fire, or the door of the vehicle in front If it is turned on, etc., the specific abnormal type of the vehicle in front can be obtained by performing data processing on the image data of the vehicle in front collected in real time. It should be understood that the above examples of vehicle abnormality types are only exemplary examples, and the vehicle abnormality types applicable in the present invention are not limited thereto. In addition, the instruction template is a pre-defined set of rule templates including necessary word slots. The construction of the instruction template is based on the data set of message instructions reported by voice in traffic, including the traffic anomaly data reported by the driver’s voice that needs to be processed. For example, the instruction template for a fire at the rear of the preceding vehicle may consist of the license plate number of the preceding vehicle, the time of occurrence, and the address of the occurrence (or specific to the latitude and longitude). Necessary word slots refer to keywords that are indispensable for describing events, including any one of address word slots, time word slots, and vehicle word slots. For example, address word slots include specific roads, directions, latitude and longitude, etc., and vehicle word slots Slots include things like vehicle color and license plate number. It should be understood that the above examples of instruction templates and necessary word slots are only exemplary examples, and the instruction templates and necessary word slots that can be used in the present invention are not limited thereto.

According to an embodiment of the present invention, performing anomaly detection on the collected image data is performing vehicle anomaly classification on the collected image data according to a pre-trained convolutional neural network model. Here, the convolutional neural network model is a model used in an artificial intelligence algorithm, and the pre-trained convolutional neural network model is obtained by training on pre-set training data. The steps of training the convolutional neural network model are as follows: obtain abnormal event images and normal traffic images encountered on the road, mark each acquired image with abnormal type and normal situation, and use the marked image data as a convolutional neural network The training set of the network. The workflow of the convolutional neural network is: first perform a convolution operation on the input image data, then use the ReLU activation function to extract low-level features, and then reduce the dimensionality of the features by pooling. Usually, the convolutional neural network model repeats the above algorithm process to make the feature dimension smaller and closer to the high-order features, and finally obtains the classification results of the pictures through the fully connected layer. The principle of abnormal classification is as follows: shown in Figure 3. Specifically, the original collected sample data (image data of abnormal events in the road and image data of normal traffic acquired in real time) are randomly divided into two parts, one part is the training set and the other part is the test set. The test set is substituted into the pre-trained convolutional neural network model for training, and the output of the convolutional neural network model is compared with the actual marked result, and the parameter values in the convolutional neural network model are updated through the backpropagation algorithm. Then, the data is subjected to multiple iterative training, for example, 500 iterative training is performed on the data, and after every 10 iterations of training, the test data is substituted to check the accuracy, accuracy, recall rate or F value of the network, etc. The model evaluation standard, and when the convolutional neural network performs well on both the training set and the test set, and the calculated evaluation standard value is better, it indicates that the convolutional neural network model has been trained well. Finally, the image data collected in real time is input into the convolutional neural network model for anomaly detection to obtain whether the vehicle is abnormal and the type of vehicle anomaly.

According to an embodiment of the present invention, when processing an abnormal event image through a convolutional neural network, first confirm the type of picture that needs to be identified, that is, the type of identifiable abnormal event captured by the camera, for example, traffic accidents, road obstacles, Abnormal event categories such as temporary construction of the road surface, fire of the vehicle in front, abnormal tire pressure of the vehicle in front, open door of the vehicle in front, and falling of things in the vehicle in front. Then, perform image processing on the collected pictures of various abnormal events, for example, select 1,000 pictures of the same type of abnormal traffic event and scale them to the same size, and then perform grayscale changes, horizontal flips, vertical flips, and rotations on the original pictures and color value jumping to achieve the effect of data enhancement. Finally, mark the corresponding pictures with event types, and continue to mark the relevant information for each event type, for example, continue to mark the information of the road obstacle event type as "in which lane", etc., so that the surrounding vehicles can move around When receiving the time message, make corresponding preparations in advance or in time to reduce the probability of accidents. It should be understood that the above-mentioned examples of abnormal events and information annotations are only exemplary examples, and the types of abnormal events and the content of information annotations that can be used in the present invention are not limited thereto.

According to an embodiment of the present invention, the process of data processing the collected image data also includes extracting the license plate image data in the image data, and completing the automatic license plate number recognition function through license plate positioning, character segmentation and character recognition. Specifically, the license plate location process is first performed on the static image in the collected image data to locate the image at the license plate position, and then the character segmentation process is performed on the image data at the license plate position to segment the characters including letters, Chinese characters, and numbers. License plate information, for example, character segmentation results in "Chuan, 2, 5, A" and so on. Then, identify and confirm the segmented license plate information to obtain a detailed license plate number, for example, obtain the license plate number as "Chuan A555A2". Finally, the resulting detailed license plate number is output. Specifically as shown in Figure 4. It should be understood that the above examples for character segmentation and license plate numbers are only exemplary examples, and the character segmentation and license plate numbers that can be used in the present invention are not limited thereto.

According to the embodiment of the present invention, performing text conversion and word slot filling on the collected speech data refers to converting the collected speech data into corresponding text information, and obtaining clear traffic anomalies through instruction template matching and word slot filling processing The data are shown in Figure 5 in detail. Here, parsing the text is to convert the recognized voice data broadcast by the driver into corresponding text information to obtain the intention of the driver to broadcast the voice, and the matching template is to perform instruction template matching on the converted text information to filter out the needs The text information of the reported traffic anomaly data, word slot filling is to fill the text information with necessary word slots and/or related word slots to make the traffic anomaly data reported by the driver more accurate and detailed. According to the embodiment of the present invention, assuming that the driver says "there was a car accident at Xuanwumen", the text conversion of the speech data can be used to obtain the driver's intention, that is, the traffic failure that the driver wants to report is "car accident". Then, according to the obtained text information "A car accident occurred at Xuanwumen", find out the matching car accident template in the instruction template, and determine that the voice data needs to be reported. Finally, according to the text information "A car accident occurred at Xuanwumen", it can be seen that except for the address word slot "Xuanwumen", other necessary word slots are missing. At this time, the necessary word slots need to be filled. When filling the necessary word slots, the voice reporting system installed in the vehicle will ask the driver to obtain the missing necessary word slots. For example, the voice reporting system asks the driver "Please confirm the specific road and vehicle information where the accident occurred". When the driver answers "the right two lanes of the central road, a red off-road vehicle collided with a white car", a clear traffic anomaly can be obtained. data. For another example, if the driver says "open the trunk of the car in front", the voice data lacks necessary word slots after being converted into text information. The reporting system will ask the driver "Please confirm the license plate information of the vehicle ahead", and when the user answers "Su A22222", clear traffic abnormal data can be obtained. It should be understood that the above examples of the voice data broadcast by the driver are only exemplary examples, and the voice data that can be used in the present invention is not limited thereto.

In step S3, the abnormal traffic data is wirelessly transmitted to the cloud server, and broadcast to the target vehicles in the predetermined area through the cloud server. Specifically, standard format processing is performed on the abnormal traffic data to construct event information, and the constructed event information is wirelessly transmitted to the cloud server. Then, analyze the constructed event information through the cloud server to obtain the information of the target vehicle, and send the traffic anomaly data to the target vehicle. Finally, the target vehicle converts the received traffic anomaly data into voice data and broadcasts the traffic anomaly information to the driver. According to the embodiment of the present invention, the acquired traffic anomaly data is processed in a standard format to construct event information, including collected image data, detected abnormal image data and voice data reported by the driver, all of which will be extracted into text, and passed After standard formatting and processing, it is sent to the cloud server. For example, sending the acquired traffic anomaly data to the cloud server in the form of json formatting, compared with sending voice data or image data directly, the amount of transmitted data is greatly reduced, effectively improving the real-time performance of data transmission. It should be understood that the above examples of the standard format are only exemplary examples, and the standard format that can be used in the present invention is not limited thereto. The representation format of events based on intent and word slots is defined by constructing event information. For example, the common format of abnormal traffic events ahead is shown in Table 1. Five word slots are defined in Table 1, namely "report license plate", "License plate of the vehicle in front", "time", "latitude and longitude" and "lane", among them, in Table 1, the value of the word slot corresponding to the above word slot obtained by constructing the event information of the traffic anomaly data as "the vehicle's brake failure" They are "Su A12345", "Su A12223", "1511598661956", "(118.76, 32.04)" and "Right 1 Lane". It should be understood that the above examples of event formats and word slots are only exemplary examples, and the event formats and word slots that can be used in the present invention are not limited thereto.

Table 1

According to the embodiment of the present invention, when the constructed event information is wirelessly transmitted to the cloud server, the constructed event information can be successfully sent to the cloud server in an area with network coverage by using 4G or 5G communication technology. At present, most of the vehicles on the market have the function of connecting to the Internet, and with the cloud server, the vehicle can achieve the effect of communication and interconnection without adding other peripheral hardware devices, and relying on the strong transmission rate of 4G or 5G, the event information constructed It can quickly transmit between the vehicle terminal and the cloud server to realize real-time sending and receiving of information. Here, the cloud server provides connection services for each vehicle, and the cloud server binds and stores the license plate information of each vehicle and the communication address of the terminal. Each vehicle is registered in the on the cloud server. After the cloud server receives the constructed event information, it analyzes the constructed event information to obtain the target vehicle that will be affected by the traffic anomaly, and sends a traffic anomaly notification to the target vehicle through wireless transmission technology, and the target vehicle will receive the traffic anomaly notification. The abnormal traffic data is converted into voice data and the information is delivered to the driver in the form of voice broadcast. In addition, users can also register their portable smart terminal devices on the cloud server with license plate information, such as wearable devices, mobile phones, ipads, etc. When the cloud server notifies the target vehicle of the acquired event information, if the target vehicle's license plate If the information is bound to the smart terminal device, the notification message will be pushed to the smart terminal device registered by the vehicle and the driver at the same time. According to an embodiment of the present invention, if a vehicle parked on the side of the road is detected to be abnormal by a passing vehicle, for example, the door is not closed or the trunk is opened, the driver of the parked vehicle can receive a notification via a smart watch or a mobile phone. Notification of vehicle abnormalities reported by other vehicles to the cloud server. It should be understood that the above examples of portable smart terminal devices are only exemplary examples, and the smart terminal devices that can be used in the present invention are not limited thereto.

According to an embodiment of the present invention, assuming that the vehicle-mounted camera automatically detects a traffic anomaly, for example, the vehicle-mounted camera takes a picture of the vehicle in front every 0.5 seconds, and the captured picture is scaled to a fixed size and proportional size, through data enhancement Processing and convolutional neural network model processing enables anomaly detection of captured pictures. As shown in Figure 6, abnormal detection is performed on the captured pictures to classify the abnormal information of the vehicle in front, and the result of the abnormal type classification is that the cargo in the front vehicle has fallen, and the lane is the right lane. Then, the license plate information is extracted from the images collected by the on-board camera, and the license plate information of the dropped goods is obtained through license plate location, character segmentation and character recognition. Here, two event information can be constructed. The first event information is to accurately notify the truck in front that a cargo has fallen from the vehicle through the license plate information, and the second event information is to notify the surrounding vehicles of the front truck that there is an abnormal situation on the road. Please Drivers of surrounding vehicles should be vigilant and drive cautiously. The second event information needs to ensure that the surrounding vehicles and the shooting vehicle are registered on the cloud server. After receiving the two event information constructed above and analyzing and processing the constructed event information, the cloud server sends the first event information to the truck in front, and sends the second event information to other trucks on the same journey through location information. vehicle. The truck in front and other vehicles on the same journey convert the received event information data into voice messages, and remind the driver to drive carefully through voice broadcasts, thereby effectively reducing the probability of accidents.

According to the embodiment of the present invention, when the driver is supposed to actively report the abnormal traffic data, for example, when the driver finds that the brake system of his vehicle cannot be braked, he will actively send out the voice message "brake failure, pay attention to avoiding" The text conversion and word slot filling of the voice data emitted by the driver can conclude that the driver's intention is to send an avoidance warning. As shown in Figure 7, the driver of the white car actively sent out a voice message saying "Brakes are out of order, pay attention to avoiding", and the on-board camera of the white car automatically collects the image data of the surrounding environment of the vehicle, and identifies the vehicle in front according to the acquired image data. The license plate information is Su A12345. Construct the event information of the voice reporting event by filling in the necessary word slots, assuming that the constructed event information is: {Intention: avoidance warning, location: Longpan Road (118.76, 32.04), the license plate of the vehicle in front: Su A12345, the reported license plate: Su A67890}, then send the above-mentioned event information constructed to the cloud server through wireless transmission, and the cloud server analyzes the event information, and finds that the target vehicle that needs to be notified is Su A12345, and the cloud server will broadcast an abnormal prompt to the target vehicle SU A12345. Here, the vehicles behind and next to the reported vehicle are not affected, therefore, they will not be notified of the abnormal prompt. If the intent of the constructed event information is that the construction ahead is impassable, the cloud server will broadcast an exception notification to all vehicles in the predetermined area that need to be notified. Here, both the white car and the black car are registered on the cloud server with the license plate number as information. The cloud server sends the event information to the target vehicle Su A12345, and the target vehicle converts the received traffic anomaly data into a voice message, and reminds the driver to pay attention to avoiding through voice broadcast. If the driver of the black car also binds and registers the wearable device on the cloud server with the license plate information at the same time, the wearable device will also receive a reminder or notification of avoidance information.

FIG. 8 is a block diagram showing a vehicle communication device according to an embodiment of the present invention.

As shown in FIG. 8 , a vehicle communication device 800 includes a data collection module 801 , a data processing module 802 and a wireless communication module 803 . Wherein, the data collection module 801 is configured to collect live information in a predetermined area of the vehicle in real time. The data processing module 802 is configured to perform data processing on the live information to obtain traffic anomaly data. The wireless communication module 803 is configured to wirelessly transmit the abnormal traffic data to the cloud server, and broadcast to target vehicles in a predetermined area through the cloud server. Among them, the real-time information includes the image data of the vehicle and the surrounding environment of the vehicle in the predetermined area and the voice data broadcast by the driver. The data undergoes text transformation and word slot filling. The vehicle communication device according to the embodiment of the present invention can be implemented by a dedicated hardware device, and can also be implemented in the form of software on a terminal device such as a smart phone. For example, a vehicle communication device according to an embodiment of the present invention may be implemented as a driving recorder. When the vehicle communication device according to the embodiment of the present invention is implemented on a smart phone, the terminal device can be controlled in a specific application to realize the functions of various modules of the vehicle communication device.

The data collection module 801 collects live information in a predetermined area of the vehicle in real time. According to the embodiment of the present invention, specifically, the data collection module 801 collects the image data of the vehicle itself and the image data of the surrounding environment of the vehicle in a predetermined area in real time through the camera installed in the vehicle and the driving computer, and the image data of the surrounding environment of the vehicle through the camera installed in the vehicle. The microphone device acquires the voice data broadcast by the driver.

The data processing module 802 according to the embodiment of the present invention will be described in detail below with reference to FIG. 9 .

As shown in FIG. 9 , the data processing module 802 includes an image processing unit 901 and a speech processing unit 902 , wherein the image processing unit 901 classifies the collected image data according to the type of vehicle abnormality according to the pre-trained convolutional neural network model. The voice processing unit 902 performs text conversion and word slot filling on the collected voice data of the driver. Specifically, the image processing unit 901 uses a convolutional neural network model to perform anomaly detection on the collected image data to realize the classification of vehicle anomalies, wherein the vehicle anomalies include failure of the brakes of the vehicle, loss of control of the steering of the vehicle, and abnormal tire pressure of the front vehicle. , at least one of the cargo falling off from the vehicle in front, an obstacle in front, the door of the vehicle in front opening, a fire at the rear of the vehicle in front, and an accident in front. It should be understood that the above examples of vehicle abnormality types are only exemplary examples, and the vehicle abnormality types applicable in the present invention are not limited thereto. The voice processing unit 902 performs text conversion on the collected voice data to obtain corresponding text information, then matches the text information with a predefined instruction template, and fills word slots for the successfully matched text information. Here, the instruction template is a collection of traffic failure data reported by the driver's voice that needs to be processed. Word slot filling is to fill the necessary word slots for the successfully matched text information. The necessary word slots include any one of address word slots, time word slots, and vehicle word slots. For example, address word slots include roads and latitude and longitude, and vehicle Word slots include vehicle color and license plate number information, etc. It should be understood that the above examples of instruction templates and necessary word slots are only exemplary examples, and the instruction templates and necessary word slots that can be used in the present invention are not limited thereto.

According to the embodiment of the present invention, the image processing unit 901 also extracts the license plate information in the collected image data through license plate location, character segmentation and character recognition. Specifically, the image processing unit 901 first performs license plate location processing on the static image in the collected image data to locate the image data at the position of the license plate, and then performs character segmentation processing on the image data at the position of the license plate to segment out characters including letters and Chinese characters. , license plate number information including numbers. Finally, the image processing unit 901 recognizes and confirms the segmented license plate number information to obtain a complete license plate number.

Returning to FIG. 8, the wireless communication module 803 performs standard formatting processing on the abnormal traffic data to construct event information, and then wirelessly transmits the constructed event information to the cloud server, and the cloud server analyzes the constructed event information to obtain target vehicle information , and send traffic anomaly data to the target vehicle. The target vehicle receives the abnormal traffic data and converts the received information into voice data, and then transmits the information to the driver in the form of voice broadcast. According to the embodiment of the present invention, if the user registers the smart terminal device carried with him on the cloud server with the license plate information, such as wearable device, mobile phone, ipad, etc., the cloud server analyzes the event information constructed and simultaneously sends a The smart terminal device bound to the target vehicle sends traffic anomaly data, that is, when the cloud server notifies the target vehicle of the acquired event information, if the license plate information of the target vehicle has been bound to the smart terminal device, the notification message will be simultaneously pushed to the vehicle and on the smart terminal device registered by the driver. It should be understood that the above examples of portable smart terminal devices are only exemplary examples, and the smart terminal devices that can be used in the present invention are not limited thereto.

According to the embodiment of the present invention, the wireless communication module 803 also sends the location information of the vehicles in the area to the cloud server at a fixed frequency, so that the server can obtain the location information of the vehicles in real time. The wireless communication module 803 can use 4G or 5G communication technology, so that the event information constructed can be successfully sent to the cloud server in the area covered by the network, and relying on the strong transmission rate of 4G or 5G, the event information constructed can be quickly sent to the vehicle terminal and the cloud server to achieve the effect of sending and receiving information in real time.

According to the vehicle communication method and the device using the method according to the embodiment of the present invention, real-time acquisition of own vehicle information and surrounding vehicle information is obtained through the camera, driving computer and driver's voice broadcast, and the acquired abnormal traffic data and The traffic abnormal data reported by the driver's active voice is wirelessly transmitted to the cloud server, and the traffic abnormal data is broadcast to the target vehicles in the predetermined area through the cloud server, so as to realize the information sharing of traffic abnormal data. This invention not only liberates the driver's hands, eliminates the problem of driving due to operation, but also makes the information sharing more accurate and real-time by the cloud server's targeted broadcast notification to the target vehicle, avoiding potential traffic hazards, It reduces the probability of traffic accidents and contributes to safe driving.

While the invention has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that changes may be made without departing from the spirit and scope of the invention as defined by the claims and their equivalents. Various changes in form and detail.

Claims (16)

1. A vehicle communication method, characterized in that the method comprises the following steps: S1: Collect real-time live information in a predetermined area of the vehicle, wherein the live information includes image data of the vehicle and the surrounding environment of the vehicle in the predetermined area and voice data broadcast by the driver; S2: Perform data processing on the live information to obtain traffic anomaly data, wherein the data processing includes using a pre-trained artificial intelligence model to perform abnormal detection on the collected image data and perform text conversion and word slots on the collected voice data filling; S3: Wirelessly transmit the abnormal traffic data to a cloud server, and broadcast to target vehicles in a predetermined area through the cloud server. 2. The method according to claim 1, wherein the pre-trained artificial intelligence model comprises a convolutional neural network model, and the step S2 comprises: Carrying out vehicle anomaly type classification to the collected image data according to the convolutional neural network model; Perform text conversion on the collected speech data to obtain corresponding text information, and match the text information with a predefined instruction template, and fill necessary word slots for the successfully matched text information. 3. The method according to claim 2, wherein the vehicle abnormality types include brake failure of the vehicle, out-of-control steering of the vehicle, abnormal tire pressure of the vehicle in front, shedding of goods in the vehicle in front, obstacles in front, door opening of the vehicle in front, At least one of a fire in the rear of the vehicle in front and an accident ahead. 4. The method according to claim 2, wherein the instruction template is a collection of abnormal traffic data that needs to be processed as reported by the driver's voice, and the necessary word slots include address word slots, time word slots and vehicle word slots. Any one of the word slots, wherein the address word slot includes road and latitude and longitude, and the vehicle word slot includes vehicle color and license plate number. 5. The method according to claim 1, wherein said step S2 further comprises: Extracting the license plate image data in the image data, and obtaining the license plate number through license plate positioning, character segmentation and character recognition, Wherein, the license plate number is a unique key value for storing vehicle information in the cloud server. 6. The method according to claim 1, characterized in that, said step S3 comprises: Carrying out standard formatting processing on the abnormal traffic data to construct event information; wirelessly transmitting the constructed event information to a cloud server, and the cloud server analyzes the constructed event information to obtain target vehicle information, and sends traffic anomaly data to the target vehicle; The target vehicle converts the received abnormal traffic data into voice data and transmits information to the driver in the form of voice broadcast. 7. The method according to claim 1, wherein said step S3 further comprises: Analyze the constructed event information through the cloud server and send traffic anomaly data to the intelligent terminal device bound to the target vehicle. 8. The method of claim 1, further comprising: Send the location information of vehicles in the area to the cloud server at a fixed frequency. 9. A vehicle communication device, characterized in that the device comprises: The data collection module is configured to collect live information in a predetermined area of the vehicle in real time, wherein the live information includes image data of the vehicle and the surrounding environment of the vehicle in the predetermined area and voice data broadcast by the driver; The data processing module is configured to perform data processing on the live information to obtain traffic anomaly data, wherein the data processing includes using a pre-trained artificial intelligence model to perform anomaly detection on the collected image data and to perform abnormal detection on the collected voice data. Text transformation and word slot filling; The wireless communication module is configured to wirelessly transmit the abnormal traffic data to a cloud server, and broadcast to target vehicles in a predetermined area through the cloud server. 10. The device according to claim 9, wherein the pre-trained artificial intelligence model comprises a convolutional neural network model, and the data processing module is configured to: The image processing unit classifies the collected image data according to the type of vehicle abnormality according to the convolutional neural network model; The voice processing unit performs text conversion on the collected voice data to obtain corresponding text information, matches the text information with a predefined instruction template, and fills necessary word slots for the successfully matched text information. 11. The device according to claim 10, characterized in that, the abnormal types of the vehicle include brake failure of the vehicle, loss of control of the steering of the vehicle, abnormal tire pressure of the vehicle in front, cargo falling off in the vehicle in front, obstacles in front, door opening of the vehicle in front, At least one of a fire in the rear of the vehicle in front and an accident ahead. 12. The device according to claim 10, wherein the instruction template is a collection of abnormal traffic data reported by the driver's voice and needs to be processed, and the necessary word slots include address word slots, time word slots and vehicle word slots. Any one of the word slots, wherein the address word slot includes road and latitude and longitude, and the vehicle word slot includes vehicle color and license plate number. 13. The device according to claim 9, wherein the image processing unit further comprises: Extracting the license plate image data in the image data, and obtaining the license plate number through license plate positioning, character segmentation and character recognition, Wherein, the license plate number is a unique key value for storing vehicle information in the cloud server. 14. The device of claim 9, wherein the wireless communication module is configured to: Carrying out standard formatting processing on the abnormal traffic data to construct event information; wirelessly transmitting the constructed event information to a cloud server, and the cloud server analyzes the constructed event information to obtain target vehicle information, and sends traffic anomaly data to the target vehicle; The target vehicle converts the received abnormal traffic data into voice data and transmits information to the driver in the form of voice broadcast. 15. The device according to claim 9, wherein the wireless communication module is further configured to: Analyze the constructed event information through the cloud server and send traffic anomaly data to the intelligent terminal device bound to the target vehicle. 16. The device of claim 9, further comprising: Send the location information of vehicles in the area to the cloud server at a fixed frequency.