JP7797995B2 - Abnormal noise diagnosis system - Google Patents

Description

This disclosure relates to an abnormal noise diagnosis system that diagnoses abnormal noises generated in a vehicle.

Conventionally, a system that uses a network to identify faults in vehicles equipped with electronically controlled components (ECUs) is known (see, for example, Patent Document 1). In this system, a service company's server computer acquires vehicle identification information from the user's terminal and identifies the vehicle by referencing a vehicle information database. The server computer then refers to a fault investigation database by fault symptom, sends a questionnaire screen corresponding to the details of the fault in the vehicle to the terminal, and identifies the fault based on the response (questionnaire information).

Japanese Patent Application Laid-Open No. 2006-182319

When diagnosing the cause of an abnormal noise generated in a vehicle, more accurate diagnostic results can be obtained by using medical interview information regarding the circumstances under which the abnormal noise occurred in addition to sound data. However, because the medical interview information useful for diagnosing abnormal noises is relatively diverse, inputting the information into an input screen, etc., is time-consuming, placing a heavy burden on the workers involved in diagnosing abnormal noises.

The primary objective of this disclosure is to provide an abnormal noise diagnosis system that can easily obtain diagnostic information useful for diagnosing abnormal noises occurring in a vehicle while reducing the burden on workers and others.

The abnormal sound diagnosis system disclosed herein includes a terminal that acquires data on sounds emitted from a vehicle and diagnostic information related to abnormal sounds generated in the vehicle, and an information processing device that diagnoses abnormal sounds generated in the vehicle based on the sound data and diagnostic information from the terminal. The terminal of this abnormal sound diagnosis system acquires vehicle condition data indicating the vehicle's condition in synchronization with the sound data, and automatically extracts information to be provided to the information processing device as diagnostic information from the acquired vehicle condition data. This reduces the burden on workers and makes it easy to obtain diagnostic information useful for diagnosing abnormal sounds generated in the vehicle. The information processing device may also include a diagnostic unit constructed using machine learning to diagnose the cause of the abnormal sound based on the provided information.

1 is a schematic configuration diagram illustrating an abnormal sound diagnosis system according to the present disclosure. FIG. 10 is an explanatory diagram illustrating an example of an input screen for medical interview information. 4 is a flowchart showing a series of processes executed in a terminal constituting the abnormal sound diagnosis system of the present disclosure.

Next, we will explain the form for implementing the invention of this disclosure with reference to the drawings.

Figure 1 is a schematic diagram showing the configuration of an abnormal sound diagnosis system 1 of the present disclosure. The abnormal sound diagnosis system 1 shown in the figure is used to diagnose the cause of abnormal sounds generated in a vehicle V, such as a vehicle equipped with only an engine as a power source, a hybrid vehicle (HEV, PHEV), or an electric vehicle (BEV, FCEV), and includes a mobile terminal 10 and a server 20 that can exchange information with the mobile terminal 10 via communication.

The mobile terminal 10 is used by a worker (user of the abnormal sound diagnosis system 1) at a vehicle dealership, repair shop, or the like when responding to the user (owner) of the vehicle V in which an abnormal sound has occurred, or when performing a reproduction test in which the vehicle V is driven (operated) on a roadway or test bench to reproduce the abnormal sound. In this embodiment, the mobile terminal 10 is a smartphone that includes an SoC, ROM, RAM, an auxiliary storage device (flash memory) M, a touch-panel display 11, a communication module 12 capable of exchanging various information with a server 20 or the vehicle V's electronic control unit via wired or wireless communication, and a microphone (not shown). An abnormal sound diagnosis support application (program) is also installed on the mobile terminal 10. As shown in FIG. 1 , the mobile terminal 10 includes a medical interview information acquisition unit 13, a sound acquisition unit 14, a vehicle state acquisition unit 15, a calculation processing unit 16, an extraction unit 17, and a display control unit 18, each of which is constructed through cooperation between the abnormal sound diagnosis support application (software) and hardware such as the SoC of the mobile terminal 10.

The medical interview information acquisition unit 13 acquires, via the display unit 11, information indicating the state of the vehicle V at the time of the occurrence of the abnormal noise (hereinafter referred to as "medical interview information"), provided by the user of the vehicle V, etc. The sound acquisition unit 14 acquires time axis data of the sound (sound pressure) when a reproduction test is performed by an operator. The vehicle condition acquisition unit 15 acquires information indicating the state of the vehicle V (hereinafter referred to as "vehicle condition data") in synchronization with the acquisition of the time axis data of the sound by the sound acquisition unit 14 when a reproduction test is performed. The vehicle condition data includes multiple physical quantities (vehicle speed, engine speed, etc.) corresponding to items of the medical interview information. The calculation processing unit 16 performs analysis processing of the time axis data of the sound acquired by the sound acquisition unit 14. The extraction unit 17 narrows down the analysis results by the calculation processing unit 16 in accordance with the operator's selection, etc., and performs processing on the vehicle condition data. The display control unit 18 controls the display unit 11.

Figure 2 shows an input screen (questionnaire) for medical interview information displayed on the display unit 11 of the mobile terminal 10, along with an example of input. As shown in part in Figure 2, the medical interview information includes vehicle identification information, requests, date and time of occurrence, frequency of occurrence, location of the abnormal noise, type of sound, physical quantities that change while the vehicle V is traveling, such as vehicle speed, the operating condition of the vehicle V, the warm-up effect in engine-equipped vehicles, options selected by the driver while driving the vehicle V, and information about the driving environment of the vehicle V, and is input by the operator or the user of the vehicle V.

Vehicle identification information is information for identifying vehicle V, such as the vehicle model and vehicle identification number (chassis number). The order details the occurrence of abnormal noise provided by the user of vehicle V. The frequency of occurrence is selected from a drop-down list containing multiple options, such as always, several times per day, once per day, etc. The location of the abnormal noise is selected from a drop-down list containing multiple vehicle parts, such as the engine, powertrain, body, brakes, doors, and interior. The type of sound is selected by the worker from a drop-down list containing multiple onomatopoeia (e.g., rattle, clatter, etc.) corresponding to any abnormal sound generated by vehicle V, based on the sound recognized by the user of vehicle V as being similar to the actual abnormal sound. The physical quantity is selected from a drop-down list containing multiple options, such as vehicle speed, engine RPM, brake light switch ON/OFF time, steering angle, and the SOC of the high-voltage battery of a hybrid or electric vehicle. The operating state of vehicle V is selected from a drop-down list including multiple options such as starting, idling, stopping, starting, accelerating, cruising, decelerating (brakes off), braking (brakes on), reversing, and turning. Warm-up effect is selected from a drop-down list including cold, warm, and cold and warm. Selection items are selected from a drop-down list including shift position, driving mode, auxiliary equipment operating status, etc. Driving environment information is selected from a drop-down list including road surface conditions such as flat road, uphill road, downhill road, and weather. It goes without saying that not all of the above multiple items are provided by the user of vehicle V, and the medical interview information can be provided to the extent that the worker, etc., understands it.

The server 20 of the abnormal sound diagnosis system 1 is a computer (information processing device) including a CPU, ROM, RAM, input/output devices, a communications module, etc., and is installed and managed, for example, by the automobile manufacturer that produces the vehicle V. The server 20 is configured with an abnormal sound diagnosis unit 21 that diagnoses abnormal sounds generated in the vehicle V through cooperation between hardware such as the CPU and a pre-installed abnormal sound diagnosis application. The abnormal sound diagnosis unit 21 includes a neural network (convolutional neural network) constructed using supervised learning (machine learning) to diagnose the cause of abnormal sounds generated in the vehicle V and the component that is the source of the abnormal sound based on the medical history information and sound time axis data acquired by the mobile device 10. Furthermore, when the server 20 detects the occurrence of a new abnormal sound in the vehicle V, it re-trains the abnormal sound diagnosis unit 21 using the sound time axis data acquired for the new abnormal sound and the contents of each item of the medical history information as training data.

Furthermore, server 20 includes a storage device 22 that stores a database containing information on multiple abnormal noises that are known to occur in each vehicle model. The database stores information linked to each of the multiple abnormal noises, such as time axis data for the noise, the cause of the abnormal noise, the part that is the source of the noise, the details of medical interview information provided by the user, etc., and measures to resolve the abnormal noise. Server 20 also updates the database based on information obtained from multiple vehicles, including vehicle V, and information on newly discovered abnormal noises sent from automobile manufacturers (developers, etc.), vehicle dealerships, repair shops, etc.

Next, the abnormal sound diagnosis procedure using the abnormal sound diagnosis system 1 will be explained with reference to Figures 2 and 3.

When a worker at a vehicle dealership, repair shop, or similar facility receives a request from a user of vehicle V to resolve an abnormal noise, the worker listens to the user's medical history and then performs a reproduction test to obtain the information necessary to diagnose the abnormal noise. To perform the reproduction test, the worker connects a mobile device 10 to the vehicle V's electronic control unit and places or secures the mobile device 10 or an external microphone connected to the mobile device 10 in an appropriate location on the vehicle V. The worker then launches an abnormal noise diagnosis support application and turns on the vehicle V's start switch. The mobile device 10 then acquires information from the electronic control unit, such as the vehicle identification number or chassis number of the vehicle V. The worker then taps the recording start button displayed on the display 11 and drives (operates) the vehicle V on a roadway or test stand to reproduce the driving conditions under which the abnormal noise occurred, based on the medical history provided by the user of the vehicle V.

While the vehicle V is running (operating), the sound acquisition unit 14 of the mobile device 10 acquires time axis data of the sound emitted from the vehicle V at predetermined time intervals (micro-time intervals) and stores the data in the auxiliary storage device M. The vehicle state acquisition unit 15 acquires vehicle state data specified by the operator in response to medical interview information from the user of the vehicle V, etc., from the electronic control device of the vehicle V at predetermined time intervals (micro-time intervals) in synchronization with the acquisition of the sound time axis data by the sound acquisition unit 14, and stores the vehicle state data in the auxiliary storage device M. When the operator taps the recording stop button displayed on the display unit 11, the acquisition of the sound time axis data and vehicle state data is completed, and the mobile device 10 executes the series of processes shown in FIG. 3.

As shown in FIG. 3 , after the reproduction test is completed, the calculation processing unit 16 of the mobile device 10 acquires the time axis data of the sound acquired by the sound acquisition unit 14 (step S100), and performs a Short-Time Fourier Transform (STFT) on the acquired time axis data of the sound to acquire a spectrogram (acoustic spectrogram) showing the relationship between time, frequency, and sound pressure (step S110). Furthermore, the display control unit 18 of the mobile device 10 causes the display unit 11 to display the spectrogram acquired by the calculation processing unit 16 (step S120). In this embodiment, the spectrogram is a color map with the horizontal axis as the time axis and the vertical axis as the frequency axis, and shows the relationship between time and sound pressure level for each frequency by color-coding the sound pressure level.

When the spectrogram is displayed on the display unit 11 of the mobile terminal 10, the worker selects (specifies) on the display unit 11 the range of the spectrogram that should be diagnosed (analyzed) by the abnormal sound diagnosis unit 21 (server 20) (hereinafter referred to as the "diagnosis range"). In response to the worker's screen operation, the extraction unit 17 acquires the diagnosis range selected by the worker and instructs the display control unit 18 to display the diagnosis range on the display unit 11 (step S130). In step S130, the display control unit 18 grays out the portions not selected by the worker, thereby displaying the spectrogram on the display unit 11 so that only the diagnosis range selected by the worker is visible.

The extraction unit 17 also reads vehicle condition data within the diagnostic range (step S140) and extracts information to be provided to the server 20 as medical interview information from the read vehicle condition data (step S150). In step S150, the extraction unit 17, for example, performs a differential process on the vehicle speed (physical quantity) acquired as vehicle condition data to calculate acceleration, and extracts (acquires) the driving state of the vehicle V within the diagnostic range (acceleration, deceleration, or constant speed) based on the value and sign of the acceleration, and stores (retains) this as medical interview information in RAM, etc. In step S150, the extraction unit 17 also extracts (acquires) the driving state of the vehicle V (idling, etc.) from the engine speed acquired as vehicle condition data, and extracts (acquires) the driving state of the vehicle V (accelerator OFF or brake ON, etc.) from the ON/OFF times of the brake lamp switch. In this embodiment, the vehicle condition data (physical quantities) to be extracted from the medical interview information are determined in advance, and the extraction unit 17 performs the process of step S150 on the target data from the vehicle condition data within the diagnostic range.

After processing step S150, the display control unit 18 displays a message on the display unit 11 instructing the operator to input medical interview information. After the operator has completed inputting the medical interview information, the medical interview information acquisition unit 13 confirms the information extracted in step S150 and the information entered by the operator as the final medical interview information (step S160). In step S160, when the operator displays the medical interview information input screen in response to the medical interview information input instruction, the information extracted as medical interview information from the vehicle condition data in step S150 (e.g., "acceleration" in "driving state" in Figure 2) can be visually confirmed on the input screen. Therefore, in step S160, the operator only needs to input the remaining medical interview information obtained from the user of vehicle V or learned by the operator during the reproduction test. After the medical interview information is confirmed, the operator taps the information transmission button displayed on the display unit 11. The information necessary for diagnosing the abnormal noise is transmitted from the communication module 12 of the mobile terminal 10 to the server 20 (step S170). In this embodiment, the information transmitted from the mobile terminal 10 to the server 20 includes at least the time axis data of the sound acquired by the sound acquisition unit 14, the medical interview information confirmed in step S160, and information defining the diagnostic range selected by the operator.

When the information necessary for diagnosing the abnormal noise is sent from the mobile device 10 to the server 20, the abnormal noise diagnosis unit 21 of the server 20 diagnoses the cause of the abnormal noise occurring in the vehicle V based on the information provided by the mobile device 10 and sends the diagnosis results to the mobile device 10. The diagnosis results include the cause of the abnormal noise occurring in the vehicle V, the part that is the source of the abnormal noise, and measures to resolve the abnormal noise read from the storage device 22. When the diagnosis results from the server 20 are received by the mobile device 10 (step S180), the diagnosis results are displayed on the display unit 11 (step S190), and the series of processes executed by the mobile device 10 to diagnose the abnormal noise are completed. By executing the process shown in FIG. 3, the worker can accurately explain the diagnosis results from the server 20 to the user of the vehicle V and promptly proceed with measures to address the abnormal noise.

As described above, the mobile device 10 that constitutes the abnormal sound diagnosis system 1 acquires vehicle condition data indicating the state of the vehicle V in synchronization with the sound time axis data, automatically extracts information to be provided to the server 20 as medical interview information from the acquired vehicle condition data, and displays it on the medical interview information input screen (steps S140-S160). This allows the operator, etc. to simply input into the input screen as medical interview information what they hear from the user, etc. of the vehicle V, or what they themselves learn during the reproduction test. Therefore, the operator, etc., does not need to input medical interview information that is difficult to accurately grasp or understand, such as the operating status of the vehicle V, into the input screen. As a result, the abnormal sound diagnosis system 1 reduces the burden on the operator, etc., of inputting medical interview information, while making it possible to easily obtain medical interview information useful for diagnosing abnormal sounds occurring in the vehicle V. Furthermore, by acquiring part of the medical interview information from actually measured vehicle condition data, the accuracy of the medical interview information used to diagnose abnormal sounds can be further improved.

The abnormal sound diagnosis support application installed on the mobile terminal 10 may also be installed on a tablet terminal, laptop or desktop personal computer, etc., and such a tablet terminal may be used in place of the mobile terminal 10. Furthermore, the abnormal sound diagnosis system 1 may be configured as a single information processing device.

The disclosed invention is extremely useful for diagnosing abnormal noises occurring in vehicles.

1 Abnormal sound diagnosis system, 10 Mobile terminal, 11 Display unit, 12 Communication module, 13 Medical interview information acquisition unit, 14 Sound acquisition unit, 15 Vehicle condition acquisition unit, 16 Arithmetic processing unit, 17 Extraction unit, 18 Display control unit, 20 Server (information processing device), 21 Abnormal sound diagnosis unit, 22 Storage device, V Vehicle.

Claims (1)

An abnormal noise diagnosis system including: a terminal that acquires data on sounds emitted from a vehicle and inquiry information related to abnormal noises generated in the vehicle; and an information processing device that diagnoses the abnormal noise generated in the vehicle based on the sound data and the inquiry information from the terminal,
The terminal acquires vehicle condition data indicating the condition of the vehicle in synchronization with the sound data, automatically extracts a portion of the medical interview information from the acquired vehicle condition data , displays the portion of the information, and also displays an input screen that instructs the user to input the remaining information of the medical interview information, and once the remaining information is input, provides the portion of the information and the remaining information to the information processing device as the final medical interview information .