JP7730740B2 - Sound source detection method and sound source detection device - Google Patents

Description

The present invention relates to a sound source detection method and a sound source detection device.

For example, Patent Document 1 discloses a technology for displaying on a display screen a visual image of a sound source, in which an audio heat map generated based on sounds picked up by a microphone array is superimposed on an image captured by an omnidirectional camera.

International Publication No. 2018/20841

However, in Patent Document 1, when generating an audio heat map, which is map data that visually indicates the location of a sound source, no consideration is given to the deviation of the sound source position measured by the microphone array.

In other words, there is room for further improvement in detecting the sound source position to improve the accuracy of sound source measurement.

The sound source detection of the present invention involves acquiring sound generated from a measurement object and converting it into an electrical signal, calculating the sound source position from this electrical signal, measuring the air condition in the space from the position where the sound generated from the measurement object is acquired to the measurement object, estimating the sound field state of the space from the air condition, calculating the deviation of the sound source position from the sound field state, and correcting the sound source position.The sound source detection of the present invention is characterized by acquiring an image and temperature information of the measurement object, and superimposing a sound pressure heat map generated based on the corrected sound source position and a temperature heat map of the measurement object on the image of the measurement object, the sound pressure heat map of the sound source position showing differences in sound pressure with contour lines, and the temperature heat map of the measurement object showing differences in temperature using different colors .

According to the present invention, the position of a sound source can be accurately estimated by taking into account the state of the medium through which the sound propagates.

FIG. 1 is a functional block diagram of a first embodiment to which the present invention is applied. 4 is a flowchart showing the flow of processing in the first embodiment. FIG. 4 is an explanatory diagram showing a specific display example in the first embodiment. 10A and 10B are explanatory diagrams showing specific display examples in the first embodiment, in which FIG. 10A shows a display in which only the heat map of sound pressure is superimposed, and FIG. 10B shows a display in which only the heat map of air condition is superimposed. FIG. 10 is an explanatory diagram showing a specific display example in the first embodiment, where a shows various heat maps superimposed with a spectrogram showing the temporal change in the sound pressure heat map, and b shows an enlarged view of the spectrogram showing the temporal change in the sound pressure heat map. FIG. 10 is a functional block diagram of a second embodiment to which the present invention is applied. 10 is a flowchart showing the flow of processing according to a second embodiment. FIG. 10 is a functional block diagram of a third embodiment to which the present invention is applied. 10 is a flowchart showing the flow of processing according to a third embodiment. FIG. 11 is an explanatory diagram showing a specific display example in the third embodiment. FIG. 11 is an explanatory diagram showing a specific display example in the third embodiment. 10A and 10B are explanatory diagrams showing specific display examples in the third embodiment, in which (a) shows the names of abnormal noise determination items and results superimposed in text, and the vehicle status superimposed in a time series graph, and (b) shows an enlarged view of the superimposed time series graph showing the vehicle status. FIG. 10 is an explanatory diagram showing an outline of a system configuration of a fourth embodiment to which the present invention is applied.

An embodiment of the present invention will be described in detail below with reference to the drawings.
An embodiment in which the present invention is applied to detecting abnormal noise from a vehicle will be described below. This embodiment of abnormal noise detection is implemented, for example, to detect the location of a sound source generated from a completed vehicle, which is the object of measurement, during the completed vehicle inspection process, which is the final stage of an automobile production line. Generally, during the completed vehicle inspection process, an operator (inspector) test drives the completed vehicle to be inspected on free rollers, and inspects a number of items, including the engine, meters, brakes, etc.

During this inspection, the sound generated from the finished vehicle being measured is detected, and it is determined which part of the finished vehicle the sound is coming from.

Hereinafter, the device of this embodiment will also be referred to as the "sound source detection device," and this sound source detection device is configured as part of the inspection equipment in the finished vehicle inspection process.

The completed vehicles to be inspected are not limited to vehicles powered solely by gasoline engines (in other words, spark-ignition internal combustion engines) or diesel engines (in other words, compression-ignition internal combustion engines), but may also be so-called hybrid vehicles or electric vehicles.

Figure 1 shows a functional block diagram of the sound source detection device of the first embodiment. The sound source detection device of the first embodiment is configured to include a sound acquisition unit 10, a sound source position estimation unit 20, an image acquisition unit 30, an air condition measurement unit 40, a sound field condition estimation unit 50, a sound source position correction unit 60, a synthetic image generation unit 70, a display control unit 80, and a display unit 90.

The sound acquisition unit 10 acquires the sound generated by the vehicle under inspection, which is the measurement object, and converts it into an electrical signal, i.e., sound data.

The sound acquisition unit 10 includes a microphone array with multiple microphones (microphones) arranged in an array that acquires sounds generated by the vehicle under inspection and converts them into sound data, and a recording unit that temporarily stores the sound data from the microphone array. The microphone array is positioned outside the vehicle so that it can collect sounds such as the engine sound of the vehicle under inspection running on free rollers. The microphone's directivity and frequency characteristics are selected to suit its position relative to the vehicle under inspection that is the subject of measurement and the frequency band of the engine sound. The microphones in the microphone array may, for example, have a single directivity toward the vehicle under inspection.

Instead of a microphone array, the sound acquisition unit 10 may, for example, move a single microphone to acquire sound from a sound source at multiple different positions and use it as sound data. Also, instead of a microphone array, the sound acquisition unit 10 may provide multiple sound reflectors in space and use a single microphone to acquire the sound as sound data. The sound acquisition unit 10 is not limited to a microphone array, and as long as it can collect sound from a sound source at multiple positions, the collected sound may be used as sound data. Furthermore, the microphone used by the sound acquisition unit 10 may be, for example, an existing microphone installed to collect horn sounds during the finished vehicle inspection process.

The sound source position estimation unit 20 measures the time delay between multiple sounds measured by the sound acquisition unit 10 and estimates the sound source position from the amount of time delay.

More specifically, the sound source position estimation unit 20 measures the difference Δt in the arrival time of sound from the vehicle under inspection to multiple microphones from the sound data acquired by the sound acquisition unit 10, and estimates the sound source position using this value. A specific method for estimating the sound source position is to estimate the sound source position using an angle θ that can be expressed using the time difference ΔT between sound arriving at microphone A and microphone B, which are separated by a distance d (m), at a sound speed of C, and trigonometry. The angle θ can be expressed using Equation 1: θ = sin ^-1 (C × ΔT/d) (1). Note that the sound source position estimation unit 20 may estimate the sound source position with higher accuracy by using the delay sum of the sound or the plane wave/spherical wave characteristics of the sound.

Furthermore, techniques such as FFT (Fast Fourier Transform) or wavelet analysis can be used to extract only sounds of specific frequencies, and the time shift between sounds of each frequency can be calculated to estimate the sound source position with high accuracy.

The video acquisition unit 30 is a video camera or the like that is used to indicate the sound source position calculated by the sound source position estimation unit 20 and the sound source position correction unit 60, and the position of the vehicle under inspection from which the sound is coming. The video acquisition unit 30 acquires images or video.

The air condition measuring unit 40 measures the temperature of the vehicle under inspection and the condition of the air that acts as a sound medium in the space between the vehicle under inspection and the sound acquisition unit 10. In other words, the air condition measuring unit 40 measures the spatial conditions that affect the sound from the vehicle under inspection.

The air condition measuring unit 40 in the first embodiment is an infrared camera capable of non-contact temperature measurement. The infrared camera acquires the temperature conditions of the vehicle under inspection and the area from the vehicle under inspection to the sound acquisition unit 10. In addition to an infrared camera, the air condition measuring unit 40 may also be a measuring device capable of measuring temperature conditions, such as an air temperature measuring device using ultrasound.

Note that spatial conditions that affect sound, i.e., air conditions related to sound, include, for example, air temperature, humidity, air pressure, direction and strength of air flow, as well as gas coefficients, the mixture ratio, specific heat ratio, and molecular weight of gases in the air. Therefore, the air condition measuring unit 40 may be a device that measures these air conditions related to sound.

The sound field state estimation unit 50 estimates the sound field state in the space from the vehicle under inspection to the sound acquisition unit 10 based on the data acquired by the air condition measurement unit 40.

For example, the speed of sound C is expressed by Equation 2: C = (kRT/M) ^0.5 ... (2) where k is the specific heat ratio of air, R is the gas constant, T is the temperature of air, and M is the molecular weight of air.

The effect of air humidity on sound absorption is described, for example, in ISO 9613-1: Acoustics - Attenuation of sound during propagation outdoors - Part 1: Calculation of the absorption of sound by the atmosphere.

In addition, the wind direction based on the temperature distribution will be from areas with low temperatures to areas with high temperatures. Wind speed can be estimated based on factors such as the volume of the heat source, and can be calculated using, for example, TIV (Thermal Image Velocimetry).

When the air condition measuring unit 40 is an infrared camera, if there are high and low heat sources, the sound field state estimating unit 50 estimates that the temperature of the space nearby is close to that of the heat source, estimates the sound field state due to temperature changes, estimates the speed of sound from the temperature up to each microphone of the sound acquiring unit 10, and calculates the time lag Δtt. Similarly, if the temperature distribution in the space is known by the air condition measuring unit 40, the sound field state estimating unit 50 may estimate the speed of sound up to each microphone in the same manner as above and calculate the time lag Δtt.

The sound source position correction unit 60 corrects the sound source position calculated by the sound source position estimation unit 20 when the sound field state estimation unit 50 detects a change in the sound propagation state.

The sound source position is corrected using factors such as the speed of sound, which affects the time lag of the sound for each microphone in the sound acquisition unit 10 used when identifying the sound source. For example, in the above-mentioned formula 1, if the temperature T is 30°C higher, the sound speed C will be about 8% faster. The sound source position is corrected using these characteristics and calculation results. Specifically, the value Δtt calculated by the sound field state estimation unit 50 is used to correct the sound source position estimated by the sound source position estimation unit 20.

The composite image generation unit 70 generates a thermographic-like heat map with sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by applying processing such as semi-transparency. In other words, the composite image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30, with the sound source position displayed semi-transparently as a heat map of sound pressure at the sound source position.

The composite image generation unit 70 also generates a heat map of the air condition from the temperature information measured by the infrared camera, which is the air condition measurement unit 40, and generates a semi-transparent image or video superimposed on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

The display control unit 80 controls and displays multiple types of display, such as switching between the image (first composite image) or video (first composite video) generated by the composite image generation unit 70 and the heat map image (second composite image) or heat map video (second composite video) acquired by the infrared sensor of the air condition measurement unit 40. The display control unit 80 processes the image (first composite image) or video (second composite video) generated by the composite image generation unit 70 to make it easier to view.

Specifically, since it would be difficult to understand if both the sound source position information and the air information were superimposed, they could be displayed as separate superimposed images, with the heat map of sound pressure at the sound source position only displayed as contour lines, with the semi-transparent areas filled in with color and made transparent.

Note that the contour display is not limited to a heat map of sound pressure at the sound source location, but may also be applied to heat maps of air conditions such as temperature information. Furthermore, when the heat map of sound pressure at the sound source location and the heat map of air conditions are superimposed on an image or video of the vehicle being inspected, text, pictograms, images, etc. that indicate the displayed content may also be added.

The display control unit 80 may have the same functions and perform the same processing as the composite image generation unit 70.

The display unit 90 displays sound source location information and air information to relevant parties around the vehicle under inspection, such as workers (including the vehicle driver), managers, and data scientists who will use the data. The display unit 90 is, for example, an LCD display or an organic EL display. Furthermore, if audio is also included, the display unit 90 may also include a sound source, amplifier, speaker, etc. for generating and emitting sound.

Figure 2 is a flowchart showing the processing flow in the sound source detection device of the first embodiment.

In step S1, the microphone in the sound acquisition unit 10 collects sounds generated by the vehicle under test as it test drives on the free rollers, and acquires them as sound data.

In step S2, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

In step S3, an image or video is acquired from the infrared camera.

In step S4, the sound field state of the space is estimated (sound field state estimation unit 50).

In step S5, the sound source position estimated in step S2 is corrected in accordance with the sound field state of the space (sound source position correcting unit 60).
In step S6, a heat map similar to thermography is generated with sound pressure as a parameter for the sound source position estimated in step S2, and is superimposed on the image or video acquired by the video acquisition unit 30 after being processed for semi-transparency or the like (first composite image or first composite video). Also in step S6, a heat map of the air condition is generated from the data acquired in step S3 (temperature information measured by the infrared camera), and is superimposed on the image or video of the vehicle under inspection acquired by the video acquisition unit 30 after being processed for semi-transparency (second composite image or second composite video).

In step S7, processing is performed so that the first composite image or first composite video generated in step S6 and the second composite image or second composite video generated in step S6 are alternately displayed on the display unit 90.

In step S8, it is determined whether a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine ends. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S1. The decision to stop the display on the display unit 90 can be made, for example, by a worker or other relevant person operating a switch that stops the display function, thereby turning the display on or off.

Specific display examples for the first embodiment described above are shown in Figures 3 to 5. The attached Figures 3 to 5 are black and white images, but in actual images, the temperatures are color-coded, with high temperatures being red, medium temperatures being yellow, and low temperatures being blue, for example, and the temperature is indicated by changes in brightness within each color.

The first display example shown in Figure 3 is a composite video or composite image in which a heat map of sound pressure at the sound source location and a heat map of temperature (air conditions) generated from temperature information measured by an infrared camera are superimposed on an image or video of the vehicle under inspection acquired by the video acquisition unit 30.

In this first display example, the heat map of sound pressure at the sound source position is shown as contour lines only, making it easier to distinguish from the heat map of temperature (air conditions) generated from temperature information measured by an infrared camera.

In addition, for the temperature heat map, pictograms and legends for each measurement quantity (e.g., red indicates high temperatures and blue indicates low temperatures on a thermometer) have been added.

The second display example shown in Figure 4 is a method in which Figure 4(a) and Figure 4(b) are not displayed constantly, but are alternately displayed over time. This display has the advantage that the display for each measurement quantity is organized and the differences between them are easily seen the moment the display is switched. Figure 4(a) is a composite video or image in which a heat map of sound pressure at the sound source position is superimposed as contour lines only on an image or video of the vehicle under inspection acquired by the video acquisition unit 30. Figure 4(b) is a composite video or image in which a heat map of air conditions (temperature) generated from data acquired by the air condition measurement unit 40 (infrared camera) is superimposed on an image or video of the vehicle under inspection acquired by the video acquisition unit 30.

The third display example shown in Figure 5 displays the time-varying contour line representation of the sound source as a spectrogram, allowing you to see the changes in the sound's frequency characteristics and sound pressure.

A spectrogram is three-dimensional data that includes time, and here, following the general format, the horizontal axis represents time and the vertical axis represents frequency. The total time length is, for example, 20 seconds, and frequencies up to around 1000 Hz are included.

The brightness and color of each point represent the power (dBA) (or sound pressure or amplitude) of a certain frequency at a certain point in time. The attached image is a black and white image, but in an actual spectrogram display, for example, high power is color-coded in red, medium power in yellow-green, and low power in blue, and power is indicated by changes in brightness within each color.

Figure 5(a) shows a composite video or image in which a heat map of sound pressure at the sound source position is superimposed as contour lines only, a spectrogram showing the change in the contour line display of the sound source over time, and a heat map of air conditions (temperature) generated from data acquired by the air condition measurement unit 40 (infrared camera) are superimposed on an image or video of a vehicle under inspection acquired by the video acquisition unit 30.

Figure 5(b) shows an enlarged spectrogram showing the temporal change in the contour representation of the sound source.

Displays such as those in the first to third display examples are controlled by the display control unit 80 to be displayed at the appropriate timing, for the appropriate length of time, and under the appropriate display conditions (brightness, color, etc.).

In the second display example, the sound source and temperature are displayed cyclically over time. This makes it easier to see the difference between the sound source and the temperature, and makes it easier to compare the two displays, such as when they are displayed side by side.

The display processed in this manner is then presented as the measurement results to the test operator, manager, and other relevant parties on a display device such as an LCD or organic EL display on the display unit 90. The sound source monitoring results may also be presented as sound or audio to the test operator, manager, and other relevant parties as the measurement results using an audio presentation device such as a speaker.

In addition, the results of sound source monitoring may be visually displayed using video or images, and the results of sound source monitoring may be announced using sound or audio, so that the measurement results can be presented visually and audibly to relevant parties such as inspection workers and managers.

After performing this process, sound source monitoring continues unless an operation such as a stop button is pressed to stop sound source monitoring. If sound source monitoring continues, sound data is collected again as per the above process, and sound source monitoring is performed. If sound source monitoring is to be terminated, an end message is displayed on the display unit 90. Along with the display, an end message may also be announced by a buzzer or voice.

As explained above, in the first embodiment described above, the sound acquisition unit 10 acquires sound generated from the vehicle under inspection and converts it into an electrical signal, the sound source position estimation unit 20 calculates the sound source position from this electrical signal, the air condition measurement unit 40 measures the air condition of the vehicle under inspection and the space from the sound acquisition unit 10 to the vehicle under inspection, the sound field condition estimation unit 50 estimates the sound field condition of the space from the air condition, and the sound source position correction unit 60 calculates the deviation of the sound source position from the sound field condition and corrects the sound source position. Therefore, the sound source detection device of the first embodiment can estimate the sound source position more accurately than conventional methods by reflecting the state of the medium through which the sound propagates.

In the first embodiment described above, the sound field state of the space is estimated using the sound propagation speed from the sound source position calculated using a calculation formula with parameters being the physical quantities of the air in the space from the sound acquisition unit 10 to the vehicle under inspection. Therefore, the sound source detection device of the first embodiment can measure and estimate the state of the air, which is the medium of sound, quickly and in a short time with a low calculation load.

In the first embodiment described above, the image acquisition unit 30 acquires an image of the vehicle under inspection, and the composite image generation unit 70 superimposes a heat map of sound pressure generated based on the corrected sound source position onto the image or video of the vehicle under inspection to generate a composite image or video, which is then displayed on the screen of the display unit 90. As a result, the sound source detection device of the first embodiment can clearly display the accurately estimated sound source position.

In the first embodiment described above, a heat map of air conditions generated by acquiring information on the air conditions in the space from the sound acquisition unit 10 to the vehicle under inspection is superimposed on a video or image of the vehicle under inspection to generate a composite video or image, and the display control unit 80 processes the sound pressure heat map and air condition heat map in this composite video or image so that they are easy to see, and displays them on the screen of the display unit 90. Therefore, the sound source detection device of the first embodiment can clearly display the accurately estimated sound source position, as well as the factors that corrected the sound source position.

In the first embodiment described above, the air condition in the space from the sound acquisition unit 10 to the vehicle under inspection is measured in a non-contact manner. Therefore, the sound source detection device of the first embodiment can inexpensively and easily measure and estimate the temperature condition of the sound medium.

In the first embodiment described above, the display of the heat map of sound source position and sound pressure is corrected based on the temperature distribution in the space. As a result, the sound source detection device of the first embodiment can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

In the first embodiment described above, the higher the temperature in the space, the farther the sound source position is corrected, and the acquired sound pressure may be corrected to decrease, correcting the display of the sound pressure heat map. In this case, the sound source detection device can easily and accurately estimate the sound source position and display it in an easy-to-understand manner.

In the first embodiment described above, the propagation speed of sound from the sound source position may be corrected so that it increases as the temperature at the sound source position increases. In this case, the sound source detection device can easily and accurately estimate the sound source position and display it in an easy-to-understand manner.

In the first embodiment described above, a video or image (composite video or composite image) of the vehicle under inspection is displayed on a visual display device such as an LCD display. Information can be presented in an easy-to-understand manner in an appropriate modal depending on the situation at the site, the characteristics of the informant, etc. Therefore, the sound source detection device of the first embodiment can present information in an easy-to-understand manner in an appropriate modal depending on the situation at the site, the characteristics of the informant, etc.

In the second display example of the first embodiment described above, a first composite video or first composite image is generated by superimposing a heat map of sound pressure generated based on the corrected sound source position on a video or image of the vehicle under inspection, and a second composite video or second composite image is generated by superimposing a heat map of air conditions generated by acquiring information on the air conditions in the space from the sound acquisition unit 10 to the vehicle under inspection on a video or image of the vehicle under inspection, and the first composite video or first composite image and the second composite video or second composite image are displayed alternately for a fixed period of time. Therefore, in the second display example of the first embodiment, there is no need to forcibly change the display content to generate a single composite image, and the spatial differences in relation to the measurement target become clear the moment the switch is made, allowing information to be presented in an easy-to-understand manner.

In the first and third display examples of the first embodiment described above, a heat map of sound pressure generated based on the corrected sound source position and a heat map of air conditions generated by acquiring information on the air conditions in the space from the sound acquisition unit 10 to the vehicle under inspection are displayed superimposed in parallel on a video or image of the vehicle under inspection. Therefore, in the first and third display examples of the first embodiment, information can be presented in an easy-to-understand manner.

In the first embodiment described above, the heat map of sound pressure generated based on the corrected sound source position is not colored but is represented only by contour lines. Therefore, in the first embodiment, even if different items (sound pressure at the sound source position and the air condition in the space between the sound acquisition unit 10 and the vehicle under inspection) are each displayed on a heat map, viewers can easily distinguish between them without becoming confused.

Furthermore, in the first embodiment described above, the phase of the sound generated by the vehicle under inspection may be used when calculating the sound source position.

Another embodiment of the present invention will now be described. Note that the same components as those in the first embodiment described above will be assigned the same reference numerals, and redundant explanations will be omitted.

A sound source detection device according to a second embodiment of the present invention will be described using Figures 6 and 7. Figure 6 shows a functional block diagram of the sound source detection device according to the second embodiment. Figure 7 is a flowchart showing the processing flow in the sound source detection device according to the second embodiment.

The sound source detection device of the second embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but the air condition measurement unit 41 is an ultrasonic aerometer (ultrasonic anemometer) rather than an infrared camera.

In the second embodiment, the air condition measuring unit 41 measures the state of the air that serves as the medium for sound in the space from the vehicle under inspection to the sound acquisition unit 10. In other words, the air condition measuring unit 41 measures the spatial state that affects the sound from the vehicle under inspection.

The air condition measuring unit 41 of the second embodiment non-contactly measures the wind direction and wind speed of the airflow in the space from the vehicle under inspection to the sound acquisition unit 10 and in the space around the vehicle under inspection. Note that the air condition measuring unit 41 may use measuring means other than an ultrasonic anemometer as long as it can measure the state of the airflow.

In the second embodiment, when the air condition measuring unit 41 is an ultrasonic anemometer, the sound field state estimating unit 50 estimates the sound speed and the direction and amount of sound movement using the wind speed magnitude F and wind directions Fx, Fy, and Fz, and calculates the time delay Δtt between the sound arriving at each microphone of the sound acquiring unit 10 from the sound source position. The sound speed C can be expressed as in Equation 3, where _C0 is the speed of sound when the wind speed is zero, and F is the wind speed: C = _C0 + F... (3). F, which represents the wind speed, is a positive value when it is in the same direction as the sound's traveling direction, and a negative value when it is opposite the sound's traveling direction. Furthermore, the influence of wind direction on the sound's traveling direction can be calculated in the same way as wind speed, using the direction and magnitude of a vector in three-dimensional space.

In the second embodiment, the sound source position correction unit 60 corrects the sound source position calculated by the sound source position estimation unit 20 based on the sound field state acquired by the sound field state estimation unit 50.

In the second embodiment, the synthetic image generation unit 70 generates a thermographic-like heat map with sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by applying processing such as semi-transparency. In other words, the synthetic image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video in which the sound source position is displayed semi-transparently as a heat map of sound pressure at the sound source position and is superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30.

In addition, in the second embodiment, the composite image generation unit 70 uses information about the airflow measured by the ultrasonic measuring instrument, which is the air condition measurement unit 41, to represent the wind direction and speed of the airflow at each point around the vehicle under inspection using arrow directions and lengths, and generates an image or video by superimposing this on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

The processing flow of the sound source detection device of the second embodiment will be explained using Figure 7.

In step S21, the microphone in the sound acquisition unit 10 collects sounds generated by the vehicle under test as it runs on the free rollers, and acquires the sounds as sound data.

In step S22, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

In step S23, the measurement results of the ultrasonic anemometer are obtained.

In step S24, the sound field state of the space is estimated (sound field state estimation unit 50).

In step S25, the sound source position estimated in step S22 is corrected according to the sound field state of the space (sound source position correction unit 60).

In step S26, a thermographic-like heat map is generated using the sound pressure corrected in step S25 as a parameter for the sound source position corrected in step S25, and the heat map is superimposed on the image or video acquired by the video acquisition unit 30 after processing such as making it semi-transparent (first composite image or first composite video). Also in step S26, the data acquired in step S23 (wind direction and wind speed of airflow measured by the ultrasonic anemometer) is converted into a pictogram and superimposed semi-transparently on the image or video of the vehicle under inspection acquired by the video acquisition unit 30 (second composite image or second composite video).

In step S27, processing is performed so that the first composite image or first composite video generated in step S26 and the second composite image or second composite video generated in step S26 are alternately displayed on the display unit 90.

In step S28, it is determined whether a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine is terminated. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S21.

The sound source detection device of this second embodiment can achieve substantially the same effects as the sound source detection device of the first embodiment described above.

Furthermore, in the second embodiment described above, the display of the sound source position and sound pressure heat map is corrected based on the wind direction and wind speed of the airflow in the space. As a result, the sound source detection device of the first embodiment can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

A sound source detection device according to a third embodiment of the present invention will be described using Figures 8 and 9. Figure 8 shows a functional block diagram of the sound source detection device according to the third embodiment. Figure 9 is a flowchart showing the processing flow in the sound source detection device according to the third embodiment.

The sound source detection device of the third embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but in addition to the air condition measurement unit 40, which is an infrared camera, it has an air condition measurement unit 41, which is an ultrasonic anemometer.

In other words, the sound source detection device of the third embodiment measures temperature and air flow as spatial conditions that affect the sound from the vehicle under inspection.

In addition, the sound source detection device of the third embodiment has a vehicle condition acquisition unit 100, which can also display the condition of the vehicle being inspected.

Furthermore, the sound source detection device of the third embodiment has an abnormal sound determination unit 110, which determines whether the sound of the measured sound source is normal or abnormal, and can display the results.

In the third embodiment, the sound field state estimation unit 50 calculates the time lag Δtt between the sound arriving at each microphone of the sound acquisition unit 10 from the sound source position, taking into account the temperature measured by the air state measurement unit 40 and the state of the air flow measured by the air state measurement unit 41.

In the third embodiment, the sound source position correction unit 60 corrects the sound source position calculated by the sound source position estimation unit 20 based on the sound field state acquired by the sound field state estimation unit 50, i.e., based on temperature and airflow.

The vehicle state acquisition unit 100 acquires information for correlating the sound measured by the sound acquisition unit 10 with the state of the vehicle being inspected. The vehicle state acquisition unit 100 acquires, for example, the vehicle speed of the vehicle being inspected, the engine speed of the internal combustion engine installed in the vehicle, the accelerator opening of the vehicle being inspected, the amount of depression of the brake pedal of the vehicle being inspected, the steering angle of the vehicle being inspected, the acceleration of the vehicle being inspected, and the shift position of the vehicle being inspected.

The vehicle status information acquired by the vehicle status acquisition unit 100 is not limited to the information described above, but may include any information related to the vehicle being inspected, such as the coolant temperature of the internal combustion engine installed in the vehicle being inspected, the operating status and set temperature of the air conditioning system (air conditioner) of the vehicle being inspected, and the number of occupants in the vehicle being inspected.

The abnormal sound detection unit 110 determines whether the sound from the sound source corrected by the sound source position correction unit 60 is a normal sound. The abnormal sound detection unit 110 is, for example, an abnormal sound detection algorithm modeled using an unsupervised learning machine learning model, which is one type of machine learning method. As an unsupervised learning machine learning model, for example, an unsupervised learning model that only learns normal data and does not learn abnormal data can be applied.

The abnormal sound detection unit 110 can also use, for example, a supervised learning machine learning model that learns and detects both normal and abnormal sound data.

Furthermore, the abnormal sound detection unit 110 is not limited to machine learning, but may determine whether something is normal or abnormal by setting thresholds for the physical quantities of sound, such as sound pressure, frequency characteristics, and time modulation of sound, or may determine whether something is normal or abnormal using the MT method used in quality engineering.

In the third embodiment, the synthetic image generation unit 70 generates a thermographic-like heat map with sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by applying processing such as semi-transparency. In other words, the synthetic image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video in which the sound source position is displayed semi-transparently as a heat map of sound pressure at the sound source position and is superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30.

In addition, in the third embodiment, the composite image generation unit 70 uses temperature information generated by the infrared camera, which is the air condition measurement unit 40, and air flow information measured by the ultrasonic measurement device, which is the air condition measurement unit 41, to display the temperature at each point around the vehicle under inspection using color, and the wind direction and speed of the air flow using the direction and length of arrows, and generates an image or video by superimposing this on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

Furthermore, in the third embodiment, the composite image generation unit 70 superimposes the results of the abnormal noise determination and the condition of the vehicle under inspection onto an image or video in which a heat map of the sound pressure of the sound source, temperature information, and air volume information are superimposed on an image or video of the vehicle under inspection.

The processing flow of the sound source detection device of the third embodiment will be explained using Figure 9.

In step S31, the microphone in the sound acquisition unit 10 collects sounds generated by the vehicle under test as it runs on the free rollers, and acquires them as sound data.

In step S32, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

In step S33, an image or video is acquired from the infrared camera.

In step S34, the measurement results of the ultrasonic anemometer are obtained.

In step S35, the sound field state of the space is estimated (sound field state estimation unit 50).

In step S36, the sound source position estimated in step S22 is corrected according to the sound field state of the space (sound source position correction unit 60).

In step S37, a thermographic-like heat map is generated using the sound pressure corrected in step S36 as a parameter for the sound source position corrected in step S36, and the heat map is superimposed on the image or video acquired by the video acquisition unit 30 after processing such as making it semi-transparent (first composite image or first composite video). Also in step S37, the data acquired in step S34 (wind direction and wind speed of the airflow measured by the ultrasonic anemometer) is converted into a pictogram and superimposed semi-transparently on the image or video of the vehicle under inspection acquired by the video acquisition unit 30 (second composite image or second composite video).

In step S38, processing is performed so that the first composite image or first composite video generated in step S37 and the second composite image or second composite video generated in step S37 are alternately displayed on the display unit 90.

In step S39, it is determined whether a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine is terminated. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S31.

Specific display examples for the third embodiment described above are shown in Figures 10 to 12. The attached Figures 10 to 12 are black and white images, but in actual images, the temperatures are color-coded, with high temperatures being red, medium temperatures being yellow, and low temperatures being blue, for example, and the temperature is indicated by changes in brightness within each color.

The fourth display example shown in Figure 10 is a composite video or image in which a heat map of sound pressure at the sound source location, predetermined graphic symbols (circles) that indicate the temperature at each point using their respective colors, and symbols (arrows) that indicate the wind direction (direction) and wind speed (magnitude) of the airflow at each point are superimposed on an image or video of the vehicle under inspection acquired by the video acquisition unit 30.

In the fourth display example shown in Figure 10, the heat map of sound pressure at the sound source position is displayed as contour lines only. Also, in the fourth display example shown in Figure 10, the direction and length of the arrow indicate the wind direction and speed of the airflow at each point. That is, the direction of the arrow indicates the wind direction of the airflow at that point. And the length of the arrow indicates the wind speed at that point.

In other words, the fourth display shown in Figure 10 generates a circular symbol indicating the temperature at each of multiple points on the vehicle under inspection and around the vehicle under inspection, and an arrow protruding from the circular symbol indicating the direction (wind direction) and magnitude (wind speed) of the air flow. The color of the circle represents the temperature at that point, the orientation of the arrow represents the direction (wind direction) of the air flow at that point, and the length or thickness of the arrow represents the magnitude (wind speed) of the air flow at that point.

Note that the display in Figure 10 is just one example, and the wind speed at each point may be represented by the thickness of the arrow line. Furthermore, the way in which temperature and air flow are represented is not limited to Figure 10; the temperature at each point may be represented by a symbol other than a circle (for example, a rectangle), and the air flow at each point may be represented by a symbol other than an arrow (for example, a line diagram with a flag fluttering).

For example, the wind speed at each point may be represented by the color of the arrow or the size of the circle in Figure 10. Furthermore, the temperature at each point may be represented by the darkness of the color of the temperature symbol, the thickness of the line, or the number of lines. In short, temperature and air flow can be represented in any way as long as each physical quantity is displayed graphically.

The fifth display example shown in Figure 11 is the first display example described above, with the item names and results of the abnormal noise detection superimposed in text.

The sixth display example shown in Figure 12 is an example of display example 1 described above, with the names and results of the abnormal noise detection items superimposed in text, and the vehicle status superimposed in a time-series graph.

Figure 12(a) is a composite video or image in which the names and results of the abnormal noise detection items are superimposed in text on the display example 1 described above, and the vehicle status is superimposed on a time series graph. Figure 12(b) is an enlarged view of the superimposed time series graph showing the vehicle status.

The sound source detection device of this third embodiment can achieve substantially the same effects as the sound source detection devices of the first and second embodiments described above.

Furthermore, in the third embodiment described above, a circular symbol indicating temperature and an arrow protruding from the circular symbol indicating the direction (wind direction) and magnitude (wind speed) of airflow are generated for each of the vehicle under inspection and multiple points around the vehicle. The color of the circle represents the temperature at that point, the direction of the arrow represents the direction of the airflow at that point, and the length or thickness of the arrow represents the magnitude (wind speed) of the airflow at that point. The circle and arrow are superimposed on a video or image of the vehicle under inspection and displayed on the screen. Therefore, the sound source detection device of the third embodiment can clearly display the state of the air, which is the medium of sound, in the composite image generation unit 70.

In the third embodiment described above, the sound from the sound source location is judged to be normal or abnormal, and the judgment result is superimposed on the video or image of the vehicle being inspected. As a result, the sound source detection device of the third embodiment can detect with high accuracy the location from which abnormal sounds are being generated and display it in an easy-to-understand manner.

In the third embodiment described above, information about the condition of the vehicle being inspected is superimposed on a video or image of the vehicle being inspected. As a result, the sound source detection device of the third embodiment can accurately detect areas where abnormal sounds are being generated in relation to the vehicle condition, and can display these in an easy-to-understand manner.

The sound source detection device of the fourth embodiment of the present invention will be described using Figure 13. Figure 13 is an explanatory diagram showing an outline of the system configuration of the sound source detection device of the fourth embodiment.

The sound source detection device of the fourth embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but further includes an oscillator 120, speakers 121a and 121b, and an LED 122 to confirm the accuracy of the sound field measurement and to calibrate the measurement equipment.

The sound source detection device of the fourth embodiment generates a reference sound on the vehicle under inspection, which is the object to be measured, in order to check the accuracy of the device and to calibrate the device.

Oscillator 120 generates sound of a predetermined frequency and sound pressure, and may also include an amplifier so that sound can be output from speakers 121a and 121b.

Speakers 121a and 121b are installed on the vehicle under inspection (the object being measured) to confirm the accuracy and calibrate the equipment, and generate reference sounds. The sound source detection device of the fourth embodiment has two speakers 121a and 121b, but to support sound source position estimation, it is necessary to have at least two or more speakers so that sounds can be generated from multiple points simultaneously. Note that when simultaneously measuring in the three axial directions of up, down, left, right, and depth, six or more speakers that generate reference sounds should be installed.

The LED (light-emitting diode) 122 can record the frequency and sound pressure of the sound from the speaker in both the image acquisition unit 30 and the infrared camera (air condition measurement unit 40), and also collects and confirms information from the oscillator 120. The LED 122 is a light source that changes intensity and color according to the physical characteristics of the reference sound source. Note that the sound source detection device of the fourth embodiment may use a lighting fixture whose brightness and color can be changed instead of the LED 122.

Calibration of the sound source detection device of the fourth embodiment is performed in a booth, such as an anechoic chamber, where the temperature and airflow are controlled to prevent changes in the sound field. If such an environment cannot be prepared, walls made of sound-absorbing material may be erected to prevent influence from the surrounding area.

Speakers 121a and 121b receive a signal of a predetermined frequency and sound pressure from oscillator 120 and emit a sound for a predetermined duration. In other words, speakers 121a and 121b are multiple reference sound sources that emit reference sounds.

This sound is simultaneously captured by microphone 10a and microphone 10b that make up sound capture unit 10, and the time difference between them is measured to estimate the sound source position, and it is confirmed whether it matches the position of speaker 121a or speaker 121b. If there is a positional misalignment, the problem is addressed by adjusting the equipment or adding a correction coefficient.

Since the characteristics of measuring instruments vary depending on the frequency of the sound, the above measurements are performed while changing the frequency of oscillator 120, and adjustments are made each time.

After the calibration is completed as described above, the accuracy (performance) of the sound field measurement is confirmed in an actual measurement environment. Here, we will explain the case where an infrared camera (air condition measurement unit 40) is used.

Similar to calibration, oscillator 120 generates sound of a specified frequency, sound pressure, and duration, and the sound is emitted from speakers 121a and 121b.

This sound is measured by microphones 10a and 10b of the sound acquisition unit 10, and temperature is measured by an infrared camera (air condition measurement unit 40) to measure the sound field state at that time. Furthermore, an image is taken by a camera (image acquisition unit 30) for superimposed display.

In order to visually record the sound conditions on the video, indicators such as LED 122 are used to project frequency, sound pressure, etc. onto the camera image using the color and brightness of LED 122, text display, etc.

Then, the actual corrected sound source position obtained in the first to third examples described above is compared with the speaker position to confirm the performance of the sound field measurement.

The sound source detection device of this fourth embodiment can achieve substantially the same effects as the sound source detection device of the first embodiment described above.

In the fourth embodiment described above, the process of calculating and correcting the sound source position is calibrated using multiple reference sound sources that emit reference sounds and a light source that changes intensity and color depending on the physical properties of the reference sound source. Therefore, the sound source detection device of the fourth embodiment can simply and efficiently measure the sound field state, evaluate the performance of measuring equipment, and calibrate using only sound and video, without obtaining information about the reference sound source.

Specific embodiments of the present invention have been described above, but the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention. The above-described embodiments relate to sound source detection methods and sound source detection devices.

10...sound acquisition unit 10a...microphone 10b...microphone 20...sound source position estimation unit 30...image acquisition unit 40...air condition measurement unit 41...air condition measurement unit 50...sound field condition estimation unit 60...sound source position correction unit 70...synthetic image generation unit 80...display control unit 90...display unit 100...vehicle condition acquisition unit 110...abnormal sound determination unit 120...oscillator 121a...speaker 121b...speaker 122...LED

Claims (14)

The sound generated by the vehicle, which is the object of measurement, is acquired and converted into an electrical signal.
The sound source position is calculated from this electrical signal,
measuring the air condition in the space from a position where sound generated from the measurement object is acquired to the measurement object;
Estimating the sound field state of the space from the air state;
A sound source detection method for calculating a deviation of the sound source position from the sound field state and correcting the sound source position,
Acquire the image and temperature information of the object to be measured,
a heat map of sound pressure generated based on the corrected sound source position and a heat map of temperature of the measurement object are superimposed on an image of the measurement object, and displayed;
The sound pressure heat map of the sound source position shows the difference in sound pressure with contour lines.
A sound source detection method characterized in that the heat map of the temperature of the measurement object is displayed in different colors to indicate differences in temperature . The sound source detection method described in claim 1, characterized in that the sound field state of the space is estimated using the sound propagation velocity from the sound source position calculated using a calculation formula with air physical quantities as parameters. superimposing a heat map of the air condition generated by acquiring information on the air condition in the space up to the measurement object onto an image of the measurement object;
The sound source detection method according to claim 1 , characterized in that the sound pressure heat map and the air condition heat map superimposed on the image of the measurement object are processed and displayed so as to be easily visible. The sound source detection method according to claim 3 , wherein the air condition in the space from a position where the sound generated from the measurement object is acquired to the measurement object is measured in a non-contact manner. The sound source detection method according to any one of claims 1 to 4 , wherein the display of the heat map of the sound source position and sound pressure is corrected based on the temperature distribution or air flow in the space. The sound source detection method according to any one of claims 1 to 4 , wherein the higher the temperature of the space, the farther the sound source position is corrected, and the acquired sound pressure is corrected to be smaller, thereby correcting the display of the sound pressure heat map. 7. The sound source detection method according to claim 2 , wherein the propagation speed of the sound from the sound source position is corrected so that it becomes faster as the temperature of the sound source position increases. The sound source detection method according to claim 1, characterized in that a predetermined graphic indicating temperature and arrows protruding from the graphic indicating the direction and magnitude of air flow are generated for each of the measurement object and a plurality of points around the measurement object, the color of the graphic indicates the temperature at that point, the direction of the arrow indicates the direction of the air flow at that point, and the length or thickness of the arrow indicates the magnitude of the air flow at that point , and the graphic and arrows are superimposed on an image of the measurement object. The sound source detection method according to any one of claims 1 to 8, characterized in that a process for calculating and correcting a sound source position is calibrated by using a plurality of reference sound sources that emit reference sounds and a light source that changes intensity or color according to physical properties of the reference sound sources. 10. The sound source detection method according to claim 1 , further comprising determining whether the sound from the sound source position is normal or abnormal, and superimposing the determination result on the image of the object to be measured. 11. The sound source detection method according to claim 1 , wherein information about the state of the measurement object is superimposed on an image of the measurement object. 12. The sound source detection method according to claim 1 , wherein the image of the object to be measured is displayed on a visual display device. The sound source detection method according to any one of claims 2 to 12 , wherein the phase of the sound generated from the object to be measured is utilized when calculating the sound source position. a sound acquisition unit that acquires sound generated from a vehicle, which is a measurement object, and converts it into an electrical signal;
a sound source position estimation unit that calculates the sound source position from the electrical signal;
an air condition measuring unit that measures the air condition in the space from a position where sound generated from the measurement object is acquired to the measurement object;
a sound field state estimation unit that estimates a sound field state of the space from the air state;
a sound source position correction unit that calculates a deviation of the sound source position from the sound field state and corrects the sound source position;
an image acquisition unit that acquires an image of the measurement object;
a state measurement unit that acquires temperature information of the measurement object;
a display control unit that displays a heat map of sound pressure generated based on the corrected sound source position and a heat map of the temperature of the measurement object by superimposing them on an image of the measurement object,
A sound source detection device characterized in that a heat map of sound pressure at a sound source position displays differences in sound pressure using contour lines, and a heat map of the temperature of the object being measured displays differences in temperature using different colors .