JP7837211B2 - Audio signal processing device and audio signal processing method - Google Patents

Description

The embodiments of the disclosure relate to an audio signal processing device and an audio signal processing method.

For example, if one occupant makes a phone call to someone outside the vehicle using a mobile device, other passengers inside the vehicle can hear the caller's voice but cannot hear the received voice, which may cause discomfort.

Therefore, there is a technology that makes the conversation difficult for passengers to hear by collecting the transmitted voice using a microphone placed near the person making the call inside the vehicle, generating an inverse-phase sound of the transmitted voice, and outputting it from a speaker placed near passengers other than the person making the call (see, for example, Patent Document 1).

Japanese Patent Publication No. 2006-303721

However, the person making the call inside the vehicle may speak to a passenger during the call. In this case, the out-of-phase sound, which is intended to suppress any discomfort to the passenger, may interfere with the temporary conversation between the person making the call and the passenger.

One embodiment, made in view of the above, aims to provide an audio signal processing device and an audio signal processing method that can make the transmitted audio inside the vehicle less audible to the passenger without interfering with the temporary conversation between the person making the call and the passenger inside the vehicle.

According to one embodiment, the audio signal processing device outputs a sound from a speaker to the passengers in the vehicle that makes it difficult for the caller's voice to be heard by other passengers in the vehicle. When the audio signal processing device detects speech from the caller to the passenger, or from the passenger to the caller, during a call, it stops outputting the sound.

A voice signal processing device and voice signal processing method according to one embodiment of this system have the effect of making the transmitted voice inside the vehicle less audible to the passenger without interfering with the temporary conversation between the caller and the passenger inside the vehicle.

Figure 1 is an explanatory diagram showing an overview of the audio signal processing method according to an embodiment. Figure 2 is a functional block diagram showing an example configuration of an audio signal processing device according to an embodiment. Figure 3 is a flowchart showing an example of a process performed by the controller of the audio signal processing device according to the embodiment. Figure 4 is a flowchart showing an example of a process performed by the controller of the audio signal processing device according to the embodiment. Figure 5 is a flowchart showing an example of a process performed by the controller of the audio signal processing device according to this embodiment.

The following describes in detail embodiments of the audio signal processing device and audio signal processing method with reference to the attached drawings. However, this invention is not limited to the embodiments described below.

[1. Overview of the audio signal processing method]
First, with reference to Figure 1, an overview of the audio signal processing method performed by the audio signal processing device according to the embodiment will be described. Figure 1 is an explanatory diagram showing an overview of the audio signal processing method according to the embodiment. Here, we will describe a case where an occupant in the passenger seat of vehicle 100 is making a call with someone outside the vehicle using a mobile terminal, and the driver is seated in the driver's seat of vehicle 100.

In the following, the passenger in the front seat will be referred to as the caller 10, the person on the other end of the call outside the vehicle as the external caller 11, and the passenger in the driver's seat as the passenger 12. The audio signal processing method according to this embodiment also applies when the driver of the vehicle 100 is the caller, and other passengers are passengers other than the caller. Furthermore, the audio signal processing method according to this embodiment also applies when there are multiple passengers other than the caller 10 inside the vehicle.

As shown in the upper diagram of Figure 1, the audio signal processing device 1 is connected to a microphone 20, a speaker 21, and a camera 22 installed in the vehicle 100. When a caller 10 inside the vehicle and a caller 11 outside the vehicle begin a conversation using their mobile devices, the audio signal processing device 1 generates a sound that makes the voice of the caller inside the vehicle difficult to hear for other passengers 12.

For example, the audio signal processing device 1 acquires the transmitted voice of the caller 10 from the microphone 20 and generates an inverted-phase sound of the transmitted voice (hereinafter referred to as "cancellation sound"). The audio signal processing device 1 then outputs the cancellation sound from the speaker 21.

As a result, the voice signal processing device 1 can suppress the passenger 12 from becoming uncomfortable due to the voice of the caller 10, because the transmitted voice is canceled out by the cancellation tone.

Furthermore, when a call begins, the audio signal processing device 1 can output a masking tone instead of a cancellation tone, or it can output both the cancellation tone and the masking tone simultaneously. The masking tone can be, for example, a natural ambient sound or background music (BGM).

Subsequently, during the call, caller 10 may interrupt the transmission to speak to passenger 12. Also, passenger 12 may speak to caller 10 during the call. In this case, if the cancellation or masking sound continues to be output, passenger 12 will have difficulty hearing caller 10's speech.

Therefore, as shown in the lower diagram of Figure 1, when the audio signal processing device 1 detects speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, during a call, it stops outputting the cancellation sound and masking sound. This allows the audio signal processing device 1 to make the transmitted speech inside the vehicle less audible to the passenger 12 without interfering with the temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

[2. Example of Audio Signal Processing Device Configuration]
Next, with reference to Figure 2, an example of the configuration of the audio signal processing device according to the embodiment will be described. Figure 2 is a functional block diagram showing an example of the configuration of the audio signal processing device according to the embodiment.

As shown in Figure 2, the audio signal processing device 1 comprises a controller 2 and a storage unit 7. The storage unit 7 is, for example, an information storage device such as a data flash, and stores masking sound information 71 and crew information 72.

The masking sound information 71 includes, for example, audio information such as natural ambient sounds or background music (BGM). The occupant information 72 includes, for example, voiceprint information, facial image information, and name information of the occupants of the vehicle 100, which are registered in advance.

Furthermore, when the caller 10 uses the vehicle 100's hands-free calling system, the memory unit 7 stores the voiceprint information of the caller 11 outside the vehicle as user information. In this case, the controller 2 may pre-store the voiceprint information of the caller 11 outside the vehicle in the memory unit 7, or it may acquire the voice of the caller 11 outside the vehicle output from the speaker 21 during hands-free calling using the microphone 20 and store it in the memory unit 7 as the voiceprint information of the caller 11 outside the vehicle.

The controller 2 includes a microcomputer with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various circuits. The controller 2 comprises a call detection unit 3, an audio output control unit 4, an image analysis unit 5, and an audio analysis unit 6, all of which function by the CPU executing a program stored in the ROM using the RAM as a working area.

Furthermore, the call detection unit 3, voice output control unit 4, image analysis unit 5, and voice analysis unit 6 of the controller 2 may be partially or entirely composed of hardware such as ASICs (Application Specific Integrated Circuits) or FPGAs (Field Programmable Gate Arrays).

The call detection unit 3, voice output control unit 4, image analysis unit 5, and voice analysis unit 6 of the controller 2 each realize or execute the information processing functions described below. Note that the internal configuration of the controller 2 is not limited to the configuration shown in Figure 2; other configurations are also acceptable as long as they perform the information processing described later.

The call detection unit 3 detects the start and end of a call made by the caller 10. The call detection unit 3 detects the start and end of a call by analyzing at least one of the following: the audio inside the vehicle acquired by the microphone 20, the image captured by the camera 22, or the radio waves from a mobile terminal detected inside the vehicle.

Furthermore, the call detection unit 3 is connected to a mobile terminal inside the vehicle via USB®, Bluetooth®, or Wi-Fi®. When a call is initiated using the hands-free calling system, the unit detects the start and end of the call via USB, Bluetooth, or Wi-Fi. Upon detecting the start of a call, the call detection unit 3 notifies the voice output control unit 4 of this fact.

When the start of a call is notified, the voice output control unit 4 generates a cancellation tone from the transmitted voice input from the microphone 20 via the voice analysis unit 6 and outputs it through the speaker 21. At this time, the voice output control unit 4 can also read a masking tone from the masking tone information 71 in the storage unit 7 and output it through the speaker 21. Furthermore, when the end of a call is notified, the voice output control unit 4 stops outputting both the cancellation tone and the masking tone.

The image analysis unit 5 analyzes images of the caller 10 and passenger 12 captured by the camera 22, and detects speech made when a specific gesture of the caller 10 or passenger 12 is detected inside the vehicle, recognizing it as speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10.

Specific gestures include, for example, the gesture of the caller 10 looking towards the passenger 12, the gesture of the caller 10 facing the passenger 12, the gesture of the passenger 12 looking towards the caller 10, or the gesture of the passenger 12 facing the caller 10.

Furthermore, specific gestures may include the caller 10 moving the mobile device away from their mouth during a call, or covering the sound-receiving part of the mobile device. This allows the image analysis unit 5 to detect that the caller 10 and passenger 12 inside the vehicle may be in the process of initiating a brief conversation.

Furthermore, the call detection unit 3 detects speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, when the microphone is muted or the call is put on hold during a hands-free call via USB, Bluetooth, or Wi-Fi. This allows the call detection unit 3 to detect that the caller 10 and passenger 12 inside the vehicle may be about to begin a temporary conversation.

The image analysis unit 5 and the call detection unit 3, when they detect speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, notify the voice output control unit 4 of this. When the voice output control unit 4 is notified that speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, has been detected during a call, it stops outputting the cancellation and masking tones. This allows the voice output control unit 4 to make the transmitted voice inside the vehicle less audible to the passenger 12 without interfering with the temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

The voice analysis unit 6 analyzes the speech of the caller 10, which is collected by the microphone 20. When a specific word is included in the speech, it detects that speech as a message from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10. The specific word here is the name of the passenger 12 included in the caller 10's speech, or the name of the caller 10 included in the passenger 12's speech. This allows the voice analysis unit 6 to detect that the caller 10 and passenger 12 inside the vehicle may be about to begin a temporary conversation.

The voice analysis unit 6, upon detecting speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, notifies the voice output control unit 4 accordingly.

The voice output control unit 4 stops outputting cancellation and masking tones when it detects that a speech has been made from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, during a call. This allows the voice output control unit 4 to make the transmitted voice inside the vehicle less audible to the passenger 12 without interfering with the temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

[3. Processes executed by the controller]
Next, with reference to Figures 3 to 5, an example of processing performed by the controller of the audio signal processing device according to the embodiment will be described. Figures 3 to 5 are flowcharts showing an example of processing performed by the controller of the audio signal processing device according to the embodiment.

When a call is initiated inside the vehicle, controller 2 starts the process shown in Figure 3. As shown in Figure 3, controller 2 detects the start of a call by analyzing at least one of the following: the audio inside the vehicle acquired by microphone 20, the image captured by camera 22, or the radio waves from a mobile device detected inside the vehicle.

Furthermore, the controller 2 is connected to the mobile device inside the vehicle via USB, Bluetooth, or Wi-Fi, and detects the start of a call via USB, Bluetooth, or Wi-Fi when a call is initiated using the hands-free calling system.

When a call is initiated inside the vehicle, the controller 2 outputs at least one of a cancellation tone and a masking tone from the speaker 21 (step S101). This allows the voice signal processing device 1 to prevent the passenger 12 from becoming uncomfortable due to the speaker's voice.

Next, controller 2 executes the cancellation/masking sound output processing (step S102) and then terminates the processing. When controller 2 starts the cancellation/masking sound output processing, as shown in Figure 4, it first determines whether the call has ended (step S201). Controller 2 detects the end of the call using the same method as when detecting the start of the call.

Then, if controller 2 determines that the call has ended (step S201, Yes), it terminates the cancellation tone/masking tone output processing. If controller 2 determines that the call has not ended (step S201, No), it determines whether a specific gesture is present (step S202).

At this time, the controller 2 analyzes the image captured by the camera 22 to determine if a specific gesture occurred when the caller 10 or passenger 12 made a specific gesture, and determines if no specific gesture occurred when the caller 10 or passenger 12 did not make a specific gesture.

Specific gestures include, for example, the gesture of the caller 10 looking towards the passenger 12, the gesture of the caller 10 facing the passenger 12, the gesture of the passenger 12 looking towards the caller 10, or the gesture of the passenger 12 facing the caller 10. This allows the audio signal processing device 1 to detect that the caller 10 and passenger 12 inside the vehicle may be having a brief conversation during the call.

Furthermore, specific gestures may include the caller 10 moving the mobile device away from their mouth during a call, or covering the sound-receiving part of the mobile device. Even with these gestures, the audio signal processing device 1 can detect that the caller 10 and passenger 12 inside the vehicle may be having a brief conversation.

Then, if the controller 2 determines that a specific gesture has occurred (step S202, Yes), it determines whether or not it has detected speech from the caller 10 or the passenger 12 (step S203). If the controller 2 determines that it has not detected speech from the caller 10 or the passenger 12 (step S203, No), it proceeds to step S202.

Furthermore, if the controller 2 determines that it has detected speech from the caller 10 or the passenger 12 (step S203, Yes), it stops outputting the cancellation sound and the masking sound that are currently being output (step S206). This allows the audio signal processing device 1 to make the transmitted voice inside the vehicle less audible to the passenger 12 without interfering with the temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

Subsequently, controller 2 executes the cancellation sound/masking sound output suspension process (step S207), and then terminates the cancellation sound/masking sound output suspension process. A specific example of the cancellation sound/masking sound output suspension process will be described later with reference to Figure 5.

Thus, the controller 2 detects speech from the caller 10 or passenger 12 inside the vehicle as speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, when the camera 22 detects a specific gesture of the caller or passenger 12 inside the vehicle. This allows the audio signal processing device 1 to detect that the caller 10 and passenger 12 inside the vehicle may actually be initiating a temporary conversation.

Furthermore, when the controller 2 detects speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10, during a call, it stops outputting the cancellation sound and masking sound that are currently being output. This allows the audio signal processing device 1 to make the transmitted audio inside the vehicle less audible to the passenger 12 without interfering with the temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

Furthermore, if the controller 2 determines that there is no specific gesture (step S202, No), it determines whether or not a specific word is present in the utterance (step S204). The specific word is, for example, the name of passenger 12 included in the utterance of caller 10, or the name of caller 10 included in the utterance of passenger 12.

Then, if the controller 2 determines that a specific word is present in the utterance (step S204, Yes), it detects that specific word as an utterance from the caller 10 to the passenger 12, or from the passenger 12 to the speaker. This allows the voice signal processing device 1 to detect that there is a possibility of a temporary conversation between the caller 10 and the passenger 12 inside the vehicle.

Subsequently, controller 2 proceeds to step S206. Furthermore, if controller 2 determines that the utterance does not contain a specific word (step S204, No), it determines whether the microphone is muted or the call is put on hold in the hands-free calling system (step S205).

Then, if the controller 2 determines that the microphone is muted or the call is on hold in the hands-free calling system (step S205, Yes), it detects the speech of the caller 10 and passenger 12 at that time as speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10. This allows the voice signal processing device 1 to detect that the caller 10 and passenger 12 inside the vehicle may be having a temporary conversation.

Subsequently, controller 2 proceeds to step S206. Furthermore, if controller 2 determines that there is no microphone mute or call hold in the hands-free calling system (step S205, No), it proceeds to step S201.

Next, with reference to Figure 5, an example of the cancellation/masking sound output cessation processing performed by the controller 2 will be described. As shown in Figure 5, when the controller 2 starts the cancellation/masking sound output cessation processing, it first determines whether the call has ended or not (step S301). The controller 2 detects the end of the call in the same way as when detecting the start of the call.

Then, if controller 2 determines that the call has ended (step S301, Yes), it terminates the cancellation tone/masking tone output stop processing. If controller 2 determines that the call has not ended (step S301, No), it determines whether a specific gesture is present (step S302).

The specific gesture referred to here is the same gesture that is determined during the cancellation sound/masking sound output processing. If controller 2 determines that the specific gesture is present (step S302, Yes), it proceeds to step S301.

Furthermore, if Controller 2 determines that there is no specific gesture (Step S302, No), it determines whether a specific word is present in the utterance (Step S303). The specific word here is the same word determined during the cancellation sound/masking sound output processing. If Controller 2 determines that the specific word is present in the utterance (Step S303, Yes), it proceeds to Step S301.

Furthermore, if controller 2 determines that the utterance does not contain a specific word (step S303, No), it determines whether to mute the microphone or put the call on hold (step S304). The microphone mute or call hold operation here is the same as the microphone mute or call hold operation determined during the cancellation sound/masking sound output processing.

If controller 2 determines that the microphone is muted or the call is on hold (step S304, Yes), it proceeds to step S301. If controller 2 determines that the microphone is not muted or the call is not on hold (step S304, No), it outputs at least one of the cancellation sound and/or masking sound from speaker 21 (step S305).

Subsequently, controller 2 executes the cancellation sound/masking sound output processing (step S306). This cancellation sound/masking sound output processing is the same as the cancellation sound/masking sound output processing shown in Figure 4. Afterward, controller 2 terminates the cancellation sound/masking sound output stop processing.

The above-described embodiment is merely an example, and various modifications are possible. For example, the controller 2 may, during a call, detect the utterance of the caller 10, and then, within a predetermined time, detect the passenger 12's response to the caller 10's utterance. In this case, the controller 2 may detect the passenger 12's response as an utterance from the passenger 12 to the caller 10 and stop the output of the cancellation and masking tones.

Furthermore, the caller 10 tends to have a higher voice pitch when speaking to the person they are talking to than when speaking to the passenger 12. Therefore, the controller 2 may analyze the caller 10's voice tone and, if the caller 10's voice pitch becomes lower than the pitch at the start of the call, detect that speech from the caller 10 to the passenger 12 and stop outputting the cancellation and masking tones.

Furthermore, in the embodiment described above, the audio signal processing device 1 output a cancellation tone that is the opposite phase of the speech of the caller 10 inside the vehicle. However, during a hands-free call using the hands-free calling system, the device may also output a cancellation tone that is the opposite phase of the speech of the caller 11 outside the vehicle.

Furthermore, in the embodiment described above, the output of the cancellation and masking tones is stopped upon detection of speech from the caller 10 to the passenger 12, or from the passenger 12 to the caller 10. However, the output of the cancellation and masking tones may also be stopped upon detection of speech from an outside caller 11 to the passenger 12, or from the passenger 12 to the outside caller 11.

Specifically, the controller 2 detects a conversation between the outside caller 11 and the passenger 12 if the outside caller's speech includes the passenger 12's name. Alternatively, the controller 2 detects a conversation between the passenger 12 and the outside caller 11 if the passenger 12's speech includes the outside caller 11's name.

When controller 2 detects speech from an external caller 11 to a passenger 12, or from a passenger 12 to an external caller 11, it stops outputting cancellation or masking sounds.

For example, when a person 10 inside the car and a person 11 outside the car are talking using a hands-free calling system, a passenger 12 who is listening to the call might speak to the person 11 outside the car (the person 10 inside the car is talking to), or the person 11 outside the car might speak to the passenger 12 who is likely near the person 10 inside the car.

In such cases, since a conversation is expected to occur between the caller 11 outside the vehicle and the passenger 12, the output of cancellation or masking sounds can be stopped to prevent interference with the conversation between the caller 11 and the passenger 12.

Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents.

1. Audio signal processing unit 2. Controller 3. Call detection unit 4. Audio output control unit 5. Image analysis unit 6. Audio analysis unit 7. Memory unit 10. Caller 11. Caller outside the vehicle 12. Passenger 20. Microphone 21. Speaker 22. Camera 71. Masking sound information 72. Occupant information 100. Vehicle

Claims (8)

The system outputs a sound through the speaker to the passengers that makes it difficult for other passengers to hear the conversation audio of the person making the call inside the vehicle.
If the system detects speech from the caller to the passenger, or from the passenger to the caller, during a call, it stops the output of the sound.
Audio signal processing device. The system detects speech made when a specific gesture of the caller or passenger is detected by the camera, and this speech is detected as speech from the caller to the passenger, or from the passenger to the caller.
The audio signal processing device according to claim 1. The aforementioned specific gesture is,
The gesture of the caller looking towards the passenger, the gesture of the caller facing the passenger, the gesture of the passenger looking towards the caller, or the gesture of the passenger facing the caller.
The audio signal processing device according to claim 2. The aforementioned specific gesture is,
The gesture of the caller moving the mobile device away from their mouth while on a call, or covering the sound-receiving part of the mobile device,
The audio signal processing device according to claim 2. When an utterance contains a specific word, the utterance is detected as an utterance from the caller to the passenger, or from the passenger to the caller.
The audio signal processing device according to claim 1. The aforementioned specific word is,
The name of the passenger included in the utterance of the caller, or the name of the caller included in the utterance of the passenger.
The audio signal processing device according to claim 5. During a call using a hands-free calling system, if the microphone is muted or the call is put on hold, the system detects this as speech from the caller to the passenger, or from the passenger to the caller.
The audio signal processing device according to claim 1. The audio signal processing unit
The system outputs a sound through the speaker to the passengers that makes it difficult for other passengers to hear the conversation audio of the person making the call inside the vehicle.
If the system detects speech from the caller to the passenger, or from the passenger to the caller, during a call, it stops the output of the sound.
Audio signal processing method.