JP7707793B2 - SONIC WAVE OUTPUT DEVICE, SONIC WAVE OUTPUT METHOD, AND SONIC WAVE OUTPUT PROGRAM - Google Patents

Description

The present invention relates to a sonic output device, a sonic output method, and a sonic output program.

When listening to a speaker's voice without meeting face-to-face, or in an online conference, the volume and frequency of the voice differs from speaker to speaker, which can make it difficult for the listener to hear the speaker's voice. In addition, there are individual differences in the internal function of the listener's ear and how they perceive sound, and the frequency components or volume of sound waves that each listener finds easy to hear differ from listener to listener. This creates the problem that it is difficult for a listener to hear a speaker's voice without meeting face-to-face.

Patent Document 1 describes a communication device that identifies a speaker and makes the voice spoken by the identified speaker easier to hear compared to other speakers. Specifically, the communication device described in Patent Document 1 changes the voice signal processing of the voice to voice signal processing that is preset for each speaker, and outputs the voice that has been subjected to the changed voice signal processing.

JP 2014-233000 A

In the method described in Patent Document 1, it may be difficult for the listener to hear the voice that has been processed by the voice signal processing set for each speaker. This is because, as mentioned above, there are individual differences in the internal function of the listener's ear and how they perceive sound, and the frequency components or volume of sound waves that each listener finds easy to hear differ from listener to listener. Therefore, if the listener has difficulty hearing the voice, the communication device cannot make the speaker's voice easier to hear unless the listener changes the settings for the voice signal processing. However, in the method described in Patent Document 1, the communication device cannot output sound waves that have been adjusted to be easier for the listener to hear.

One example of the objectives of the present invention is to provide a sound wave output device, a sound wave output method, and a sound wave output program that enable sound waves that are adjusted to be easily audible to a user to be output from a sound wave output means.

In one aspect of the present invention, the sound wave output device includes a generating means for generating sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by the user by a sound wave output means that converts an acoustic signal into a sound wave and outputs it, and a control means for controlling the characteristics of the acoustic signal based on the voice of the other party and inputting it to the sound wave output means via an amplifying means so as to reduce the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by reference sound characteristic information relating to a reference sound having a frequency component that can be heard by the user, the frequency characteristics being set as being easy for the user to hear when the sound wave of the reference sound is reflected inside the user's ear.

In another aspect of the present invention, a sound wave output method generates sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by the user by a sound wave output means that converts an acoustic signal into a sound wave and outputs it, controls the characteristics of the acoustic signal based on the voice of the other party and inputs it to the sound wave output means via the amplification means so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by reference sound characteristic information regarding a reference sound having a frequency component that the user can hear, the sound wave of the reference sound being reflected inside the user's ear and set as easy for the user to hear, is reduced.

In another aspect of the present invention, the sound wave output program causes a computer to realize a generation function of generating sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output means that converts an acoustic signal into a sound wave and outputs it, and a control function of controlling the characteristics of the acoustic signal based on the voice of the other party and inputting it to the sound wave output means via the amplification means so as to reduce the discrepancy between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by reference sound characteristic information regarding a reference sound having a frequency component that the user can hear, the sound wave of the reference sound being reflected inside the user's ear and set as easy for the user to hear.

The sound wave output device, sound wave output method, and sound wave output program of the present invention make it possible to output sound waves from the sound wave output means that are adjusted to be easily audible to the user.

1 is a block diagram showing an example of the configuration of a sonic wave output device according to a first embodiment of the present invention. 5 is a flowchart showing an example of the operation of the sonic wave output device according to the first embodiment of the present invention. FIG. 11 is a block diagram showing an example of the configuration of a sonic wave output system according to a second embodiment of the present invention. FIG. 11 is a sequence diagram showing an example of the operation of the sonic wave output system according to the second embodiment of the present invention. FIG. 11 is a sequence diagram showing an example of the operation of the sonic wave output system according to the second embodiment of the present invention. 10 is a flowchart showing an example of the operation of an external device according to the second exemplary embodiment of the present invention. 10 is a flowchart showing an example of the operation of a sonic wave output device according to a second embodiment of the present invention. 10 is a flowchart showing an example of the operation of a sonic wave output device according to a second embodiment of the present invention. FIG. 11 is a block diagram showing an example of the configuration of a sonic wave output system according to a third embodiment of the present invention. FIG. 11 is a sequence diagram showing an example of the operation of a sonic wave output system according to a third embodiment of the present invention. 13 is a flowchart showing an example of the operation of a sonic wave output device according to a third embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of each embodiment of the present invention.

[First embodiment]
A first embodiment of the present invention will be described.

Figure 1 is a block diagram showing an example of the configuration of a sonic wave output device 1 according to this embodiment.

The sonic wave output device 1 of this embodiment includes a generation unit 11 and a control unit 12.

For example, the sonic wave output device 1 is connected to an earphone (not shown) that includes a sonic wave output means (not shown) that converts an acoustic signal into a sound wave and outputs it, and a sonic wave input means (not shown) that receives input of a sound wave reflected from inside the user's ear. Alternatively, the sonic wave output device 1 is connected to a headphone that includes a sonic wave output means that converts an acoustic signal into a sound wave and outputs it, and a sonic wave input means that receives input of a sound wave reflected from inside the user's ear. The sonic wave output means is, for example, a speaker. The sonic wave output means converts an acoustic signal into a sound wave and outputs it. The acoustic signal input to the sonic wave output means is, for example, an audio signal that indicates the voice of a speaker (speaker) other than the user. The speaker's voice is live voice during a call or meeting, voice recorded in memory, or voice from an audio signal received via a network. The sonic wave input means is, for example, a microphone.

The generator 11 generates sound wave characteristic information indicating the characteristics of sound waves reflected inside the user's ear among the sound waves heard by the user by a sound wave output means that converts an acoustic signal into a sound wave and outputs it. Specifically, the generator 11 generates the sound wave characteristic information from the output of a microphone to which sound waves reflected inside the ear are input.

The control unit 12 controls the characteristics of the acoustic signal based on the voice of the other party so that the deviation between the frequency characteristics indicated by the acoustic wave characteristics information and the frequency characteristics indicated by the reference sound characteristic information relating to a reference sound having frequency components that can be heard by the user is reduced. The frequency characteristics indicated by the reference sound characteristic information relating to the reference sound are the characteristics of the sound waves of the reference sound that are reflected inside the user's ear and are set to be easy for the user to hear. The control unit 12 inputs the acoustic signal with the controlled characteristics to the sound wave output means via an amplification means (not shown).

In this way, the sound wave output device 1 generates sound wave characteristic information of the sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output means that converts the sound signal into sound waves and outputs them. The sound wave output device 1 controls the characteristics of the sound signal based on the voice of the other party so as to reduce the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are the frequency characteristics of the sound waves of the reference sound reflected inside the user's ear and set as easy for the user to hear. The sound wave output device 1 inputs the sound signal with the controlled characteristics to the sound wave output means via the amplification means. As a result, the sound wave output means can output to the user sound waves based on the sound signal whose characteristics have been adjusted in accordance with the reference sound characteristic information.

Next, an example of the operation of the sonic wave output device 1 of this embodiment will be described with reference to FIG. 2. FIG. 2 is a flowchart showing an example of the operation of the sonic wave output device 1.

The generation unit 11 generates sound wave characteristic information indicating the characteristics of sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output means that converts an acoustic signal into a sound wave and outputs it (step S101).

The control unit 12 controls the characteristics of an acoustic signal based on the voice of the other party and inputs it to the sound wave output means via the amplification means (step S102). In step S102, the control unit 12 controls the characteristics of the acoustic signal so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information becomes small.

In this way, the sound wave output device 1 inputs an acoustic signal whose characteristics have been controlled to the sound wave output means so that the sound waves that are easy for the user to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. Therefore, the sound wave output device 1 can output sound waves that have been adjusted to be easy for the user to hear from the sound wave output means.

[Second embodiment]
Next, the sonic wave output device 3 according to the second embodiment of the present invention will be specifically described.

FIG. 3 is a block diagram showing an example of the configuration of a sonic output system according to a second embodiment of the present invention. In the second embodiment, the sonic output system includes a sonic output device 3. In the second embodiment, the generation unit 31 corresponds to, for example, the generation unit 11 of the first embodiment. In the second embodiment, the control unit 32, the determination unit 36, and the reference sound characteristic information storage unit 37 correspond to, for example, the control unit 12 of the first embodiment. The second embodiment further differs from the sonic output device 1 of the first embodiment in that the sonic output device 3 includes a sonic output unit 33, a receiving unit 34, and a sonic input unit 35.

The sound wave output system of this embodiment is a conference communication system. Before the start of a conference, the sound wave output device 3 performs initial settings to generate reference sound characteristic information indicating the characteristics of sound waves reflected inside the user's ear that are easy for the user to hear. When performing the initial settings described below, the sound wave output device 3 is connected to the external device 2 by wireless communication or wired communication, as shown in FIG. 3. After the initial settings of the sound wave output device 3 are completed, the external device 2 may no longer be connected to the sound wave output device 3. In other words, the external device 2 may be disconnected from the sound wave output device 3.

The sonic output device 3 is connected to a sonic input device (sound wave input device 4-1 in this example) that receives voice input from a user (user A1) via wired or wireless communication. As shown in FIG. 3, the sonic output device 3 is connected to sonic input devices (sound wave input devices 4-2, ..., 4-n (n is an integer of 2 or more)) that receive voice input from other users (users A2 to An) via wired or wireless communication. The sonic input devices (sound wave input devices 4-2 to 4-n in this example) that receive voice input from other users (users A2 to An) may be connected to the sonic output device 3 after the above-mentioned initial setting is completed.

The external device 2 performs initial settings of the sound wave output device 3. The external device 2 is, for example, a smartphone. Alternatively, the external device 2 may be a computer. Application software that realizes the functions of an input/output unit 21, a transmission unit 22, and a determination unit 23, which will be described later, is pre-installed in the external device 2. Details of the initial settings using the external device 2 will be described later.

The sound wave output device 3 is, for example, a headphone. Alternatively, the sound wave output device 3 may be an earphone. Thus, in this embodiment, the sound wave output device 3 is built into the headphone or the earphone.

The sonic output device 3 is used, for example, in online conferences or conferences involving two or more people over the phone. For example, each participant in an online conference uses the same sonic output device as the sonic output device 3 of user A1, and a sonic input device that receives voice input. In this example, among the participants in the online conference, user A1 of the sonic output device 3 inputs voice using the sonic input device 4-1. Each of the other users (users A2 to An) inputs voice using one of the sonic input devices 4-2 to 4-n. The receiver 34 of the sonic output device 3 receives a voice signal, which is a signal corresponding to the voice, from the sonic input device that receives the voice input of the other users (in this example, each of the sonic input devices 4-2 to 4-n).

The receiving unit 34 may receive an audio signal corresponding to the voice from a sound wave input device (sound wave input device 4-1 in this example) that receives the voice input of user A1. The receiving unit 34 may transmit the voice signal of user A1 to the sound wave output device of the other users (user A2 to user An). In this embodiment, the sound wave input device 4-1 transmits the voice signal to the sound wave output device of the other users (user A2 to user An).

The sound wave output device 3 of this embodiment performs the above-mentioned initial setting. In the initial setting, the control unit 32 causes the sound wave output unit 33 to output a reference sound based on an output request notification from the external device 2 requesting the output of a reference sound for the initial setting. The reference sound is a sound wave used to judge whether or not a sound is easy to hear by the user. The reference sound is a plurality of types of sound waves having different frequencies and volumes. The sound wave input unit 35 inputs, for each of the reference sounds, sound waves output from the sound wave output unit 33 to the inside of the ear and reflected inside the ear, and outputs an input acoustic signal indicating the input sound wave to the generation unit 31. In the initial setting, the receiving unit 34 receives evaluation information indicating whether or not each of the reference sounds is easy to hear from the external device 2. The evaluation information includes information indicating the result of the user's judgment of whether or not the reference sound is easy to hear. The generation unit 31, which will be described later, stores the evaluation information and the reference sound characteristic information in the reference sound characteristic information storage unit 37. The reference sound characteristic information may include the evaluation information. The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds. In the initial setting, the sound wave output device 3 stores, in the reference sound characteristic information storage unit 37, reference sound characteristic information including frequency information indicating the frequency characteristics of the sound waves of the reference sound reflected inside the user's ear. When used in an online conference, the sound wave output device 3 controls the characteristics of the sound signal.

Also, when the initial setting is completed and an online conference is started, the receiving unit 34 of the sound wave output device 3 receives a voice signal, which is an acoustic signal based on the voice of the other user An from the other user A2, who is the call partner, from the sound wave input devices 4-2 to 4-n. The acoustic signal is input to the sound wave output unit 33 via the control unit 32. Immediately after the start of the online conference, the control unit 32 has not yet controlled the characteristics of the acoustic signal so that the sound waves that user A1 finds easy to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. The sound wave output unit 33 converts the acoustic signal input from the control unit 32 into a sound wave and outputs it to the inside of the ear of user A1. The sound wave input unit 35 inputs the sound waves of the call partner that are output from the sound wave output unit 33 to the inside of the ear and reflected inside the ear, and outputs them to the generation unit 31. The generation unit 31 generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves output by the sound wave output unit 33 to the inside of the ear of user A1. Next, the sonic wave output device 3 executes the following process using the determination unit 36, the reference sound characteristic information storage unit 37, and the control unit 32. This process controls the characteristics of the acoustic signal so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the characteristics of the sound wave reflected inside the user's ear, which is the frequency characteristic indicated by the reference sound characteristic information and is easy for user A1 to hear, is reduced. The details of the process will be described later. The control unit 32 inputs the acoustic signal with the controlled characteristics to the sonic wave output unit 33 via an amplifier (not shown).

[Configuration of sound wave input device]
The sonic wave input device 4-1 includes a sonic wave input unit 41-1 and a transmission unit 42-1. The sonic wave input device 4-2 includes a sonic wave input unit 41-2 and a transmission unit 42-2. The sonic wave input device 4-n includes a sonic wave input unit 41-n and a transmission unit 42-n. Thus, each of the sonic wave input devices 4-1 to 4-n includes a sonic wave input unit and a transmission unit.

The sonic wave input devices 4-1 to 4-n are, for example, smartphones. Alternatively, the sonic wave input devices may be built into external devices. For example, the sonic wave input device 4-1 may be built into the external device 2 used by the user A1. Specifically, the external device 2 may further include the function of a sonic wave input unit 41-1, which will be described later, and the transmission unit 22, which will be described later, may realize the function of a transmission unit 42-1 of the sonic wave input device 4-1, which will be described later.

Each of the sound wave input devices 4-1 to 4-n receives voice input from the participants of the online conference (users A1 to An) and outputs a voice signal that is a signal corresponding to the voice.

Here, the configuration of the sonic wave input devices 4-1 to 4-n of this embodiment will be described in detail, taking the sonic wave input device 4-1 as a representative.

The sound wave input unit 41-1 receives voice input from user A1 and generates a voice signal that is a signal corresponding to the voice. The sound wave input unit 41-1 outputs the generated voice signal to the transmission unit 42-1.

The sound wave input unit 41-1 inputs the voice signal of user A1 to the transmission unit 42-1. The transmission unit 42-1 transmits the input voice signal to a sound wave output device (in this example, a sound wave output device (not shown) other than the sound wave output device 3) used by another user participating in the online conference. The sound wave input device 4-1 that receives the voice input of user A1 does not output a voice signal to the sound wave output device 3 used by user A1.

[Configuration of external device 2]
The configuration of the external device 2 of the present embodiment will be described in detail with reference to Fig. 3. The external device 2 includes an input/output unit 21, a transmission unit 22, and a determination unit 23.

The input/output unit 21 receives an instruction to start initial settings in response to a user operation. The input/output unit 21 outputs the instruction to start initial settings to the transmission unit 22.

An instruction to start the initial setting is input to the transmission unit 22 from the input/output unit 21. When the instruction to start the initial setting is input, the transmission unit 22 transmits an output request notification to the sound wave output device 3, requesting the output of an initial reference sound wave for the initial setting. The reference sound is multiple types of sound waves with different frequencies and volumes.

The determination unit 23 receives frequency information about the sound waves of the reference tones and volume information about the reference tones from the transmission unit 22. The frequency information is information indicating one or more frequencies. This frequency information and volume information is characteristic information of the reference tones for which evaluation information has been transmitted, input by the transmission unit 22 when evaluation information described later is transmitted to the sound wave output device 3. When the frequency information and volume information are input, the determination unit 23 determines whether evaluation information has been generated for all the reference tones. When it is determined that evaluation information has been generated for all the reference tones, the determination unit 23 outputs a notification of the end of the initial settings to the transmission unit 22. Furthermore, when there is a reference tones for which evaluation information has not been generated among the sound waves of all the reference tones, the determination unit 23 associates the frequency information and volume information of the reference tones for which evaluation information has not been generated, and outputs them to the transmission unit 22.

The transmission unit 22 receives frequency information and volume information from the determination unit 23. The transmission unit 22 transmits an output request notification for initial setting, including the input frequency information and volume information, to the sound wave output device 3. As a result, the external device 2 causes the sound wave output unit 33 of the sound wave output device 3 to output a sound wave of the reference sound.

The transmission unit 22 also receives an end notification of the initial settings from the determination unit 23. The transmission unit 22 transmits the end notification of the initial settings to the sound wave output device 3.

When an instruction to start initial setup is input, the input/output unit 21 may receive reference sound characteristic information in response to a user's operation. When the reference sound characteristic information is input to the input/output unit 21, the transmission unit 22 may transmit the reference sound characteristic information to the sound wave output device 3.

By transmitting the output request notification, the sound wave of the reference sound is output from the sound wave output unit 33 of the sound wave output device 3 to the inside of the ear of the user A1. The input/output unit 21 of the external device 2 receives an operation input for each of the sound waves of the multiple reference sounds, in response to an operation by the user, regarding an evaluation indicating whether the reference sound is easy to hear or not, and generates evaluation information corresponding to the operation input. The evaluation information may include the following information. The evaluation information may include information indicating whether the user can easily hear a sound of the pitch of the reference sound output from the sound wave output unit 33. Furthermore, when the evaluation information indicates that the sound is difficult to hear, the input/output unit 21 may include information indicating that the volume of the sound waves is high or that the volume is low in the evaluation information. The input/output unit 21 outputs the generated evaluation information to the transmission unit 22.

The transmission unit 22 transmits the generated evaluation information to the sonic output device 3. Furthermore, when transmitting the evaluation information to the sonic output device 3, the transmission unit 22 outputs to the determination unit 23 the frequency information and the volume information included in the output request notification transmitted immediately before the evaluation information was generated, in association with each other.

[Configuration of sonic wave output device 3]
Next, the configuration of the sonic wave output device 3 of the present embodiment will be described in detail with reference to Fig. 3. The sonic wave output device 3 includes a generator 31 and a controller 32.

The sound wave output unit 33 is connected to the control unit 32. The receiving unit 34 is connected to the generation unit 31 and the control unit 32. The sound wave input unit 35 is connected to the generation unit 31. The determination unit 36 is connected to the generation unit 31, the control unit 32, and the reference sound characteristic information storage unit 37. The reference sound characteristic information storage unit 37 is connected to the generation unit 31 and the determination unit 36.

The sonic wave output device 3 may be realized as two or more devices. For example, among the components of the sonic wave output device 3, the sonic wave output unit 33 and the sonic wave input unit 35 may be earphones or headphones, and the other components may be realized as an information processing device. In addition, the earphones or headphones may be implemented with the left ear sonic wave input unit 351 and the left ear sonic wave output unit 331 implemented integrally, and the right ear sonic wave input unit 352 and the right ear sonic wave output unit 332 implemented integrally.

First, the components of the sonic wave output device 3 of this embodiment will be described in detail when performing initial settings.

The receiving unit 34 receives an output request notification for initial setup, including frequency information and volume information. The receiving unit 34 associates the frequency information with the volume information and outputs the information to the control unit 32.

The control unit 32 inputs an acoustic signal to the sound wave output unit 33 via an amplifier means to output a reference sound wave having a frequency indicated in the input frequency information and a volume indicated in the input volume information.

The sound wave output unit 33 includes a left ear sound wave output unit 331 and a right ear sound wave output unit 332.

The left ear sound wave output unit 331 receives an acoustic signal from the control unit 32 to output a sound wave of the reference sound. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the left ear of the user. For example, the left ear sound wave output unit 331 is a speaker for the left ear that converts an electrical signal into a sound wave.

The right ear sound wave output unit 332 receives an acoustic signal from the control unit 32 to output a sound wave of the reference sound. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the right ear of the user. For example, the right ear sound wave output unit 332 is a speaker for the right ear.

The sound wave input unit 35 includes a left ear sound wave input unit 351 and a right ear sound wave input unit 352.

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. For example, the input acoustic signal is a signal indicating the waveform of the sound waves reflected inside the ear. For example, the left ear sound wave input unit 351 is a microphone for the left ear. Specifically, the left ear sound wave input unit 351 is a bone conduction microphone for the left ear.

The right ear sound wave input unit 352 inputs sound waves reflected inside the user's right ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. For example, the right ear sound wave input unit 352 is a microphone for the right ear. Specifically, the right ear sound wave input unit 352 is a bone conduction microphone for the right ear.

In the case of initial settings, the generation unit 31 receives an input sound signal related to the sound signal of the reference sound, which is generated by the control unit 32 based on the frequency information and volume information transmitted from the transmission unit 22, from the sound wave input unit 35. The generation unit 31 analyzes the signal strength of each frequency component constituting the input input sound signal. The generation unit 31 performs frequency resolution of the input sound signal to generate reflected sound wave characteristic information indicating frequency characteristics including information indicating a frequency distribution or information indicating frequency components according to the analysis result. The reflected sound wave characteristic information is frequency information indicating the frequency characteristics of sound waves reflected inside the user's ear from the reference sound. The generation unit 31 outputs sound wave characteristic information including frequency information. For example, the sound wave characteristic information includes frequency information indicating at least one frequency. The generation unit 31 also detects the volume at that frequency based on the intensity of the amplitude of the input sound signal. Therefore, the reflected sound wave characteristic information in this embodiment includes volume information indicating the volume of the sound wave indicated in the input sound signal.

The left ear sound wave input unit 351 and the right ear sound wave input unit 352 may generate reflected sound wave characteristic information. When the left ear sound wave input unit 351 and the right ear sound wave input unit 352 generate reflected sound wave characteristic information, the generation unit 31 may acquire sound wave characteristic information from each of the left ear sound wave input unit 351 and the right ear sound wave input unit 352, rather than generating the reflected sound wave characteristic information.

The receiving unit 34 also receives evaluation information. The receiving unit 34 associates the evaluation information with frequency information and volume information included in the output request notification received before receiving the evaluation information, and outputs the associated information to the generating unit 31.

The generating unit 31 receives frequency information, volume information, and evaluation information from the receiving unit 34. The frequency information and volume information input from the receiving unit 34 are output acoustic information that indicates the frequency characteristics of the acoustic signal when each of the reference sounds is output.

The receiving unit 34 also receives a notification of completion of the initial settings from the external device 2. When a notification of completion of the initial settings is received, the receiving unit 34 outputs the notification of completion of the initial settings to the generating unit 31.

When a notification of the end of the initial settings is input, the generation unit 31 performs the following operation. The generation unit 31 stores, as reference sound characteristic information, output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds, in the reference sound characteristic information storage unit 37. In this embodiment, the generation unit 31 associates the reference sound characteristic information with the evaluation information and stores them in the reference sound characteristic information storage unit 37.

The reference sound characteristic information storage unit 37 stores evaluation information and reference sound characteristic information in association with each other by the generation unit 31. The reference sound characteristic information storage unit 37 stores at least reference sound characteristic information associated with evaluation information indicating that it is easy to hear. Specifically, the reference sound characteristic information storage unit 37 of this embodiment stores reference sound characteristic information associated with evaluation information indicating that it is easy to hear, and reference sound characteristic information associated with evaluation information indicating that it is difficult to hear. The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds is output. The reference sound characteristic information also includes reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds.

The reference sound characteristic information storage unit 37 may store reference sound characteristic information for the left ear and reference sound characteristic information for the right ear. When the reference sound characteristic information for the left ear and the reference sound characteristic information for the right ear are stored in the reference sound characteristic information storage unit 37, the control unit 32 may operate as follows. The control unit 32 may control the characteristics of the acoustic signal for the left ear sound wave output unit 331 based on the reference sound characteristic information for the left ear, and control the characteristics of the acoustic signal for the right ear sound wave output unit 332 based on the reference sound characteristic information for the right ear. In this way, the sound wave output device 3 may individually control each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 when performing initial setup and when used in an online conference.

Next, each component of the sound wave output device 3 of this embodiment will be described in detail when used in an online conference.

The receiver 34 receives audio signals from the sonic wave input devices (in this example, the sonic wave input devices 4-2 to 4-n). The receiver 34 outputs the audio signals to the controller 32.

The control unit 32 receives an audio signal from the receiving unit 34. The control unit 32 also receives a notification from the determining unit 36 requesting control of the characteristics of the audio signal. If the notification requesting control of the characteristics of the audio signal is not received, the control unit 32 inputs an audio signal for outputting sound waves based on the audio signal to the sound wave output unit 33 via an amplifier. If the notification requesting control of the characteristics of the audio signal is received, the control unit 32 controls the characteristics of the audio signal to be output for the audio signal from the receiving unit 34. Specifically, the control unit 32 converts the audio signal into a frequency domain signal or data via a Fourier transform circuit. The control unit 32 changes the characteristics of the audio signal based on the voice of the other party for the frequency domain signal or data so that the deviation between the frequency characteristics indicated by the reference sound characteristic information and the characteristics of the sound waves that are easy for the user to hear and the characteristics of the sound waves reflected inside the user's ear becomes small. For example, the control unit 32 changes the frequency and the volume, which is the magnitude of the frequency and the volume. After that, the control unit 32 restores the frequency domain signal or data whose characteristics have been controlled by the change to an audio signal on the time axis using an inverse Fourier transform circuit. The control unit 32 then inputs the acoustic signal, which is an audio signal with controlled characteristics, to the sound wave output unit 33 via the amplification means.

For example, when sound waves having frequency characteristics that are difficult for the user to hear are output from the sound wave output unit 33, a notification requesting control of the characteristics of the sound signal is input from the determination unit 36. The control unit 32 controls the characteristics of the sound signal for the audio signal input from the receiving unit 34 so that the frequency characteristics indicated by the reference sound characteristic information and the sound waves that are easy for the user to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. Specifically, the control unit 32 controls the sound signal so that the frequency characteristics of the sound wave characteristic information are less likely to deviate from the frequency characteristics of the reference sound characteristic information. For example, when the frequency characteristics of the sound wave characteristic information are such that the signal strength of a certain frequency component is weaker than the signal strength of the frequency component of the sound waves that are easy for the user to hear and are reflected inside the user's ear, the control unit 32 controls the characteristics of the sound signal as follows. The control unit 32 controls the characteristics of the sound signal so as to increase the signal strength of the certain frequency component.

The sound wave output unit 33 converts the acoustic signal input from the control unit 32 into a sound wave and outputs it.

The sound wave input unit 35 inputs sound waves reflected inside the user's ear and outputs an input acoustic signal representing the input sound waves to the generation unit 31.

The generator 31 generates sound wave characteristic information according to the analysis results of the signal strength of each frequency component that constitutes the input acoustic signal input from the sound wave input unit 35. The generator 31 outputs the generated sound wave characteristic information to the determiner 36.

Based on the input sound wave characteristic information, the determination unit 36 determines whether or not sound waves that are easy for the user to hear are output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. When the frequency characteristic indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is easy for the user to hear and stored in the reference sound characteristic information storage unit 37 deviates from the frequency characteristic indicated by the sound wave characteristic information by a predetermined threshold value or more, the determination unit 36 makes the following determination. The determination unit 36 determines that sound waves that are difficult for the user to hear are output by the sound wave output unit 33. When it is determined that sound waves that are difficult for the user to hear are output by the sound wave output unit 33, the determination unit 36 performs the following operation. The determination unit 36 specifies output acoustic information associated with the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37, associated with evaluation information indicating that the frequency characteristic indicated by the sound wave characteristic information determined to be difficult to hear is easy to hear. The determination unit 36 outputs a notification requesting control of the characteristics of the acoustic signal to the control unit 32. The notification requesting control of the characteristics of the audio signal includes the specified output audio information.

Furthermore, when the deviation between the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information stored in the reference sound characteristic information storage unit 37 that is deemed easy for the user to hear and the frequency characteristics indicated by the sound wave characteristic information is less than a predetermined threshold, the determination unit 36 makes the following determination: The determination unit 36 determines that sound waves that are easy for the user to hear are being output by the sound wave output unit 33. When it determines that sound waves that are easy for the user to hear are being output by the sound wave output unit 33, it does not perform processing to output a notification to the control unit 32 requesting control of the characteristics of the acoustic signal.

As described above, when a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 controls the characteristics of the acoustic signal so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information becomes small, and inputs the acoustic signal to the sound wave output unit 33 via the amplification means. Specifically, the control unit 32 controls the characteristics of the acoustic signal so that the deviation from the frequency characteristics indicated by the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal becomes small.

The sonic output device 3 may output a notification to the control unit 32 requesting control of the characteristics of the sound signal including two or more pieces of output sound information for the sound wave characteristic information of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. The control unit 32 may control the sound signal based on the frequency characteristics of each of the multiple pieces of output sound information so that the frequency characteristics of the sound signal become frequency characteristics corresponding to the frequency characteristics of the sound wave characteristic information and the frequency characteristics of the reference sound characteristic information.

As described above, the sound wave output device 3 generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 that converts the sound signal into a sound wave and outputs it. The sound wave output device 3 controls the characteristics of the sound signal based on the voice of the other party so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are the frequency characteristics of the sound wave of the reference sound reflected inside the user's ear and set as easy to hear for the user, is reduced. In addition, the sound wave output device 3 inputs the sound signal with the controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output to the user a sound wave based on the sound signal whose characteristics have been adjusted according to the reference sound characteristic information. This allows the sound wave output device 3 to output from the sound wave output unit 33 a sound wave that has been adjusted to be easy to hear for the user.

[Operation]
Next, an example of the operation of the sonic wave output system of this embodiment will be described with reference to FIGS.

FIGS. 4 and 5 are sequence diagrams showing an example of the operation of the sonic output system. FIG. 6 is a flowchart showing an example of the operation of the external device 2. FIGS. 7 and 8 are flowcharts showing an example of the operation of the sonic output device 3.

First, referring to FIG. 4, the operation of the external device 2 to perform initial settings of the sound wave output device 3 will be described together with the operation of the sound wave output device 3.

The input/output unit 21 of the external device 2 receives an instruction to start initial settings in response to a user operation (step S201).

An instruction to start the initial setup is input to the transmission unit 22 from the input/output unit 21. The transmission unit 22 transmits an output request notification to the sound wave output device 3, which requests the output of a reference sound for the initial setup, the output request notification including frequency information indicating the initial frequency and volume information indicating the initial volume of sound waves with different characteristics to be evaluated (step S202).

The receiver 34 of the sonic wave output device 3 receives an output request notification for initial setup, including frequency information and volume information. The receiver 34 associates the frequency information with the volume information and outputs the information to the controller 32.

The control unit 32 receives frequency information and volume information from the receiving unit 34. The control unit 32 inputs, to the left ear sound wave output unit 331 and the right ear sound wave output unit 332, an acoustic signal for outputting sound waves of the frequency indicated in the input frequency information and of the volume indicated in the input volume information.

The left ear sound wave output unit 331 receives an acoustic signal from the control unit 32 for outputting sound waves of the reference sound. The right ear sound wave output unit 332 receives an acoustic signal from the control unit 32 for outputting sound waves of the reference sound. The left ear sound wave output unit 331 converts the acoustic signal into sound waves and outputs the sound waves into the inside of the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves into the inside of the user's right ear (step S203). This allows the user to hear the sound waves of the reference sound based on the output request notification.

The left ear sound wave input unit 351 inputs sound waves reflected inside the left ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31. The right ear sound wave input unit 352 inputs sound waves reflected inside the right ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31 (step S204).

The generation unit 31 receives an input acoustic signal from the left ear sound wave input unit 351. The generation unit 31 also receives an input acoustic signal from the right ear sound wave input unit 352. The generation unit 31 analyzes the signal strength of each frequency component that constitutes the input acoustic signal (step S205). The generation unit 31 generates reflected sound wave characteristic information indicating frequency characteristics according to the analysis result.

The input/output unit 21 of the external device 2 receives an operational input for an evaluation indicating whether the sound is easy to hear in response to a user's operation, and generates evaluation information corresponding to the operational input (step S206). The input/output unit 21 outputs the generated evaluation information to the transmission unit 22.

The transmission unit 22 transmits the evaluation information generated in step S206 to the sonic output device 3 (step S207). The transmission unit 22 outputs to the determination unit 23 the frequency information and the volume information included in the output request notification transmitted immediately before the evaluation information was generated, in association with each other.

Note that the operations of steps S206 and S207 are performed in parallel with the operations of steps S204 and S205. The order in which the operations of steps S204 to S207 are performed is not limited to the order shown in FIG. 4.

The receiver 34 of the sonic wave output device 3 receives the evaluation information. The receiver 34 associates the evaluation information with the frequency information and volume information contained in the output request notification received before receiving the evaluation information, and outputs the association to the generator 31.

The generating unit 31 receives frequency information, volume information, and evaluation information from the receiving unit 34. The frequency information and volume information input from the receiving unit 34 are output acoustic information that indicates the frequency characteristics of the acoustic signal when each of the reference sounds is output.

The determination unit 23 of the external device 2 determines whether or not evaluation information has been generated by the input/output unit 21 for all the reference sounds (step S208). If evaluation information has been generated for all the reference sounds, the determination unit 23 outputs an end notification of the initial settings to the transmission unit 22. If an end notification of the initial settings is input, the transmission unit 22 transmits an end notification of the initial settings to the sound wave output device 3.

The receiver 34 of the sound wave output device 3 receives a notification of the end of the initial settings from the external device 2. When the receiver 34 receives a notification of the end of the initial settings, it outputs the notification of the end of the initial settings to the generator 31.

When a notification of the end of the initial settings is input, the generation unit 31 associates the evaluation information with the reference sound characteristic information for each reference sound and stores it in the reference sound characteristic information storage unit 37. The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each reference sound is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each reference sound.

In step S208, if there is a reference sound for which no evaluation information is generated among all the reference sounds, the determination unit 23 of the external device 2 associates the frequency information and volume information of the reference sound for which no evaluation information is generated and outputs them to the transmission unit 22. When the frequency information and volume information are input, the transmission unit 22 of the external device 2 performs the operation of step S202. That is, the transmission unit 22 transmits a reference sound output request notification for initial setup, including the input frequency information and volume information, to the sound wave output device 3. Then, the external device 2 performs the operations of steps S206, S207, and S208. Furthermore, the sound wave output device 3 performs the operations of steps S203 to S205 until it receives a notification of completion of the initial setup.

Next, referring to FIG. 5, the operation of the sonic wave output device 3 controlling the characteristics of the acoustic signal and inputting the audio signal received from the sonic wave input device (sound wave input devices 4-2 to 4-n in this example) to the sonic wave output unit 33 will be described.

During an online conference, the sound wave output device 3 of this embodiment generates sound wave characteristic information of sound waves that have been previously output and that have been reflected inside the ear. The sound wave output device 3 then controls the characteristics of the acoustic signal to be output next to the sound wave output unit 33.

In FIG. 5, an example is described in which a notification requesting control of the characteristics of the acoustic signal has not been input to the control unit 32 of the sonic output device 3 before the sonic output device 3 receives the audio signals transmitted from the sonic input devices 4-2 to 4-n in step S302 described below.

Each of the sonic input devices 4-2 to 4-n receives voice input from another user (step S301). Each of the sonic input devices 4-2 to 4-n that has received the voice input generates a voice signal that is a signal corresponding to the voice. Each of the sonic input devices 4-2 to 4-n that has generated the voice signal transmits the voice signal to a sonic output device (including the sonic output device 3 in this example) other than the sonic output device of the other user who input the voice to the sonic input device (step S302).

The receiver 34 of the sonic wave output device 3 receives audio signals from the sonic wave input devices (in this example, the sonic wave input devices 4-2 to 4-n). The receiver 34 outputs the audio signals to the control unit 32.

The control unit 32 receives an audio signal from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting sound waves based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier means.

The left ear sound wave output unit 331 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's right ear (step S303).

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31. The right ear sound wave input unit 352 inputs sound waves reflected inside the user's right ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31 (step S304).

The generation unit 31 receives input acoustic signals from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 31 analyzes the signal strength of each frequency component that constitutes the input input acoustic signal (step S305). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 outputs to the determination unit 36 the sound wave characteristic information generated from the input acoustic signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right ear sound wave input unit 352.

Based on the input sound wave characteristics information, the determination unit 36 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S306). Specifically, the determination unit 36 determines whether the frequency characteristics indicated by the reflected sound wave characteristics information included in the reference sound characteristic information that is easy for the user to hear and stored in the reference sound characteristic information storage unit 37 deviate by a predetermined threshold or more from the frequency characteristics indicated by the sound wave characteristics information. If the deviation is equal to or greater than the predetermined threshold, the sound wave corresponding to the input sound wave characteristics information is a sound wave that is difficult for the user to hear, and if the deviation is less than the predetermined threshold, the sound wave is easy for the user to hear.

When the determination unit 36 determines in step S306 based on the input sound wave characteristic information that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33, the determination unit 36 performs the following operation. For the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear, the determination unit 36 identifies output sound information associated with reflected sound wave characteristic information associated with evaluation information indicating ease of hearing and stored in the reference sound characteristic information storage unit 37. Alternatively, the determination unit 36 may calculate the similarity between the frequency characteristics indicated by the input sound wave characteristic information and the frequency characteristics indicated by the reflected sound wave characteristic information associated with evaluation information indicating ease of hearing, and identify reflected sound wave characteristic information whose similarity is higher than a threshold value. The determination unit 36 may also identify output sound information associated with the identified reflected sound wave characteristic information and stored in the reference sound characteristic information storage unit 37.

The determination unit 36 outputs a notification to the control unit 32 requesting control of the characteristics of the audio signal, including the identified output audio information.

In addition, when the determination unit 36 determines in step S306 based on the input sound wave characteristic information that sound waves that are easy for the user to hear are being output by the sound wave output unit 33, the determination unit 36 does not identify the output sound information. In addition, the determination unit 36 does not perform processing to output a notification to the control unit 32 requesting control of the characteristics of the sound signal.

The control unit 32 receives an audio signal from the receiving unit 34. If no notification is received requesting control of the characteristics of the audio signal, the control unit 32 generates an audio signal for outputting sound waves based on the audio signal. On the other hand, if a notification is received from the determining unit 36 requesting control of the characteristics of the audio signal, the control unit 32 controls the characteristics of the audio signal to be output for the audio signal from the receiving unit 34. Specifically, the control unit 32 controls the characteristics of the audio signal to be input to the sound wave output unit 33 so that the deviation from the output audio information included in the notification requesting control of the characteristics of the audio signal is reduced. As a result, the control unit 32 generates an audio signal for outputting sound waves based on the audio signal with the characteristics of the audio signal controlled from the sound wave output unit 33 (step S307). The control unit 32 inputs the audio signal generated in step S307 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier. In this way, the control unit 32 controls the characteristics of the acoustic signal so that the reflected sound wave characteristic information, which is the frequency characteristic indicated by the reference sound characteristic information, and which is a sound wave that is easily audible to the user, are less likely to deviate from the characteristics of the sound wave reflected inside the user's ear.

The output acoustic information included in the notification requesting control of the characteristics of the acoustic signal indicates the frequency characteristics of the acoustic signal when a reference sound that is easily audible to the user is output. When the characteristics of the acoustic signal are controlled so that the deviation from the frequency characteristics of the output acoustic information becomes small, the frequency characteristics of the sound wave characteristic information of the input acoustic signal whose characteristics have been controlled will deviate less from the frequency characteristics of the reflected sound wave characteristic information compared to the frequency characteristics of the sound wave characteristic information of an acoustic signal whose characteristics have not been controlled.

The left ear sound wave output unit 331 and the right ear sound wave output unit 332 receive an acoustic signal from the control unit 32. The left ear sound wave output unit 331 and the right ear sound wave output unit 332 convert the acoustic signal into a sound wave and output it (step S308).

Note that if the receiving unit 34 receives an audio signal after step S303, the acoustic signal generated in step S307 is an acoustic signal for outputting a sound wave based on the audio signal received after step S303.

In addition, when the sound waves based on the audio signal transmitted in step S302 are being output at the start of step S307, the control unit 32 may perform the following operation in step S307. The control unit 32 may generate an acoustic signal for outputting the sound waves that are to be output from among the sound waves based on the audio signal transmitted in step S302.

Next, an example of the operation of the external device 2 will be described with reference to FIG. 6. The operation in FIG. 6 is a detailed description of the operations in steps S201, S202, S206, S207, and S208 in FIG. 4.

The input/output unit 21 receives an instruction to start initial settings in response to a user operation (step S401). The input/output unit 21 outputs the instruction to start initial settings to the transmission unit 22.

An instruction to start the initial setup is input to the transmission unit 22 from the input/output unit 21. When the instruction to start the initial setup is input, the transmission unit 22 performs the following operation. The transmission unit 22 transmits an output request notification, which includes frequency information and volume information, to the sound wave output device 3 and requests the output of a reference sound for the initial setup (step S402). For example, the output request notification that is output first during the initial setup includes frequency information indicating the initial frequency of the reference sound and volume information indicating the initial volume.

The input/output unit 21 receives operational input for an evaluation indicating whether the sound is easy to hear or not in response to a user's operation. When operational input for an evaluation indicating that the sound is easy to hear is received (step S403, YES), the input/output unit 21 generates evaluation information indicating that the sound is easy to hear and outputs the generated evaluation information to the transmission unit 22. When operational input for an evaluation indicating that the sound is difficult to hear is received (step S403, NO), the input/output unit 21 generates evaluation information indicating that the sound is difficult to hear and outputs the generated evaluation information to the transmission unit 22.

If evaluation information indicating that the sound is easy to hear is input, the transmission unit 22 transmits the evaluation information indicating that the sound is easy to hear to the sound wave output device 3 (step S404). If evaluation information indicating that the sound is difficult to hear is input, the transmission unit 22 does not perform the operation of step S404. The transmission unit 22 transmits the evaluation information indicating that the sound is difficult to hear to the sound wave output device 3 (step S405).

After performing the operation of step S404 or step S405, the transmission unit 22 associates the frequency information and volume information included in the output request notification transmitted in step S402 and outputs them to the determination unit 23.

The determination unit 23 receives frequency information and volume information from the transmission unit 22. The determination unit 23 determines whether evaluation information has been generated for all volumes of the reference sounds to be evaluated for the reference sounds of the frequency indicated in the input frequency information (step S406). The volume to be evaluated here is a volume that is set in advance as the volume to be evaluated for the reference sound of a predetermined frequency. For example, the external device 2 causes the sound wave output device 3 to output the reference sound of a predetermined frequency at different volumes at predetermined intervals, from the minimum volume to be evaluated to the maximum volume to be evaluated. The external device may also set the volume of the reference sound to be evaluated for each frequency based on the A characteristic, which is a frequency weighting characteristic.

If there is a reference sound with a volume for which no evaluation information has been generated among the reference sounds of the frequency indicated in the input frequency information (step S406, NO), the determination unit 23 performs the following operation. The determination unit 23 associates the frequency information and volume information of the reference sound to be evaluated next, and outputs them to the transmission unit 22 (step S407). Specifically, in step S407, the determination unit 23 associates the frequency information input from the transmission unit 22 with volume information indicating a volume for which no evaluation information has been generated, and outputs them to the transmission unit 22.

The frequency information and the volume information are input to the transmission unit 22 from the determination unit 23. In step S402, the transmission unit 22 transmits an output request notification including the frequency information and the volume information output in step S407 to the sound wave output device 3. The external device 2 performs the operations from step S402 to step S407 until evaluation information is generated for all the volume reference sounds to be evaluated for the reference sounds of the frequency indicated in the predetermined frequency information.

When evaluation information has been generated for all the volume levels of the reference tones to be evaluated for the reference tones of the frequencies indicated in the input frequency information (step S406, YES), the determination unit 23 performs the following operation. The determination unit 23 checks whether evaluation information has been generated for all the frequency levels of the reference tones to be evaluated (step S408). The frequency to be evaluated here is a frequency that is set in advance as the frequency to be evaluated. For example, the external device 2 causes the sound wave output device 3 to output reference tones of different frequencies at a predetermined interval, from the minimum frequency to be evaluated to the maximum frequency to be evaluated.

If there is a reference sound of a frequency for which evaluation information has not been generated (step S408, NO), the determination unit 23 performs the following operation. The determination unit 23 associates the frequency information and volume information of the sound wave to be evaluated next, and outputs them to the transmission unit 22 (step S409). Specifically, in step S409, the determination unit 23 associates frequency information indicating the frequency to be evaluated next, for which evaluation information has not been generated, with volume information indicating the volume to be evaluated, and outputs them to the transmission unit 22 (step S409). In step S402, the transmission unit 22 transmits an output request notification including the frequency information and volume information output in step S409 to the sound wave output device 3. The external device 2 performs the operations from step S402 to step S409 until evaluation information is generated for all reference sounds of the frequencies to be evaluated.

When evaluation information has been generated for all reference tones of the frequencies to be evaluated (step S408, YES), the determination unit 23 inputs a notification of the end of the initial settings to the transmission unit 22.

The transmission unit 22 receives an end notification of the initial settings from the determination unit 23. The transmission unit 22 transmits the end notification of the initial settings to the sound wave output device 3 (step S410). In addition, since evaluation information has been generated for all the reference sounds, the external device 2 ends the operation of FIG. 6.

Next, referring to FIG. 7, an example of an operation in which the sound wave output device 3 performs initial settings under the control of the external device 2 will be described. The operation in FIG. 7 is a detailed description of the operations from step S203 to step S205 in FIG. 4.

The receiving unit 34 receives an output request notification for initial setting, including frequency information and volume information (step S501). The receiving unit 34 associates the frequency information with the volume information and outputs the information to the control unit 32.

The control unit 32 receives frequency information and volume information from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting sound waves of the frequency indicated in the input frequency information and the volume indicated in the input volume information to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier means.

The left ear sound wave output unit 331 receives an acoustic signal from the control unit 32. The right ear sound wave output unit 332 receives an acoustic signal from the control unit 32. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave into the inside of the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into a sound wave and outputs the sound wave into the inside of the user's right ear (step S502).

In addition, in step S502, the left ear sound wave output unit 331 may output sound waves at a different timing than the right ear sound wave output unit 332, or may output sound waves at the same timing as the right ear sound wave output unit 332. When the left ear sound wave output unit 331 outputs sound waves at a different timing than the right ear sound wave output unit 332, the sound wave output device 3 may receive evaluation information for each of the left and right ears and store reference sound characteristic information for each of the left and right ears in the reference sound characteristic information storage unit 37.

The left ear sound wave input unit 351 inputs sound waves reflected inside the left ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31. The right ear sound wave input unit 352 inputs sound waves reflected inside the right ear and outputs an input acoustic signal indicating the input sound waves to the generation unit 31 (step S503). The right ear sound wave input unit 352 and the left ear sound wave input unit 351 output input acoustic signals to the generation unit 31 at the same timing. Note that the right ear sound wave input unit 352 may output an input acoustic signal to the generation unit 31 at a different timing than the left ear sound wave input unit 351.

The generation unit 31 receives an input acoustic signal from the left ear sound wave input unit 351. The generation unit 31 also receives an input acoustic signal from the right ear sound wave input unit 352. The generation unit 31 analyzes the signal strength of each frequency component that makes up the frequency of the input acoustic signal from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 31 generates reflected sound wave characteristic information according to the analysis results (step S504).

The receiving unit 34 also receives evaluation information (step S505). The receiving unit 34 associates the evaluation information with frequency information and volume information included in the output request notification received before receiving the evaluation information, and outputs the associated information to the generating unit 31. The receiving unit 34 receives the evaluation information at any timing. The frequency information and volume information included in the output request notification that are output to the generating unit 31 are output acoustic information.

The generating unit 31 receives the evaluation information and the output acoustic information from the receiving unit 34. The generating unit 31 does not perform operations from step S507 onward until a notification of the end of the initial settings is received (step S506, NO).

Each time the receiving unit 34 receives an output request notification (step S501), the left ear sound wave output unit 331 and the right ear sound wave output unit 332 perform the operation of step S502. In addition, the left ear sound wave input unit 351 and the right ear sound wave input unit 352 perform the operation of step S503. The generating unit 31 performs the operation of step S504. The receiving unit 34 performs the operation of step S505.

The receiving unit 34 receives a notification of the end of the initial settings from the external device 2. When the receiving unit 34 receives a notification of the end of the initial settings, it outputs the notification of the end of the initial settings to the generating unit 31.

The generation unit 31 receives a notification of the end of the initial settings from the receiving unit 34 (step S506, YES). When the notification of the end of the initial settings is received from the receiving unit 34, the following operation is performed. The generation unit 31 associates the evaluation information with the reference sound characteristic information for each of the reference sounds and stores them in the reference sound characteristic information storage unit 37 (step S507). The reference sound characteristic information includes output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds.

In addition, each time the output acoustic information and the evaluation information are input, the generation unit 31 may associate the evaluation information with the reference sound characteristic information and store the evaluation information in the reference sound characteristic information storage unit 37.

Next, referring to FIG. 8, an example of the operation in which the sonic wave output device 3 inputs an acoustic signal to the sonic wave output unit 33 for an audio signal received from a sonic wave input device (in this example, sonic wave input devices 4-2 to 4-n) will be described. The operation in FIG. 8 details the operation from step S303 to step S308 in FIG. 5.

In FIG. 8, an example is described in which a notification requesting control of the characteristics of the acoustic signal has not been input to the control unit 32 of the sonic output device 3 before the sonic output device 3 receives the audio signal in step S601 described below.

The receiver 34 receives an audio signal from the sonic wave input device (in this example, the sonic wave input devices 4-2 to 4-n) (step S601). The receiver 34 outputs the audio signal to the controller 32.

The control unit 32 receives an audio signal from the receiving unit 34. The control unit 32 inputs an acoustic signal for outputting sound waves based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier (step S602).

The left ear sound wave output unit 331 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's right ear (step S603).

The left ear sound wave input unit 351 inputs sound waves reflected inside the left ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31. The right ear sound wave input unit 352 inputs sound waves reflected inside the right ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 31 (step S604).

The generation unit 31 receives input acoustic signals from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 31 analyzes the signal strength of each frequency component that constitutes the input input acoustic signal (step S605). The generation unit 31 generates sound wave characteristic information according to the analysis result. The generation unit 31 outputs to the determination unit 36 the sound wave characteristic information generated from the input acoustic signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right ear sound wave input unit 352.

Based on the input sound wave characteristic information, the determination unit 36 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S606).

In step S606, the determination unit 36 makes the following determination. The determination unit 36 determines whether the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is deemed easy for the user to hear and stored in the reference sound characteristic information storage unit 37 deviate by a predetermined threshold or more from the frequency characteristics indicated by the sound wave characteristic information. If the deviation is equal to or greater than the predetermined threshold, the determination unit 36 determines that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33. If the deviation is less than the predetermined threshold, the determination unit 36 determines that sound waves that are easy for the user to hear are being output by the sound wave output unit 33.

When the determination unit 36 determines that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33 (step S606, NO), the determination unit 36 performs the following operation. For the frequency characteristics indicated by the sound wave characteristics information determined to be difficult to hear, the determination unit 36 identifies output acoustic information associated with reflected sound wave characteristic information that is associated with evaluation information indicating that it is easy to hear and that is stored in the reference sound characteristic information storage unit 37. For example, the determination unit 36 calculates the similarity between the frequency characteristics indicated by the reflected sound wave characteristic information and the frequency characteristics indicated by the input sound wave characteristic information among the reflected sound wave characteristic information associated with the evaluation information indicating that it is easy to hear, and identifies reflected sound wave characteristic information whose similarity is higher than a threshold value.

The determination unit 36 outputs a notification to the control unit 32 requesting control of the characteristics of the audio signal, including the identified output audio information (step S607).

If the determination unit 36 determines based on the input sound wave characteristic information that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear (step S606, YES), the operation of step S607 is not performed.

If a notification requesting control of the characteristics of the acoustic signal is not input, the control unit 32 generates an acoustic signal for outputting sound waves based on the audio signal in step S602. If a notification requesting control of the characteristics of the acoustic signal is input, the control unit 32 performs the following operation in step S602. The control unit 32 controls the characteristics of the acoustic signal to be output in response to the audio signal from the receiving unit 34. Specifically, the control unit 32 controls the characteristics of the acoustic signal so that there is less deviation from the output acoustic information included in the notification requesting control of the characteristics of the acoustic signal. In this way, the control unit 32 generates an acoustic signal for causing the sound wave output unit 33 to output sound waves based on the audio signal with the characteristics of the acoustic signal controlled.

The determination unit 36 may make a determination in step S606 as follows. The determination unit 36 may determine whether or not the frequency characteristics indicated by the reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with the evaluation information indicating that the sound is difficult to hear deviate by a predetermined threshold value or more from the frequency characteristics indicated by the sound wave characteristic information. If the deviation is equal to or greater than the predetermined threshold value, the determination unit 36 determines that sound waves that are easy for the user to hear are being output by the sound wave output unit 33. If the deviation is less than the predetermined threshold value, the determination unit 36 determines that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33. In addition to these methods, the determination unit 36 can use any method to determine whether or not sound waves that are easy for the user to hear are being output by the sound wave output unit 33.

As described above, the sound wave output device 3 of this embodiment generates sound wave characteristic information indicating the characteristics of the sound wave reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 that converts the sound signal into a sound wave and outputs it. The sound wave output device 3 controls the characteristics of the sound signal based on the voice of the other party so that the difference between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are the frequency characteristics of the sound wave of the reference sound reflected inside the user's ear and set as easy to hear for the user, is reduced. In addition, the sound wave output device 3 inputs the sound signal with the controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output to the user a sound wave based on the sound signal whose characteristics have been adjusted according to the reference sound characteristic information. This allows the sound wave output device 3 to output from the sound wave output unit 33 a sound wave that has been adjusted to be easy to hear for the user.

The sound wave output device 3 of this embodiment generates a sound signal for outputting sound waves based on a voice signal, with the characteristics of the sound signal controlled so that the deviation from the output sound information included in the notification requesting control of the characteristics of the sound signal is small. The output sound information included in the notification requesting control of the characteristics of the sound signal indicates the frequency characteristics of the sound signal when each of the reference sounds that are easy for the user to hear is output. The sound wave characteristic information of the input sound signal of the sound signal with the characteristics controlled has a smaller deviation from the reflected sound wave characteristic information of the reference sound characteristic information compared to the sound wave characteristic information of the input sound signal of the sound signal when the characteristics are not controlled. By operating in this manner, the control unit 32 can output sound waves that are easy for the user to hear to the left ear sound wave output unit 331 and the right ear sound wave output unit 332.

In addition, the sound wave output device 3 of this embodiment continuously controls the characteristics of the acoustic signal input to the sound wave output unit 33, so that the sound wave output device 3 can automatically output sound waves that are easy for the user to hear during a call or online conference.

In addition, the sound wave output device 3 of this embodiment controls the characteristics of the sound signal for the left ear sound wave output unit 331 based on reference sound characteristic information for the left ear, and controls the characteristics of the sound signal for the right ear sound wave output unit 332 based on reference sound characteristic information for the right ear. As a result, even if the frequency characteristics of sound waves that are easily heard by each of the user's left and right ears differ, the sound wave output device 3 can output from the sound wave output unit 33 sound waves that have been adjusted to be easily heard in accordance with the ease of hearing of each of the left and right ears.

[Third embodiment]
Next, the sonic wave output device 5 according to the third embodiment of the present invention will be specifically described.

In the third embodiment, the generation unit 51 corresponds to, for example, the generation unit 11 of the first embodiment. In the third embodiment, the control unit 52, the determination unit 53, the reference sound characteristic information storage unit 37, and the speaker information storage unit 54 correspond to, for example, the control unit 12 of the first embodiment. The third embodiment further differs from the sonic wave output device 1 of the first embodiment in that the sonic wave output device 3 includes a sonic wave output unit 33, a receiving unit 34, and a sonic wave input unit 35.

The third embodiment also differs from the sonic wave output device 3 of the second embodiment in that the left ear sonic wave output unit 331 and the right ear sonic wave output unit 332 of the sonic wave output device 5 output sound waves based on a voice signal at different timings for each speaker. In this embodiment, the sonic wave output device 5 controls the timing at which sound waves based on a voice signal are output from the left ear sonic wave output unit 331 and the right ear sonic wave output unit 332 for each speaker as a characteristic of the acoustic signal. This embodiment will be described mainly with reference to the differences from the second embodiment, but the various examples described in the second embodiment can also be applied.

An example of the configuration of the sonic output system of this embodiment will be described with reference to FIG. 9. FIG. 9 is a block diagram showing an example of the configuration of the sonic output system of this embodiment.

The sonic output system of this embodiment includes a sonic output device 5.

Since each of the configurations of the external device 2 in this embodiment is similar to each of the configurations of the external device 2 in the second embodiment shown in FIG. 3, the corresponding elements are given the same reference numerals as in FIG. 3 and the description is omitted.

The configurations of the sonic wave input devices 4-1 to 4-n in this embodiment are similar to the configurations of the sonic wave input devices 4-1 to 4-n in the second embodiment shown in FIG. 3, so the same reference numerals as in FIG. 3 are used for corresponding elements and descriptions thereof are omitted.

Next, the configuration of the sonic wave output device 5 of this embodiment will be described in detail with reference to FIG. 9.

The sound wave output device 5 includes a generation unit 51 and a control unit 52.

The sound wave output unit 33 is connected to the control unit 52. The receiving unit 34 is connected to the generation unit 51 and the control unit 52. The sound wave input unit 35 is connected to the generation unit 51. The determination unit 53 is connected to the generation unit 51, the control unit 52, the reference sound characteristic information storage unit 37, and the speaker information storage unit 54. The reference sound characteristic information storage unit 37 is connected to the generation unit 51 and the determination unit 53. The speaker information storage unit 54 is connected to the determination unit 53.

The sound wave output unit 33, the receiving unit 34, the sound wave input unit 35, and the reference sound characteristic information storage unit 37 in this embodiment are similar to the configurations of the sound wave output device 3 in the second embodiment shown in FIG. 3, so corresponding elements are given the same reference numerals as in FIG. 3 and common descriptions are omitted.

The process in which each component of the sonic wave output device 5 performs initial settings under the control of the external device 2 is similar to the process in which each component of the sonic wave output device 3 of the second embodiment performs initial settings. Therefore, a description of the process in which the sonic wave output device 5 performs initial settings under the control of the external device 2 will be omitted.

The following describes in detail each configuration of the sound wave output device 5 when used in an online conference.

The receiver 34 receives an audio signal, which is a signal corresponding to a voice, from the sonic wave input device (in this example, the sonic wave input devices 4-2 to 4-n). The receiver 34 outputs the audio signal to the controller 52.

The sound wave output unit 33 includes a left ear sound wave output unit 331 and a right ear sound wave output unit 332.

The left ear sound wave output unit 331 receives an acoustic signal from the control unit 52, which will be described later. The left ear sound wave output unit 331 converts the acoustic signal into a sound wave and outputs the sound wave to the inside of the user's left ear.

An acoustic signal is input to the right ear sonic wave output unit 332 from the control unit 52. The right ear sonic wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's left ear. Each of the right ear sonic wave output unit 332 and the left ear sonic wave output unit 331 outputs sound waves at a predetermined timing based on the acoustic signal. The predetermined timing is a timing indicated by time difference information indicating the timing at which sound is output from the right ear sonic wave output unit 332 and the timing at which sound is output from the left ear sonic wave output unit 331, which are set for each speaker using the sonic wave output system. Note that, similar to the timing, the sonic wave output device 5 may output sound waves so that sound can be heard from a specific direction for each speaker by outputting sound waves at different volumes from each of the right ear sonic wave output unit 332 and the left ear sonic wave output unit 331.

The generation unit 51 in this embodiment is similar to the generation unit 31 of the sonic wave output device 3 in the second embodiment shown in FIG. 3, so a description thereof will be omitted.

The determination unit 53 receives sound wave characteristic information generated from the input acoustic signal of the sound wave input unit 35 from the generation unit 51. Based on the input sound wave characteristic information, the determination unit 53 determines whether or not sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332. The process of determining whether or not sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 is the same as in the second embodiment, so a description thereof will be omitted.

When the determination unit 53 determines, based on the input sound wave characteristic information, that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33, it identifies the output sound information. The operation of identifying the output sound information is the same as in the second embodiment, and therefore a description thereof will be omitted.

In addition, when the determination unit 53 determines, based on the input sound wave characteristic information, that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear, the determination unit 53 does not perform processing to identify the output acoustic information.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. The speaker information indicates the frequency characteristics of the voice of each speaker. The speaker information storage unit 54 stores the speaker information in association with time difference information indicating the timing at which the voice is output from the right ear sound wave output unit 332 and the timing at which the voice is output from the left ear sound wave output unit 331, which are set for each speaker. The speaker information stored in the speaker information storage unit 54 is the sound wave characteristic information generated by the generation unit 51. The determination unit 53 may calculate the similarity between information indicating the frequency distribution included in the sound wave characteristic information and information indicating the frequency distribution included in the speaker information. The determination unit 53 identifies speaker information whose similarity is higher than a threshold value, and identifies the time difference information associated with the identified speaker information and stored in the speaker information storage unit 54.

The determination unit 53 may also use machine learning to identify the speaker. For example, the determination unit 53 uses learning data including input acoustic signals of the voices of multiple people and information indicating which person's voice the input acoustic signals of the voices of the multiple people are. The determination unit 53 may generate an extraction model that extracts features specific to each speaker from the input acoustic signal based on the learning data, and extract information indicating the features from the input acoustic signal using the extraction model. The determination unit 53 may calculate the similarity between the features extracted from the input acoustic signal and the features stored in the speaker information storage unit 54 as speaker information. The determination unit 53 identifies speaker information whose similarity is higher than a threshold value, and identifies time difference information associated with the identified speaker information and stored in the speaker information storage unit 54. In addition, if the time difference information cannot be identified, the information indicating the features extracted from the input acoustic signal is stored in the speaker information storage unit 54 as speaker information.

If the time difference information cannot be identified, the determination unit 53 performs the following operation. For speakers for whom the time difference information cannot be identified, the determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54. For example, the determination unit 53 assigns time difference information that has not yet been assigned from a plurality of pieces of time difference information set in advance in the order in which the speakers speak. In this way, the determination unit 53 specifies the timing at which the voice is output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 for each speaker who has spoken in the online conference so as not to overlap with other speakers. This allows the control unit 52 to output sound waves for the voice of a user who has newly joined the online conference so that the voice of the user who has newly joined the online conference can be heard from a direction different from that of other users. The determination unit 53 stores the assigned time difference information and sound wave characteristic information as speaker information in the speaker information storage unit 54.

When the output acoustic information is identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the characteristics of the acoustic signal including the identified output acoustic information and the time difference information. When the output acoustic information is not identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the output of the sound wave including the time difference information.

The speaker information storage unit 54 stores speaker information and time difference information in association with each other. The speaker information stored in the speaker information storage unit 54 is sound wave characteristic information generated by the generation unit 51, which indicates the frequency characteristics of the voice of each speaker. The speaker information may also be information indicating a feature amount extracted from an input acoustic signal. The speaker information storage unit 54 stores time difference information that differs for each speaker in association with the speaker information. The speaker information and time difference information may be stored in the speaker information storage unit 54 in advance by a user.

The control unit 52 receives an audio signal from the receiving unit 34. The control unit 52 receives a notification from the determining unit 53 requesting control of the characteristics of the audio signal. Alternatively, the control unit 52 receives a notification from the determining unit 53 requesting control of the output of sound waves. When a notification requesting control of the output of sound waves is received from the determining unit 53, the control unit 52 controls the characteristics of the audio signal to be output in response to the audio signal from the receiving unit 34 as follows. The control unit 52 generates an audio signal for outputting sound waves based on the audio signal from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information included in the notification requesting control of the output of sound waves. The control unit 52 inputs the generated audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier.

When a notification requesting control of the characteristics of the acoustic signal is input, the control unit 52 controls the characteristics of the acoustic signal to be output in response to the audio signal from the receiving unit 34 as follows. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the other party so that the frequency characteristics indicated by the reference sound characteristic information and the sound waves that are easy for the user to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for outputting sound waves based on the audio signal from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. The control unit 52 inputs the acoustic signal for outputting sound waves based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier. In this way, the control unit 52 controls the acoustic signal so that sound waves are output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the specified time difference information.

When neither a notification requesting control of the characteristics of the acoustic signal nor a notification requesting control of the output of sound waves is input, the control unit 52 may perform the following operation. The control unit 52 may input an acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to cause the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to simultaneously output sound waves based on an audio signal. When neither a notification requesting control of the characteristics of the acoustic signal nor a notification requesting control of the output of sound waves is input, this may be, for example, the start of an online conference.

In this way, the sound wave output device 5 generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 that converts the sound signal into a sound wave and outputs it. The sound wave output device 5 controls the characteristics of the sound signal based on the voice of the other party so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are set as easy for the user to hear when the sound wave of the reference sound is reflected inside the user's ear, is reduced. The sound wave output device 5 inputs the sound signal with the controlled characteristics to the sound wave output unit 33 via an amplification means. As a result, the sound wave output unit 33 can output to the user a sound wave based on the sound signal whose characteristics have been adjusted according to the reference sound characteristic information. This allows the sound wave output device 5 to output sound waves adjusted to be easy for the user to hear from the sound wave output unit 33.

Next, an example of the operation of the sonic wave output system of this embodiment will be described with reference to Figures 10 and 11.

Figure 10 is a sequence diagram showing an example of the operation of the sonic output system. Figure 11 is a flowchart showing an example of the operation of the sonic output device 5.

First, referring to FIG. 10, the operation of the sound wave output device 5 controlling the characteristics of the sound signal received from the sound wave input device (sound wave input devices 4-2 to 4-n in this example) and inputting the sound signal to the sound wave output unit 33 will be described.

During an online conference, the sound wave output device 5 of this embodiment generates sound wave characteristic information of sound waves that have been previously output and that have been reflected inside the ear. The sound wave output device 5 then controls at least the timing of output from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 as the characteristics of the acoustic signal of the sound waves that will next be output by the sound wave output unit 33.

Note that FIG. 10 illustrates an example in which a notification requesting control of the sound wave output is input to the control unit 52 of the sound wave output device 5 before step S703, which will be described later.

Each of the sonic input devices 4-2 to 4-n receives voice input from another user (step S701). Each of the sonic input devices 4-2 to 4-n that has received the voice input generates a voice signal that is a signal corresponding to the voice. Each of the sonic input devices 4-2 to 4-n that has generated the voice signal transmits the voice signal to a sonic output device (including the sonic output device 5 in this example) other than the sonic output device of the other user who input the voice to the sonic input device (step S702).

The receiver 34 of the sonic wave output device 5 receives audio signals from the sonic wave input devices (in this example, the sonic wave input devices 4-2 to 4-n). The receiver 34 outputs the audio signals to the control unit 52.

The control unit 52 receives an audio signal from the receiving unit 34. A notification requesting control of the output of sound waves is also received. The control unit 52 generates an acoustic signal for causing the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at the timing indicated by the time difference information included in the notification requesting control of the output of sound waves. The control unit 52 inputs the generated acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplification means.

The left ear sound wave output unit 331 converts the acoustic signal into sound waves and outputs the sound waves inside the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves inside the user's right ear (step S703). In step S703, the left ear sound wave output unit 331 and the right ear sound wave output unit 332 output sound waves at a predetermined timing based on the acoustic signal. Note that the sound waves output in step S703 may be sound waves whose characteristics have been controlled by the sound wave output device 5.

The left ear sound wave input unit 351 inputs sound waves reflected inside the user's left ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 51. The right ear sound wave input unit 352 inputs sound waves reflected inside the user's right ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 51 (step S704).

The generation unit 51 receives input acoustic signals from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 51 analyzes the signal strength of each frequency component that constitutes the input input acoustic signal (step S705). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 outputs to the determination unit 53 the sound wave characteristic information generated from the input acoustic signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right ear sound wave input unit 352.

Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are being output by each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S706). Specifically, the determination unit 53 determines whether the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is deemed easy for the user to hear and stored in the reference sound characteristic information storage unit 37 deviate by a predetermined threshold or more from the frequency characteristics indicated by the sound wave characteristic information. If the deviation is equal to or greater than the predetermined threshold, the sound wave corresponding to the input sound wave characteristic information is a sound wave that is difficult for the user to hear, and if the deviation is less than the predetermined threshold, the sound wave is easy for the user to hear.

If the determination unit 53 determines in step S706 based on the input sound wave characteristic information that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33, the determination unit 53 performs the following operation. For the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear, the determination unit 53 identifies output acoustic information associated with reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with evaluation information indicating that the sound waves are easy to hear.

If the determination unit 53 determines in step S706 based on the input sound wave characteristic information that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear, it does not identify the output acoustic information.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. The determination unit 53 identifies the time difference information that is associated with the identified speaker information and stored in the speaker information storage unit 54.

If the time difference information cannot be identified, the determination unit 53 performs the following operation. The determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54. The determination unit 53 stores the time difference information and the frequency information included in the sound wave characteristics information as speaker information in the speaker information storage unit 54. In this manner, the determination unit 53 identifies or assigns the time difference information (step S707).

When the output acoustic information is identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the characteristics of the acoustic signal including the identified output acoustic information and the time difference information. When the output acoustic information is not identified, the determination unit 53 outputs a notification to the control unit 52 requesting control of the output of the sound wave including the time difference information.

The control unit 52 receives an audio signal from the receiving unit 34. The control unit 52 receives a notification from the determining unit 53 requesting control of the characteristics of the audio signal. Alternatively, the control unit 52 receives a notification from the determining unit 53 requesting control of the output of the sound waves. When a notification requesting control of the output of the sound waves is input from the determining unit 53, the control unit 52 performs the following operation on the audio signal to be output from the receiving unit 34. The control unit 52 generates an audio signal for outputting sound waves based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information included in the notification requesting control of the output of the sound waves. The control unit 52 inputs the generated audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier. Also, when a notification requesting control of the characteristics of the audio signal is input, the control unit 52 performs the following operation on the audio signal to be output from the receiving unit 34. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the other party so that the frequency characteristics indicated by the reference sound characteristic information and the sound waves that the user finds easy to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for outputting sound waves based on the audio signal from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. In this way, the control unit 52 generates an acoustic signal for outputting sound waves based on the audio signal (step S708). The control unit 52 inputs the acoustic signal generated in step S708 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier.

An acoustic signal is input from the control unit 52 to the left ear sound wave output unit 331 and the right ear sound wave output unit 332. The right ear sound wave output unit 332 and the left ear sound wave output unit 331 convert the acoustic signal into sound waves and output the sound waves into the inside of the user's ears (step S709).

The determination unit 53 may perform the operations of steps S706 and S707 in parallel. Alternatively, the determination unit 53 may perform the operation of step S706 after performing the operation of step S707.

Next, referring to FIG. 11, an example of the operation in which the sonic wave output device 5 inputs an acoustic signal to the sonic wave output unit 33 for an audio signal received from a sonic wave input device (sound wave input devices 4-2 to 4-n in this example) will be described. The operation in FIG. 11 details the operations from step S703 to step S709 in FIG. 10. In FIG. 11, an example will be described in which a notification requesting control of the output of sound waves is input to the control unit 52 of the sonic wave output device 5 before step S802, which will be described later.

The receiver 34 receives an audio signal from the sonic wave input device (in this example, the sonic wave input devices 4-2 to 4-n) (step S801). The receiver 34 outputs the audio signal to the controller 52.

The control unit 52 receives an audio signal from the receiving unit 34. The control unit 52 also receives a notification requesting control of the output of sound waves. The control unit 52 inputs an acoustic signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier means, so as to cause the left ear sound wave output unit 331 and the right ear sound wave output unit 332 to output sound waves based on the audio signal at a predetermined timing (step S802). The predetermined timing is the timing indicated by the time difference information.

When a notification requesting control of the characteristics of the acoustic signal is input in step S802, the control unit 52 performs the following operation on the acoustic signal to be output in response to the voice signal from the receiving unit 34. The control unit 52 controls the characteristics of the acoustic signal based on the voice of the other party so that the frequency characteristics indicated by the reference sound characteristic information and the sound waves that the user finds easy to hear are less likely to deviate from the characteristics of the sound waves reflected inside the user's ear. In addition to controlling the characteristics of the acoustic signal, the control unit 52 generates an acoustic signal for outputting sound waves based on the audio signal from the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. The control unit 52 inputs the acoustic signal for outputting sound waves based on the audio signal to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 via an amplifier.

The left ear sound wave output unit 331 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's left ear. The right ear sound wave output unit 332 converts the acoustic signal into sound waves and outputs the sound waves to the inside of the user's right ear (step S803).

The left ear sound wave input unit 351 inputs sound waves reflected inside the left ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 51. The right ear sound wave input unit 352 inputs sound waves reflected inside the right ear and outputs an input acoustic signal indicative of the input sound waves to the generation unit 51 (step S804).

The generation unit 51 receives input acoustic signals from the left ear sound wave input unit 351 and the right ear sound wave input unit 352. The generation unit 51 analyzes the signal strength of each frequency component that constitutes the input input acoustic signal (step S805). The generation unit 51 generates sound wave characteristic information according to the analysis result. The generation unit 51 outputs to the determination unit 53 the sound wave characteristic information generated from the input acoustic signal input from the left ear sound wave input unit 351 and the sound wave characteristic information generated from the input acoustic signal input from the right ear sound wave input unit 352.

Based on the input sound wave characteristic information, the determination unit 53 determines whether sound waves that are easy for the user to hear are being output from each of the left ear sound wave output unit 331 and the right ear sound wave output unit 332 (step S806). The specific processing in step S806 is the same as the processing in step S706 in FIG. 10, so a description thereof will be omitted.

When the determination unit 53 determines, based on the input sound wave characteristic information, that sound waves that are difficult for the user to hear are being output by the sound wave output unit 33 (step S806, NO), it performs the following operation. For the frequency characteristics indicated by the sound wave characteristic information determined to be difficult to hear, the determination unit 53 identifies output acoustic information associated with reflected sound wave characteristic information stored in the reference sound characteristic information storage unit 37 in association with evaluation information indicating that the frequency characteristics are easy to hear (step S807).

If the determination unit 53 determines based on the input sound wave characteristic information that the sound wave output unit 33 is outputting sound waves that are easy for the user to hear (step S806, YES), the operation of step S807 is not performed.

The determination unit 53 identifies the speaker of the voice based on the input sound wave characteristic information and the speaker information stored in the speaker information storage unit 54. If the speaker can be identified (step S808, YES), the determination unit 53 performs the following operation. The determination unit 53 identifies the time difference information associated with the speaker information of the identified speaker and stored in the speaker information storage unit 54 (step S809).

If the speaker cannot be identified (step S808, NO), the determination unit 53 does not perform the operation of step S809. For speakers for whom the determination unit 53 cannot identify the speaker and therefore cannot identify the time difference information, the determination unit 53 assigns time difference information different from the time difference information stored in the speaker information storage unit 54. The determination unit 53 stores the assigned time difference information and the sound wave characteristic information as speaker information in the speaker information storage unit 54 (step S810).

The determination unit 53 that has performed the operation of step S809 or step S810 performs the following operation. If the output acoustic information has been identified, that is, the determination unit 53 that has performed the operation of step S807, outputs a notification to the control unit 52 requesting control of the characteristics of the acoustic signal. The notification requesting control of the characteristics of the acoustic signal includes the identified output acoustic information and time difference information. If the output acoustic information has not been identified, that is, the determination unit 53 that does not perform the operation of step S807, performs the following operation. The determination unit 53 outputs a notification to the control unit 52 requesting control of the output of sound waves including time difference information. In this way, the determination unit 53 outputs a notification requesting control of the characteristics of the acoustic signal or a notification requesting control of the output of sound waves to the control unit 52 (step S811).

The determination unit 53 may perform the operations from step S806 to step S807 and the operations from step S808 to step S810 in parallel. Alternatively, the determination unit 53 may perform the operations from step S806 to step S807 after performing the operations from step S808 to step S810.

As described above, the sound wave output device 5 of this embodiment generates sound wave characteristic information indicating the characteristics of the sound waves reflected inside the ear among the sound waves heard by the user by the sound wave output unit 33 that converts the sound signal into a sound wave and outputs it. The sound wave output device 5 controls the characteristics of the sound signal based on the voice of the other party so that the deviation between the frequency characteristics indicated by the sound wave characteristic information and the frequency characteristics indicated by the reference sound characteristic information, which are the frequency characteristics of the sound wave of the reference sound reflected inside the user's ear and set as easy to hear for the user, is reduced. The sound wave output device 5 inputs the sound signal with the controlled characteristics to the sound wave output unit 33 via the amplification means. As a result, the sound wave output unit 33 can output to the user a sound wave based on the sound signal whose characteristics have been adjusted according to the reference sound characteristic information. This allows the sound wave output device 5 to output to the sound wave output unit 33 a sound wave that has been adjusted to be easy to hear for the user.

The sound wave output device 5 of this embodiment also outputs sound waves to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at the timing indicated by the time difference information. Specifically, the sound wave output device 5 outputs sound waves to the left ear sound wave output unit 331 and the right ear sound wave output unit 332 at different timings for each speaker. This allows sound waves to be output so that the voice of each speaker can be heard from a specific direction, making it easier for the user of the sound wave output device 5 to determine which of the speakers is speaking.

In addition, the sound wave output device 5 of this embodiment continuously controls the characteristics of the acoustic signal output from the sound wave output unit 33, so that the sound wave output device 5 can automatically output sound waves that are easy for the user to hear during a call or online conference.

[Modification of the third embodiment]
The control unit of the sonic wave output device of the modified example of the third embodiment, in addition to controlling the characteristics of the acoustic signal in the third embodiment, controls each of the left ear sonic wave output unit and the right ear sonic wave output unit so that the volume of the sound wave corresponding to the consonant is output within a predetermined threshold. The method of controlling the characteristics of the acoustic signal in the third embodiment is the same as in the third embodiment, so a description thereof will be omitted. In addition to controlling the characteristics of the acoustic signal in the second embodiment, the sonic wave output device 3 in the second embodiment may control each of the left ear sonic wave output unit and the right ear sonic wave output unit so that the volume of the sound wave corresponding to the consonant is output within a predetermined threshold.

The generation unit of this modified example analyzes the signal strength of each frequency component that constitutes the input sound signal. Depending on the analysis result, the generation unit outputs to the determination unit sound wave characteristic information that further includes volume information indicating the volume of the sound wave corresponding to the consonant.

The determination unit of this modified example receives sound wave characteristic information that further includes volume information of sound waves corresponding to consonants. The determination unit outputs a notification to the control unit requesting control of the volume of the consonants.

The determination unit may determine whether the volume of the sound waves corresponding to the consonant is within a predetermined threshold based on the volume information of the sound waves corresponding to the consonant. If it is within the predetermined threshold, the determination unit may operate in the same manner as the sound wave output device 5 of the third embodiment. If it is not within the predetermined threshold, the determination unit may output a notification to the control unit requesting control of the volume of the consonant.

The control unit of this modified example receives a notification from the determination unit requesting control of the volume of a consonant. When a notification requesting control of the volume of a consonant is received, the control unit controls the characteristics of the acoustic signal to be output in response to the audio signal from the receiving unit as follows: The control unit generates an acoustic signal so that the volume of a sound wave corresponding to a consonant falls within a predetermined threshold value. The control unit inputs acoustic signals for outputting sound waves based on the audio signal to the left ear sound wave output unit and the right ear sound wave output unit.

The sound wave output unit outputs sound waves based on the acoustic signal into the user's ear.

The control unit of the sound wave output device of this modified example generates an acoustic signal so that the volume of sound waves corresponding to consonants is within a predetermined threshold. This allows the control unit of the sound wave output device of this modified example to output to the sound wave output unit sound waves that have been adjusted so that the volume of consonants is approximately the same. Since the control unit controls the characteristics of the acoustic signal input to the sound wave output unit so that there is no variation in the volume of sound waves corresponding to consonants, the user can hear the conversation clearly.

[Hardware configuration example]
An example of the configuration of hardware resources for implementing each of the sonic output devices (1, 3, 5) in each of the above-mentioned embodiments of the present invention using one information processing device (computer) will be described. Note that the sonic output device may be physically or functionally implemented using at least two information processing devices. The sonic output device may also be implemented as a dedicated device. Also, only a part of the functions of the sonic output device may be implemented using an information processing device.

Figure 12 is a diagram showing an example of the hardware configuration of an information processing device capable of realizing the sonic wave output device of each embodiment of the present invention. The information processing device 6 includes a communication interface 61, an input/output interface 62, a computing device 63, a storage device 64, a non-volatile storage device 65, and a drive device 66.

For example, the generation unit 11 of the sonic wave output device 1 in FIG. 1 can be realized by a communication interface 61 and a calculation device 63. Also, the control unit 12 of the sonic wave output device 1 in FIG. 1 can be realized by the calculation device 63.

The communication interface 61 is a communication means for the sonic output device of each embodiment to communicate with an external device in a wired and/or wireless manner. When the sonic output device is realized using at least two information processing devices, the devices may be connected so as to be able to communicate with each other via the communication interface 61.

The input/output interface 62 is a man-machine interface such as a keyboard, which is an example of an input device, and a display, which is an output device.

The arithmetic unit 63 is realized by a general-purpose CPU (Central Processing Unit) or a processing device such as a microprocessor, and a plurality of electric circuits. The arithmetic unit 63 is capable of, for example, reading various programs stored in the non-volatile storage device 65 into the storage device 64 and executing processing according to the read programs.

The storage device 64 is a memory device such as a RAM (Random Access Memory) that can be referenced by the computing device 63, and stores programs, various data, etc. The storage device 64 may be a volatile memory device.

The non-volatile storage device 65 is a non-volatile storage device such as a ROM (Read Only Memory), a flash memory, etc., and is capable of storing various programs, data, etc.

The drive device 66 is, for example, a device that processes the reading and writing of data from and to the recording medium 67 described below.

The recording medium 67 is any recording medium capable of recording data, such as an optical disk, a magneto-optical disk, or a semiconductor flash memory.

Each embodiment of the present invention may be realized by forming a sonic wave output device using, for example, the information processing device 6 illustrated in FIG. 12. Each embodiment of the present invention may be realized by supplying a program capable of realizing the functions described in each of the above embodiments to this sonic wave output device.

In this case, the embodiment can be realized by having the computing device 63 execute a program supplied to the sonic wave output device. It is also possible to configure some, but not all, of the functions of the sonic wave output device in the information processing device 6.

Furthermore, the sound wave output device may be configured so that the above program is recorded on the recording medium 67 and is appropriately stored in the non-volatile storage device 65 at the shipping stage or operation stage of the sound wave output device. In this case, the method of supplying the above program may be a method of installing the program in the sound wave output device using an appropriate tool at the manufacturing stage before shipping or at the operation stage. The method of supplying the above program may also be a general method such as downloading the program from an external source via a communication line such as the Internet.

The above-described embodiments are preferred embodiments of the present invention, and various modifications can be made without departing from the spirit of the present invention.

1, 3, 5 Sound wave output device 11, 31, 51 Generation unit 12, 32, 52 Control unit 33 Sound wave output unit 331 Left ear sound wave output unit 332 Right ear sound wave output unit 34 Reception unit 35 Sound wave input unit 351 Left ear sound wave input unit 352 Right ear sound wave input unit 36, 53 Determination unit 37 Reference sound characteristic information storage unit 54 Speaker information storage unit 2 External device 21 Input/output unit 22 Transmission unit 23 Determination unit 4-1 to 4-n Sound wave input device 41-1 to 41-n Sound wave input unit 42-1 to 42-n Transmission unit 6 Information processing device 61 Communication interface 62 Input/output interface 63 Arithmetic unit 64 Storage device 65 Non-volatile storage device 66 Drive device 67 Recording medium

Claims (8)

a generating means for generating sound wave characteristic information indicating characteristics of sound waves reflected inside the ear among the sound waves heard by the user by a sound wave output means for converting an acoustic signal into a sound wave and outputting the sound wave;
a control means for controlling the characteristics of the acoustic signal based on the voice of the other party so that the difference between the frequency characteristics indicated by the sound wave characteristics information and the frequency characteristics indicated by reference sound characteristic information relating to a reference sound having a frequency component audible to the user, the frequency characteristics being set as being easy for the user to hear when the sound wave of the reference sound is reflected inside the ear of the user, and inputting the control signal to the sound wave output means via an amplification means;
A sonic output device comprising: a sound wave input means for inputting the sound waves of the other party that are output from the sound wave output means to the inside of the ear and reflected at the inside of the ear, and outputting an input acoustic signal indicating the input sound waves;
The reference sound characteristic information includes a result of a judgment by the user as to whether the reference sound is easy to hear or not, and frequency information indicating a frequency characteristic of a sound wave of the reference sound reflected inside the ear of the user,
2. The sonic wave output device according to claim 1, wherein the generating means generates the sonic wave characteristic information in accordance with an analysis result of the signal strength of each frequency component that constitutes the input sound signal. 3. The sonic wave output device according to claim 2, wherein the control means controls the acoustic signal so that a deviation of a frequency characteristic of the sound wave characteristic information from a frequency characteristic of the reference sound characteristic information becomes small. the reference sound is a plurality of types of sound waves having different frequencies and volumes outputted by the sound wave output means,
the sound wave input means outputs the input sound signal for each of the reference sounds;
a receiving unit that receives, from an external device, evaluation information indicating whether each of the reference sounds is easy to hear,
the generating means stores, as the reference sound characteristic information, output sound information indicating the frequency characteristics of the sound signal when each of the reference sounds is output, and reflected sound wave characteristic information indicating the frequency characteristics of the input sound signal corresponding to each of the reference sounds, in a reference sound characteristic information storage means;
The sonic output device according to claim 2 or 3, characterized in that, when there is a deviation between the frequency characteristics indicated by the reflected sound wave characteristic information included in the reference sound characteristic information that is easy for the user to hear and stored in the reference sound characteristic information storage means and the frequency characteristics indicated by the sound wave characteristic information, the control means controls the characteristics of the acoustic signal so as to reduce the deviation from the frequency characteristics indicated by the output acoustic information associated with the reflected sound wave characteristic information. the generating means generates the sound wave characteristic information further including volume information indicating a volume of a sound wave corresponding to a consonant according to an analysis result of the signal strength of each frequency component constituting the input sound signal;
5. The sonic wave output device according to claim 2, wherein the control means controls a characteristic of the acoustic signal so that a volume of the sound wave corresponding to the consonant falls within a predetermined threshold value. the sound wave output means includes a right ear sound wave output means for inputting sound to the right ear of the user, and a left ear sound wave output means for inputting sound to the left ear of the user,
a speaker information storage means for storing speaker information indicating the frequency characteristics of the voice of each speaker and time difference information indicating the timing at which the voice is output from the right ear sound wave output means and the timing at which the voice is output from the left ear sound wave output means, which are set for each speaker;
a determination means for determining a speaker of the voice based on the speaker information stored in the speaker information storage means, and determining a timing at which the voice is output from the right ear sound wave output means and the left ear sound wave output means in accordance with the determined speaker and the time difference information,
6. The sonic wave output device according to claim 1, wherein the control means controls the acoustic signal so that sound waves are output from the left ear sonic wave output means and the right ear sonic wave output means at a timing indicated by the identified time difference information. generating sound wave characteristic information indicating characteristics of sound waves reflected inside the ear among the sound waves heard by the user by a sound wave output means which converts an acoustic signal into a sound wave and outputs the sound wave;
A sound wave output method, comprising: controlling the characteristics of the acoustic signal based on the voice of the other party in a call, and inputting the controlled acoustic signal to the sound wave output means via an amplification means so as to reduce the discrepancy between the frequency characteristics indicated by the sound wave characteristics information and the frequency characteristics indicated by reference sound characteristic information relating to a reference sound having frequency components audible by the user, the frequency characteristics being set as being easy for the user to hear when the sound wave of the reference sound is reflected inside the user's ear. On the computer,
a generating function for generating sound wave characteristic information indicating characteristics of sound waves reflected inside the ear among the sound waves heard by the user by a sound wave output means which converts an acoustic signal into a sound wave and outputs the sound wave;
a control function for controlling the characteristics of the acoustic signal based on the voice of the other party so that the difference between the frequency characteristics indicated by the sound wave characteristics information and the frequency characteristics indicated by reference sound characteristic information relating to a reference sound having a frequency component audible to the user, the frequency characteristics being set as being easy for the user to hear when the sound wave of the reference sound is reflected inside the ear of the user, and inputting the characteristics to the sound wave output means via an amplification means, is reduced;
A sound wave output program that makes this possible.