JP7632526B2 - Audio processing device, audio processing method, and audio processing program - Google Patents

Description

The present invention relates to an audio processing device, an audio processing method, and an audio processing program .

In a device such as an audio device that has an audio input/output means, when an earphone with a microphone or an earphone without a microphone is inserted into the external connection terminal for audio input/output, in the case of an earphone with a microphone, both the audio input and output automatically switch from the microphone and speaker of the device body to the microphone and earphone of the earphone with a microphone, and in the case of an earphone without a microphone, the audio output automatically switches from the speaker of the device body to the earphone without a microphone.

Conversely, if you unplug a microphone-equipped earphone or similar device from the external connection terminal, audio input and output will automatically switch to the device's built-in microphone and speaker.

JP 2003-058977 A

Traditionally, earphones with microphones have come in a variety of manufacturers, types, specifications, and characteristics.

However, it is not anticipated that users will be able to arbitrarily combine the device itself with external audio input and output means, such as earphones with a microphone.

The present invention has been made in consideration of such problems, and aims to provide an audio processing device, an audio processing method, and an audio processing program that allow the device body to be used in combination with external audio input means and audio output means, such as earphones with a microphone.

One aspect of the audio processing device of the present invention includes an external connection terminal into which an earphone can be inserted and removed, and a control unit, wherein the control unit executes audio playback when no earphone is inserted into the external connection terminal, and when it detects that an earphone has been inserted into the external connection terminal during audio playback, stops the audio playback , and when it detects that an earphone has been inserted into the external connection terminal during audio playback, determines whether the inserted earphone is an earphone with a microphone or an earphone without a microphone, and when it determines that the inserted earphone is an earphone with a microphone, stops the audio playback, and when it determines that the inserted earphone is an earphone without a microphone, continues the audio playback .
In addition, one aspect of the audio processing device of the present invention includes an external connection terminal into which an earphone can be inserted and removed, and a control unit, wherein the control unit performs audio playback without an earphone being inserted into the external connection terminal, and when it detects that an earphone has been inserted into the external connection terminal during the audio playback, stops the audio playback, and performs audio recording without an earphone being inserted into the external connection terminal, and when it detects that an earphone has been inserted into the external connection terminal during the audio recording, stops the audio recording.

1 is a block diagram showing a configuration of an electronic circuit of a learning support device 10 according to an embodiment of the recording device of the present invention. A figure showing the correspondence between the current state of audio input/output according to the setting data stored in the main body/earphone microphone setting data memory unit 12d of the learning support device 10 and the external connection status data stored in the external connection status data memory unit 12e, and the state of audio input/output after changing the connection to the external connection terminal 20. 13 is a diagram showing a main unit/earphone microphone setting screen GMs displayed on a display unit 17 in response to a user operation of the learning support device 10. FIG. 13 is a diagram showing an earphone/microphone setting window Wc that is displayed on the display unit 17 when the earphone/microphone 21 is inserted in the learning support device 10. FIG. 6 is a flowchart showing a microphone selection process in response to a user operation of the learning assistance device 10. 13 is a diagram showing an earphone microphone symbol EM and a main body microphone symbol BM displayed on a user voice recording screen GR in response to a microphone selection process of the learning support device 10. FIG. 13 is a diagram showing an example of a learning support process based on pronunciation lesson data 12c1 by the learning support device 10. 10 is a flowchart showing a microphone selection process in response to a user operation of the learning assistance device 10 according to a modified example.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is a block diagram showing the configuration of the electronic circuit of a learning support device 10 according to an embodiment of the recording device of the present invention.

The electronic circuit of the learning support device 10 includes a CPU 11, which is a computer. The CPU 11 controls the operation of each part of the circuit according to a learning support processing program 12a prestored in a storage unit 12 such as a flash ROM, or a learning support processing program 12a read by a storage medium reading unit 14 from an external storage medium 13 such as a memory card and stored in the storage unit 12, or a learning support processing program 12a downloaded via a communication unit 15 from a web server (here, a program server) 30 on a communication network N and stored in the storage unit 12.

The CPU 11 is connected to the memory unit 12, recording medium reading unit 14, and communication unit 15 via a data and control bus, as well as to a key input unit 16, a display unit 17 with a touch panel, an audio input unit 18, and an audio output unit 19.

The voice input unit 18 includes a main body microphone 18M (first voice input means) that inputs the voice of a user, etc.

The audio output unit 19 includes a main speaker 19S (first audio output means) that outputs audio based on audio data stored in the memory unit 12 or recorded.

The audio input unit 18 and audio output unit 19 are provided with a shared external connection terminal (EX) 20, and the user connects an earphone microphone (earphone with microphone) 21 (second audio input means/second audio output means), an earphone (earphone without microphone) 22 (second audio output means), or an external speaker 23 (second audio output means) to the external connection terminal (EX) 20 of the learning support device 10 as necessary. Note that in this specification, for simplicity, the earphone microphone 21, the earphone 22, or the external speaker 23 may be collectively referred to as the "external audio input/output device."

Here, the connection between the external connection terminal 20 of the learning support device 10 and the external audio input/output device may be via a wired cable such as an audio cable or a universal serial bus (USB), or it may be via wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark).
1 , in the case of connection via an audio cable, a connection between the learning support device 10 and the external audio input/output device is established by physically inserting a connection terminal of the external audio input/output device into the external connection terminal 20 of the learning support device 10. Conversely, the connection between the learning support device 10 and the external audio input/output device is released by physically removing the connection terminal of the external audio input/output device from the external connection terminal 20 of the learning support device 10.

In addition, when connecting via a USB cable, the connection terminal (USB terminal) of the external audio input/output device is physically inserted into the external connection terminal 20 of the learning support device 10, and a connection is established between the learning support device 10 and the external audio input/output device after a predetermined communication process is performed between the two. The connection between the learning support device 10 and the external audio input/output device is released by physically removing the connection terminal of the external audio input/output device from the external connection terminal 20 of the learning support device 10.

Furthermore, in the case of a connection via wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), a connection between the learning support device 10 and the external audio input/output device is established after a process of pairing the two is performed. The connection between the learning support device 10 and the external audio input/output device is disconnected when either the learning support device 10 or the external audio input/output device is turned off, when an operation for disconnecting is performed on either the learning support device 10 or the external audio input/output device, or when wireless communication between the learning support device 10 and the external audio input/output device is interrupted due to radio wave conditions.

In the following, an example will be described in which the learning support device 10 and the external audio input/output device are connected via an audio cable, but as mentioned above, it goes without saying that the connection between the learning support device 10 and the external audio input/output device may be via a wired cable other than an audio cable, or via wireless communication. Furthermore, in this specification and the attached drawings, establishing a connection between the learning support device 10 and the external audio input/output device may be expressed as inserting, plugging in, or attaching the external audio input/output device to the learning support device 10. Conversely, disconnecting the connection between the learning support device 10 and the external audio input/output device may be expressed as unplugging, pulling out, or removing the external audio input/output device from the learning support device 10.

The earphone microphone (earphone with microphone) 21 includes an earphone 21s (second audio output means) that outputs the audio output from the audio output unit 19, and a microphone 21m (second audio input means) that inputs the voice of a user, etc.

The audio output from the main speaker 19S and the audio output from the earphone 21s of the earphone microphone 21, the earphone 22, or the external speaker 23 is automatically switched upon detection of the establishment of a connection (insertion) of the earphone microphone 21, the earphone 22, or the external speaker 23 to the external connection terminal 20.

On the other hand, audio input through the main body microphone 18M and audio input through the microphone 21m of the earphone microphone 21 are switched according to the setting data for [main body microphone] or [earphone microphone] set in response to user operation in the main body/earphone microphone setting data storage unit 22d described below, regardless of whether the earphone microphone 21 is connected (inserted) to the external connection terminal 20 or not, and audio input through only one of them is exclusively selected.

The storage unit 12 stores the learning support processing program 12a as well as a voice recognition processing program 12b, and also includes a learning content storage unit 12c, a main unit/earphone microphone setting data storage unit 12d, an external connection status data storage unit 12e, a recorded voice data storage unit 12f, a judgment result data storage unit 12g, and a display data storage unit 12h.

The learning support processing program 12a stores a system program that controls the overall operation of the learning support device 10, a program for connecting to external electronic devices via the communication unit 15, and a program for executing learning functions according to the learning content data stored in the learning content storage unit 12c together with the voice recognition processing program 12b.

The learning content storage unit 12c stores learning content data such as, for example, pronunciation lesson data 12c1, response lesson data 12c2, etc.

The pronunciation lesson data 12c1 has, for example, text data and audio data corresponding to model words and sentences for the pronunciation lesson, and has a function of displaying the text data of the words or sentences on the display unit 17, outputting the audio data from the audio output unit 19, and then analyzing the user's audio data input from the audio input unit 18, and displaying the judgment results of the pronunciation quality, etc. on the display unit 17.

The response lesson data 12c2 has, for example, text data and voice data corresponding to the other person who is a model for the response lesson and the user himself/herself, and has a function of displaying the text data of the other person's question, etc. on the display unit 17, outputting the voice data from the voice output unit 19, and then analyzing the user's voice data input from the voice input unit 18, and displaying the judgment result, such as whether the answer is correct, on the display unit 17.

The algorithm for analyzing and determining the user's voice data required for processing the pronunciation lesson data 12c1 and the response lesson data 12c2 is generated based on the specifications and acoustic characteristics of the main microphone 18M and the microphone 21m of the earphone microphone (earphone with microphone) 21, which is an original product of this learning support device 10.

Therefore, in the pronunciation lessons and conversation lessons, the use of genuine main body microphone 18M or earphone microphone 21, whose recording specifications and acoustic characteristics are known in advance, allows for more accurate analysis of the user's voice data than using the microphone 21m of a non-genuine earphone microphone (earphone with microphone) 21 or other external microphones, and ultimately allows for more accurate judgment results.

In addition to the pronunciation lesson data 12c1 and the response lesson data 12c2, the learning content storage unit 12c also stores various learning content data that require the user to read and write text data and listen to model voice data, but do not require the user to input or analyze voice data.

The main body/earphone microphone setting data storage unit 12d identifies and stores the setting data [main body/earphone microphone] according to user operation, indicating whether the user's voice data is to be input from the main body microphone 18M or from the microphone 21m of the earphone microphone (earphone with microphone) 21.

The external connection status data storage unit 12e detects the current connection status of the external connection terminal 20 by the CPU 11, and stores the status as a status in which the earphone microphone (earphone with microphone) 21 is connected (microphone-equipped plugged-in status), or a status in which the earphone (earphone without microphone) 22 is connected (no microphone (speaker) plugged-in status), or a status in which no connection is established (disconnected status; unplugged status).

The recorded voice data storage unit 12f records and stores the user's voice data input from the voice input unit 19 by the CPU 11.

The judgment result data storage unit 12g stores data on the judgment result determined by analyzing the user's voice data in accordance with processing by the CPU 11 according to the pronunciation lesson data 12c1 and the response lesson data 12c2.

The display data storage unit 22h stores display data, for example in bitmap format, that the CPU 11 causes to be displayed on the display unit 12 in response to the operation of the learning support device 10.

In the learning support device 10 configured in this manner, the CPU 11 controls the operation of each part of the circuit according to the instructions written in the learning support processing program 12a and the voice recognition processing program 12c, and the software and hardware operate in cooperation to realize a learning support function according to the learning content data (12c), including a switching function between the main body microphone 18M and the earphone microphone 21, as described in the operation description below.

Next, we will explain the operation of the learning support device 10 configured as described above.

Figure 2 shows the current state of audio input/output according to the setting data stored in the main unit/earphone microphone setting data storage unit 12d of the learning support device 10 and the external connection state data stored in the external connection state data storage unit 12e, in association with the state of audio input/output after changing the connection to the external connection terminal 20.

Figure 3 shows the main unit/earphone microphone setting screen GMs displayed on the display unit 17 in response to user operation of the learning support device 10.

As shown in FIG. 3, the main unit/earphone microphone setting screen GMs displays an item [Main unit] for setting audio input (recording) to the main unit microphone 19M and an item [Earphone microphone] for setting to the microphone 21m of the earphone microphone 21, and either the [Main unit] or [Earphone microphone] item is selected and specified by operating the cursor Cu. At this time, the main unit/earphone microphone setting screen GMs displays a message encouraging the user to use a genuine product: "If you use an earphone microphone, it will affect the judgment results, so please use a genuine microphone."

As shown in FIG. 2, the learning support device 10 of this embodiment has three types of audio input/output modes [A], [B], and [C] when the setting data [Main Unit] for inputting audio data from the main unit microphone 18M is stored in the main unit/earphone microphone setting data storage unit 12d, and has three types of audio input/output modes [D], [E], and [F] when the setting data [Earphone Microphone] for inputting audio data from the earphone microphone 21 is stored.

Each audio input/output mode [A] to [F] is explained below.

Mode [A]: When the setting data is [Main unit] and the earphone microphone (earphone with microphone) 21 is connected (plugged in with microphone), the user's voice data is input (recorded) from the main unit microphone 18M, and the voice data to be played is output (played) from the earphone 21s of the earphone microphone 21.

Mode [B]: When the setting data is [Main unit] and the earphone (earphone without microphone) 22 (or external speaker 23) is connected (no microphone (speaker) plugged in), the user's voice data is input (recorded) from the main unit microphone 18M, and the voice data to be played is output (played) from the earphone 22 or external speaker 23.

Mode [C]: When the setting data is [Main unit] and nothing is connected (disconnected from any external audio input/output device; unplugged), the user's voice data is input (recorded) from the main unit microphone 18M, and the voice data to be played is output (played) from the main unit speaker 19S.

Mode [D]: Setting data [earphone microphone]. When the earphone microphone (earphone with microphone) 21 is connected (plugged in with microphone), the user's voice data is input (recorded) from the microphone 21m of the earphone microphone 21, and the voice data to be played is output (played) from the earphone 21s of the earphone microphone 21.

Mode [E]: When the setting data is [earphone microphone] and the earphone (earphone without microphone) 22 (or external speaker 23) is connected (no microphone (speaker) plugged in), the user's voice data cannot be input (recorded), and the voice data to be played is output (played) from the earphone 22 or the external speaker 23.

Mode [F]: When the setting data is [Earphone Microphone] and nothing is connected (disconnected from any external audio input/output device; unplugged), the user's voice data cannot be input (recorded), and the audio data to be played is output (played) from the main speaker 19S.

The following describes what happens when the connection to the external connection terminal 20 is changed in each of the audio input/output modes [A] to [F].

Figure 4 shows the earphone/microphone settings window Wc that is displayed on the display unit 17 when the earphone/microphone 21 is inserted and a connection is established in mode [C] in which audio input (recording) is set to [Main unit] in the learning support device 10.

When the earphone microphone 21 or earphone 22 (external speaker 23) is unplugged and disconnected from the mode [A] or [B], the mode changes to the mode [C]. At this time, the audio data to be played is switched from being output only to the user from the earphone 21s or earphone 22 of the earphone microphone 21 to being output (played) from the main speaker 19S, so the output (playback) is temporarily stopped in consideration of the user missing audio and the impact on the surroundings. In addition, when the user unplugs the earphone microphone 21 or earphone 22 (external speaker 23) to disconnect, the input (recording) of the user's voice from the main microphone 18M is also temporarily stopped in consideration of discontinuities in the voice uttered by the user.

When the earphone microphone (earphone with microphone) 21 is inserted and a connection is established in the mode [C], the earphone microphone setting window Wc is displayed on the display unit 17 as shown in FIG. 4. The setting window Wc displays a message asking whether to change the setting from the main body microphone 18M to the earphone microphone 21, "Do you want to set the microphone setting to the earphone microphone? [Yes/No]", as well as a message encouraging the user to use a genuine product, "If you use an earphone microphone, it will affect the judgment results, so please use a genuine microphone."

Here, for example, if the earphone microphone 21 that was inserted and connected is a genuine product, and the user operates the cursor Cu to select [Yes] in the setting window Wc, the mode changes to [D], and the setting data of the main unit/earphone microphone setting data storage unit 12d is changed to the setting data for [earphone microphone].

On the other hand, if the user operates the cursor Cu in the setting window Wc to select [No] because the earphone microphone 21 that was inserted and connected is a non-genuine product, the mode changes to [A], and the setting data stored in the main unit/earphone microphone setting data storage unit 12d remains as [Main unit].

At this time, the audio data to be played back is switched from output from the main speaker 19S to output (playback) from the earphone 21s of the earphone microphone 21, so the output (playback) is temporarily suspended to prevent the user from missing any audio. Also, as the user inserts the earphone microphone 21 to establish a connection, input (recording) of the user's voice from the earphone microphone 21 or the main microphone 18M is temporarily suspended to prevent discontinuities in the user's voice.

In addition, when the earphone (earphone without microphone) 22 is inserted from the mode [C] and a connection is established, the mode changes to the mode [B]. At this time, the audio data to be played back is simply switched from output from the main speaker 19S to output (playback) to the user only through the earphone 22, so output (playback) continues as is, but the output (playback) may be paused in consideration of the user missing audio. In addition, when the user inserts the earphone 22 to establish a connection, input (recording) of the user's voice from the main microphone 18M is paused in consideration of discontinuities in the voice uttered by the user.

When the earphone microphone 21 is unplugged from the mode [D] state to disconnect, the mode changes to the mode [C], and the setting data of the main unit/earphone microphone setting data storage unit 12d is changed to the setting data of [main unit]. At this time, the audio data to be played is switched from being output only to the user from the earphone 21s of the earphone microphone 21 to being output (played) from the main unit speaker 19S, so the output (playback) is temporarily stopped in consideration of the user missing the sound and the impact on the surroundings. In addition, when the user unplugs the earphone microphone 21 to disconnect, the input (recording) of the user's voice from the main unit microphone 18M is also temporarily stopped in consideration of discontinuities in the voice uttered by the user.

When the earphone 22 (external speaker 23) is removed and disconnected from the mode [E] state, the mode changes to the mode [F]. At this time, the audio data to be played is switched from being output only to the user from the earphone 22 to being output (played) from the main speaker 19S, and the output (playback) is temporarily stopped, just as when the mode changes to the mode [C]. Note that input (recording) of the user's voice is no longer possible.

When the earphone microphone (earphone with microphone) 21 is inserted and a connection is established in the mode [F], a setting window Wc for the earphone microphone is displayed on the display unit 17 as shown in FIG. 4. The setting window Wc displays a message asking whether the earphone microphone 21 needs to be set from the main body microphone 18M, "Do you want to set the microphone to the earphone microphone? [Yes/No]," as well as a message encouraging the user to use a genuine product, "If you use an earphone microphone, it will affect the judgment results, so please use a genuine microphone."

Here, for example, if the earphone microphone 21 that was inserted and connected is a genuine product, and the user operates the cursor Cu to select and specify [Yes] in the setting window Wc, the mode changes to [D], and the setting data stored in the main unit/earphone microphone setting data storage unit 12d remains as [earphone microphone].

On the other hand, if the user operates the cursor Cu in the setting window Wc to select [No] because the earphone microphone 21 that was inserted and connected is a non-genuine product, the mode changes to [A], and the setting data of the main unit/earphone microphone setting data storage unit 12d is changed to the setting data for [main unit].

At this time, the audio data to be played is switched from output from the main speaker 19S to output (playback) from the earphone 21s of the earphone microphone 21, so the output (playback) is temporarily stopped, just as when mode [C] is changed to mode [D] or [A]. In addition, input (recording) of the user's voice from the earphone microphone 21 or the main microphone 18M is also temporarily stopped.

In addition, when the earphone (earphone without microphone) 22 is inserted and a connection is established in the mode [F] state, the mode changes to the mode [E]. At this time, the audio data to be played back is simply switched from output from the main body speaker 19S to output (playback) to the user only via the earphone 22, just as when the mode changes from [C] to [B], so output continues as is, but the output (playback) may be temporarily halted in consideration of the user missing audio. Note that input (recording) of the user's audio is no longer possible.

Figure 5 is a flowchart showing the microphone selection process in response to user operation of the learning support device 10.

Figure 6 shows the earphone microphone symbol EM and the main microphone symbol BM displayed on the user voice recording screen GR in response to the microphone selection process of the learning support device 10.

For example, when a user is studying based on the learning content data stored in the learning content storage unit 12c, when the user establishes a connection (inserts) the earphone microphone 21 to the external connection terminal 20 of the audio input unit 18 and the audio output unit 19, the CPU 11 displays the setting window Wc for the earphone microphone on the display unit 17, as shown in FIG. 4 (step S1).

When the setting data of the main body/earphone microphone setting data storage unit 12d is [Main body], if the user selects and specifies the item [Yes] to be set for the earphone microphone in the setting window Wc for the earphone microphone because the earphone microphone 21 inserted into the external connection terminal 20 and established a connection is a genuine product (step S2 (YES)), the CPU 11 changes the setting data stored in the main body/earphone microphone setting data storage unit 12d from [Main body] to [Earphone microphone] and sets the state (mode [D]) in which the user's voice can be input (recorded) from the microphone 21m of the earphone microphone 21 (step S3).

Then, when the CPU 11 needs to record the user's voice during the learning support process based on the learning content data, for example, as shown in FIG. 6, when the user voice recording screen GR is displayed on the display unit 17, the CPU 11 displays the earphone microphone symbol EM and the recording message "Recording with earphone microphone" me1 on the screen GR together with the recording time indicator TR, as shown in FIG. 6 (A) (step S4).

On the other hand, in the earphone microphone setting window Wc (see FIG. 4), if the user selects and specifies the item [No] that is not to be set for the earphone microphone because the earphone microphone 21 inserted into the external connection terminal 20 and established a connection is a non-genuine product (step S2 (NO)), the CPU 11 will maintain the setting data stored in the main unit/earphone microphone setting data storage unit 12d in [Main unit] and set the main unit microphone 18M to a state (mode [A]) in which the user's voice can be input (recorded) (step S5).

Then, when the CPU 11 needs to record the user's voice during the learning support process based on the learning content data, for example, as shown in FIG. 6, when the user voice recording screen GR is displayed on the display unit 17, the built-in microphone symbol BM and the recording message "Recording with built-in microphone" me2 are displayed on the screen GR along with the recording time indicator TR, as shown in FIG. 6 (B) (step S6).

Also, when the setting data of the main body/earphone microphone setting data storage unit 12d is [earphone microphone], if the user establishes a connection (inserts) of the earphone microphone 21 to the external connection terminal 20 of the audio input unit 18 and audio output unit 19, the CPU 11 displays the setting window Wc for the earphone microphone on the display unit 17 as shown in FIG. 4 (step S1).

In the earphone microphone setting window Wc, when the user selects and specifies the item [Yes] to be set for the earphone microphone because the earphone microphone 21 inserted into the external connection terminal 20 and established a connection is a genuine product (step S2 (YES)), the CPU 11 maintains the setting data stored in the main body/earphone microphone setting data storage unit 12d as [earphone microphone] and sets the earphone microphone 21m in a state (mode [D]) in which the user's voice can be input (recorded) from the microphone (step S3).

Then, when the CPU 11 needs to record the user's voice during the learning support process based on the learning content data, for example, as shown in FIG. 6, when the user voice recording screen GR is displayed on the display unit 17, the CPU 11 displays the earphone microphone symbol EM and the recording message "Recording with earphone microphone" me1 on the screen GR together with the recording time indicator TR, as shown in FIG. 6 (A) (step S4).

On the other hand, in the earphone microphone setting window Wc (see FIG. 4), if the user selects and specifies the option [No] to not set the earphone microphone (set to the main unit microphone) because the earphone microphone 21 inserted into the external connection terminal 20 and established a connection is a non-genuine product (step S2 (NO)), the CPU 11 changes the setting data stored in the main unit/earphone microphone setting data storage unit 12d from [earphone microphone] to [main unit], and sets the device in a state (mode [A]) in which the user's voice can be input (recorded) from the main unit microphone 18M (step S5).

Then, when the CPU 11 needs to record the user's voice during the learning support process based on the learning content data, for example, as shown in FIG. 6, when the user voice recording screen GR is displayed on the display unit 17, the built-in microphone symbol BM and the recording message "Recording with built-in microphone" me2 are displayed on the screen GR along with the recording time indicator TR, as shown in FIG. 6 (B) (step S6).

Figure 7 shows an example of learning support processing based on pronunciation lesson data 12c1 by the learning support device 10.

Here, it is assumed that the main unit/earphone microphone setting data (12d) is [Main unit] and the device is in one of the audio input/output modes [A], [B], or [C].

When the user selects pronunciation lesson data 12c1 on a list screen (not shown) of learning content data 12c1, 12c2, ... displayed on the display unit 17, the CPU 11 causes the display unit 17 to display a pronunciation lesson initial screen G1 showing the selection options "Judge by word" and "Judge by sentence", as shown in FIG. 7(A).

When the "Judge by Sentence" option is selected in response to a user operation on the pronunciation lesson initial screen G1, the CPU 11 causes the display unit 17 to display a model screen G2 showing the model sentence "Do you like it?" and a pronunciation request message "Please pronounce the sentence following the model sentence", as shown in FIG. 7(B), and causes the audio output unit 19 to output (play) audio data of the model pronunciation of the model sentence "Do you like it?".

When the output (playback) of the model voice data corresponding to the model screen G2 is completed, the CPU 11 causes the display unit 17 to display a user voice recording screen G3 showing the recording message "Recording" as shown in FIG. 7(C), and prompts the user to pronounce the voice in accordance with the model pronunciation. At this time, the main body microphone symbol BM is displayed on the user voice recording screen G3, and the user is notified that recording is currently being performed using the main body microphone 18M.

When the recording of the user's pronunciation is completed, the CPU 11 displays a user voice analysis screen G4 with the analysis message "Analyzing" displayed, as shown in FIG. 7 (D), and performs voice recognition on the user's voice data recorded in the recorded voice data storage unit 12f. It also analyzes the pronunciation quality, etc. by comparing it with the voice data of the model pronunciation, and judges it as a score, and stores the score of the judgment result in the judgment result data storage unit 12g.

Then, the CPU 11 causes the display unit 17 to display a pronunciation judgment screen G5 showing the score of the judgment result (e.g., "84 points") stored in the judgment result data storage unit 12g, as shown in FIG. 7(E).

When the user selects [Next Sentence] from among the multiple items displayed on the pronunciation judgment screen G5, the CPU 11 causes the display unit 17 to display the next model screen G2, which displays the next model sentence, "Can I take your order?", and a pronunciation request message, as shown in FIG. 7 (F), and causes the audio output unit 19 to output (play) audio data of the model pronunciation.

As described above, among the learning content data stored in the learning content storage unit 12c, at least the pronunciation lesson data 12c1 and the response lesson data 12c2 require recording and analyzing the voice data pronounced by the user. Therefore, using the genuine main body microphone 18M or earphone microphone 21, whose recording specifications and acoustic characteristics are clear in advance, allows for more accurate analysis of the user's voice data than using the microphone 21m of a non-genuine earphone microphone (earphone with microphone) 21 or other external microphones, and ultimately allows for more accurate determination results.

For this reason, in the learning support device 10 of this embodiment, when an earphone microphone 21 is inserted into the external connection terminal 20 of the audio input unit 18 and audio output unit 19 and a connection is established, it is necessary to display a setting window Wc for the earphone microphone so that the voice data spoken by the user can be recorded by an original microphone.

Therefore, according to the learning support device 10 configured as described above, when the CPU 11 detects that an earphone microphone (earphone with microphone) 21 has been connected (inserted) to the external connection terminal 20 of the audio input unit 18 and the audio output unit 19, it displays an earphone microphone setting window Wc on the display unit 17, which recommends the use of a genuine microphone and allows the user to specify whether or not to set the earphone microphone 21 as the target for audio input.

As a result, even when learning in accordance with the learning content (12c) involves listening to audio for learning from the earphone microphone 21, the user's voice that needs to be input (recorded) can be analyzed more accurately and a more accurate assessment result can be obtained, so that when the earphone microphone 21 is an original product, the audio input state can be switched to the audio input state from the earphone microphone 21, and when it is not, the audio input state can be switched to the audio input state from the main microphone 18M, depending on the user's operation.

In addition, according to the learning support device 10 configured as described above, when the CPU 11 is inputting (recording) the user's voice from the main microphone 18M, it displays this fact by displaying the main microphone symbol BM and the recording message "Recording with the main microphone" me2, and when the CPU 11 is inputting (recording) the user's voice from the earphone microphone 21, it displays this fact by displaying the earphone microphone symbol EM and the recording message "Recording with the earphone microphone" me1.

This makes it easy to tell whether user voice is coming from the main unit microphone 18M or the earphone microphone 21 when it is necessary to input (record) the user's voice.

Furthermore, according to the learning support device 10 configured as above, regardless of whether the target of audio input is set to the main microphone 18M or the earphone microphone 21 according to the main body/earphone microphone setting data (12d), when the CPU 11 detects the removal (disconnection) of the earphone microphone 21, earphone 22, or external speaker 23 that is inserted into and connected to the external connection terminal 20 of the audio input unit 18 and audio output unit 19, the target of audio output can be automatically switched from the earphone microphone 21, earphone 22, or external speaker 23 to the main speaker 19S.

(Modification)
FIG. 8 is a flowchart showing a microphone selection process in response to a user operation of the learning assistance device 10 according to a modified example.

In the embodiment, in the microphone selection process in response to user operation described with reference to FIG. 5, regardless of whether the setting data in the main unit/earphone microphone setting data storage unit 12d is set to [main unit] or [earphone microphone], when the earphone microphone 21 is connected (inserted) to the external connection terminal 20, the setting window Wc for the earphone microphone shown in FIG. 4 is displayed on the display unit 17, and the user is allowed to select whether or not to set the microphone to the earphone microphone depending on whether the earphone microphone 21 that has been inserted and connected is a genuine product.

In contrast, as shown in the microphone selection process according to the user's operation of the modified example in FIG. 8, when the earphone microphone 21 is connected (inserted) to the external connection terminal 20, the CPU 11 determines whether the setting data in the main unit/earphone microphone setting data storage unit 12d is set to [main unit] (step S0), and if it is set to [main unit] (step S0 (YES)), as in the above embodiment, the setting window Wc for the earphone microphone shown in FIG. 4 is displayed on the display unit 17, and the user is allowed to select whether to set the microphone to the earphone microphone depending on whether the earphone microphone 21 that was inserted and connected is a genuine product (step S1).

If the setting data of the main body/earphone microphone setting data storage unit 12d is set to [earphone microphone] (step S0 (NO)), the CPU 11 may be configured to maintain the setting data of [earphone microphone] (steps S3, S4) without displaying the setting window Wc for the earphone microphone. In this case, if the earphone microphone 21 that is inserted and connected is a genuine product, the voice data of the set user can be analyzed more accurately without the user having to take the trouble of setting up the earphone microphone, and thus a more accurate judgment result can be obtained.

The methods of each process performed by the learning support device 10 described in the above embodiment, i.e., the input/output mode setting and switching process shown in FIG. 2, and the microphone selection process in response to user operations shown in the flowcharts of FIG. 5 and FIG. 8, can all be stored as programs that can be executed by a computer and distributed on recording media such as memory cards (ROM cards, RAM cards, etc.), magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), and semiconductor memories.

In addition, the program data for implementing each of the above techniques can be transmitted over the communications network N in the form of program code, and the learning support function including the microphone switching function described above can be realized by importing this program data into the computer of an electronic device connected to the communications network N via a communications unit.

The present invention is not limited to the above-mentioned embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the above-mentioned embodiment includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple constituent elements disclosed. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, or some constituent elements are combined in a different form, if the problem described in the "Problem to be Solved by the Invention" column can be solved and the effect described in the "Effect of the Invention" column can be obtained, the configuration in which the constituent elements are deleted or combined can be extracted as the invention.

The inventions described in the original claims of this application are set forth below.

[Appendix 1]
a first audio input unit and a first audio output unit provided in a device body;
a control unit that accepts an operation to specify either the first audio input unit or the second audio input unit in response to a connection being established between an external second audio input unit and a device body;
A recording device equipped with
[Appendix 2]
The control unit is
when the operation for identifying the first voice input unit is accepted and voice input is executed by the first voice input unit, a first identification indication corresponding to the first voice input unit is displayed on a display unit;
when the operation for identifying the second voice input unit is accepted and voice input is executed by the second voice input unit, a second identification indication corresponding to the second voice input unit and different from the first identification indication is displayed on the display unit;
2. The recording device of claim 1.
[Appendix 3]
The control unit is
when the operation for identifying the first voice input unit is received, regardless of a state in which the second voice input unit is connected to the device body, the input of voice by the second voice input unit is prohibited and the input of voice by the first voice input unit is permitted.
3. The recording device according to claim 1 or 2.
[Appendix 4]
The control unit is
Regardless of whether the voice input unit specified by the received operation is the first voice input unit or the second voice input unit,
accepting an operation to specify either the first audio input unit or the second audio input unit when the second audio input unit changes from a disconnected state to a connected state with the device body;
4. A recording device according to any one of claims 1 to 3.
[Appendix 5]
The control unit is
when the second audio input unit changes from a disconnected state to a connected state with the device body after the operation for identifying the first audio input unit is received, displaying on a display unit an audio input identification unit that allows a user to identify either the first audio input unit or the second audio input unit;
when the second voice input unit changes from a disconnected state to a connected state with the device body after the operation for identifying the second voice input unit is accepted, the voice input identifying unit is not displayed on a display unit.
4. A recording device according to any one of claims 1 to 3.
[Appendix 6]
The control unit is
when an external second audio output unit that has been in a state of connection establishment with the device main body becomes in a state of disconnection from the device main body after the operation for identifying either the first audio input unit or the second audio input unit is accepted, automatically allowing the first audio output unit to output audio.
6. A recording device according to any one of claims 1 to 5.

10 ... learning support device 11 ... CPU
12 ... Memory unit 12a ... Learning support processing program 12b ... Voice recognition processing program 12c ... Learning content memory unit 12d ... Main unit / earphone microphone setting data memory unit 12e ... External connection status data memory unit 12f ... Recorded voice data memory unit 12g ... Judgment result data memory unit 12h ... Display data memory unit 17 ... Display unit 18 ... Voice input unit 18M ... Main unit microphone 19 ... Voice output unit 19S ... Main unit speaker 20 ... External connection terminal 21 ... Earphone microphone (earphone with microphone)
22...Earphones (earphones without microphone)
23 ...External speaker GMs ...Main unit/earphone microphone setting screen Wc ...Setting window for earphone microphone EM ...Earphone microphone symbol BM ...Main unit microphone symbol me1 ...Message during recording with earphone microphone me2 ...Message during recording with main unit microphone.

Claims (6)

An external connection terminal for inserting and removing earphones,
A control unit;
Equipped with
The control unit is
Executing audio playback with earphones not inserted in the external connection terminal,
when detecting that an earphone is inserted into the external connection terminal during playback of the audio, stopping the playback of the audio;
when detecting that an earphone is inserted into the external connection terminal during playback of the sound, determining whether the inserted earphone is an earphone with a microphone or an earphone without a microphone;
When it is determined that the earphones are earphones with a microphone, the reproduction of the audio is stopped, and when it is determined that the earphones are earphones without a microphone, the reproduction of the audio is continued .
Audio processing device. An external connection terminal for inserting and removing earphones,
A control unit;
Equipped with
The control unit is
Executing audio playback with earphones not inserted in the external connection terminal,
when detecting that an earphone is inserted into the external connection terminal during playback of the audio, stopping the playback of the audio;
Execute audio recording with earphones not plugged into the external connection terminal,
when it is detected that an earphone is inserted into the external connection terminal during recording of the audio, the recording of the audio is stopped.
Audio processing device. The control unit is
Record the voice using the microphone on the device,
when it is detected that an earphone is inserted into the external connection terminal during recording of the audio, recording of the audio by the microphone of the main body is stopped.
The audio processing device according to claim 2 . The control unit is
when it is determined that the inserted earphone is an earphone with a microphone, the recording of the sound is stopped, and the user is prompted to select whether or not to switch the microphone used for recording the sound from the microphone of the main body to the microphone of the earphone with a microphone.
The audio processing device according to claim 3 . A control unit of a sound processing device having an external connection terminal into which an earphone can be inserted and removed,
Executing audio playback with earphones not inserted in the external connection terminal,
when detecting that an earphone is inserted into the external connection terminal during playback of the audio, stopping the playback of the audio ;
when detecting that an earphone is inserted into the external connection terminal during playback of the sound, determining whether the inserted earphone is an earphone with a microphone or an earphone without a microphone;
When it is determined that the earphones are earphones with a microphone, the reproduction of the audio is stopped, and when it is determined that the earphones are earphones without a microphone, the reproduction of the audio is continued .
Audio processing methods. A computer of a voice processing device having an external connection terminal into which an earphone can be inserted and removed,
Executing audio playback with earphones not inserted in the external connection terminal,
when detecting that an earphone is inserted into the external connection terminal during playback of the audio, stopping the playback of the audio ;
when detecting that an earphone is inserted into the external connection terminal during playback of the sound, determining whether the inserted earphone is an earphone with a microphone or an earphone without a microphone;
When it is determined that the earphones are earphones with a microphone, the reproduction of the audio is stopped, and when it is determined that the earphones are earphones without a microphone, the reproduction of the audio is continued .
A voice processing program for executing the processing.