JP6447474B2 - Electronics - Google Patents

Description

  The present invention relates to an electronic device having a voice recognition function.

  Conventionally, in an electronic device, an operation by a mouse operation or a touch operation is performed. In mouse operation or touch operation, it is necessary for the user to perform an operation by aligning the operation key to be operated, but there are cases where the operation key is small or the character size is small, so that the operation is difficult. Therefore, in order to prevent user's operation mistakes, a technology for assisting operation of an electronic device by voice recognition has been developed. For example, when the user speaks a search item into the microphone, a technology is proposed that searches the information by recognizing the search item, or automatically adjusts the volume of the audio device when a specific sound registered in advance is detected. (For example, see Patent Documents 1 and 2).

Japanese Patent Laid-Open No. 10-021254 JP 2002-305421 A

  However, the techniques of Patent Documents 1 and 2 are techniques for preventing a user's operation mistake, and when the user makes a mistake in the operation, the process executed by the mistaken operation can be easily restored. could not.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of easily returning an operation mistaken by a user.

The present invention provides a display unit that displays an operation screen on which a plurality of icons are arranged, an operation unit that receives a drag-and-drop operation on the icon, and an operation that executes an operation process corresponding to the received drag-and-drop operation A processing unit; a device state holding unit that holds a device state before the operation processing; a microphone that acquires voice; a keyword registration unit that registers voice indicating operation failure via the microphone as a keyword; and the operation unit between the drag and drop preset voice recognition period after the operation is started in the voice recognition unit for detecting the keyword from the voice acquired by the microphone, the keyword is detected by the voice recognition unit case, and a state return unit for returning the device state held by the device state storage section, the speech certification The unit detects a voice whose voice level is equal to or higher than a predetermined threshold as a voice indicating an operation failure, and the state returning unit detects the voice whose voice level is equal to or higher than a predetermined threshold by the voice recognition unit The keyword registration unit detects a voice whose voice level is equal to or higher than a predetermined threshold by the voice recognition unit, and the detected voice is used as the keyword. When not registered, a confirmation screen for confirming whether to detect the detected voice as the keyword is displayed on the display unit, and when the registration is instructed, the detected voice is registered as the keyword. It is characterized by that .
Also, when the keyword by the speech recognition unit is detected, to display the confirmation screen to confirm whether to return the device status on the display unit, when an instruction to return the device state via the confirmation screen is The apparatus state held by the apparatus state holding unit may be returned to.

  According to the present invention, an operation mistaken by the user can be easily restored.

It is a functional block diagram which shows the structure of the electronic device which concerns on this invention. It is a flowchart which shows the flow of a process when the electronic device shown in FIG. 1 is operated. It is an image figure which shows the example of operation when a key is operated correctly. It is an image figure which shows the example of operation when a key operation is carried out accidentally. It is the figure which showed the execution timing of the operation part shown in FIG. 1, an operation process part, and a speech recognition part.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that, in the following embodiments, the same reference numerals are given to configurations showing similar functions.

  The electronic device 1 is an operation device capable of touch operation and mouse operation, such as a smartphone, a tablet terminal, and a personal computer. Referring to FIG. 1, the operation unit 11, the storage unit 12, and the communication unit 13 The microphone 14 and the control unit 15 are provided. Hereinafter, the electronic device 1 may be described as an apparatus.

  The operation unit 11 includes a touch panel and operation buttons. The touch panel is provided with a transparent pressure-sensitive sensor on the surface of a display unit that displays various operation keys and operation status, and functions as a display unit and an input unit. The touch panel detects a touch operation on the display surface and outputs a signal corresponding to the position where the touch operation is detected, thereby accepting an operation on the operation key displayed on the display unit. The operation buttons also include various operation keys such as a numeric keypad for inputting numerical values and character keys for inputting characters such as alphabets. The operation unit 11 may be connected to a mouse and may accept a pointer operation via the mouse.

  The storage unit 12 is a storage unit such as a semiconductor memory or an HDD (Hard Disk Drive). In the storage unit 12, a voice indicating an operation failure is registered in advance as a keyword 121.

  The communication unit 13 has a function of transmitting / receiving various data to / from an external terminal via a network such as a LAN (Local Area Network).

  The microphone 14 is a microphone or the like and is a sound collecting device that acquires sound.

  The control unit 15 is connected to the operation unit 11, the storage unit 12, the communication unit 13, and the microphone 14, and executes operation control of the entire electronic device according to instruction information input from the operation unit 11. The control unit 15 is an information processing unit such as a microcomputer including a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The ROM stores a control program for controlling the operation of the electronic device. The control unit 15 reads the control program stored in the ROM and develops the control program in the RAM, thereby controlling the entire device according to the instruction information input from the operation unit 11.

  In addition, the control unit 15 functions as a keyword registration unit 151, a voice recognition unit 152, an operation processing unit 153, a device state holding unit 154, and a state returning unit 155.

  The keyword registration unit 151 has a function of registering voice indicating operation failure as the keyword 121. The keyword registration unit 151 registers the voice that the user utters toward the microphone 14 in advance as the keyword 121. As the keyword 121, for example, “Ah” or “I made a mistake” can be registered as words that the user unintentionally says when the operation fails.

The voice recognition unit 152 has a function of performing voice recognition of voice acquired via the microphone 14 and detecting the keyword 121 registered by the keyword registration unit 151. The speech recognition unit 152 performs the lead to a key operation on the operation unit 11 has been started is detected, starts a speech recognition process, a preset speech recognition phase period of the speech recognition process, the keyword 121 Is detected. By limiting the voice recognition period, the load on the electronic device can be reduced.

  The operation processing unit 153 executes an operation process corresponding to the key operation (operation on the operation key) received by the operation unit 11 to operate the electronic device.

  The device state holding unit 154 includes a RAM and the like, and has a function of temporarily holding the device state of the electronic device 1 before the operation processing corresponding to the key operation is executed by the operation processing unit 153.

  When the speech recognition unit 152 detects the keyword 121 from the voice, the state return unit 155 causes the operation unit 11 to display a confirmation screen for confirming whether the operation is to be restored. When an instruction to restore the operation is given via the confirmation screen, the status returning unit 155 changes the device status of the electronic apparatus 1 based on the device status held by the device status holding unit 154 to the operation unit 11. To return to the state before the key operation was accepted. When the continuation of operation is instructed via the confirmation screen, the status returning unit 155 maintains the current device status.

  Next, with reference to FIG. 2, the flow of processing when the electronic apparatus of this embodiment is operated will be described.

  First, the device state holding unit 154 holds the device state of the electronic device 1 (step s11), and the operation unit 11 waits until it detects the start of a key operation (No in step s12 and step s12). When the operation unit 11 detects the start of key operation (Yes in step s12), the voice recognition unit 152 starts voice recognition of the voice acquired by the microphone 14 (step s13).

  Subsequently, the operation unit 11 stands by until the key operation is finished (No in Step s14 and Step s14). For example, when it is detected that the user has pressed an operation key (key operation start) and released from the pressed state, the end of the key operation is detected. In this case, if the user keeps pressing the operation key for a short time, the operation is performed as a click operation, and if the pressed state is maintained for a long time, the operation is performed as a long press operation. A key operation is accepted by the operation unit 11 as a drop operation. When the key operation ends (Yes in step s14), the operation processing unit 153 executes an operation process corresponding to the key operation accepted by the operation unit 11 (step s15).

  Subsequently, the voice recognition unit 152 determines whether or not the keyword 121 is detected by voice recognition (step s16). When the keyword 121 is detected (Yes in step s16), the state returning unit 155 displays a confirmation screen 50 (see FIG. 4) for confirming whether or not to return the operation on the operating unit 11 (step s17). ). The confirmation screen 50 is provided with a “return” key 51 for issuing an instruction to return the operation, and a “continue” key 52 for continuing the operation.

  When the “return” key 51 is pressed (Yes in step s18), the state return unit 155 changes the current device state to the device state before the key operation based on the device state held by the device state holding unit 154. Return (step s19), and the process is terminated.

  When the keyword 121 is not detected or when the “continue” key 52 is pressed (No in step s16, No in step s18), the status returning unit 155 maintains the current device status and performs this process. finish.

  As a result, for example, as shown in the operation screen G80 of FIG. 3, the dragged icon 40d is dropped between the icon 40a and the icon 40b by the user's drag-and-drop operation. Assume that it is desired to rearrange the order of 40d in the order of icons 40a, 40d, 40b, and 40c as shown in the operation screen G81. However, by mistake, the dragged icon 40d is dropped on the icon 40b as shown in the operation screen G82 in FIG. 4, and the icon 40b and the icon 40d are combined as shown in the operation screen G83. Assume that the icon has become 40bd. Therefore, if the user unintentionally says “Oh, I made a mistake”, “Oh” is detected as the keyword 121 by the voice recognition unit 152, and a confirmation screen 50 is displayed to confirm whether the operation should be restored as shown in the operation screen G84. Is done. When the “return” key 51 is pressed on the confirmation screen 50, the operation screen G85 returned to the device state before the drag-and-drop operation is detected based on the device state held by the device state holding unit 154 is displayed. Is displayed.

  As described above, the electronic apparatus 1 according to the present embodiment holds the operation unit 11 that receives a key operation, the operation processing unit 153 that executes an operation process corresponding to the received key operation, and the apparatus state before the operation process. The apparatus state holding unit 154, the microphone 14 for acquiring the voice, and the voice indicating the operation failure from the voice acquired by the microphone 14 during the voice recognition period set in advance after the key operation is started by the operation unit 11. A voice recognition unit 152 to be detected, and a state return unit 155 for returning to the device state held by the device state holding unit 154 when the voice recognition unit 152 detects a voice indicating an operation failure. Accordingly, when the user makes a voice indicating that the operation has failed during the operation, the apparatus returns to the state before the user performs the operation, and thus the operation can be easily performed again.

  The voice recognition unit 152 detects a voice (keyword 121) registered in advance via the microphone 14 as a voice indicating the operation failure, but detects a voice whose voice level is equal to or higher than a predetermined threshold. Also good. This is because when the operation fails, a loud voice may be produced unexpectedly.

  The keyword registration unit 151 also detects the detected voice when the voice recognition unit 152 detects a voice whose voice level is equal to or higher than a predetermined threshold and the detected voice is not registered as the keyword 121. May be displayed on the operation unit 11 for confirming whether or not to be registered as the keyword 121. When instructed to register via this confirmation screen, the keyword registration unit 151 registers the detected voice as the keyword 121. According to this, it is possible to register, as the keyword 121, a voice indicating an operation failure actually issued by the user during the operation.

  Further, as illustrated in FIG. 5, the operation processing unit 153 is configured such that the operation process corresponding to the key operation received by the operation unit 11 is a normal operation process (an operation process that is not set as an important operation process). As soon as a key operation is accepted by the operation unit 11, an operation process corresponding to the key operation is executed. On the other hand, when the operation process corresponding to the key operation received by the operation unit 11 is an important operation process set in advance, the operation processing unit 153, for example, has a certain holding period t61 such as a few seconds. Suspend execution until

  As important operation processes set in advance, the user can set telephone call, LINE (registered trademark) transmission, SNS transmission, mail transmission, charging execution, data deletion, and the like. For example, when a calling process is executed like a telephone call, there is a possibility that an incoming call history may remain in the other party even if the state returning unit 155 returns the apparatus state. Therefore, by providing the holding period t61, a preset important operation process is not immediately executed. Therefore, when a voice with an operation failure is detected in the voice recognition period t62 of the voice recognition unit 152, the state return unit 155 causes the device to return to the device. The failed operation can be restored by returning the status. When the voice of the operation failure is detected by the voice recognition unit 152, important operation processing set in advance by the operation processing unit 153 until the “return” key 51 or the “continue” key 52 is pressed via the confirmation screen 50. Is put on hold. When the “return” key 51 is pressed, the preset important operation processing is stopped, and when the “continue” key 52 is pressed, the preset important operation processing is executed. .

  The present invention is not limited to the above-described embodiment, and it goes without saying that various changes can be made without departing from the spirit of the present invention. For example, the electronic device 1 may be applied to an image forming apparatus.

DESCRIPTION OF SYMBOLS 1 Electronic device 11 Operation part 12 Storage part 13 Communication part 14 Microphone 15 Control part 121 Keyword 151 Keyword registration part 152 Speech recognition part 153 Operation processing part 154 Device state holding | maintenance part 155 State return part 40a-40d, 40bd icon G80-G85 Operation Screen 50 Confirmation screen 51, 52 Key t61 Holding period t62 Voice recognition period

Claims (2)

A display unit for displaying an operation screen on which a plurality of icons are arranged;
An operation unit for accepting a drag-and-drop operation on the icon ;
An operation processing unit that executes an operation process corresponding to the accepted drag-and-drop operation ;
A device state holding unit for holding a device state before the operation processing;
A microphone to capture audio,
A keyword registration unit for registering voice indicating operation failure as a keyword via the microphone;
A speech recognition unit that detects the keyword from speech acquired by the microphone during a speech recognition period set in advance after the drag-and-drop operation is started in the operation unit;
A state returning unit for returning to the device state held by the device state holding unit when the keyword is detected by the voice recognition unit ;
The voice recognition unit detects a voice having a voice level equal to or higher than a predetermined threshold as a voice indicating an operation failure,
The state returning unit returns to the device state held by the device state holding unit when a voice whose voice level is equal to or higher than a predetermined threshold is detected by the voice recognition unit,
The keyword registration unit registers a detected voice as the keyword when the voice recognition unit detects a voice whose voice level is equal to or higher than a predetermined threshold and the detected voice is not registered as the keyword. An electronic device characterized by displaying a confirmation screen for confirming whether or not to register the detected voice as the keyword when an instruction to register is displayed . The state returning unit is
When the keyword is detected by the voice recognition unit, a confirmation screen for confirming whether to return the device state is displayed on the display unit, and when an instruction to return the device state is given via the confirmation screen, the device The electronic device according to claim 1 , wherein the electronic device is returned to the device state held by the state holding unit.

Images