JP3364569B2 - Voice information processing device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice information processing apparatus which is applied to an interface device such as an answering machine, a voice mail, a portable information terminal or the like, and which displays and edits voice information.

[0002]

2. Description of the Related Art Currently, various sound source editors for editing sound data by transforming a waveform are put into practical use. Also, a multimedia editor that puts a button-like mark in the text and specifies the button to reproduce sound data has been put into practical use.

For example, an audio data processing device for displaying the sound data with the strength of the recorded sound data on the vertical axis and the time on the horizontal axis and performing operations such as cutting, copying and pace is disclosed in Japanese Patent Application Laid-Open No. HEI-1. No. 172900.

On the other hand, as a data input method, pen input,
Devices such as mobile terminals, which use ultra-compact keyboards and the like, and which emphasize portability at the expense of input efficiency, have been put into practical use. Voice input / output is the most familiar input / output method for users, and has the advantage that the equipment required can be made extremely compact without sacrificing input efficiency. Suitable for devices.

However, since voice information processing such as displaying, editing, and reusing messages of voice data is difficult, voice input / output is not fully utilized in mobile terminals at present. In addition, the same cause hinders the spread of voice mail.

[0006] Even in the answering machine, due to the similar difficulty of voice information processing, no one having the function of correcting and editing the input voice has been realized, and it is understood that the answering machine is an answering machine. There are many users who terminate the call at the time. That is, for example, if you make a mistake in the answering machine, repeat it, or enter a meaningless expression such as "ah" or "hmm", you cannot correct or delete the input and perform voice information processing such as deletion. Therefore, many users are afraid of these inputs and abandon the inputs themselves. Therefore, by adding a voice information processing function to such an answering machine, it is possible to correct or delete the above-mentioned input error, repetition, meaningless expression, etc., so that the number of users who refrain from inputting can be reduced. It is expected.

However, the above-mentioned Japanese Patent Laid-Open No. 1-1729.
It is difficult to apply what is disclosed in Japanese Patent Publication No. 00 to mobile terminals, voice mails, and answering machines. Therefore, for example, it is conceivable to adopt the technique disclosed in Japanese Patent Publication No. 63-30645. In this Japanese Examined Patent Publication No. 63-30645, voice data is displayed in a mixed manner with a text by using a certification mark, and voice data can be freely moved, deleted, inserted, reproduced and the like like the text data. A possible information processing system is disclosed. That is, the one disclosed in Japanese Patent Publication No. Sho 63-30645 is capable of editing and reusing a voice message by displaying and editing a mixed character string and voice.

[0008]

However, in the one disclosed in the above Japanese Patent Publication No. Sho 63-30645, since the voice is handled in a large block called the certification mark, when the voice data is edited, the words in the voice are edited. There is a very high possibility that a break will occur in the middle position of, and it will be difficult to handle the sound as a meaningful unit during editing, and therefore the quality of the edited sound will be very low. Also, if there is no voice input, intermittently input voice data is treated as continuous input to prevent data storage, and the user does not know the voice gap, unless the user manually instructs the break. Therefore, voice editing becomes very difficult.

The present invention has been made in order to solve the above problems, and greatly improves the display, which is a drawback of conventional voice information, and facilitates editing and reuse of voice information. It is an object of the present invention to provide a voice information processing device that performs

[0010]

In order to solve the above problems, the present invention provides a data analysis unit for analyzing voice information.
And the time of voice information analyzed by the data analysis unit
Control to display as a symbol with a size corresponding to the length.
Voice information processing device composed of a symbol display control unit
In the display, the symbol display control unit displays 0.1
The symbol length is defined as a unit of time length from 2 seconds to 0.3 seconds.
Control the display of the voice information for about 0.05 seconds
Display a symbol with a height-to-width ratio of 4: 1
It is configured to do.

[0011]

[0012]

[0013]

The basic processing of the present invention having the above configuration is as follows. In the data analysis unit, the voice information to be displayed is divided into specific time lengths and transmitted to the symbol display control unit. The symbol display control unit selects a symbol corresponding to the data and performs two-dimensional display in the display area.
If there is a past display, the display position of the next symbol is calculated from the display state and transmitted to the display unit. At this time, the symbol display control unit controls the symbol display with the time length of 0.1 seconds to 0.3 seconds of the voice information as one unit. The display unit displays a symbol based on audio information at a predetermined position.

Here, in the present invention, a description will be given of the case where the symbol display is performed with the time length of 0.1 seconds to 0.3 seconds of the voice information as one unit. The speaking speed of Japanese is 8 Mora per second by the announcer.
It is said that 12 aura from a) and 5 to 8 aura by the general public. This is a hiragana display, indicating that the announcer speaks 5 to 8 characters per second and the general public speaks 5 to 8 characters. By the way, when we look at sentences, the ratio of display lengths between the kanji and kana kanji mixed display and the hiragana display of the same content is approximately "kanji kana mixed sentence: hiragana sentence" from "2: 3" to "4: 5".
It is said that. Based on these conditions, the relationship between the voice time length and the symbol display length is calculated. From the speech rate, one second of voice corresponds to 5 to 12 characters of Hiragana.

On the other hand, in the kanji kana mixed sentence, "2: 3"
3.3 characters to 8 characters, "4: 5" 4 characters to 9 characters.
It corresponds to 6 characters, that is, 3.3 to 9.6 characters. From these, the time length per character is the reciprocal of the number of characters, so 1 / 9.6 seconds to 1 / 3.3 per character.
Seconds, that is, 0.1 seconds to 0.3 seconds. Therefore, one unit of symbol display length is 0.1 seconds to 0.3
By making the seconds correspond to the voice information as one unit, the utterance content can be expressed with a display length similar to a kanji kana mixed sentence, and a visually very natural display is possible. In addition,
In general, Japanese display characters are squares having an aspect ratio of "1: 1", and therefore it is preferable that one unit of the symbol display has an aspect ratio of 1: 1. If one unit is expressed by a plurality of smaller parts, a natural editing operation becomes possible.

Further, according to the present invention, in the above-mentioned voice information processing apparatus, a text input section for inputting a character string is provided, and the symbol display control section causes a character input to the text input section to be a symbol based on voice information. The columns are multiplexed to control the symbol display.

The basic processing of the present invention having the above configuration is as follows. In the data analysis unit, the voice information to be displayed is divided into specific time lengths and transmitted to the symbol display control unit. The symbol display control unit selects a symbol corresponding to the data and performs two-dimensional display in the display area.
If there is a past display, the display position of the next symbol is calculated from the display state and transmitted to the display unit. The display unit displays a symbol based on audio information at a predetermined position. on the other hand,
When a character string related to voice information is input from the text input unit and the character string is found, the target voice information and the related character string are transmitted to the symbol display control unit. The symbol display control unit selects a symbol corresponding to the target voice information, calculates the display position of the next symbol from the past display state, and transmits the calculated display position to the display unit, and also displays the same display position and the related character string on the display unit. Communicate to. The display unit displays the symbol and the character string at a predetermined position. At this time, the symbol and the character string are multiplexed and displayed.

Further, according to the present invention, in the above-described voice information processing device, the symbol display control unit is configured to control the display length of the symbol in correspondence with the display length of the character string.

According to the present invention, when the display length of the symbol corresponds to the display length of the symbol and the character is reduced and becomes difficult to read, the display length of the character string is made to correspond to the display length of the symbol. Since the display length is controlled to be displayed, it is possible to display characters in an easy-to-read manner.

Further, according to the present invention, in the above-described voice information processing apparatus, the symbol display control unit is configured to determine the display attribute of the symbol according to the attribute of the voice information and control the display of the symbol. .

According to the present invention, in the symbol display controller,
The display attributes of symbols such as line width, display size, display interval, display color, underline, tilt, and shading are determined according to the attributes of audio information such as volume, frequency, pitch (periodicity), and the like. Since it is configured to perform display control,
It is possible to display such that the contents of the voice information can be easily visually recognized.

Further, according to the present invention, in the above-described voice information processing apparatus, the symbol display control unit determines a plurality of display attributes of the symbol according to a plurality of attributes of the voice information and controls the display of the symbol. I am configuring.

According to the present invention, in the symbol display controller,
The display attributes of the symbols are determined by determining the display attributes of the symbols according to the attributes of the audio information, so it is easy to visually see the information of the audio information contents. A display that can be recognized by the user becomes possible.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of a first embodiment of a voice information processing apparatus of the present invention. This voice information processing device assumes an interface similar to a text editor.

In FIG. 1, 101 is a voice input unit for inputting voice information, 102 is a data analysis unit for analyzing data of voice information input from the voice input unit 101, and 103 is voice analyzed by the data analysis unit 102. Symbol display control unit for controlling display of symbols representing data, 10
Reference numeral 4 denotes a silent portion display control unit that controls the display of the silent portion analyzed by the data analysis unit 103, and reference numeral 105 denotes a display unit that displays the symbols representing the audio data controlled by the symbol display control unit 103. As the voice input unit 101, an external memory holding previously input voice data, a microphone, or the like can be used. However, when a voice input unit 101 that does not have a memory function, such as a microphone, is used, a memory for storing voice data is required for later editing and reuse of voice data.

As shown in FIG. 1, the voice data input from the voice input unit 101 is analyzed by the data analysis unit 102. Of the results obtained by this analysis, information on the time length of the audio data is transmitted to the symbol display control unit 103. Then, the symbol display control unit 103 prepares one unit shape of the symbol to be displayed in a strip shape having a width of ¼ with respect to the height, and for about 0.05 seconds of voice information, The display is controlled to be displayed at the rate of one symbol. As a result, a symbol having an aspect ratio of 1: 1 is about 0.
Since it corresponds to 2 seconds of voice, the display is similar to the speaking speed and the number of characters when reading a sentence containing Japanese kanji and kana. An example of the display is shown in FIG. This is a symbol that corresponds to the text "Speech synthesis from text" (speech "Speak from the text").
It is shown in correspondence with text and voice, and it indicates that the time required for uttering the voice "I'll come to you" is about 2.8 seconds.

Further, in the case of the present embodiment, in the calculation of the display position, the display position is linearly specified from the upper left to the right so as to realize an appearance similar to that of the text editor, and when the display position reaches the right end, It is designed to continue from the right end of the line.

When the voice data has a silent portion, punctuation marks, spaces, line feeds, etc. are displayed as symbols representing the silent portion. That is, when the data analysis unit 101 detects a silent section, the time length of the silent section is transmitted to the silent section display control unit 104, and as a symbol representing silence corresponding to the silent time, a punctuation mark, a reading point, a line break, etc. Selected and transmitted to the symbol controller. Silence symbol is 0.005
It corresponds to a silent part of 0.2 seconds for the reading point and 0.4 seconds for the punctuation mark, with a maximum of three per second. If the silent part continues for 0.4 seconds or more, a line break is inserted after the punctuation mark, and then 0.8
Line breaks are performed by the number of seconds corresponding to the time length of the silent section. As a result, for example, the display as shown in FIGS. 3 and 4 is obtained. Here, FIG. 3 shows 2 by the strip-shaped symbol.
FIG. 4 shows the one-dimensional display, and FIG. 4 shows a part of the symbol in association with the voice data.

In addition to the above, as the display control of the silent portion, it is possible to make a setting such that the silent portion thereafter is ignored when two or more line feeds continue.

Next, as a second embodiment, in addition to that of the first embodiment, the user displays the strip-shaped symbol displayed based on the input voice data by superimposing a character string. A voice information processing apparatus having a function that can be performed will be described.

As shown in FIG. 5, the voice information processing apparatus of the second embodiment is different from that of the first embodiment in that a text input unit 107 for inputting text such as a character string and a user process. And a data processing unit 1 for processing the text data input from the text input unit 107 based on the instruction from the processing instruction unit 108.
06, and an audio output unit 109 that outputs audio data.
It has a configuration that includes and.

As described in the first embodiment, the strip-shaped symbol can be determined by analyzing the voice data, but when the character string is multiplexed on the symbol, the character string is held in association with the voice data. There is a need to. Therefore, first, the association will be described with reference to FIG. 6 showing a display example according to the second embodiment. First, the voice input unit 10
Voice data V10001, V10002 input to
On the display unit 105, the symbols S1001 and S1
It is assumed that 002 is displayed (FIG. 6A).

Next, the user analogizes the target voice data portion and selects the portion from the processing instruction unit 108. Then, the audio data of the portion is reproduced by the audio output unit 109 from the data analysis unit 102 via the data processing unit 106, so that it is confirmed whether or not it is the target. Then, if there is an unnecessary portion before or after that, or if the selection area is insufficient, the process instructing unit 108 once cancels the selection and then performs the selection again. When the selection is correct and the processing instruction unit 108 selects the symbol S1002, the data processing unit 106 is notified of the selection range.

When the text T1002 is input to the text input unit 107 with the symbol S1002 selected by the process designation unit 108, the data processing unit 106 displays the symbol S1002 and the text T1.
002. The data analysis unit 102 detects the associated text T1002 and transmits it to the symbol display control unit 103. Then, in the display unit 105, the display length of the symbol is adjusted to be the same as the display length of the character string according to the display length of the character string of the text data,
The symbol is multiplexed with the character string and displayed as shown in FIG. In FIG. 6B, as an example, the text T1002 is “overlaid with text”.

In the second embodiment, the display length of the symbol is adjusted to be the same as the display length of the character string according to the display length of the character string of the text data, and the symbol is multiplexed with the character string. However, this is to prevent the characters from becoming small and difficult to read. If such an inconvenience does not occur, the display length of the symbol is adjusted to be the same as the display length of the symbol string of the audio data according to the display length of the symbol based on the audio data, and the symbol is changed to the character string. You may make it multiple.

Next, in the second embodiment, referring to FIG. 7 showing an example of two-dimensional display by strip-shaped symbols,
The overall symbol display will be described. In the present embodiment, the audio data is represented by 16 types of symbols in which the display color is an achromatic color from white to black and the brightness (density) is divided into 16 equal parts. For determining the display color of the symbol,
Calculate the average value of the volume of the voice data corresponding to each symbol, compare the calculated average value with a table that is predetermined so that the larger value becomes darker (darker), and display the symbol of the corresponding display color. This is adopted, and an example of the display is as shown in FIG.

In the second embodiment, the display color (display density) of the symbol is determined according to the volume of the voice information (voice data), but instead of this, as shown in FIG. It is also possible to use a kind of shading as a display attribute of. In addition to this, symbols may be displayed using display attributes such as line width, display size, display interval, underline, and inclination. And the above 2
In the embodiment, the symbol display corresponding to the volume is performed as the voice attribute, but the symbol display may be performed using the voice attribute such as the frequency component and the pitch instead of the volume.

The audio output unit 109 can be used for reproduction (audio output) of audio data as appropriate.

As a third embodiment, a description will be given of a case where a symbol is displayed with a plurality of display attributes in association with a plurality of voice attributes. The process of the present embodiment is performed almost in the same manner as in the second embodiment, except that it is based on a plurality of audio attributes such as volume, frequency, and pitch of audio information (audio data). , Display color, line width,
Symbol display control is performed by a plurality of attributes of the symbol such as display size, display interval, underline, and inclination.

FIG. 9 shows an example of the symbol display of the third embodiment. The display shown in FIG. 9 is displayed by changing the display size in correspondence with the volume and changing the meshing in correspondence with the pitch. More specifically, since the area of the audio data V701 has a large volume and a rough pitch (periodicity), the corresponding symbol S701 has a large display size and a light shade. In the area of the audio data V702, since the volume is large and the pitch (periodicity) is fine, the corresponding symbol S702 has a large display size and a dark shade. Also, the volume of the audio data V703 is small and the pitch (periodicity) is small.
Is rough, the corresponding symbol S703 has a small display size and a light shade. Since the volume is small and the pitch (periodicity) is fine in the area of the audio data V704, the display size of the corresponding symbol S704 is small and the shade is dark.

[0042]

As described above, according to the present invention, it is possible to display voice information whose contents can be easily visually recognized.

Therefore, by adopting the present invention, it is possible to easily edit and search the voice data, and also to easily reuse the voice data such as editing a part of the voice data.

Further, according to the present invention, since silent parts symbols, punctuation marks, and line breaks are automatically displayed, it is possible to more easily analogize the contents of the voice, so that the voice data can be reused easily. Become.

Further, according to the present invention, the contents of the voice data can be displayed as character information by using the voice symbol and the text, and the reusability of the voice data can be improved.

Furthermore, the present invention is suitable for small-sized systems such as portable terminals that place importance on portability, and if applied to these systems, data input / output can be performed without sacrificing input efficiency. Is.

If the present invention is applied in combination with a pen interface, a touch panel, etc., it is possible to construct an interface that is intuitive and familiar to beginners.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a voice information processing device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a display symbol and a voice corresponding to a text of a Japanese kanji kana mixed sentence of “synthesize voice from text”.

FIG. 3 is a diagram showing an example in which two-dimensional display is performed using strip-shaped symbols.

FIG. 4 is a diagram showing a part of the symbol display of FIG. 3 in association with audio data.

FIG. 5 is a block diagram showing a schematic configuration of a voice information processing device according to a second embodiment of the present invention.

FIG. 6 is a diagram showing a text, a symbol, and a text in association with each other in a process of displaying a symbol based on a text and a symbol based on text in a multiplexed manner.

FIG. 7 is a diagram showing an example in which two-dimensional display is performed using symbols using display colors (display densities).

FIG. 8 is a diagram showing an example in which two-dimensional display is performed by using a symbol using a mesh.

FIG. 9 is an example of a display in which a plurality of display attributes of a symbol are associated with a plurality of attributes of voice information in the voice information processing apparatus according to the third embodiment of the present invention. It is a figure which shows the audio | voice data and the symbol at the time of making each mesh correspond to the pitch (periodicity) of audio | voice information.

[Explanation of symbols]

101 voice input unit 102 Data analysis unit 103 Symbol display control unit 104 Voiceless display control unit 105 display 106 data processing unit 107 text input section 108 processing instruction unit 109 audio output section

Claims (5)

(57) [Claims] 1. A data analysis unit for analyzing voice information,
The time length of the voice information analyzed by the data analysis unit
A thin control that displays as a symbol of the corresponding size.
In the voice information processing device composed of the display control unit
In addition, the symbol display control unit starts from 0.1 seconds of voice information.
Symbol display with a unit of time length up to 0.3 seconds
Control, and about 0.05 seconds of voice information, the height
Display a symbol with a ratio of width to width of 4: 1
A voice information processing device characterized by: 2. The voice information processing device according to claim 1.
A text input section for inputting a character string, and
The display control unit displays the
A symbol table that multiplexes the character strings entered in the text input section.
An audio information processing apparatus, characterized by performing the control shown. 3. The voice information processing apparatus according to claim 2,
The symbol display control unit corresponds to the display length of the character string.
And display control of the symbol display length is performed.
Voice information processing device. 4. The method according to any one of claims 1 to 3.
In the voice information processing device, the symbol display control unit changes the system according to the attribute of voice information.
Determine display attributes of symbols and control symbol display
A voice information processing device characterized by the above. 5. The voice information processing apparatus according to claim 4,
And the symbol display control unit responds to a plurality of attributes of voice information.
The symbol display to determine multiple display attributes for the symbol.
An audio information processing apparatus characterized by performing an indication control.