JP4465221B2 - Speech data summarization system and its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech data summarization system with which a portion of prescribed speech can be summarized and reproduced, for example, in which an auditor shows an interest, from memoranda etc., written by many auditors when the system is applied to a lecture etc. <P>SOLUTION: The system comprises a plurality of writing detection terminals 2 for detecting that which is written in recording media and a speech data summarization terminal 4 which inputs the speech and summarizes the speech data indicating the speech inputted based on the writing detection information detected at the writing detection terminals 2. The writing detection terminals 2 detects that which is written and transmits the detected writing detection information. The speech data summarization terminal 4 memorizes the speech data by making the same correspondent to the input time when the speech is inputted. The system receives the writing detection information transmitted by the writing detection terminals 2, memorizes the writing frequency data making the received writing detection information and time correspondent to each other, acquires the frequency time of the occurrence of the frequency when the writing frequency data exceeds the threshold and summarizes the memorized speech data based on the acquired frequency time. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to a voice data summarization system, and more particularly to a voice data summarization system that summarizes voices recorded based on written data obtained by a tablet or the like.

Conventionally, a character that recognizes characters using stroke information written with a pen, or captures image information of characters written on paper using a scanner or camera, and recognizes characters based on the image information A recognition device is known (see, for example, Patent Document 1). Furthermore, a system in which such a character recognition device is combined with a recording / reproducing device for recording and reproducing sound has been considered. This system records voice continuously in a meeting, etc., and stores the written data and the voice data in synchronism as a minutes creation memo, and reproduces the corresponding voice by instructing the written data after the meeting. It is like that.
JP-A-8-161435

  However, in the above-described conventional technology, when applied at the time of a lecture or the like, it is not possible to summarize and reproduce, for example, a part of the audio that the audience showed interest from, for example, notes written by many listeners. The problem was left.

  The present invention has been made in order to solve such problems, and the purpose of the present invention was, for example, the audience showed interest from notes written by many audiences when applied at the time of lectures, etc. An object of the present invention is to provide an audio data summarization system and program for summarizing and reproducing a part of a specific voice such as voice.

The voice data summarization system according to claim 1 is provided with a plurality of writing detection terminals that are held by a listener and that detects writing on a recording medium, and writing detection information that is input to the voice and detected at the writing detection terminal. A voice data summarizing terminal that summarizes voice data indicating the input voice based on
The writing detection terminal is a writing detection means for detecting that the listener is writing,
Transmission means for transmitting the writing detection information detected by the writing detection means,
The voice data summarizing terminal stores a time data acquisition unit for measuring time, a voice data storage for storing the voice data indicating the input voice and the input time when the voice measured by the time acquisition unit is input. Means, receiving means for receiving the handwritten detection information transmitted by the transmitting means, correspondence between the handwritten detection information received by the receiving means and the time measured by the time acquiring means, and for each attribute of the listener The writing frequency data storage means for storing the writing frequency data, and the writing frequency data determined by the user who requests the summary of the voice data among the writing frequency data for each attribute stored by the writing frequency data storage means , A frequency time acquisition unit that acquires a frequency time at which a frequency at which the frequency data exceeds a threshold occurs, and the frequency time acquisition unit. And a summary means for summarizing the voice data stored in the voice data storage means on the basis of the frequency time. Further, as in the speech data summarization system according to claim 2, in the speech data summarization system according to claim 1, the handwriting frequency data determined by the user who requests summarization of the speech data is the handwriting best matching the attribute of the user. It may be frequency data.

  According to the present invention, since voice data can be summarized based on writing frequency data obtained from writing detection information transmitted from a plurality of writing detection terminals, for example, when applied at the time of a lecture, etc. A part of the voice that the listener showed interest in can be summarized and reproduced from a memo written by the listener.

Further, the voice data summarizing system according to claim 3 is the voice data summarizing system according to claim 1 and 2 , wherein the summarizing means determines an unvoiced section and a voiced section of the voice data stored in the voice data storing means. Then, the reproduction start time of the audio data is calculated based on the voiced section.

  According to the present invention, since the unvoiced and voiced sections of the accumulated voice data are determined and the reproduction start time of the voice data is calculated from the voiced sections, it is possible to summarize the voice data only in the voiced sections.

  The above audio data summarization system can also be realized by a computer and a program. That is, if a program for causing a computer to function as means for configuring the audio data summarization system according to any one of claims 1 to 3 is recorded on a computer-readable recording medium or provided through a network, the computer It can be implemented. Examples of the recording medium include a flexible disk, MO, ROM, memory card, CD, DVD, and removable disk.

  As described above, according to the audio data summarization system and the program thereof of the present invention, when applied at the time of a lecture or the like, from a memo etc. written by a large number of listeners, for example, a voice in which the listener has shown interest It is possible to provide a system capable of summarizing and reproducing a part of a specific voice.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an audio data summarizing system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a functional configuration of the audio data summarizing system.

  The voice data summarization system 1 shown in FIGS. 1 and 2 includes a plurality of writing detection terminals 2 for detecting writing on a recording medium, a microphone device 3 as voice input means for inputting voice, and writing detection. A voice data summarizing terminal 4 that summarizes voice data indicating voice input based on the handwriting detection information detected in the terminal 2 and a speaker as voice output means for outputting voice data reserved by the voice data summarizing terminal 4 And a device 5.

  For example, when the present invention is applied at the time of a lecture, the lecturer continuously records voice using the microphone device 3. Each listener has a writing detection terminal 2, puts a recording medium 6 such as paper on the writing detection terminal 2, and takes notes on the recording medium 6 using the pen 7 while listening to the lecture. The writing detection terminal 2 may be configured by a pointing device such as a tablet.

  The writing detection terminal 2 shown in FIG. 2 includes a writing detection unit 20 that detects writing, and a transmission unit 21 that transmits writing detection information detected by the writing detection unit 20.

  The writing detection information includes an ID for identifying the writing detection terminal 2 and information indicating that writing has started, and the writing detection means 20 is stored in the recording medium 6 placed on the writing detection terminal 2. When the tip of the pen 7 is touched, the start of writing is detected, and writing detection information including information indicating that writing has started is generated.

  When the writing detection terminal 2 has a clock, the writing detection means 20 may include time information in the writing detection information. When the time information is included in the handwriting detection information, the transmission means 21 may temporarily store the handwriting detection information and transmit a plurality of handwriting detection information collectively.

  The transmission means 21 includes an antenna, a driver, and a receiver, and may transmit and receive information wirelessly, or includes an interface such as a USB (Universal Serial Bus) and a network, and transmits and receives information via a wire. Also good.

  On the other hand, the voice data summarizing terminal 4 associates the time acquisition means 41 for measuring time with the voice data indicating the voice input by the microphone device 3 and the input time when the voice obtained by the time acquisition means 41 is input. Voice data storage means 42 to be stored, receiving means 43 for receiving the handwriting detection information transmitted by the transmitting means 21, handwriting detection information received by the receiving means 43 and the time obtained by the time acquisition means 41. Writing frequency data storage means 44 for storing the corresponding writing frequency data, frequency time acquisition means 45 for acquiring a frequency time at which the writing frequency data stored by the writing frequency data storage means 44 exceeds the threshold, and Based on the frequency time acquired by the frequency time acquisition unit 45, the audio data stored in the audio data storage unit 42 is stored. And summary means 46 promises and audio data summarized by summary means 46 and an audio output control unit 48 for controlling to output to the speaker apparatus 5.

  For example, the audio data storage means 42 is configured to continuously accumulate audio data in a lecture or the like in real time. The audio data is data corresponding to the uncompressed PCM format, data encoded in conformity with MP3, and the like.

  The voice data storage means 42 stores the voice data corresponding to the input time and the input time in association with each other by associating the storage address of the voice data with the input time.

  Here, an example of data in which the input time and the storage address stored by the voice data storage unit 42 are associated is shown in Table 1. The storage time is the time when the audio data is accumulated at the storage address, and is represented by time Tn (n is a natural number). The storage address is an address indicating a storage area when audio data is stored at the time of storage, and is expressed by an address n (n is a natural number). Note that the data shown in Table 1 may be stored in association with storage times and storage addresses at regular time intervals.

  The voice data storage means 42 stores analysis data required for each analysis time based on the voice data in addition to the voice data described above. The voice data storage means 42 stores the analysis data. A voice input control means 47 for controlling the voice data summary terminal 4 is further provided.

  FIG. 5 is a graph showing the analysis data obtained for each analysis time.

  For example, as shown in FIG. 5, the voice data storage means 42 stores analysis data obtained from every analysis time, that is, from the section A, the section B, the section C, and the like. . Here, the analysis data means an average of respective values obtained by squaring the sound pressure value obtained from the sound data within the analysis time and squaring.

  FIG. 6 is a graph showing the writing frequency data obtained for each analysis time.

  The writing frequency data stored in the writing frequency data storage means 44 corresponds to the writing detection information received by the receiving means 43 and the time obtained by the time acquisition means 41, and is shown in FIG. As described above, the number of times of reception of the handwriting detection information by the receiving means 43 is accumulated every analysis time.

  The frequency time acquisition means 45 acquires the frequency time when the frequency that the writing frequency data exceeds the threshold value occurs. In the example shown in FIG. 6, since the writing frequency data exceeds the threshold value in the section J and the section P, the time T <b> 10 and the time T <b> 16 are acquired by the frequency time acquisition unit 45.

  The summarizing means 46 has a function for determining an unvoiced section and a voiced section. That is, the unvoiced and voiced sections of the voice data stored in the voice data storage means 42 are determined, and the playback start time of the voice data is obtained from the voiced sections. A method for determining an unvoiced section and a voiced section is known, for example, “acoustic / voice engineering” (Sadaaki Furui, Modern Science Co., 1992). Hereinafter, a summary process based on sound pressure will be described.

  The summarizing means 46 selects a voice region corresponding to the frequency time from the voice region where the analysis data exceeds the threshold, and based on the start time and the end time when the analysis data exceeds the threshold in the selected voice region, The voice data stored in the voice data storage means 42 is summarized.

  Further, the summarizing unit 46 groups the voice regions whose analysis data exceeds the threshold into a voice region group, and responds to the frequency time from the grouped voice region group based on the frequency time acquired by the frequency time acquiring unit 45. A voice region group is selected, and the voice data stored in the voice data storage means 42 is summarized based on the start point time and the end point time when the analysis data exceeds the threshold in the selected voice region group. Also good.

  Here, the speech area group is divided into sections where the average data of the sections does not exceed the threshold continuously for a predetermined number of times. For example, if a section in which the average data of two consecutive sections does not exceed the threshold is defined as a section, as shown in FIG. 5, the section C and the section D, and the section L and the section M are defined as a section. Are divided from section E to section K as section N and section Q.

  For example, as shown in FIG. 6, since the writing frequency data exceeds the threshold value in the section J, the frequency time T10 is acquired by the frequency time acquiring means 45, and the summarizing means 46 starts from the section E corresponding to the frequency time T10. Select up to K. In addition, since the writing frequency data exceeds the threshold value in the section P, the frequency time T16 is acquired by the frequency time acquisition unit 45, and the summarization unit 46 selects the section N and the section Q corresponding to the frequency time T16.

  Further, when summarizing the voice data of the selected voice region or voice region group, the summarizing means 46 adjusts and summarizes the above-described start point time and end point time. For example, by operating the operating means 4a provided in the voice data summarizing terminal 4, it can be set according to the user's preference.

  Further, the summarizing means 46 uses a value obtained by adding the minimum value to a value obtained from an arbitrary ratio with respect to the difference between the maximum value and the minimum value of the analysis data as a threshold value.

  This threshold value is merely an example, and instead of this threshold value, the summarizing unit 46 acquires the difference between the maximum value and the minimum value of the analysis data at an arbitrary time interval, and arbitrarily selects the difference between the acquired maximum value and minimum value. A value obtained by adding the minimum value to the value obtained from the ratio may be used as the threshold value.

  Further, the summarizing means 46 acquires the difference between the maximum value and the average value of the analysis data at an arbitrary time interval, and adds the average value to the value obtained from an arbitrary ratio with respect to the acquired difference between the maximum value and the average value. A value may be used as a threshold value.

  The operation of the audio data summarizing system 1 configured as described above will be described with reference to the flowchart shown in FIG.

  The voice data summarization control operation in the voice data summarization system 1 starts when the writing frequency data is completely stored in the writing frequency data storage means 44.

  First, the writing time data stored in the writing frequency data storage unit 44 is analyzed by the frequency time acquiring unit 45 (S1), and the frequency time is acquired from the analyzed writing frequency data (S2). Then, the summarizing means 46 summarizes the voice data based on the acquired frequency time (S3).

  Next, the procedure of summarization processing by the summarizing means 46 will be described with reference to FIG. 4a. FIG. 4 a is a flowchart for explaining an example (first example) of voice data summarization processing by the summarizing means 46.

  First, based on the analysis data calculated | required from the audio | voice data memorize | stored in the audio | voice data storage means 42, the audio | voice area | region exceeding a threshold value is extracted (S11). Next, a voice area is selected based on the frequency time acquired by the frequency time acquisition means 45 from the extracted voice area (S12). Next, a start point time and an end point time exceeding the threshold are calculated from the selected voice region (S13). Then, after adjusting the calculated start point time and end point time (S14), the voice data stored in the voice data storage means 42 is summarized based on the adjusted start point time and end point time (S15).

  Next, other voice data summarization processing by the summarizing means 46 will be described.

  FIG. 4B is a flowchart for explaining an example (second example) of voice data summarization processing by the summarizing means 46.

  First, based on the analysis data calculated | required from the audio | voice data memorize | stored in the audio | voice data storage means 42, the audio | voice area | region exceeding a threshold value is extracted (S21). Next, the extracted voice regions are grouped (S22). Next, a speech region group is selected based on the frequency time acquired by the frequency time acquisition unit 45 from the grouped speech region group (S23). Next, a start point time and an end point time exceeding the threshold are calculated from the selected voice region group (S24). Then, after adjusting the calculated start point time and end point time (S25), the voice data stored in the voice data storage means 42 is summarized based on the adjusted start point time and end point time (S26). ).

  According to the speech data summarizing system 1 of this embodiment, speech data can be summarized based on writing frequency data and analysis data obtained from writing detection information transmitted from a plurality of writing detection terminals 2. Therefore, for example, when applied at the time of a lecture or the like, it is possible to summarize and reproduce a part of the audio that the listener has shown interest from notes written by many listeners.

  The writing frequency data storage means 44 stores writing frequency data for each technical field, sex, and age of the listener, and what data the user selects at the time of voice request or the user's technical field, sex, age, etc. Therefore, the voice data requesting terminal selects the frequency data that most closely matches, and the summary voice that matches the user's preference can be reproduced.

  The audio data summarization system of the present invention supplies a computer-readable recording medium that records a program code of software that realizes the functions of the apparatus according to the above-described embodiment to a computer. This can also be realized by reading and executing the program code stored in the. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and a storage medium storing this program code, such as a flexible disk, CD-ROM, DVD-ROM, CD -R, CD-RW, MO, HDD, etc. constitute the present invention.

1 is a schematic configuration diagram of an audio data summarization system according to an embodiment of the present invention. 1 is a block diagram showing a functional configuration of an audio data summarization system according to an embodiment of the present invention. The flowchart explaining operation | movement of the audio | voice data summary system which concerns on one example of embodiment of this invention. The flowchart explaining operation | movement of the audio | voice data summary system which concerns on one example of embodiment of this invention. The flowchart explaining operation | movement of the audio | voice data summary system which concerns on one example of embodiment of this invention. A graph of analysis data obtained for each analysis time. A graph of writing frequency data obtained for each analysis time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Voice data summarization system 2 ... Writing detection terminal 3 ... Microphone apparatus 4 ... Voice data summarization terminal 5 ... Speaker apparatus 20 ... Writing detection means 21 ... Transmission means 41 ... Time acquisition means 42 ... Voice data storage means 43 ... Reception means 44 ... Writing frequency data storage means 45 ... Frequency time acquisition means 46 ... Summary means 47 ... Voice input control means 48 ... Voice output control means

Claims (4)

A plurality of writing detection terminals for detecting the possession of the listener and writing on the recording medium;
A voice data summarization terminal that inputs voice and summarizes voice data indicating the input voice based on the handwriting detection information detected in the handwriting detection terminal;
The handwriting detection terminal is
Writing detection means for detecting that the listener is writing;
Transmission means for transmitting the writing detection information detected by the writing detection means,
The voice data summary terminal
Time acquisition means for measuring time;
Voice data storage means for storing voice data indicating the voice inputted and the input time when the voice measured by the time acquisition means is input;
Receiving means for receiving writing detection information transmitted by the transmitting means;
A writing frequency data storage means for storing writing frequency data for each attribute of the listener, by associating the writing detection information received by the receiving means with the time measured by the time acquisition means;
Among the writing frequency data for each attribute stored by the writing frequency data storage means , using the writing frequency data determined by the user who requests the summary of the voice data, the frequency time at which the frequency that the writing frequency data exceeds the threshold occurs. Frequency time acquisition means to acquire;
A voice data summarizing system comprising: summarizing means for summarizing voice data stored in the voice data storage means based on the frequency time acquired by the frequency time acquiring means. 2. The voice data summarization system according to claim 1, wherein the writing frequency data determined by a user who requests summarization of voice data is writing frequency data that most closely matches the attribute of the user . The summary section, the audio data unvoiced and voiced sections of the accumulated audio data to determine the storage means, according to claim 1 or 2, characterized in that to calculate the reproduction start time of the audio data based on the voiced The voice data summarization system described.   An audio data summarization program for causing a computer to function as means for constituting the audio data summarization system according to any one of claims 1 to 3.

Images