JP6661710B2 - Electronic device and control method for electronic device - Google Patents

Description

  The embodiment of the present invention relates to a technique for estimating a direction of a speaker and displaying an object corresponding to the speaking time of each identified speaker on a screen.

  2. Description of the Related Art There has been developed an electronic device that estimates a direction of a speaker based on a phase difference for each frequency component of voice input to a plurality of microphones.

JP 2006-254226 A

  If voice is collected while the user is holding the electronic device, the accuracy of estimating the direction of the speaker may be reduced.

  An object of the present invention is to provide an electronic device and an electronic device control method for estimating the direction of a speaker and displaying an object corresponding to the speaking time of each identified speaker on a screen.

An electronic apparatus according to an embodiment includes a first microphone and a second microphone arranged at a predetermined distance, a recording processing unit, a frequency decomposition unit, a section detection unit, a speech direction estimation unit, a speaker The system includes a clustering unit, a user interface display processing unit, an acceleration sensor for detecting acceleration, and a control unit . The recording processing means records two acoustic signals from the first microphone and the second microphone. The frequency decomposition unit receives digitized time-series amplitude data of two audio signals from the first microphone and the second microphone, and determines a power value and a phase value for each frequency of the amplitude data. Generate in chronological order. It said interval detection means, based on a result of the power value and the phase value of the frequency decomposition unit, for detecting a speech section. The speech direction estimating means based on a detection result of said interval detecting means detects the speech direction of the speech segment. The speaker clustering means, based on the temporal lifetime of the sound the sound of said detected utterance direction by the speech direction estimating means emitted to generate a speaker identification information for each time. The user interface display processing unit, based on said speaker identification information from the speaker clustering means, the corresponding object visibly displayed on speech time of each speaker on the display screen of the display. The control means determines whether the difference between the current device inclination obtained from the acceleration sensor and the device inclination at the start of speaker identification exceeds a threshold value, and the difference exceeds the threshold value. When it is determined that the speech direction is estimated, the utterance direction estimating means is required to initialize data relating to speaker identification, it is determined that the difference does not exceed the threshold, and x of the device obtained from the acceleration sensor is determined. , Y, z-axis direction takes a periodic value, the recording processing means, the frequency decomposition means, the section detection means, the speech direction estimation means, and the speaker clustering means Request to stop processing.

FIG. 1 is an exemplary perspective view illustrating an example of the external appearance of an electronic apparatus according to an embodiment. FIG. 2 is an exemplary block diagram illustrating the configuration of the electronic apparatus according to the embodiment. Functional block diagram of a recording application. The figure which shows a sound source direction and the arrival time difference observed in an acoustic signal. FIG. 4 is a diagram illustrating a relationship between a frame and a frame shift amount. The figure which shows the procedure of FFT processing, and short-time Fourier-transform data. FIG. 3 is a functional block diagram of a speech direction estimation unit. FIG. 3 is a functional block diagram showing the internal configuration of each of a two-dimensional data conversion unit and a graphic detection unit. The figure which shows the procedure of phase difference calculation. The figure which shows the procedure of coordinate value calculation. FIG. 3 is a functional block diagram illustrating an internal configuration of a sound source information generation unit. The figure for demonstrating direction estimation. The figure which shows the relationship between (theta) and (DELTA) T. FIG. 4 is a diagram illustrating an example of a screen displayed by a user interface display processing unit. 9 is a flowchart illustrating a procedure for initializing data relating to speaker identification.

  Hereinafter, embodiments will be described with reference to the drawings.

  First, the configuration of the electronic device of the present embodiment will be described with reference to FIG. The electronic device can be realized as a portable terminal, for example, a tablet personal computer, a laptop or notebook personal computer, or a PDA. Hereinafter, it is assumed that the electronic device is realized as a tablet personal computer 10 (hereinafter, referred to as a computer 10).

  FIG. 1 is a diagram illustrating an appearance of the computer 10. The computer 10 includes a computer main body 11 and a touch screen display 17. The computer main body 11 has a thin box-shaped housing. The touch screen display 17 is arranged on the surface of the computer main body 11. The touch screen display 17 includes a flat panel display (for example, a liquid crystal display (LCD)) and a touch panel. The touch panel is provided so as to cover the screen of the LCD. The touch panel is configured to detect a position on the touch screen display 17 touched by a user's finger or pen.

FIG. 2 is a block diagram illustrating a system configuration of the computer 10.
As shown in FIG. 2, the computer 10 includes a touch screen display 17, a CPU 101, a system controller 102, a main memory 103, a graphics controller 104, a BIOS-ROM 105, a nonvolatile memory 106, an embedded controller (EC) 108, a microphone 109A, 109B, an acceleration sensor 110, and the like.

  The CPU 101 is a processor that controls operations of various modules in the computer 10. The CPU 101 executes various software loaded from the nonvolatile memory 106 as a storage device to the main memory 103 as a volatile memory. The software includes an operating system (OS) 200 and various application programs. Various application programs include a recording application 300.

  The CPU 101 also executes a basic input / output system (BIOS) stored in the BIOS-ROM 105. The BIOS is a program for controlling hardware.

  The system controller 102 is a device that connects between a local bus of the CPU 101 and various components. The system controller 102 also incorporates a memory controller for controlling access to the main memory 103. Further, the system controller 102 also has a function of executing communication with the graphics controller 104 via a PCI EXPRESS standard serial bus or the like.

  The graphics controller 104 is a display controller that controls the LCD 17A used as a display monitor of the computer 10. The display signal generated by the graphics controller 104 is sent to the LCD 17A. The LCD 17A displays a screen image based on the display signal. A touch panel 17B is arranged on the LCD 17A. The touch panel 17B is a capacitive pointing device for performing input on the screen of the LCD 17A. The touch position on the screen where the finger touches and the movement of the touch position are detected by the touch panel 17B.

  The EC 108 is a one-chip microcomputer including an embedded controller for power management. The EC 108 has a function of turning on or off the power of the computer 10 according to the operation of the power button by the user.

The acceleration sensor 110 detects the acceleration applied to the electronic device 10 in the x, y, and z axis directions.
The direction of the electronic device 10 can be detected by detecting the acceleration in the x, y, and z axis directions.

FIG. 3 is a functional block diagram of the recording application 300.
It includes a frequency decomposition section 301, a voice section detection section 302, a speech direction estimation section 303, a speaker clustering section 304, a user interface display processing section 305, a recording processing section 306, and a control section 307.

  The recording processing unit 306 performs a recording process by performing compression processing and the like on the audio data input from the microphones 109A and 109B and storing the audio data in the storage device 106.

  The control unit 307 can control the operation of each unit of the recording application 300.

[Basic concept of sound source estimation based on phase difference for each frequency component]
The microphone 109A and the microphone 109B are two microphones arranged at a predetermined distance in a medium such as air, and convert the vibrations of the medium (sound waves) at two different points into electric signals (acoustic signals). Means. Hereinafter, when the microphone 109A and the microphone 109B are collectively handled, this is referred to as a microphone pair.

  The acoustic signal input unit digitizes the two acoustic signals 403 and 404 from the microphones 109A and 109B by periodically performing A / D conversion on the two acoustic signals from the microphones 109A and 109B at a predetermined sampling cycle Fr. The generated amplitude data is generated in time series.

  Assuming that the sound source is located far enough than the distance between the microphones, as shown in FIG. 4A, the wavefront 401 of the sound wave emitted from the sound source 400 and reaching the microphone pair is substantially flat. . When this plane wave is observed at two different points by using the microphone 109A and the microphone 109B, the microphone is set in accordance with the direction R of the sound source 400 with respect to a line 402 connecting the microphone 109A and the microphone 109B (this is called a base line). A predetermined arrival time difference [Delta] T should be observed in the acoustic signal converted in pairs. Note that when the sound source is sufficiently far, the arrival time difference ΔT becomes 0 when the sound source 400 exists on a plane perpendicular to the baseline 402, and this direction is defined as the front direction of the microphone pair.

[Frequency decomposition unit]
Now, there is a fast Fourier transform (FFT) as a general method for decomposing the amplitude data into frequency components. As a typical algorithm, a Cooley-Turkey DFT algorithm or the like is known.

  As shown in FIG. 5, the frequency decomposition unit 301 extracts N consecutive amplitude data as a frame (T-th frame 411) from the amplitude data 410 from the audio signal input unit and performs a fast Fourier transform. This extraction position is repeated while being shifted by the frame shift amount 413 (T + 1-th frame 412).

  As shown in FIG. 6A, the amplitude data forming the frame is windowed 601 and then subjected to the fast Fourier transform 602. As a result, short-time Fourier transform data of the input frame is generated in the real part buffer R [N] and the imaginary part buffer I [N] (603). An example of the windowing function (Hamming windowing or Hanning windowing) 605 is shown in FIG.

  The short-time Fourier transform data generated here is data obtained by decomposing the amplitude data of the frame into N / 2 frequency components, and the real part R [k] in the buffer 603 for the k-th frequency component fk. The numerical value of the imaginary part I [k] represents a point Pk on the complex coordinate system 604 as shown in FIG. The square of the distance of Pk from the origin O is the power Po (fk) of the frequency component, and the signed rotation angle θ {θ: −π> θ ≧ π [radian]} of Pk from the real part axis is This is the phase Ph (fk) of the frequency component.

  When the sampling frequency is Fr [Hz] and the frame length is N [sample], k takes an integer value from 0 to (N / 2) -1, k = 0 is 0 [Hz] (DC), k = (N / 2) -1 represents Fr / 2 [Hz] (highest frequency component), and the interval is equally divided by frequency resolution Δf = (Fr / 2) ÷ ((N / 2) -1) [Hz]. Is the frequency at each k, and is represented by fk = k · Δf.

  As described above, the frequency decomposing unit 301 continuously performs this processing at a predetermined interval (frame shift amount Fs), so that the frequency composed of the power value and the phase value of each frequency of the input amplitude data is obtained. Generate a decomposition data set in time series.

[Voice section detector]
Voice section detection section 302 detects a voice section based on the result of frequency decomposition section 301.

[Speech direction estimation unit]
The utterance direction estimation unit 303 detects the utterance direction of the voice section based on the detection result of the voice section detection unit 302.
FIG. 7 is a functional block diagram of the speech direction estimation unit 303.
The utterance direction estimating unit 303 includes a two-dimensional data converting unit 701, a graphic detecting unit 702, a sound source information generating unit 703, and an output unit 704.

(2D data conversion unit and figure detection unit)
As shown in FIG. 8, the two-dimensional data conversion unit 701 includes a phase difference calculation unit 801 and a coordinate value determination unit 802. The figure detecting unit 702 includes a voting unit 811 and a straight line detecting unit 812.

[Phase difference calculator]
The phase difference calculating unit 801 compares the two frequency-resolved data sets a and b of the same period obtained by the frequency decomposing unit 301 and calculates the difference between the two phase values for each same frequency component. Generate phase difference data between a and ab. For example, as shown in FIG. 9, the phase difference ΔPh (fk) of a certain frequency component fk is calculated by calculating the difference between the phase value Ph1 (fk) of the microphone 109A and the phase value Ph2 (fk) of the microphone 109B. {ΔPh (fk): calculated as a 2π remainder system so as to be within −π <ΔPh (fk) ≦ π}.

[Coordinate value determination unit]
Based on the phase difference data obtained by the phase difference calculation unit 801, the coordinate value determination unit 802 calculates the phase difference data obtained by calculating the difference between the two phase values for each frequency component in a predetermined two-dimensional XY coordinate system. This is a means for determining a coordinate value to be treated as a point on the system. The X coordinate value x (fk) and the Y coordinate value y (fk) corresponding to the phase difference ΔPh (fk) of a certain frequency component fk are determined by the equations shown in FIG. The X coordinate value is the phase difference ΔPh (fk), and the Y coordinate value is the frequency component number k.

[Voting Department]
The voting unit 811 applies a linear Hough transform to each frequency component to which the (x, y) coordinates are given by the coordinate value determining unit 802, and votes its locus in a Hough voting space by a predetermined method. is there.

[Line detector]
The straight line detecting unit 812 is a unit that analyzes the vote distribution in the Hough voting space generated by the voting unit 811 and detects a powerful straight line.

[Sound source information generator]
As shown in FIG. 11, the sound source information generating unit 703 includes a direction estimating unit 1111, a sound source component estimating unit 1112, a sound source resynthesizing unit 1113, a time series tracking unit 1114, a duration evaluating unit 1115, It has a conversion unit 1116, an adaptive array processing unit 1117, and a speech recognition unit 1118.

[Direction estimation unit]
The direction estimating unit 1111 receives the straight line detection result by the straight line detecting unit 812 described above, that is, the θ value of each straight line group, and calculates the existence range of the sound source corresponding to each straight line group. At this time, the number of detected straight line groups becomes the number of sound sources (all candidates). When the distance to the sound source is sufficiently far from the baseline of the microphone pair, the existence range of the sound source is a conical surface having a certain angle with respect to the baseline of the microphone pair. This will be described with reference to FIG.

  The arrival time difference ΔT between the microphone 109A and the microphone 109B can change within a range of ± ΔTmax. As shown in FIG. 12A, when incident from the front, ΔT is 0, and the azimuth φ of the sound source is 0 ° with respect to the front. In addition, when the sound is incident right from the right, that is, from the direction of the microphone 109B as shown in FIG. 12B, ΔT is equal to + ΔTmax, and the azimuth φ of the sound source is + 90 ° with the clockwise direction being positive with respect to the front. . Similarly, when the sound is incident from the left side, that is, the direction of the microphone 109A as shown in FIG. 12C, ΔT is equal to −ΔTmax, and the azimuth φ is −90 °. In this way, ΔT is defined to be positive when the sound is incident from the right and negative when the sound is incident from the left.

Based on the above, a general condition as shown in FIG. Assuming that the position of the microphone 109A is A and the position of the microphone 109B is B, and that sound is incident from the direction of the line segment PA, ＢPAB is a right-angled triangle with the vertex P being a right angle. At this time, the center O between the microphones and the line segment OC are defined as the front direction of the microphone pair, and an angle in which the counterclockwise direction with the OC direction being the azimuth of 0 ° is defined as the azimuth φ. Since ΔQOB is similar to ΔPAB, the absolute value of the azimuth φ is equal to ΔOBQ, that is, ΔABP, and the sign matches the sign of ΔT. ΔABP can be calculated as sin ⁻¹ of the ratio of PA to AB. At this time, if the length of the line segment PA is represented by ΔT corresponding thereto, the length of the line segment AB corresponds to ΔTmax. Therefore, the azimuth including the sign can be calculated as φ = sin ⁻¹ (ΔT / ΔTmax). Then, the existence range of the sound source is estimated as a conical surface 1200 opened by (90-φ) ° with the point O as the vertex and the baseline AB as an axis. The sound source is somewhere on this conical surface 1200.

  As shown in FIG. 13, ΔTmax is a value obtained by dividing the distance L [m] between microphones by the sound speed Vs [m / sec]. At this time, it is known that the sound speed Vs can be approximated as a function of the temperature t [° C.]. Now, it is assumed that the straight line detection unit 812 has detected the straight line 1300 at the Hough inclination θ. Since the straight line 1300 is inclined rightward, θ is a negative value. When y = k (frequency fk), the phase difference ΔPh indicated by the straight line 1300 can be obtained by k · tan (−θ) as a function of k and θ. At this time, ΔT [sec] is a time obtained by multiplying the ratio of the phase difference ΔPh (θ, k) to 2π by one cycle (1 / fk) [sec] of the frequency fk. Since θ is a signed quantity, ΔT is also a signed quantity. That is, when the sound is incident from the right in FIG. 12D (the phase difference ΔPh has a positive value), θ has a negative value. When a sound enters from the left in FIG. 12D (the phase difference ΔPh becomes a negative value), θ becomes a positive value. Therefore, the sign of θ is inverted. In the actual calculation, the calculation may be performed at k = 1 (frequency just above the DC component k = 0).

[Sound source component estimation unit]
The sound source component estimating unit 1112 evaluates the distance between the (x, y) coordinate value for each frequency component given by the coordinate value determining unit 802 and the straight line detected by the straight line detecting unit 812, thereby determining the vicinity of the straight line. The located point (that is, the frequency component) is detected as the frequency component of the straight line (that is, the sound source), and the frequency component for each sound source is estimated based on the detection result.

[Sound source resynthesis unit]
The sound source sound resynthesizing unit 1113 resynthesizes the sound source sound (amplitude data) in the frame section starting from the time by performing the inverse FFT processing on the frequency components of the same acquisition time constituting each sound source sound. As shown in FIG. 5, one frame overlaps with the next frame with a time difference of a frame shift amount. As described above, in a section where a plurality of frames overlap, the amplitude data of all overlapping frames can be averaged to form final amplitude data. Such processing makes it possible to separate and extract a sound source sound as its amplitude data.

[Time series tracking unit]
A straight line group is obtained by the straight line detecting unit 812 for each Hough voting by the voting unit 811. The Hough voting is performed collectively for m consecutive (m ≧ 1) FFT results. As a result, the straight line group is obtained in time series with a period corresponding to m frames as a period (this is referred to as a “figure detection period”). Further, since the straight line group θ has a one-to-one correspondence with the sound source direction φ calculated by the direction estimating unit 1111, it corresponds to a stable sound source regardless of whether the sound source is stationary or moving. The locus of θ (or φ) on the time axis should be continuous. On the other hand, the straight line group detected by the straight line detecting unit 812 may include a straight line group corresponding to background noise (this is referred to as a “noise straight line group”) depending on how the threshold value is set. is there. However, the locus on the time axis of θ (or φ) of such a noise straight line group is not continuous or can be expected to be short even if continuous.

  The time-series tracking unit 1114 is a unit that obtains a locus of φ on the time axis by dividing φ obtained for each figure detection cycle into a continuous group on the time axis.

[Duration evaluation section]
The duration evaluating unit 1115 calculates the duration of the trajectory from the start time and the end time of the trajectory data for which the tracking has been completed, output by the time-series tracking unit 1114, and determines that the duration exceeds a predetermined threshold value. Trajectory data based on the noise, and the others are recognized as trajectory data based on the noise. The trajectory data based on the sound source sound is referred to as sound source stream information. The sound source stream information includes a start time Ts, an end time Te of the sound source sound, and time-series trajectory data of θ, ρ, and φ representing the sound source direction. The number of straight lines by the graphic detection unit 702 gives the number of sound sources, which includes noise sources. The number of sound source stream information by the duration evaluation unit 1115 gives the number of reliable sound sources excluding those based on noise.

[In-phase part]
The in-phase unit 1116 obtains the time transition of the sound source direction φ of the stream by referring to the sound source stream information from the time series tracking unit 1114, and obtains an intermediate value φmid = (φmax + φmin) from the maximum φmax and the minimum φmin of φ. ) / 2 to obtain the width φw = φmax−φmid. Then, the time-series data of the two frequency-resolved data sets a and b that are the source of the sound source stream information are extracted from a time that is a predetermined time before the start time Ts of the stream to a time that is a predetermined time after the end time Te Then, the arrival time difference calculated back by the intermediate value φmid is corrected so as to cancel the arrival time difference, thereby achieving in-phase.

  Alternatively, the direction estimating unit 1111 may set the sound source direction φ at each time to φmid so that the time-series data of the two frequency-resolved data sets a and b are always in phase. Whether to refer to the sound source stream information or the φ at each time is determined by an operation mode, and this operation mode can be set and changed as a parameter.

[Adaptive array processing unit]
The adaptive array processing unit 1117 follows the time-series data of the two frequency-resolved data sets a and b extracted and in-phase, by directing the central directivity to 0 ° in front and adding a predetermined margin to ± φw. By performing the adaptive array processing for the range, the time series data of the frequency component of the sound source sound of the stream is separated and extracted with high accuracy. Although this processing differs in method, it performs the same function as the sound source component estimating unit 1112 in that time series data of frequency components is separated and extracted. Therefore, the sound source sound resynthesis unit 1113 can resynthesize the amplitude data of the sound source sound from the time-series data of the frequency components of the sound source sound by the adaptive array processing unit 1117.

  As the adaptive array processing, as described in Reference 3, “Amada et al.,“ Microphone Array Technology for Voice Recognition ”, Toshiba Review 2004, VOL.59, NO. It is possible to apply a method of clearly separating and extracting voices within a set directivity range, such as using a “Griffith-Jim type generalized side lobe canceller” known as a method of forming a former for two main and sub-formers. it can.

  Normally, when using adaptive array processing, a tracking range is set in advance, and it is used to listen only to sound from that direction.To wait for sound from all directions, a large number of adaptive arrays with different tracking ranges are used. Had to be prepared. On the other hand, in the present embodiment, after actually obtaining the number of sound sources and their directions, it is possible to operate only the number of adaptive arrays corresponding to the number of sound sources, and the following range is also a predetermined range according to the direction of the sound sources. Since it can be set in a narrow range, speech can be separated and extracted efficiently and with good quality.

  Also, at this time, by making the time-series data of the two frequency-resolved data sets a and b in phase in advance, it is possible to process sounds in all directions only by setting the tracking range in the adaptive array processing only near the front. Become like

[Speech recognition unit]
The speech recognition unit 1118 analyzes and matches the time-series data of the frequency components of the sound source extracted by the sound source component estimating unit 1112 or the adaptive array processing unit 1117, so that the symbolic content of the stream, that is, the linguistic Symbols (strings) representing meaning, type of sound source, and speaker are extracted.

  It is assumed that each functional block from the direction estimating unit 1111 to the voice recognizing unit 1118 can exchange information by a connection not shown in FIG. 11 as necessary.

  The output unit 704 includes, as sound source information by the sound source information generation unit 703, the number of sound sources obtained as the number of straight line groups by the figure detection unit 702, and the space of each sound source that is a sound signal generation source estimated by the direction estimation unit 1111. Typical existence range (angle φ for determining conical surface), component configuration of sound emitted from each sound source (time-series data of power and phase for each frequency component) estimated by sound source component estimating section 1112, sound source sound reproduction Separated voices (time-series data of amplitude values) separated for each sound source, synthesized by the synthesis unit 1113, and determined based on the time-series tracking unit 1114 and the duration evaluation unit 1115, of the sound source excluding the noise source. The number, the temporal existence period of the sound emitted from each sound source determined by the time series tracking unit 1114 and the duration evaluation unit 1115, the in-phase unit 1116 and the adaptive array processing unit 1117 More determined, (time series data of the amplitude value) separating speech for each sound source, obtained by the speech recognition unit 1118, symbolic content of each source audio is means for outputting information including at least one of.

[Speaker clustering unit]
The speaker clustering unit 304 generates the speaker identification information 310 for each time based on the temporal existence period of the sound emitted from each sound source output from the output unit 704. The speaker identification information 310 includes a speech start time and information associated with the speaker at the speech start time.

[User interface display processing unit]
The user interface display processing unit 305 presents various setting contents necessary for the above-described acoustic signal processing to the user, receives setting input from the user, saves the setting contents to the external storage device, and reads out the setting contents from the external storage device. Or (1) display of frequency components for each microphone, (2) display of a phase difference (or time difference) plot diagram (ie, display of two-dimensional data), (3) display of various vote distributions, (4) Visualization of various processing results and intermediate results, such as display of maximum position, (5) display of straight line group on plot diagram, (6) display of frequency component belonging to straight line group, and (7) display of locus data This is a means for presenting to a user by using the data, or allowing the user to select desired data and visualize the data in more detail. By doing so, the user can check the operation of the audio signal processing device according to the present embodiment, make adjustments so that desired operations can be performed, and use the device in an adjusted state thereafter. Or become possible.

  The user interface display processing unit 305 displays, for example, a screen shown in FIG. 14 on the LCD 17A based on the speaker identification information 310.

  An object 1401 indicating speaker A, an object 1402 indicating speaker B2, and an object 1403 indicating speaker C are shown above LCD17A. Objects 1413A, 1411A, 1413B, 1412, and 1411B corresponding to the talk time of the speaker are displayed below the LCD 17A. The objects 1411A and 1411B correspond to the speaking time of the speaker A and are displayed in a color corresponding to the object 1401. The object 1412 corresponds to the speaking time of the speaker B and is displayed in a color corresponding to the object 1402. The objects 1413A and 1413B correspond to the speech time of the speaker C and are displayed in a color corresponding to the object 1403. When there is a remark, the objects 1413A, 1411A, 1413B, 1412, and 1411B are displayed flowing from right to left with time.

  By the way, the accuracy of speaker identification using the phase difference of the distance between the microphones decreases when the terminal is moved during recording. The present apparatus suppresses a decrease in convenience due to a decrease in accuracy by using the acceleration in the x, y, and z directions and the inclination of the terminal obtained from the acceleration sensor 110 for speaker identification.

  The control unit 307 requests the utterance direction estimation unit 303 to initialize data related to processing for estimating the direction of the speaker according to the acceleration detected by the acceleration sensor.

  FIG. 15 is a flowchart illustrating a procedure for initializing data relating to speaker identification.

  The control unit 307 determines whether the difference between the current inclination of the device 10 obtained from the acceleration sensor 110 and the inclination of the device 10 when the speaker identification is started exceeds a threshold value (Step B11). When it is determined that the threshold value is exceeded (Yes in step B11), the control unit 307 requests the utterance direction estimation unit 303 to initialize data related to speaker identification (step B12). The utterance direction estimating unit 303 initializes data relating to speaker identification (step B13). Then, the speech direction estimation unit 303 performs a speaker identification process based on data newly generated by each unit in the speech direction estimation unit 303.

When it is determined that the initial state has not been exceeded (No in step B12), the control unit 307
It is determined whether the acceleration values in the x, y, and z directions of the device 10 obtained from the acceleration sensor 110 have become periodic values (step B14). When it is determined that the acceleration value takes a periodic value (Yes in step B13), the control unit 307 requests the recording processing unit 306 to stop the recording process (step B15). Also, the control unit 307 requests the frequency decomposition unit 301, the voice section detection unit 302, the speech direction estimation unit 303, and the speaker clustering unit 304 to stop the processing. The recording processing unit 306 stops the recording processing (Step B16). The frequency decomposition section 301, the voice section detection section 302, the utterance direction estimation section 303, and the speaker clustering section 304 stop the processing.

  According to the present embodiment, the state held by the user is requested by requesting the utterance direction estimating unit 303 to initialize data related to the process of estimating the direction of the speaker according to the acceleration detected by the acceleration sensor 110. , It is possible to suppress a decrease in the accuracy of estimating the direction of the speaker even if the sound is collected.

  Since the various processes of the present embodiment can be realized by a computer program, the computer program is only installed and executed on a computer through a computer-readable storage medium storing the computer program. A similar effect can be easily realized.

  Although several embodiments of the present invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

  Reference Signs List 10: tablet personal computer (electronic device), 101: CPU, 103: main memory, 106: storage device, 108: embedded controller, 109A: microphone, 109B: microphone, 110: acceleration sensor, 200: operating system, 300: Recording application, 301: frequency decomposition unit, 302: voice section detection unit, 303: speech direction estimation unit, 304: speaker clustering unit, 305: user interface display processing unit, 306: recording processing unit, 307: control unit

Claims (2)

A first microphone and a second microphone arranged at a predetermined distance,
Recording processing means for recording two acoustic signals from the first microphone and the second microphone,
A frequency that receives digitized time-series amplitude data of two acoustic signals from the first microphone and the second microphone, and generates a power value and a phase value for each frequency of the amplitude data in a time-series. Disassembly means;
Based on the result of the power value and the phase value in the frequency decomposition means, a section detection means for detecting a voice section,
An utterance direction estimating means for detecting an utterance direction of the voice section based on a detection result of the section detecting means;
Speaker clustering means for generating speaker identification information for each time based on the temporal existence period of the sound emitted by each sound source in the speech direction detected by the speech direction estimation means,
Based on the speaker identification information from the speaker clustering means, a user interface display processing means for visually displaying an object corresponding to the speaking time of each speaker on a display screen of a display,
An acceleration sensor for detecting acceleration;
It is determined whether the difference between the current device inclination obtained from the acceleration sensor and the device inclination at the start of the speaker identification exceeds a threshold value, and it is determined that the difference exceeds the threshold value. In this case, the utterance direction estimating means is requested to initialize data relating to speaker identification, it is determined that the difference does not exceed the threshold value, and the x, y, z axes of a device obtained from the acceleration sensor are determined. When the value of the direction acceleration takes a periodic value, the recording processing unit, the frequency decomposition unit, the section detection unit, the speech direction estimation unit, and the speaker clustering unit are requested to stop processing. Control means for performing
An electronic device comprising: Recording two acoustic signals from a first microphone and a second microphone arranged at a predetermined distance ;
Receiving digitized time-series amplitude data of two acoustic signals from the first microphone and the second microphone , generating a power value and a phase value for each frequency of the amplitude data in a time-series,
Based on a result of the generated power value and the phase value, a voice section is detected,
Detecting the speech direction of the detected voice section,
Based on the temporal existence period of the sound emitted from each sound source in the detected speech direction, speaker identification information for each time is generated,
Based on the generated speaker identification information, an object corresponding to the speaking time of each speaker is visually displayed on a display screen of a display ,
Determine whether the difference between the current device inclination obtained from the acceleration sensor that detects the acceleration and the device inclination at the start of speaker identification exceeds the threshold,
If it is determined that the difference exceeds the threshold, request initialization of the data related to the speaker identification,
When it is determined that the difference does not exceed the threshold value and the acceleration values in the x, y, and z directions of the device obtained from the acceleration sensor take periodic values, the recording is performed. And an electronic device control method for stopping each process related to generation of the speaker identification information .

Images