JP7125599B2 - Prosody control device, prosody control method and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Kazuma Takada gave a presentation titled "Analysis of Dialogue Prosody Based on Modalities of Judgment" at Waseda University 2017 (Heisei 29) Master's Thesis and Graduation Thesis Presentation, Hideharu Nakajima, Disclosed the technology related to "prosody control device, prosody control method and program" invented by Yoshinori Sakisaka and Kazuma Takada.

The present invention relates to a prosody control device, a prosody control method, and a program for controlling prosody, particularly fundamental frequency (pitch) in speech synthesis of dialogue speech.

Conventionally, relationship analysis of fundamental frequency change shapes corresponding to intentions or meanings of speech synthesis targets (for example, Non-Patent Document 1) and control based on shape classification (for example, Non-Patent Document 2) have been performed.

David Yoshikazu Oshima, "Correlation between Intonation Type and Final Particle Function in Japanese," International Development Forum 43, March 2013, pp.47-63 Kazuhiko Iwata, Tetsunori Kobayashi, "Experimental Study on Expression of Utterance Intent by Combining Sentence Final Words and Tones for Improving Expressiveness of Dialogue Speech Synthesis", 2017, The Institute of Electronics, Information and Communication Engineers Transactions D, Vol.J100- D, No.11, pp.938-948

The methods described in Non-Patent Literatures 1 and 2 focus on words in phrases to be synthesized, particularly final particles at the end of utterances, and classify and use the changing shape of the fundamental frequency of the final particles. The method divides a phrase containing a final particle into a final particle and its front element, and selects a change shape of the fundamental frequency from the correspondence between the combination of them and the meaning or intention conveyed by the whole phrase. Gives the variation shape of the fundamental frequency.

For example, if the final particle is ``ne'', its front element is ``eat'', and the whole phrase is ``eat ne'', the above method can be used to Select the shape of change in the fundamental frequency between

However, before the final particle, there may be cases where elements other than those examined in the above Non-Patent Documents 1 or 2 are attached, but knowledge about the influence of elements not examined in the above Non-Patent Documents on the entire phrase There is no Therefore, there is a problem that it is difficult to control the fundamental frequency. For example, "Do + seems + yo" and "Do + seems + yo" both raise the fundamental frequency. However, the method described in Non-Patent Document 1 only indicates the direction of change, and the degree cannot be selected. In addition, in the case of ``suru + must be + yo'' and ``suru + mitai da + yo'', the fundamental frequency of ``yo'' drops significantly in the former and rises significantly in the latter. However, in Non-Patent Document 2, it is difficult to appropriately select the fundamental frequency because there is no correspondence with detailed intention or semantic classification. Hereinafter, the present invention will be described by referring to elements such as ``like'', ``like'', and ``nishinai'' attached to the front side of a final particle as a variable factor part.

An object of the present disclosure, which has been made in view of this point, is to provide a prosody control device, a prosody control method, and a program that are capable of various prosody control corresponding to even if the element attached to the front side of the final particle changes finely. It is in.

In order to solve the above problems, the prosody control device according to the present invention includes:
Receiving an input of an expression including a phrase, dividing the phrase into words, identifying a final particle and a verb part, and if there is a part sandwiched between the final particle and the verb part, the part is treated as a change factor part. a linguistic analyzer configured as
a scaling unit that determines a scale value corresponding to the variable factor;
a scale value control amount conversion unit that converts the scale value into a prosody control amount;
a prosody generator that controls the prosody of the expression based on the prosody control amount;
has
When the language analysis unit determines that the change factor part does not exist in the phrase, the scale value control amount conversion unit sets a predetermined constant as the prosody control amount.

In order to solve the above problems, the prosody control method according to the present invention includes:
The language analysis unit receives an input of an expression including a phrase, divides the phrase into words, identifies a final particle and a verb part, and if there is a part sandwiched between the final particle and the verb part, setting the portion as the variable factor portion;
determining, by a scaling unit, a scale value corresponding to the variable factor;
a step of converting the scale value into a prosodic control quantity by a scale value control quantity conversion unit;
controlling the prosody of the expression based on the prosody control amount by a prosody generator;
setting a predetermined constant as the prosody control amount by the scale value control amount conversion unit when the language analysis unit determines that the change factor part does not exist in the phrase;
have

According to the prosody control device, the prosody control method, and the program according to the present disclosure, even if the element that precedes the final particle changes in detail, various prosody controls corresponding to it are possible.

2 is a functional block diagram of the prosody control device of this embodiment; FIG. 4 is a flow chart of processing executed by the prosody control device of the present embodiment; FIG. 4 is a diagram showing a method of scoring impression pairs according to the present embodiment; It is a figure which shows the correspondence table of this embodiment. FIG. 5 is a diagram showing the amount of increase in fundamental frequency, etc., according to the present embodiment;

Hereinafter, the present invention will be described more specifically with reference to the drawings.

FIG. 1 is a functional block diagram of the prosody control device D of this embodiment. The prosody control device D includes a language analysis unit 1, a scaling unit 2, a scale value control amount conversion unit 3, and a prosody generation unit 4. FIG. Arrows in FIG. 1 indicate directions in which information flows. Each function of the prosody control device D will be described below, but it is not intended to exclude other functions that the prosody control device D has.

The processing executed by the language analysis unit 1, the scaling unit 2, the scale value control amount conversion unit 3, and the prosody generation unit 4 is executed by a processor (not shown) such as one or more CPUs (Central Processing Units). . A processor may include one or more memories that store programs and information during operation for various processes. Memory includes volatile memory and non-volatile memory. The memory includes memory separate from the processor and memory internal to the processor. The processor includes a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor that specializes in specific processing.

Various kinds of information and/or programs for operating the prosody control device D may be stored in the internal or external storage unit of the prosody control device D. The storage unit is composed of a semiconductor memory, a magnetic memory, or the like. The storage unit may function as a work memory.

The prosody control method executed by the prosody control device D will be described with reference to the flow chart of FIG.

At step S1, the language analysis unit 1 receives input of expressions including phrases. The language analysis unit 1 uses existing morphological analysis technology to divide the phrase into words. The language analysis unit 1 assigns a part of speech to each word based on the result of the morphological analysis, and specifies the final particle. Based on the part of speech given to each word, the language analysis unit 1 collectively sets the verb stem and conjugation change part, which are the main elements of the predicate before the final particle (at the beginning of the sentence), as a verb part. If there is a part left between the verb part and the final particle, the language analysis part 1 puts this part together as a change factor part. That is, any of the words is the variable part. The language analysis unit 1 may assign a final particle label, a verb part label, and a variable factor part label to each word, instead of such processing of putting together.

For example, when the phrase is "Suru mita da yo", the language analysis unit 1, as a result of the morphological analysis, finds the following: You get 3 elements. At this time, one element contains three pieces of information. Specifically, the three pieces of information are, from left to right, a surface form that is a character face, a large part of speech classification, and a small part of speech classification. In this phrase, the first word from the left, ``suru'', is the verb part, the third word from the left, ``yo'', is the final particle, and the word between them is ``mitaida''. is the variable factor part. In another embodiment, the language analysis unit 1 may divide the above phrase into finer units depending on the morphological analyzer used.

As another function, the language analysis unit 1 assigns a verb label to ``suru'', a variable factor label to ``miida'', and a final particle label to ``yo'', thereby performing segmentation processing. you can

In this embodiment, φ indicates that there is no change factor portion. Therefore, when the phrase is ``suruyo'', the language analysis unit 1 divides the phrase into ``suru'', ``φ'', and ``yo''.

At step S2, the linguistic analysis unit 1 determines whether or not there is a change factor part in the phrase. When the change factor part does not exist, the language analysis part 1 sends the final particle and the verb part to the scale value control amount conversion part 3 . When there is a change factor part, the language analysis unit 1 sends the final particle, the verb part and the change factor part to the scaling unit 2 .

When the linguistic analysis unit 1 determines in step S2 that there is a change factor part in the word, the scaling unit 2 determines a scale value corresponding to the change factor part in step S3. Specifically, the scaling unit 2 calculates the scale values of various phrases from the viewpoint of certainty (degree of certainty), degree of pointing out (degree of pointing action), degree of assertion (degree of strength of assertion), and the like. It can be obtained by rating. The scoring is performed, for example, by one or more persons, for example, by manually determining where the impression of each phrase is positioned in the pair of impressions shown in FIG. The scaling unit 2 can set the average value of those scores or the like as the scale value.

As another function, the scaling unit 2 can use, for example, a table such as that shown in FIG. 4 for associating the scale values with expressions. The table is created manually, for example, by one or more people.

Another function of the scaling unit 2 is to absorb differences in linguistic expressions. may be used. Thereby, the scaling unit 2 can determine a scale value corresponding to the change factor portion.

In the quantification class I, the scaling unit 2 determines the value of the continuous dependent variable by the linear sum of the categorical variables and their weighting coefficients. The weighting factors are calculated from multiple sets of pre-collected dependent variable values and corresponding categorical variables. The scaling unit 2 assigns one categorical variable to each of the words forming the change factor parts appearing in the above various phrases. When determining the scale value corresponding to a certain change factor part, the scaling unit 2 sets the categorical variable corresponding to the word contained in the change factor part to 1, and calculates the scale value by performing a product-sum operation with its weight. to decide. General quantification theory processing, such as determining weights from data, is disclosed in Reference 1 below. The description is omitted here.
Reference 1: Tsutomu Komazawa, "Quantification Theory and Data Processing", Asakura Shoten, 1982

As another function, the scaling unit 2 converts each word constituting the change factor part into a real-valued vector representing each word, and receives each word constituting the change factor part in order from the beginning to the end of the sentence. A metric (eg, a cyclic neural net, etc.) may be used to determine the scale value corresponding to the variator. Such a process of converting discrete words to real-valued vectors is called word embedding. Since this processing is disclosed in the following Reference Document 2 or the like, the explanation here is omitted.
Reference 2: Mikolov Tomas, Sutskever Ilya, Chen Kai, Corrado Greg, Dean Jeffrey, "Distributed Representations of Words and Phrases and their Compositionality", [Online], 2013, [searched June 1, 2018], arXiv : 1310.4546, Internet <URL: https://arxiv.org/abs/1310.4546>

In addition, instead of directly obtaining the degree of certainty, the degree of indication, and the degree of assertion of the linguistic expression of the change factor portion as described above, the scaling unit 2 calculates scale values such as the degree of certainty, the degree of indication, and the degree of assertion for the change factor portion. It is also possible to obtain from a general impression evaluation. For example, the scaling unit 2 uses word pairs used in general impression evaluation (for example, dull-sharp, clear-muddy, dark-bright, dirty-clean, sharp-curled) for the variable factor part. , Heavy - Light, Hard - Soft, Flashy - Plain, Thick - Thin, Rough - Smooth, etc.). can be scaled by The result may for example be given manually by a person. The scaling unit 2 may use the same quantification class I or a cyclic neural network as described above when calculating each scale value.

At step S4 in FIG. 2, the scale value control amount converter 3 converts the scale value corresponding to the change factor portion into a prosody control amount using an arbitrary conversion formula. The prosody control amount in this embodiment is, for example, the amount of change in the fundamental frequency. The amount of change is the amount of change from the fundamental frequency at the end of the change factor part to the fundamental frequency of the final particle affected by the change factor part. Note that if it is determined in step S2 that the change factor portion does not exist, the scale value control amount conversion section 3 may set the prosody control amount to a predetermined constant.

FIG. ₅ shows an example of the amount of increase in the fundamental frequency (“F0 increase (log)” in the figure) for each expression (“modality of judgment” in the figure). In FIG. 5 , representations of the change factor part are arranged in order from left to right on the horizontal axis in descending order of confidence (see the line graph), which is one of the scale values. Each expression is associated in a columnar graph with the amount of increase in the fundamental frequency during dialogue and the amount of increase in the fundamental frequency during reading.

For example, the amount of increase in the fundamental frequency of the expression "must be" shown on the far left in FIG. 5 is about -0.18 during dialogue and about -0.08 during reading. That is, the fundamental frequency from "i" at the end of the expression to the final particle immediately following the expression (for example, "yo") descends. Therefore, it can be read that the voice changes to low.

In step S4 of FIG. 2, the scale value control amount conversion unit 3 uses the values shown in FIG. Transform values to control prosody. As another function, the scale value control amount conversion unit 3 may set the variation amount of the fundamental frequency of the change factor part or the variation amount of the verb part as the prosody control amount.

The scale value control amount conversion unit 3 can control the prosody using the command response model of Reference 3 below.
Reference 3: H. Fujisaki and S. Nagashima, "A model for the synthesis of pitch contours of connected speech", 1969, Annual Report of the Engineering Research Institute, Faculty of Engineering, University of Tokyo, pp.53- 60

In this case, the scale value control amount conversion unit 3 may use a plurality of control amounts such as the magnitude of the phrase component or accent component used by the command response model and the time of occurrence and loss of effect.

In any of the above cases, the scale value control amount conversion unit 3 converts one or more scale values obtained by the scaling unit 2 into prosodic control amounts. At the time of the conversion, the scale value control amount conversion unit 3 uses a linear or nonlinear multiple regression model in which each control amount is individually converted using one or more scale values as explanatory variables and the prosody control amount as a dependent variable, or , it is possible to use a neural network that transforms simultaneously.

At step S5 in FIG. 2, the prosody generation unit 4 generates a prosody based on the control amount obtained by the scale value control amount conversion unit 3. FIG. For example, when the amount of increase in the fundamental frequency is used as the amount of control, the prosody generation unit 4 is configured such that if the amount of increase in the fundamental frequency is positive, the amount of increase in the fundamental frequency is increased (the voice becomes higher), or if the amount of increase is negative. It is possible to control the fundamental frequency in a phrase such that the fundamental frequency falls (lower voice) for values of .

When the control amount of the command response model of Reference 3 is used as the control amount, the prosody generation unit 4 controls changes in the pitch of the voice according to the magnitude of the phrase component and the accent component, and determines the occurrence of these components. It is possible to control the gradual rise of the fundamental frequency by time.

As described above, according to the present embodiment, the prosody control device D can perform various prosody controls corresponding to minute changes in the element that precedes the final particle. Specifically, the prosody control device D considers the influence of each part of the expressions that make up the phrase, such as the verb part, the final particle, and the change factor part between them, and once converts them into a scale value of 1 or more. , and further converts the scale value into the prosody control amount. This makes it possible to flexibly control the degree of prosody according to detailed intentions or meanings.

Further, according to this embodiment, the language analysis unit 1 determines whether or not the phrase has a change factor part, and when it is determined that the phrase does not have a change factor part, the scale value control amount conversion unit 3 A predetermined constant is set as the controlled variable. Therefore, flexible and diverse prosody control is possible for phrases regardless of the presence or absence of the variable factor portion.

The above-described embodiment is an example. It will be apparent to those skilled in the art that many modifications and substitutions can be made to the embodiments within the spirit and scope of the invention. Therefore, the disclosure should not be construed as limited by the above-described embodiments, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine a plurality of configuration blocks described in the configuration diagrams of the embodiments into one, or divide one configuration block.

The apparatus of the present invention can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network. For example, when the prosody control device D is configured by a computer, a program describing the processing contents for realizing each function is stored in a storage means inside or outside the computer, and the central processing unit (CPU) of the computer stores the program. By reading and executing this program, the computer can function. In addition, such programs can be distributed by selling, assigning, or lending portable recording media such as DVDs or CD-ROMs. , and can be distributed by transferring the program from a server to another computer via a network. Also, a computer that executes such a program can once store, for example, a program recorded on a portable recording medium or a program transferred from a server in its own storage means. Also, as another embodiment of this program, the computer may read the program directly from a portable recording medium and execute processing according to the program. In addition, it is also possible to sequentially execute processing according to the received program.

D Prosody control device 1 Language analysis unit 2 Scaling unit 3 Scale value control amount conversion unit 4 Prosody generation unit

Claims (3)

Receiving an input of an expression including a phrase, dividing the phrase into words, identifying a final particle and a verb part, and if there is a part sandwiched between the final particle and the verb part, the part is treated as a change factor part. a linguistic analyzer configured as
a scaling unit that determines a scale value corresponding to the variable factor;
a scale value control amount conversion unit that converts the scale value into a prosody control amount;
a prosody generator that controls the prosody of the expression based on the prosody control amount;
has
A prosody control device , wherein, when the language analysis unit determines that the change factor part does not exist in the phrase, the scale value control amount conversion unit sets a predetermined constant as the prosody control amount . The language analysis unit receives an input of an expression including a phrase, divides the phrase into words, identifies a final particle and a verb part, and if there is a part sandwiched between the final particle and the verb part, setting the portion as the variable factor portion;
determining, by a scaling unit, a scale value corresponding to the variable factor;
a step of converting the scale value into a prosodic control quantity by a scale value control quantity conversion unit;
controlling the prosody of the expression based on the prosody control amount by a prosody generator;
setting a predetermined constant as the prosody control amount by the scale value control amount conversion unit when the language analysis unit determines that the change factor part does not exist in the phrase;
A prosody control method having A program for causing a computer to function as the prosody control device according to claim 1 .

Images