JP7601649B2 - Voice synthesis adjustment device - Google Patents

Description

This disclosure relates to a voice synthesis adjustment device that adjusts (tunes) the voice synthesis process performed by a voice synthesis device.

There is known a speech synthesis device that performs natural language processing on an input sentence to obtain prosodic information of the input sentence and generates synthetic speech corresponding to the input sentence based on the obtained prosodic information. For example, a technology is known that generates synthetic speech according to predetermined prosodic information generation rules in order to make the synthetic speech closer to human speech. Note that "prosodic information" here means information that indicates the phonetic properties of human speech, such as pauses between phrases and accents of words. Conventional speech synthesis devices aim to output optimal speech for everyone, but the optimal speech varies depending on the domain in which the speech is used (which in this case means "classification determined by the usage scene and purpose"). Therefore, it is essential to adjust (tune) the prosodic information generation rules as appropriate.

Japanese Patent Application Publication No. 9-62286

However, with conventional voice synthesis devices, the adjustment work described above had the drawback of taking a lot of time and requiring specialized knowledge.

In addition, although Patent Document 1 suggests that speech synthesis is performed by setting pauses from the prosodic information according to the purpose, it does not disclose details for aiming for adjustment work based on more appropriate prosodic information, such as using pauses according to the sentence structure obtained by analyzing the input sentence, leaving room for improvement.

The present disclosure therefore aims to adjust speech synthesis based on more appropriate prosodic information while shortening the time required for adjustment work in speech synthesis without requiring specialized knowledge.

A speech synthesis adjustment device according to an embodiment of the present disclosure is a speech synthesis adjustment device that creates and outputs a speech according to a domain determined by a usage scene and a purpose of the speech by a speech synthesis device, and includes: a domain setting unit that sets the domain based on domain information from an external source; a text data acquisition unit that acquires text data including prosodic information corresponding to input data from an external source; a determination unit that determines whether the prosodic information obtained from the text data acquired by the text data acquisition unit is new prosodic information that is not stored in a prosodic information dictionary that stores prosodic information for each domain in advance; a correction information acquisition unit that transfers the text data acquired by the text data acquisition unit to the speech synthesis device when the text data is determined to be new prosodic information and acquires correction information for the prosodic information for the synthetic speech output by the speech synthesis device; and a correction control unit that registers the new prosodic information for the corresponding domain in the prosodic information dictionary based on the correction information for the prosodic information acquired by the correction information acquisition unit, and causes the speech synthesis device to recreate and output a speech corrected based on the correction information for the prosodic information.

In the above voice synthesis adjustment device, the domain setting unit sets a domain based on domain information from outside, the text data acquisition unit acquires text data including prosodic information corresponding to the input data from outside, and the judgment unit judges whether the prosodic information acquired from the acquired text data is new prosodic information not stored in the prosodic information dictionary that stores prosodic information for each domain in advance. Here, if the prosodic information acquired from the text data is judged to be new prosodic information, the correction information acquisition unit transfers the acquired text data to the voice synthesis device and acquires correction information of the prosodic information for the synthetic voice output by the voice synthesis device. The "correction information of the prosodic information" here is, for example, input by the user after the synthetic voice is output to the user by the voice synthesis device, and transferred to the voice synthesis adjustment device via the voice synthesis device, and is acquired by the correction information acquisition unit. Then, the correction control unit registers the new prosodic information related to the corresponding domain in the prosodic information dictionary based on the acquired correction information of the prosodic information, and causes the voice synthesis device to recreate and output a voice corrected based on the correction information of the prosodic information. This allows the user to listen to speech that has been corrected based on the prosodic information correction information. As described above, when the prosodic information obtained from the acquired text data is new prosodic information not stored in the prosodic information dictionary, the new prosodic information for the corresponding domain is registered in the prosodic information dictionary based on the correction information for the acquired prosodic information, so that the conventional process can be partially automated and simplified the next time the same prosodic information is obtained from the acquired text data. This makes it possible to adjust the speech synthesis based on more appropriate prosodic information while shortening the time required for adjustment work in speech synthesis without requiring specialized knowledge.

According to the present disclosure, it is possible to adjust speech synthesis based on more appropriate prosodic information while reducing the time required for adjustment work in speech synthesis without requiring specialized knowledge.

1 is a configuration diagram of a voice synthesis adjustment device and related devices according to a first embodiment. FIG. 2 illustrates examples of different domains. FIG. 1A is a diagram showing an example of a pause rule for each sentence structure for domain A, and FIG. 1B is a diagram showing an example of a pause rule for each sentence structure for domain B. FIG. 2 is a flow diagram showing a process executed by the voice synthesis adjustment device and related devices in the first embodiment. FIG. 13 is a configuration diagram of a voice synthesis adjustment device and related devices in a second embodiment. FIG. 1 illustrates examples of per-word accent types for various domains. FIG. 11 is a flow diagram showing a process executed by a voice synthesis adjustment device and related devices in the second embodiment. FIG. 13 is a diagram for explaining a process of extracting a fundamental frequency by voice analysis. FIG. 2 is a diagram illustrating an example of a hardware configuration of a voice synthesis adjustment device.

Various embodiments of the present disclosure will be described below in order with reference to the accompanying drawings. Note that in the description of the drawings, the same elements are given the same reference numerals and duplicate descriptions will be omitted.

[First embodiment]
Below, as a first embodiment, an embodiment will be described in which a pause rule (pause insertion pattern) for the sentence structure of the input text according to a set domain (audio usage scene and purpose) is newly registered or selected, and speech synthesis is adjusted based on the pause rule.

Figure 1 shows an example of the configuration of a voice synthesis adjustment device 10 and related devices in the first embodiment, including a voice synthesis device 20, a control device 30, and other related devices 40. Of these, the control device 30 is a device that controls the entire system consisting of the multiple devices shown in Figure 1, and has a user interface function that normally receives text input and domain information from the user, and receives voice input and domain information during correction, and outputs synthetic voice to the user. The other related devices 40 have a function to convert user input into text data including prosodic information by performing processes such as voice recognition and natural language processing as necessary.

The speech synthesis device 20 has a function of performing speech synthesis processing based on appropriate prosodic information for the above text data (for example, in the first embodiment, processing for creating synthetic speech with pauses added according to appropriate pause rules) while being adjusted by the speech synthesis adjustment device 10 described below, and has a built-in pause dictionary 21 that stores pause rules according to sentence structures for various domains.

The voice synthesis adjustment device 10 has a function of adjusting (tuning) the voice synthesis process by the voice synthesis device 20 so that more appropriate prosodic information (e.g., pause rules in the first embodiment) is used.

To realize the above functions, the voice synthesis adjustment device 10 includes a domain setting unit 11, a text data acquisition unit 12, a sentence structure determination unit 13, a pause correction information acquisition unit 14, a pause correction control unit 15, and a first creation control unit 16.

Of these, the domain setting unit 11 is a functional unit that sets a domain based on external domain information, the text data acquisition unit 12 is a functional unit that acquires text data including prosodic information corresponding to external input data, and the sentence structure determination unit 13 is a functional unit that analyzes the sentence structure of the text data acquired by the text data acquisition unit 12 and determines whether the sentence structure obtained by the analysis is a new sentence structure in the above-mentioned pause dictionary 21. The analysis of the sentence structure of the text data by the sentence structure determination unit 13 is performed, for example, by dividing the sentences of the text data into phrases and classifying them from the perspective of how many phrases they are divided into and where the breaks in the sentences are located (for example, the positions where commas "," would be placed if the sentence were written in text). For example, as shown in the example sentence of sentence structure id1 in Fig. 3(a) showing the pause rules for each sentence structure, the sentence structure is divided into the first to sixth phrases separated by "/" such as "It was quite cold last night, but did you sleep well?", and it is analyzed that there is a break in the sentence after the fourth phrase "It was cold, but...". Also, as shown in the example sentence of sentence structure id3, the sentence structure is divided into the first to sixth phrases separated by "/" such as "I walked 3000 steps yesterday, so I'll rest my body today!", and it is analyzed that there is a break in the sentence after the third phrase "Since I walked...". Although these sentence structures id1 and id3 have in common the point of being divided into a total of six phrases, they are considered to be different sentence structures because the locations of the breaks in the sentence are different. Note that the above sentence structure analysis method is just an example, and analysis methods other than the above may be used.

The pause correction information acquisition unit 14 is a functional unit that transfers the analyzed text data to the speech synthesis device 20 when the sentence structure determination unit 13 determines that the sentence structure is new, and then acquires text data including pause correction information for the synthesized speech corresponding to the corrected speech from the user via the speech synthesis device 20. The pause correction control unit 15 is a functional unit that registers pause rules for the relevant sentence structure related to the relevant domain in the pause dictionary 21 based on the pause correction information acquired by the pause correction information acquisition unit 14, and causes the speech synthesis device 20 to recreate and output speech with pauses corrected based on the pause correction information.

The first creation control unit 16 is a functional unit that, when the sentence structure determination unit 13 determines that the sentence structure is not a new one, transfers the pause rules and text data of the corresponding sentence structure for the corresponding domain to the speech synthesis device 20, and causes the speech synthesis device 20 to create and output speech with pauses added according to the pause rules.

The series of processes executed by the voice synthesis adjustment device 10 equipped with the above-mentioned functional units and related devices will be described later.

Figure 2 shows examples of various domains determined by combinations of audio usage scenarios and applications. As shown in Figure 2, examples of usage scenarios include "nursing care," "childcare," and "news reading," and the same usage scenario can have multiple applications, with each combination of usage scenario and application being treated as one domain. For example, for the usage scenario "nursing care," examples of applications include "healthy people in their 70s" and "people with slow cognitive judgment," and the usage scenario "nursing care" and application "healthy people in their 70s" are treated as one domain (domain A in Figure 3(a)), and the usage scenario "nursing care" and application "people with slow cognitive judgment" are treated as one domain (domain B in Figure 3(b)).

3(a) and (b) show various examples of pause rules stored in the pause dictionary 21. As shown in FIG. 3(a), the pause dictionary 21 stores pause rules for each sentence structure for the domain A of the usage scene "care" and the purpose "healthy people in their 70s" in association with the sentence structure ID. Similarly, as shown in FIG. 3(b), the pause dictionary 21 stores pause rules for each sentence structure for the domain B of the usage scene "care" and the purpose "for people with slow cognitive judgment" in association with the sentence structure ID. As is clear from a comparison of FIG. 3(a) and (b) for the same sentence structure, the pause rule in FIG. 3(b) is a rule that generally provides longer pauses than the pause rule in FIG. 3(a), taking into account the difference between the purposes "for people with slow cognitive judgment" and "for healthy people in their 70s". For example, if we look at the breaks in a sentence (e.g., the positions where a comma would be placed if written in text), i.e., the fourth phrase of sentence structure id1, the fourth phrase of sentence structure id2, and the third phrase of sentence structure id3, the pause rules for domain A define all of these points as "medium pauses," whereas the pause rules for domain B define all of these points as "long pauses."

Below, a series of processes executed by the voice synthesis adjustment device 10 and related devices will be explained using the flow diagram in Figure 4.

First, the voice synthesis adjustment device 10 receives domain information input by the user via the control device 30, and sets the domain indicated in the domain information by the domain setting unit 11 (step S1). Specifically, the domain setting unit 11 performs domain selection if the domain indicated in the domain information is an existing domain in the pause dictionary 21, and performs domain registration if the domain indicated in the domain information is a new domain not registered in the pause dictionary 21.

In addition, the input from the user is subjected to processing such as speech recognition and natural language processing by other related devices 40 and converted into text data including prosodic information, and the converted text data is output to the speech synthesis adjustment device 10 via the control device 30 and the speech synthesis device 20 (step S2).

The voice synthesis adjustment device 10 acquires the text data output in step S2 by the text data acquisition unit 12 (step S3), analyzes the sentence structure of the acquired text data by the sentence structure determination unit 13 (step S4), and determines whether the sentence structure obtained by the analysis is a new sentence structure in the pause dictionary 21 (step S5).

Here, if it is determined that the sentence structure obtained by the analysis is a new sentence structure (YES in step S5), the speech synthesis adjustment device 10 transfers the analyzed text data (including prosodic information) as is to the speech synthesis device 20 by the pause correction information acquisition unit 14, and waits for pause correction information to arrive via the speech synthesis device 20 (step S6). After that, the speech synthesis device 20 creates a voice in which pauses are inserted according to the prosodic information (here, the pause rule) included in the transferred text data, and outputs the created synthetic voice to the user via the control device 30 (step S7). When the user who listened to the output synthetic voice inputs the modified voice to the control device 30, the modified voice is subjected to processing such as speech recognition and natural language processing by other related devices 40, and is converted into text data including the modified prosodic information (here, the modified pause rule), and the converted text data (including the modified pause rule) is output to the speech synthesis adjustment device 10 via the control device 30 and the speech synthesis device 20 (step S8).

The voice synthesis adjustment device 10 acquires the converted text data (including the corrected pause rule) by the pause correction information acquisition unit 14, and registers the pause rule (corrected pause rule) of the corresponding sentence structure related to the corresponding domain obtained from the converted text data in the pause dictionary 21 by the pause correction control unit 15, and instructs the voice synthesis device 20 to correct the pause based on the corrected pause rule, recreate the synthetic voice with the corrected pause added, and output it (step S9). The voice synthesis device 20 that has received the instruction in step S9 corrects the pause based on the corrected pause rule, recreates the synthetic voice with the corrected pause added, and outputs it to the user via the control device 30 (step S10). This allows the user to listen to the voice that has been corrected based on the corrected pause rule.

If the voice synthesis device 20 does not receive the converted text data (including the corrected pause rule) from the control device 30 within a predetermined time after outputting the synthetic voice to the user via the control device 30 in step S7, it considers that there is no pause correction for the voice output in step S7 and notifies the voice synthesis adjustment device 10 of this fact (the dashed line portion from S7 to S9 in FIG. 4). In this case, in step S9, the voice synthesis adjustment device 10 registers the pause rule included in the text data acquired in step S3 for the new sentence structure obtained by analysis by the pause correction control unit 15 in the pause dictionary 21 as the pause rule for the corresponding sentence structure for the corresponding domain. In this case, the voice recreation and output process in step S10 is not necessary.

On the other hand, if it is determined in step S5 that the sentence structure obtained by analysis is not a new sentence structure (NO in step S5), the voice synthesis adjustment device 10 transfers the pause rules of the relevant sentence structure for the relevant domain and the analyzed text data to the voice synthesis device 20 by the first creation control unit 16, and instructs the voice synthesis device 20 to recreate and output synthetic voice with pauses added in accordance with the pause rules (step S11). The voice synthesis device 20 that has received the instruction in step S11 recreates synthetic voice with pauses added in accordance with the pause rules, and outputs it to the user via the control device 30 (step S12). This allows the user to listen to voice with appropriate pauses added based on the pause rules registered in the pause dictionary 21.

According to the first embodiment described above, when the sentence structure obtained from the acquired text data is new, that is, when the pause information of the sentence structure for the set domain is new information in the pause dictionary 21, the pause information for the corresponding domain corrected based on the pause correction information obtained from the corrected voice is registered in the pause dictionary 21, so that the conventional processing can be partially automated and simplified the next time the same sentence structure is input from the acquired text data. This makes it possible to adjust the voice synthesis based on more appropriate pause information while shortening the time required for adjustment work in voice synthesis without requiring specialized knowledge. In addition, the user can listen to the voice in which the pauses have been appropriately corrected based on the corrected voice.

In addition, if the sentence structure obtained from the acquired text data is not new, the first creation control unit 16 transfers the pause rules and text data of the relevant sentence structure for the relevant domain to the speech synthesis device 20, and the speech synthesis device 20 creates and outputs speech with pauses added according to the pause rules. Therefore, the user can listen to speech with appropriate pauses added based on the pause rules registered in the pause dictionary 21.

[Second embodiment]
Next, as a second embodiment, we will explain an embodiment in which rules regarding the accent of words in the input text (hereinafter referred to as "accent type") according to a set domain (speech usage scene and purpose) are newly registered or selected, and speech synthesis is adjusted based on the accent type.

Figure 5 shows an example of the configuration of the voice synthesis adjustment device 10 and related devices in the second embodiment, including the voice synthesis device 20, the control device 30, and other related devices 40. Of these, the control device 30 and other related devices 40 have the same functions as in the first embodiment, so a duplicated description will be omitted.

The speech synthesis device 20 has a function of performing speech synthesis processing based on appropriate prosodic information for the speech text (for example, in the second embodiment, processing for creating synthetic speech with word accents according to appropriate accent types) while being adjusted by the speech synthesis adjustment device 10 described below, and has a built-in accent dictionary 22 that stores accent types for each word for various domains.

The voice synthesis adjustment device 10 has a function of adjusting (tuning) the voice synthesis process by the voice synthesis device 20 so that more appropriate prosodic information (e.g., accent type in the second embodiment) is used.

To realize the above functions, the voice synthesis adjustment device 10 includes a domain setting unit 11, a text data acquisition unit 12, a word determination unit 13S, an accent correction information acquisition unit 14S, an accent correction control unit 15S, and a second creation control unit 16S. Of these, the domain setting unit 11 and the text data acquisition unit 12 have the same functions as in the first embodiment, so a duplicated description will be omitted.

The word determination unit 13S is a functional unit that determines whether the text data acquired by the text data acquisition unit 12 includes a new word in the accent dictionary 22.

The accent correction information acquisition unit 14S is a functional unit that transfers the text data acquired by the text data acquisition unit 12 as is to the speech synthesis device 20 when the word determination unit 13S determines that the text data contains a new word, and then acquires text data including accent correction information for the synthesized speech corresponding to the corrected speech from the user (here, speech including the speech of the corrected word whose accent type is to be corrected) via the speech synthesis device 20. The accent correction control unit 15S is a functional unit that registers the accent type of the relevant word for the relevant domain in the accent dictionary 22 based on the accent correction information acquired by the accent correction information acquisition unit 14S, and causes the speech synthesis device 20 to recreate and output a speech in which the accent of the word has been corrected based on the accent correction information.

The second creation control unit 16S is a functional unit that, when the word determination unit 13S determines that the domain does not contain a new word, transfers the accent type and text data of the relevant word for the relevant domain to the voice synthesis device 20, and causes the voice synthesis device 20 to create and output a voice with an accent according to the accent type of the above word.

Figure 6 illustrates examples of accent types for each word for various domains in the second embodiment, stored in the accent dictionary 22. As explained with reference to Figure 2, a domain corresponds to a combination of a "usage scene" and a "purpose," but in Figure 6, to make the explanation easier to understand, the domains are illustrated using simple expressions such as "for standard language," "for business situations," and "for the elderly."

As can be seen from Figure 6, for example, in the word "horensa" (spinach), in domain A (standard Japanese), the position of the accent nucleus (i.e., the position where the accent is placed) is "re," whereas in domain B (Kansai dialect), there is no accent nucleus. Also, in the word "maji?" (real?), in domain A (standard Japanese), the accent nucleus is "ma," whereas in domain B (Kansai dialect), the accent nucleus is "ji." In this way, even for the same word, different accent types are set depending on the domain, with the accent nucleus being in a different position.

Below, a series of processes executed by the voice synthesis adjustment device 10 and related devices will be explained using the flow diagram in Figure 7.

In steps S1 to S3, similar to the first embodiment, the domain indicated in the domain information from the user is set, the input data from the user is converted into text and output, and the converted voice text data is obtained.

Then, the voice synthesis adjustment device 10 determines, by the word determination unit 13S, whether or not the text data acquired by the text data acquisition unit 12 includes a new word in the accent dictionary 22 (step S21).

Here, if it is determined that the text data contains a new word in the accent dictionary 22 (YES in step S21), the speech synthesis adjustment device 10 transfers the text data (including prosodic information) as is to the speech synthesis device 20 by the accent correction information acquisition unit 14S, and waits for the corrected speech (speech including the speech of the accent correction word) that arrives via the speech synthesis device 20 (step S22). After that, the speech synthesis device 20 creates a speech to which a word accent is added according to the prosodic information (here, the accent type) included in the transferred text data, and outputs the created synthetic speech to the user via the control device 30 (step S23). When the corrected speech is input to the control device 30 by the user who listened to the output synthetic speech, and is transferred to the speech synthesis adjustment device 10 via the speech synthesis device 20, the speech synthesis adjustment device 10 extracts the fundamental frequency (pitch) from the corrected speech by speech analysis using the accent correction information acquisition unit 14S (step S24). That is, as shown in Figure 8, the system detects frequency changes along the time axis in the corrected speech and determines how the fundamental frequency (pitch) has changed. The position where the fundamental frequency (pitch) is high is then detected as the accent nucleus of the word.

Then, the speech synthesis adjustment device 10 uses the accent correction control unit 15S to estimate a corrected word accent type according to the position of the accent nucleus of the word from the processing result of step S24, registers the estimated result in the accent dictionary 22 as the accent type of the relevant word for the relevant domain, and instructs the speech synthesis device 20 to correct the accent of the word based on the accent correction information (here, information on the corrected word accent type obtained by estimation) and recreate and output the synthetic voice with the corrected accent (step S25). The speech synthesis device 20 that has received the instruction in step S25 corrects the accent of the word based on the accent correction information, recreates the synthetic voice with the corrected accent, and outputs it to the user via the control device 30 (step S26). This allows the user to hear the speech corrected based on the accent correction information.

If the voice synthesis adjustment device 10 does not receive corrected voice from the user within a predetermined time after transferring the text data to the voice synthesis device 20 in step S22, it assumes that there has been no correction to the accent, avoids execution of steps S24 and S25, and ends the process. In other words, registration in the accent dictionary 22 in step S25 is executed only if there has been a correction to the accent.

On the other hand, if it is determined in step S21 that the text data does not include a new word in the accent dictionary 22 (NO in step S21), the voice synthesis adjustment device 10 transfers the accent type and text data of the relevant word for the relevant domain to the voice synthesis device 20 by the second creation control unit 16S, and instructs the voice synthesis device 20 to recreate and output synthetic voice with an accent according to the accent type (step S27). Having received the instruction in step S27, the voice synthesis device 20 recreates synthetic voice with an accent according to the accent type and outputs it to the user via the control device 30 (step S28). This allows the user to hear voice with an appropriate accent based on the accent type registered in the accent dictionary 22.

According to the second embodiment described above, when a word obtained from the acquired text data is new, that is, when the accent type of the word for the set domain is new information in the accent dictionary 22, the accent type of the word for the corresponding domain is registered in the accent dictionary 22 based on the accent correction information of the word obtained from the corrected voice, so that the conventional process can be partially automated and simplified the next time the accent type of the same word is obtained from the acquired text data. This makes it possible to adjust the voice synthesis based on more appropriate accent correction information while shortening the time required for adjustment work in voice synthesis without requiring specialized knowledge. In addition, the user can listen to a voice in which the accent of the word has been appropriately corrected based on the corrected voice.

In addition, if the word obtained from the acquired text data is not new, the second creation control unit 16S transfers the accent type and text data of the relevant word related to the relevant domain to the voice synthesis device 20, and the voice synthesis device 20 creates and outputs a voice with an accent according to the accent type. Therefore, the user can listen to a voice with an accent based on the accent type of the appropriate word registered in the accent dictionary 22.

The voice synthesis adjustment device 10 may have both the pause correction function described in the first embodiment and the accent correction function described in the second embodiment.

As a modified example of the process of FIG. 4 in the first embodiment, in the first stage (learning stage), a learned pause dictionary is created by repeatedly registering pause rules for various sentence structures for various domains in a pause dictionary, and then in the second stage (rule application stage), individual cases are applied to the learned pause dictionary to adjust the speech synthesis for the pauses of the individual cases, i.e., an embodiment in which the process is performed in two stages roughly divided in terms of time may be adopted.

Similarly, as a variation of the process of FIG. 7 in the second embodiment, in the first stage (learning stage), a learned accent dictionary is created by repeatedly registering accent types of various words for various domains in an accent dictionary many times, and then in the second stage (rule application stage), individual cases are applied to the learned accent dictionary to adjust the speech synthesis for the accent of each individual case, i.e., an embodiment in which the process is performed in two stages roughly divided in terms of time may be adopted.

[Terminology, variations, etc.]
The block diagrams used in the description of the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. The method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or may be realized using two or more devices that are physically or logically separated and directly or indirectly connected (for example, using wires, wirelessly, etc.). The functional block may be realized by combining the one device or the multiple devices with software.

Functions include, but are not limited to, judgement, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, regard, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs the transmission function is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on the method of realization for either of these functions.

For example, the voice synthesis adjustment device 10 in the first and second embodiments of the present disclosure may function as a computer that performs the processing of the present disclosure. As shown in FIG. 9, the above-mentioned voice synthesis adjustment device 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, and a bus 1007. This point is similar to devices other than the voice synthesis adjustment device 10, but here it will be described as an example configuration of the voice synthesis adjustment device 10.

In the following description, the term "apparatus" may be interpreted as a circuit, device, unit, etc. The hardware configuration of the voice synthesis adjustment device 10 may be configured to include one or more of the devices shown in the figure, or may be configured to exclude some of the devices.

The functions of the voice synthesis adjustment device 10 are realized by loading a specific software (program) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communications via the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured with a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, and a register. For example, each function of the voice synthesis adjustment device 10 described above may be realized by the processor 1001.

The processor 1001 reads out programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. The programs used are those that cause a computer to execute at least some of the operations described in the above-mentioned embodiments. For example, each function of the voice synthesis adjustment device 10 may be realized by a control program stored in the memory 1002 and running on the processor 1001. Although the above-mentioned various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.

The memory 1002 is a computer-readable recording medium, and may be composed of at least one of, for example, a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), and a random access memory (RAM). The memory 1002 may also be called a register, a cache, or a main memory (primary storage device). The memory 1002 can store executable programs (program codes), software modules, and the like for implementing an information providing method according to one embodiment of the present disclosure.

Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, and a magnetic strip. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., to realize, for example, at least one of Frequency Division Duplex (FDD) and Time Division Duplex (TDD).

The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

Each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

The voice synthesis adjustment device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

Notification of information is not limited to the aspects/embodiments described in this disclosure and may be performed using other methods.

The processing steps, sequences, flow charts, etc. of each aspect/embodiment described in this disclosure may be reordered unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order and are not limited to the particular order presented.

Information, etc. may be output from a higher layer to a lower layer, or from a lower layer to a higher layer. Information, etc. may be input/output via multiple network nodes.

The input and output information may be stored in a specific location (e.g., memory) or may be managed using a management table. The input and output information may be overwritten, updated, or added to. The output information may be deleted. The input information may be transmitted to another device.

The determination may be based on a value represented by one bit (0 or 1), a Boolean (true or false) value, or a numerical comparison (e.g., with a predetermined value).

Each aspect/embodiment described in this disclosure may be used alone, in combination, or switched according to execution. Notification of specific information (e.g., notification that "X is the case") is not limited to explicit notification, but may be performed implicitly (e.g., by not notifying the specific information).

Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended as an illustrative example and does not have any limiting meaning on the present disclosure.

Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

Software, instructions, information, and the like may be transmitted and received via a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and/or wireless technologies (such as infrared, microwave), these wired and/or wireless technologies are included within the definition of a transmission medium.

The information, signals, and the like described in this disclosure may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols, chips, and the like that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

Note that terms explained in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings.

As used in this disclosure, the terms "system" and "network" are used interchangeably.

The information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using other corresponding information.

The names used for the parameters described above are not limiting in any way. Furthermore, the formulas etc. using these parameters may differ from the formulas etc. explicitly disclosed in this disclosure.

The terms "determining" and "determining" as used in this disclosure may encompass a wide variety of actions. "Determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" may include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. "Determining" may include resolving, selecting, choosing, establishing, comparing, and the like. In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." "Judgment" ("decision") can also be interpreted as "assuming," "expecting," or "considering," etc.

The terms "connected" and "coupled", or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access". As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Any reference to elements using designations such as "first" and "second" used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not imply either that only two elements may be employed or that the first element must precede the second element in some way.

The "part" in the configuration of each of the above devices may be replaced with a "circuit" or a "device", etc.

When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Additionally, the term "or," as used in this disclosure, is not intended to be an exclusive or.

In this disclosure, where articles have been added by translation, such as "a," "an," and "the" in English, this disclosure may include that the nouns following these articles are plural.

In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

10... speech synthesis adjustment device, 11... domain setting unit, 12... text data acquisition unit, 13... sentence structure determination unit, 13S... word determination unit, 14... pause correction information acquisition unit, 14S... accent correction information acquisition unit, 15... pause correction control unit, 15S... accent correction control unit, 16... first creation control unit, 16S... second creation control unit, 20... speech synthesis device, 21... pause dictionary, 22... accent dictionary, 30... control device, 40... other related devices, 1001... processor, 1002... memory, 1003... storage, 1004... communication device, 1005... input device, 1006... output device, 1007... bus.

Claims (5)

A voice synthesis adjustment device that creates and outputs a voice according to a domain determined by a usage scene and a purpose of the voice by a voice synthesis device,
a domain setting unit that sets the domain based on domain information from an external source;
a text data acquisition unit for acquiring text data including prosodic information corresponding to input data from an external source;
a determination unit that determines whether or not the prosodic information obtained from the text data acquired by the text data acquisition unit is new prosodic information that is not stored in a prosodic information dictionary that stores prosodic information for each domain in advance;
a correction information acquisition unit that transfers the text data acquired by the text data acquisition unit to the speech synthesizer when the prosodic information is determined to be new by the determination unit, and acquires correction information for the prosodic information for the synthetic speech output by the speech synthesizer;
a modification control unit that registers the new prosodic information for a corresponding domain in the prosodic information dictionary based on the modification information for the prosodic information acquired by the modification information acquisition unit, and causes the speech synthesis device to regenerate and output a speech modified based on the modification information for the prosodic information;
Equipped with
the determination unit includes a sentence structure determination unit that analyzes a sentence structure of the text data acquired by the text data acquisition unit, and determines whether or not the sentence structure obtained by the analysis is a new sentence structure in a pause dictionary that stores pause rules corresponding to the sentence structure in advance for each domain;
the correction information acquisition unit includes a pause correction information acquisition unit that, when the sentence structure determination unit determines that the sentence structure is a new sentence structure, transfers text data analyzed by the sentence structure determination unit to the speech synthesizer and acquires pause correction information for the synthetic speech output by the speech synthesizer;
the correction control unit includes a pause correction control unit that registers a pause rule of a corresponding sentence structure related to a corresponding domain in the pause dictionary based on the pause correction information acquired by the pause correction information acquisition unit, and causes the speech synthesis device to recreate and output a speech in which a pause is corrected based on the pause correction information;
Speech synthesis adjustment device. a creation control unit that transfers the prosodic information related to the corresponding domain and the text data to the speech synthesizer when the determination unit determines that the prosodic information is not new, and causes the speech synthesizer to create and output a speech to which prosody corresponding to the prosodic information has been added;
The voice synthesis adjustment device of claim 1 further comprising: a first creation control unit that, when it is determined by the sentence structure determination unit that the sentence structure is not a new sentence structure, transfers a pause rule of the sentence structure for the corresponding domain and the text data to the speech synthesizer, and causes the speech synthesizer to create and output a speech to which pauses are added in accordance with the pause rule;
The voice synthesis adjustment device of claim 1 further comprising: the determination unit includes a word determination unit that determines whether the text data acquired by the text data acquisition unit includes a new word in an accent dictionary that stores in advance accent types of words for each of the domains;
the correction information acquisition unit includes an accent correction information acquisition unit that, when it is determined by the word determination unit that the text data includes a new word, transfers the text data acquired by the text data acquisition unit to the speech synthesis device and acquires accent correction information for the synthetic speech output by the speech synthesis device;
The correction control unit includes an accent correction control unit that registers the accent type of the corresponding word related to the corresponding domain in the accent dictionary based on the accent correction information acquired by the accent correction information acquisition unit, and causes the speech synthesis device to recreate and output a speech in which the accent of the word is corrected based on the accent correction information.
2. The voice synthesis adjustment device according to claim 1. a second creation control unit that transfers the accent type of the word in question and the text data for the domain to the speech synthesizer when the word determination unit determines that the word does not contain a new word, and causes the speech synthesizer to create and output a voice with an accent according to the accent type of the word;
The voice synthesis adjustment device of claim 4 further comprising:

Images