CN116959405A - A speech synthesis method, device, electronic device and storage medium - Google Patents

Abstract

The disclosure provides a voice synthesis method, a voice synthesis device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to the technical field of deep learning and voice processing. The voice synthesis method is applied to cloud equipment provided with the artificial intelligent chip; the cloud end equipment deploys a plurality of speaker models in advance; the artificial intelligent chip comprises a plurality of computing cores for independently processing the voice synthesis request; the method comprises the following steps: receiving speech synthesis request data sent by a user, and adding the speech synthesis request data into a cache queue; the voice synthesis request data comprises at least one target text segment and a target speaker model designated by a user; and concurrently acquiring target text fragments from the cache queue, and respectively transmitting the target text fragments to different computing cores, so that the computing cores convert the target text fragments into corresponding voice data based on a target speaker model. The speech synthesis method and the speech synthesis device can improve speech synthesis efficiency under the scene of the multi-speaker model.

Description

A speech synthesis method, device, electronic device and storage medium

Technical field

The present disclosure relates to the technical field of artificial intelligence, especially to the technical field of deep learning and speech processing, and specifically to a speech synthesis method, device, electronic equipment, storage medium and computer program product.

Background technique

Speech synthesis technology, also known as Text to Speech (TTS) technology, can convert text information into speech in real time. It has been widely used in human-computer dialogue, remote voice information services, machine reading, telecommunications, entertainment, etc.

Contents of the invention

The present disclosure provides a speech synthesis method, device, electronic device, storage medium and computer program product.

According to one aspect of the present disclosure, a speech synthesis method is provided, which is applied to a cloud device configured with an artificial intelligence chip; the cloud device pre-deploys multiple speaker models for speech synthesis; the artificial intelligence chip includes multiple A computing core that independently processes speech synthesis requests; the method includes:

Receive speech synthesis request data sent by the user and add it to the cache queue; wherein the speech synthesis request data includes at least one target text segment and a target speaker model specified by the user;

Concurrently obtain the target text fragments from the cache queue, and send the target text fragments to different computing cores respectively, so that the computing core converts the target text fragments into corresponding Voice data.

According to another aspect of the present disclosure, a speech synthesis device is provided, configured in a cloud device equipped with an artificial intelligence chip; the cloud device pre-deploys a plurality of speaker models for speech synthesis; the artificial intelligence chip includes Multiple computing cores that independently process speech synthesis requests; the device includes:

A data receiving and caching module, configured to receive speech synthesis request data sent by the user and add it to the cache queue; wherein the speech synthesis request data includes at least one target text segment and a target speaker model specified by the user;

A data extraction and calculation module is used to obtain the target text fragments from the cache queue concurrently, and send the target text fragments to different computing cores respectively, so that the computing cores calculate all the target text fragments based on the target speaker model. The target text segment is converted into corresponding speech data.

According to another aspect of the present disclosure, an electronic device is provided, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the speech synthesis method according to any embodiment of the present disclosure. .

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to execute the speech synthesis method described in any embodiment of the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When executed by a processor, the computer program implements the speech synthesis method of any embodiment of the present disclosure.

According to the technology of the present disclosure, the speech synthesis efficiency in a multi-speaker model scenario can be improved.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of the drawings

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:

Figure 1 is a schematic flow chart of a speech synthesis method according to an embodiment of the present disclosure;

Figure 2 is a schematic flowchart of another speech synthesis method according to an embodiment of the present disclosure;

Figure 3 is a schematic flowchart of another speech synthesis method according to an embodiment of the present disclosure;

Figure 4 is a logical schematic diagram of another speech synthesis method according to an embodiment of the present disclosure;

Figure 5 is a schematic diagram of a speech synthesis device according to an embodiment of the present disclosure;

FIG. 6 is a block diagram of an electronic device used to implement the speech synthesis method according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

In this application solution, in order to provide users with personalized speech synthesis services, multiple speaker models (such as pre-trained deep learning models) for converting text into speech are deployed on cloud devices (such as cloud server devices), and Different speaker models have different model parameters. In order to improve the efficiency of speech synthesis, cloud devices can use CPU (Central Processing Unit, central processing unit) chips to provide computing power or GPU (Graphics Processing Unit, graphics processor) chips to provide computing power. However, these two methods have certain shortcomings. : Due to its own software and hardware characteristics, the CPU chip cannot independently control the core of the CPU chip to process speech synthesis requests, so that the CPU chip can only process the received speech synthesis requests in a serial manner, resulting in low efficiency of speech synthesis; while using When GPU chips provide computing power, the number of speaker models deployed on a cloud device is smaller, and GPU chips are more expensive, resulting in higher costs. Based on this, this application solution does not use CPU chips and GPU chips to provide computing power, but uses artificial intelligence chips suitable for cloud devices to provide computing power, that is, configuring artificial intelligence chips in cloud devices, where the artificial intelligence chips include multiple A computing core that allows independent processing of speech synthesis requests; in this way, speech synthesis based on the ability provided by the artificial intelligence chip to independently control the computing core can improve the efficiency of speech synthesis in multi-speaker model scenarios. Optionally, the artificial intelligence chip can be an artificial intelligence chip model MLU100. The MLU100 chip can not only improve computing efficiency, but also save price and cost. On this basis, for the specific process of the speech synthesis method involved in the present disclosure, please refer to the following embodiments.

Figure 1 is a schematic flowchart of a speech synthesis method according to an embodiment of the present disclosure. This embodiment can be applied to scenarios where speech synthesis services are provided to users based on multiple speaker models; typically, when users read e-books, users can select part of the e-book text and determine their own text from multiple speaker models. The required speaker model is then used to synthesize the corresponding speech for the user based on the text and speaker model selected by the user. The method can be performed by a speech synthesis device, which is implemented in software and/or hardware, and is preferably configured in an electronic device, such as a cloud server device.

As shown in Figure 1, the method specifically includes the following:

S101. Receive speech synthesis request data sent by the user and add it to the cache queue.

S102. Concurrently obtain the target text fragments from the cache queue, and send the target text fragments to different computing cores respectively, so that the computing core converts the target text fragments into Corresponding voice data.

In this embodiment, the speech synthesis request data sent by each user includes at least one target text segment that needs to be converted into speech and a target speaker model specified by the user. For example, when reading an e-book on a mobile terminal, the user can select one or more e-book text fragments and trigger the speech synthesis function. At this time, the user-selectable speaker model can be displayed in a pop-up window on the mobile terminal. , and then after the user determines the target speaker model, the mobile terminal sends at least one target text segment selected by the user and the target speaker model selected by the user to the cloud device through network communication, that is, the cloud device receives the speech synthesis request sent by the user data. It should be noted that for the speech synthesis request data sent by any user, the cloud device needs to open a corresponding thread to receive the user's speech synthesis request data, so that the speech synthesis request data of different users can be received concurrently. Furthermore, since the artificial intelligence chip that provides computing power for cloud devices allows each computing core to be independently controlled, each computing core can independently process a speech synthesis request, that is, a single computing core can convert a target text fragment sent by a user. into corresponding speech, and the artificial intelligence chip includes multiple computing cores, so that multiple users' speech synthesis requests can be processed concurrently. It should be noted that if the cloud device uses a CPU chip to provide computing power, although the CPU chip also has multiple computing cores, the computing cores of the CPU chip cannot be controlled individually. Therefore, after receiving speech synthesis requests from multiple users, the CPU chip It can only be processed in a serial manner, resulting in low efficiency of the entire speech synthesis. The use of artificial intelligence chips to provide computing power can solve this problem.

Furthermore, sometimes the speech synthesis request sent by a certain user includes a large number of target text fragments. At this time, if the user's speech synthesis request is also processed by a computing core, the computing core needs to be serially executed multiple times to complete. This user's speech synthesis request. At this time, if the number of speech synthesis requests from other users includes less target text fragments, other computing cores will finish processing them quickly. In this case, one computing core may be working while other computing cores are idle, causing each computing core to be idle. The load is not balanced, and it also affects the user's speech synthesis efficiency. In response to this possible problem, the disclosed solution pre-sets a cache queue. When a certain user's speech synthesis request data includes a large number of target text fragments, the target text fragments in the user's speech synthesis request data are first added to in the cache queue; and then distribute the user's target text fragments to different computing cores for speech synthesis processing in a concurrent manner. For example, multiple threads are opened to obtain a certain amount of target text fragments belonging to the user and distributed to different computing cores, so that each computing core converts the target text fragments into corresponding speech based on the target speaker model specified by the user. data. Then the voice data output by multiple computing cores are spliced and fed back to the user. In this way, multiple computing cores can simultaneously process a user's speech synthesis request data, which not only ensures the load balancing of the computing cores, but also ensures the efficiency of speech synthesis.

In this embodiment, a cache queue is set up to cache the user's speech synthesis request data so that the request data can be processed simultaneously by multiple computing cores. This not only ensures load balancing of the computing cores, but also improves speech synthesis efficiency.

Figure 2 is a schematic flowchart of another speech synthesis method according to an embodiment of the present disclosure. In the embodiment of the present disclosure, each speaker model is pre-associated with multiple online service threads and multiple internal computing threads. On this basis, as shown in Figure 2, the speech synthesis method specifically includes the following steps:

S201. Receive the speech synthesis request data sent by the user through the online service thread associated with the target speaker model, and add it to the cache queue.

S202. Concurrently obtain the target text fragments from the cache queue through the internal computing thread associated with the target speaker model; and send the target text fragments to different computing cores respectively, so that the computing cores are based on the target The speaker model converts the target text segments into corresponding speech data.

In this embodiment, the speech synthesis request data includes at least one target text segment and a target speaker model specified by the user. Since the speech synthesis request data sent by the user includes the target speaker model specified by the user, the speech synthesis request data sent by the user can be received through the online service thread associated with the target speaker model and added to the cache queue. It should be noted that each speaker model is pre-associated with multiple online service threads, so that it can receive speech synthesis request data for a certain speaker model from multiple different users at the same time.

In this embodiment, since the target speaker model is pre-associated with multiple internal calculation threads, the target text fragments can be concurrently obtained from the cache queue through the internal calculation threads associated with the target speaker model. It should be noted that for the situation where there are very few target text fragments in each user's speech synthesis request data, each internal computing thread can obtain the speech synthesis request data sent by a user and send it to different computing cores for processing. Speech synthesis processing enables multiple computing cores to concurrently process speech synthesis requests from different users. While the speech synthesis request data for a certain user includes a large number of target text fragments, while the speech synthesis request data for other users contains a smaller number of target text fragments, the target text sent by the user can be obtained concurrently through multiple internal computing threads. The fragments are sent to different computing cores for speech synthesis processing. This allows multiple computing cores to centrally process the target text fragments sent by one user. This not only improves speech synthesis efficiency, but also ensures load balancing of each computing core.

In addition, it should be noted that the reason why the online service thread and the internal computing thread are set up is because the cache queue is set up, and these two threads are required to add data to the cache queue and obtain data from the cache queue and distribute it. If the cache queue is not set, each online service thread will send the speech synthesis request data received by different users to different computing cores to perform speech synthesis. At this time, if a certain online service thread receives a large amount of data, the result will be: The computing core whose thread data provides computing power can only perform speech synthesis in sequence, making the user's speech synthesis inefficient. Moreover, there may be a load imbalance problem in which the computing core is still processing speech synthesis requests, while other computing cores have stopped working.

In this embodiment, each speaker model is associated with multiple online service threads and internal computing threads, so that the online service thread only receives the speech synthesis request data that needs to be processed by the speaker model associated with itself, and the internal computing thread only obtains the speech synthesis request data associated with it. Text data required by the speaker model. Such an accurate use of the speaker model selected by the user for speech synthesis.

Further, in this embodiment, before adding the speech synthesis request data to the cache queue, the online service thread can also preprocess the target text fragment in the speech synthesis request data; wherein the preprocessing is to add the target text fragment to the cache queue. Convert to text data of preset length; the preset length can be the data length required by the speaker model. For example, if the length of the target text fragment is less than the preset length, data completion is performed; if the length of the target text fragment is greater than the preset length, the target text fragment is divided. In this way, by preprocessing the target text fragment, the text data subsequently distributed to the computing core meets the requirements of the speaker model and does not require further processing by the computing core, ensuring the efficiency of speech synthesis.

Further, during the initialization stage of the cloud device, for any speaker model, the operators included in the speaker model are fused; the operator optionally is an operation function in the speaker model. In this way, through operator fusion, when subsequent speech synthesis is performed, calculations can be performed directly through a fusion operator, without the need for separate operations to generate intermediate data. This avoids the generation of intermediate data and the transfer of intermediate data between different operators, and can ensure calculation efficiency. Then, the speaker model after operator fusion is compiled and optimized to reduce online compilation time.

Figure 3 is a schematic flowchart of another speech synthesis method according to an embodiment of the present disclosure. In this embodiment, multiple speaker models share a video memory pool; the video memory pool includes a plurality of preset data blocks for storing data, and the number of the data blocks is related to the computing core of the artificial intelligence chip. The quantity is the same. As shown in Figure 3, the method specifically includes the following:

S301. Receive speech synthesis request data sent by the user and add it to the cache queue.

Wherein, the speech synthesis request data includes at least one target text segment and a target speaker model specified by the user. Optionally, receive the speech synthesis request data sent by the user through the online service thread associated with the target speaker model, and add it to the cache queue. It should be noted that the data stored in the cache queue is actually stored in the running memory of the cloud device.

S302. Concurrently obtain the target text fragment from the cache queue; for any target text fragment, send the target text fragment to an idle target data block in the video memory pool.

Optionally, the target text fragment is obtained concurrently from the cache queue through multiple internal calculation threads associated with the target speaker model. It should be noted that each internal computing thread can obtain the target text fragment from the cache queue according to the preset receipt acquisition parameter batchsize.

In this embodiment, when using the artificial intelligence chip as computing power to perform speech synthesis, the target text fragment included in the speech synthesis request needs to be saved in the video memory of the artificial intelligence chip, so that the computing core of the artificial intelligence chip can perform speech synthesis based on the corresponding text data. Composite operations. That is, the target text fragment needs to be added from the running memory to the video memory of the artificial intelligence chip.

In this embodiment, a video memory pool shared by multiple speaker models is preset. The video memory pool includes a plurality of preset data blocks for storing data, and the number of data blocks is equal to the number of computing cores of the artificial intelligence chip. same. In this way, after the internal computing thread obtains the target text fragment, it only needs to add the target text fragment to the free data block. There is no need to open up a separate video memory space each time to save the text data of the speech to be synthesized (i.e., the above mentioned target text fragment). It can ensure the efficiency of speech synthesis as a whole.

S303. Poll the current status of each computing core, and determine the target computing core that is currently idle, so that the target computing core uses the target speaker model to convert the target text fragment in the target data block into speech data. .

In this embodiment, in response to the operation of sending the target text fragment to the idle target data block in the video memory pool, the current status of each computing core is polled, and the target computing core currently in the idle status is determined, so that the The target computing core converts the target text segments in the target data block into speech data using the target speaker model.

In this embodiment, a preset video memory pool is shared to avoid opening a separate video memory space to save the text data of the speech to be synthesized every time a speech synthesis request is processed, which can ensure the efficiency of speech synthesis as a whole.

Further, after the user's target text segment is synthesized into speech data, the synthesized speech data is fed back to the user, and the target text segment in the target data block is cleared so that the internal computing thread can continue to synthesize the speech data to be synthesized. The target text fragment is stored in a free data block.

Figure 4 is a logic diagram of another speech synthesis method according to an embodiment of the present disclosure. As shown in Figure 4, where. Model0 represents a speaker model deployed in a cloud device. The online service threads associated with the speaker model are thread 0, thread 1, thread 2, and thread 3 respectively. The internal computing thread associated with the speaker model is, for example, thread A. Thread B, thread C; the computing cores of the artificial intelligence chip include, for example, computing core 0, computing core 1... and computing core 31. Other speaker models are similar to the speaker model Model0 and are not shown in the figure. Taking Model0 as an example, the speech synthesis process of the present disclosure is described: the online service threads (thread 0, thread 1, thread 2, and thread 3) respectively obtain speech synthesis request data (including target text fragments) of different users; among them, different users Refers to different users who specify using Model0 for speech synthesis. The online service thread adds the obtained speech synthesis request data of different users to the cache queue. Further, the internal computing threads (Thread A, Thread B, Thread C) concurrently obtain the target text fragments from the cache queue and send them to the video memory pool, so that different computing cores can use Model0 to perform speech synthesis based on the data in the video memory pool. . It should be noted that the target text obtained by thread A, thread B, and thread C can belong to the same user or different users; finally, the synthesis result is output and fed back to the corresponding user.

FIG. 5 is a schematic structural diagram of a speech synthesis device according to an embodiment of the present disclosure. This embodiment can be applied to a scenario in which speech synthesis is provided for users based on a multi-speaker model. The device can implement the speech synthesis method described in any embodiment of the present disclosure. The device is configured on a cloud device equipped with an artificial intelligence chip; the cloud device has pre-deployed multiple speaker models for speech synthesis; and the artificial intelligence chip includes multiple computing cores that independently process speech synthesis requests. As shown in Figure 5, the device 500 specifically includes:

The data receiving and caching module 501 is used to receive speech synthesis request data sent by the user and add it to the cache queue; wherein the speech synthesis request data includes at least one target text segment and a target speaker model specified by the user;

The data extraction and calculation module 502 is used to obtain the target text fragments from the cache queue concurrently, and send the target text fragments to different computing cores respectively, so that the computing cores calculate the target text fragments based on the target speaker model. The target text segments are converted into corresponding speech data.

In an optional implementation, each speaker model is associated with multiple online service threads;

Data receiving and caching modules include:

A receiving unit configured to receive speech synthesis request data sent by the user through an online service thread associated with the target speaker model.

In an alternative implementation, each speaker model is associated with multiple internal computing threads;

Data extraction and calculation modules include:

An extraction unit is configured to concurrently obtain the target text fragment from the cache queue through an internal computing thread associated with the target speaker model.

In an optional implementation, multiple speaker models share a video memory pool; the video memory pool includes a plurality of preset data blocks for storing data, and the number of the data blocks is consistent with the artificial intelligence The chips have the same number of computing cores;

The data extraction and calculation module also includes:

A sending unit, configured to send any target text fragment to an idle target data block in the video memory pool;

A computing unit configured to poll the current status of each computing core and determine the target computing core that is currently idle, so that the target computing core uses the target speaker model to convert the target text fragment in the target data block. into voice data.

In an optional implementation, it also includes:

A feedback and clearing module is used to feed back the synthesized voice data to the user, and clear the target text segments in the target data block.

In an optional implementation, it also includes:

A preprocessing module, configured to preprocess target text segments in the speech synthesis request data; wherein the preprocessing is to convert the target text segments into text data of a preset length.

In an optional implementation, it also includes:

An operator fusion module is used to fuse the operators included in the speaker model for any speaker model during the initialization phase of the cloud device;

The compilation module is used to compile and optimize the speaker model after operator fusion.

In an optional implementation solution, the artificial intelligence chip is an MLU100 chip.

The above-mentioned products can execute the methods provided by any embodiment of the present disclosure, and have corresponding functional modules and beneficial effects for executing the methods.

In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are in compliance with relevant laws and regulations and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

Figure 6 shows a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 6 , the device 600 includes a computing unit 601 that can execute according to a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603 Various appropriate actions and treatments. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. Computing unit 601, ROM 602 and RAM 603 are connected to each other via bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Multiple components in device 600 are connected to I/O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as magnetic disk, optical disk, etc. ; and communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.

Computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as speech synthesis methods. For example, in some embodiments, the speech synthesis method may be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 600 via ROM 602 and/or communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the speech synthesis method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the speech synthesis method in any other suitable manner (eg, by means of firmware).

Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs executing on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as cloud computing server or cloud host. It is a host product in the cloud computing service system to solve the problems of difficult management and weak business scalability in traditional physical hosts and VPS services. defect. The server can also be a distributed system server or a server combined with a blockchain.

Artificial intelligence is the study of using computers to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technology and software-level technology. Artificial intelligence hardware technology generally includes sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing and other technologies; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology and machine learning/depth Learning technology, big data processing technology, knowledge graph technology and other major directions.

Cloud computing refers to a flexible and scalable shared physical or virtual resource pool through network access. Resources can include servers, operating systems, networks, software, applications, storage devices, etc., and can be on-demand and self-service. A technical system for deploying and managing resources. Through cloud computing technology, it can provide efficient and powerful data processing capabilities for artificial intelligence, blockchain and other technology applications and model training.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution provided by the present disclosure can be achieved, there is no limitation here.

The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.

Claims (19)

1. A speech synthesis method is applied to cloud equipment configured with an artificial intelligent chip; the cloud device deploys a plurality of speaker models for speech synthesis in advance; the artificial intelligent chip comprises a plurality of computing cores for independently processing the voice synthesis request; the method comprises the following steps:

receiving speech synthesis request data sent by a user, and adding the speech synthesis request data into a cache queue; wherein the speech synthesis request data comprises at least one target text segment and a target speaker model specified by a user;

and concurrently acquiring the target text fragments from the cache queue, and respectively sending the target text fragments to different computing cores, so that the computing cores convert the target text fragments into corresponding voice data based on a target speaker model.

2. The method of claim 1, wherein each speaker model is associated with a plurality of online service threads;

receiving speech synthesis request data sent by a user, wherein the speech synthesis request data comprises:

and receiving voice synthesis request data sent by a user through the online service thread associated with the target speaker model.

3. The method of claim 1, wherein each speaker model is associated with a plurality of internal computing threads;

concurrently obtaining the target text segment from the cache queue includes:

and concurrently acquiring the target text fragments from the cache queue through the internal computing threads associated with the target speaker model.

4. The method of claim 1, wherein a plurality of speaker models share a video pool; the video memory pool comprises a plurality of preset data blocks for storing data, and the number of the data blocks is the same as the number of the computing cores of the artificial intelligent chip;

sending the target text segments to different computing cores respectively, so that the computing cores convert the target text segments into corresponding voice data based on a target speaker model, and the method comprises the following steps:

for any target text segment, the target text segment is sent to a target data block which is free in the video memory pool;

and polling the current state of each computing core, and determining the target computing core in the idle state, so that the target computing core converts the target text segment in the target data block into voice data by using the target speaker model.

5. The method of claim 4, further comprising:

and feeding the synthesized voice data back to the user, and emptying the target text segment in the target data block.

6. The method of claim 1, further comprising:

preprocessing a target text segment in the voice synthesis request data; the preprocessing is to convert the target text segment into text data with preset length.

7. The method of claim 1, further comprising:

in the cloud equipment initialization stage, aiming at any speaker model, fusing operators included in the speaker model;

and compiling and optimizing the speaker model after operator fusion.

8. The method of claim 1, wherein the artificial intelligence chip is an MLU100 chip.

9. A voice synthesis device is configured on a cloud device configured with an artificial intelligent chip; the cloud device deploys a plurality of speaker models for speech synthesis in advance; the artificial intelligent chip comprises a plurality of computing cores for independently processing the voice synthesis request; the device comprises:

the data receiving and caching module is used for receiving voice synthesis request data sent by a user and adding the voice synthesis request data into a cache queue; wherein the speech synthesis request data comprises at least one target text segment and a target speaker model specified by a user;

the data extraction and calculation module is used for obtaining the target text fragments from the cache queue in a concurrent mode, and sending the target text fragments to different calculation cores respectively, so that the calculation cores convert the target text fragments into corresponding voice data based on a target speaker model.

10. The apparatus of claim 9, wherein each speaker model is associated with a plurality of online service threads;

the data receiving and buffering module comprises:

and the receiving unit is used for receiving the voice synthesis request data sent by the user through the online service thread associated with the target speaker model.

11. The apparatus of claim 9, wherein each speaker model is associated with a plurality of internal computing threads;

the data extraction and calculation module comprises:

and the extraction unit is used for concurrently acquiring the target text fragments from the cache queue through the internal calculation threads associated with the target speaker model.

12. The apparatus of claim 9, wherein a plurality of speaker models share a video memory pool; the video memory pool comprises a plurality of preset data blocks for storing data, and the number of the data blocks is the same as the number of the computing cores of the artificial intelligent chip;

the data extraction and computation module further comprises:

the sending unit is used for sending the target text fragments to the idle target data blocks in the video memory pool aiming at any target text fragment;

and the computing unit is used for polling the current state of each computing core and determining the target computing core in the idle state, so that the target computing core converts the target text segment in the target data block into voice data by using the target speaker model.

13. The apparatus of claim 12, further comprising:

and the feedback and emptying module is used for feeding the synthesized voice data back to the user and emptying the target text segment in the target data block.

14. The apparatus of claim 9, further comprising:

the preprocessing module is used for preprocessing the target text segment in the voice synthesis request data; the preprocessing is to convert the target text segment into text data with preset length.

15. The apparatus of claim 9, further comprising:

the operator fusion module is used for fusing operators included in the speaker model aiming at any speaker model in the cloud equipment initialization stage;

and the compiling module is used for compiling and optimizing the speaker model after operator fusion.

16. The apparatus of claim 9, wherein the artificial intelligence chip is an MLU100 chip.

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the speech synthesis method of any one of claims 1-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the speech synthesis method according to any one of claims 1-8.

19. A computer program product comprising a computer program which, when executed by a processor, implements the speech synthesis method according to any of claims 1-8.