JP7624851B2 - Natural language processing device, translation device, and program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies (1) Date of issue (publication date) May 22, 2020 (Reiwa 2) Publication Proceedings of the 34th Annual Conference of the Japanese Society for Artificial Intelligence (PDF) Published by the Japanese Society for Artificial Intelligence (Website publication (download) URL: https://confit.atlas.jp/guide/event/jsai2020/proceedings/list) <Materials> Proceedings of the 34th Annual Conference of the Japanese Society for Artificial Intelligence (PDF) Published papers Excerpts <Materials> Proceedings PDF download site Webpage

Article 30, paragraph 2 of the Patent Act applies (2) Date of event (publication date) June 12, 2020 (Dates: June 9 to June 12) Name of event, location of event 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) Sponsored by the Japanese Society for Artificial Intelligence (General Incorporated Association) Held online using internet communication (Zoom) (Web publication (abstract) URL: https://confit.atlas.jp/guide/event/jsai2020/subject/4Rin1-44/categories?cryptoId= <Materials> 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) Event Guide Webpage <Materials> 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) Important Dates Webpage <Materials> <Materials> 2020 Japanese Society for Artificial Intelligence National Conference (34th) Timetable Webpage <Materials> 2020 Japanese Society for Artificial Intelligence National Conference (34th) Session Information Webpage <Materials> 2020 Japanese Society for Artificial Intelligence National Conference (34th) Lecture Information (Abstracts) Webpage

The present invention relates to a natural language processing device, a translation device, and a program.

In recent years, the level of machine translation technology has improved to a level that is practical for some applications. In addition, in order to make effective use of machine translation technology, research is being conducted into ways to improve it depending on the application.

Non-Patent Documents 1 and 2 describe devices that perform machine translation of sentences written in natural language.

Rico Sennrich, Barry Haddow, Alexandra Birch, Neural Machine Translation of Rare Words with Subword Units, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, p.1715-1725, August 2016, Association for Computational Linguistics, URL: https://www.aclweb.org/anthology/P16-1162.pdf Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention Is All You Need, 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA., 2017, URL: https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

The amount of content packed into an average sentence may vary depending on the type of language. Also, in certain applications of machine translation technology, it may be desirable to have appropriate control over the amount of content packed into a sentence.

For example, the amount of content typically contained in a single English sentence is often less than that contained in a single Japanese sentence. Furthermore, in news on television, radio, video streaming, Internet news sites, and the like, sentences in Japanese news scripts tend to be relatively long. On the other hand, sentences in English news scripts for the above-mentioned types of news tend to be relatively short. English news scripts are basically written with one meaning per sentence. Therefore, for example, when translating a Japanese news script to produce an English news script, a single Japanese sentence is often translated into several short English sentences.

In this way, it may be necessary to control the number of sentences on the output side (target language side) of the translation process depending on the language or application. Also, by controlling the number of sentences on the output side, it is possible to control the length of each sentence on the output side.

However, in conventional technology, when a sentence is given on the input side (source language side) of the translation process, it is not possible to control the number of sentences on the output side (target language side). Neither Non-Patent Document 1 nor Non-Patent Document 2 describes controlling the number of sentences on the output side (target language side). When considering machine translation for a specific language pair or machine translation processing for a specific purpose, it is desirable to be able to translate a sentence on the source language side into the desired (appropriate) number of sentences on the target language side.

To achieve this, it is desirable to be able to automatically perform the following two tasks. The first task is to estimate an appropriate number of output sentences. The second task is to control the translator so that a specific number of output sentences is achieved.

The present invention was made based on the recognition of the above problem, and aims to provide a natural language processing device, a translation device, and a program that can at least estimate the number of sentences in a target language sentence, which is the translation result, depending on the source language sentence to be translated.

[1] In order to solve the above problem, a natural language processing apparatus according to one aspect of the present invention includes a learning data supply unit that supplies learning data including a learning source sentence, a learning target sentence corresponding to the learning source sentence, and learning sentence number information indicating the number of sentences in the learning target sentence; an estimation unit that has a machine learning model and inputs a source sentence and outputs at least sentence number information indicating the number of sentences in the target sentence corresponding to the source sentence; and a source sentence supply unit that supplies the source sentence to the estimation unit. When operating in a learning mode, the estimation unit performs machine learning of the model based on the learning data supplied by the learning data supply unit, and when operating in an estimation execution mode, the estimation unit outputs the sentence number information corresponding to the source sentence based on the source sentence supplied by the source sentence supply unit.

[2] In addition, a natural language processing device according to one aspect of the present invention includes a model capable of machine learning, an estimation unit that inputs a source language sentence and outputs at least sentence number information indicating the number of sentences of a target language sentence corresponding to the source language sentence, and a source language sentence supply unit that supplies the source language sentence to the estimation unit, the model has been machine-learned based on learning data including a training source language sentence, a training target language sentence corresponding to the training source language sentence, and training sentence number information indicating the number of sentences of the training target language sentence, and the estimation unit outputs the sentence number information corresponding to the source language sentence based on the source language sentence supplied by the source language sentence supply unit.

[3] A natural language processing device according to one aspect of the present invention includes a training data supply unit that supplies training data including a training source language sentence, a training target language sentence corresponding to the training source language sentence, and training sentence number information indicating the number of sentences in the training target language sentence, and an estimation unit that has a machine learning model, inputs a source language sentence, and outputs at least sentence number information indicating the number of sentences in the target language sentence corresponding to the source language sentence, and the estimation unit performs machine learning of the model based on the training data supplied by the training data supply unit.

[4] A translation device according to one aspect of the present invention includes a sentence pair supplying unit that supplies data of a pair of a training source language sentence and a training target language sentence, a sentence number estimating unit that determines the number of sentences included in the supplied training target language sentence, a sentence number information adding unit that adds information on the number of sentences determined by the sentence number estimating unit to the training source language sentence and outputs data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added, a training data supplying unit that supplies pairs of data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added and data of the training target language sentence as training data, and an output object for output corresponding to the input source language sentence, which is input to the input source language sentence. a translation processing unit that includes a model, the model being trained using the training data, and the input source language sentence to which the information on the number of sentences of the output target language sentence has been added is input to the model, thereby outputting the output target language sentence determined by the model; and a natural language processing device according to [1] or [2] above, wherein a source language sentence supply unit of the natural language processing device supplies the input source language sentence to the estimation unit of the natural language processing device, and the number of sentences designation unit adds information on the number of sentences of the output target language sentence for output based on the number of sentences information output by the estimation unit of the natural language processing device.

[5] A translation device according to one aspect of the present invention includes a sentence number designation unit that adds, to an input source sentence, information on the number of sentences of an output target language sentence for output corresponding to the input source sentence, a model, and a translation processing unit that outputs an output target language sentence determined by the model by inputting the input source sentence to which the information on the number of sentences of the output target language sentence has been added, into the model, and the natural language processing device described in [1] or [2] above, wherein the model has been trained using training data that is a pair of data on a training source language sentence to which information on the number of sentences of the training target language sentence has been added and data on a training target language sentence, and a source language sentence supply unit of the natural language processing device supplies the input source language sentence to the estimation unit of the natural language processing device, and the sentence number designation unit adds information on the number of sentences of the output target language sentence for output based on the sentence number information output by the estimation unit of the natural language processing device.

[6] Another aspect of the present invention is a program for causing a computer to function as a natural language processing device according to any one of [1] to [3] above.

[7] Another aspect of the present invention is a program for causing a computer to function as the translation device described above in [4] or [5].

[Reference embodiment] In the above [4], the sentence number estimation unit may determine the number of sentences contained in the target language sentence for training by counting the number of sentence-final symbols contained in the supplied target language sentence for training.

[Reference aspect] In the above [4], the sentence number information adding unit may add information on the number of sentences of the target language sentence for training to the beginning of the source language sentence for training, which is text data, and the sentence number designation unit may add information on the number of sentences of the output target language sentence to the beginning of the input source language sentence, which is text data.

[Reference embodiment] In a learning device including a sentence pair supplying unit that supplies data of pairs of a training source language sentence and a training target language sentence, a sentence number estimating unit that determines the number of sentences included in the supplied training target language sentence, a sentence number information adding unit that adds the information on the number of sentences determined by the sentence number estimating unit to the training source language sentence and outputs data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added, a training data supplying unit that supplies pairs of data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added and data of the training target language sentence as training data, and a learning processing unit that uses the training data to train a model, the sentence number information adding unit may add the information on the number of sentences based on the sentence number information estimated by the estimation unit of the natural language processing device of [1] or [2].

According to the present invention, when a source language sentence is given, the sentence number estimation device can estimate the appropriate number of sentences in the target language sentence (translation result) that corresponds to the source language sentence.

In addition, by specifying the number of sentences estimated by the sentence number estimation device, it is possible to control the number of sentences in the target language sentences (translation results) output by the translation device.

1 is a block diagram showing a schematic functional configuration of a number-of-sentences estimation device according to a first embodiment of the present invention. 11 is a schematic diagram showing an example of the configuration of data supplied by a sentence pair supplying unit when the sentence number estimation device according to the embodiment operates in a learning mode. FIG. 11 is a schematic diagram showing an example of the configuration of data that a number-of-sentences information providing unit passes to a learning data supplying unit when the number-of-sentences estimation device according to the embodiment operates in a learning mode. FIG. 13 is a schematic diagram showing an example of the structure of data that a source language sentence supply unit passes to an estimation unit when the sentence number estimation device according to the embodiment operates in an estimation execution mode. FIG. 11 is a schematic diagram showing an example of data (including information on the estimated number of sentences) output by the estimation unit when the sentence number estimation device according to the embodiment operates in an estimation execution mode. FIG. FIG. 11 is a block diagram showing a schematic functional configuration of a translation device according to a second embodiment of the present invention. 11 is a schematic diagram showing an example of text pair data supplied by a text pair supplying unit in the embodiment. FIG. 4 is a schematic diagram showing an example of learning data (sentence pairs) that a learning data supply unit in the embodiment passes to a translation processing unit. FIG. 11 is a flowchart showing the procedure of a process for generating learning data for machine learning by a translation processing unit in the embodiment. 2 is a block diagram showing an example of an internal configuration of the translation device according to the embodiment. FIG. 11 is a schematic diagram showing an example of a result of a translation process performed by the translation device according to the embodiment. FIG.

Next, several embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
The number-of-sentences estimation device of the first embodiment estimates number-of-sentences information that is useful when translating a sentence in a source language into a sentence in a target language. Specifically, when a source language sentence to be translated is given, this number-of-sentences estimation device estimates an appropriate number of sentences in a target language sentence (translation result) corresponding to the source language sentence. Note that this number-of-sentences estimation device includes a model capable of machine learning, as described below.

In the following explanation, an example is used in which the source language sentence is a Japanese sentence and the target language sentence is an English sentence. However, the source language and the target language may each be another language. The source language and the target language may each be, for example, English, German, Japanese, French, Russian, Spanish, Italian, Chinese, Korean, or another language.

1 is a block diagram showing a schematic functional configuration of a sentence number estimation device according to this embodiment. As shown in the figure, the sentence number estimation device 320 includes an estimation unit 321, a source language sentence supply unit 322, a sentence number information extraction unit 323, a sentence pair supply unit 326, a sentence number information assignment unit 327, and a learning data supply unit 328. Each of these functional units can be realized, for example, by a computer and a program. Each functional unit also has a storage means as necessary. The storage means is, for example, a variable in the program or a memory allocated by the execution of the program. Also, non-volatile storage means such as a magnetic hard disk drive or a solid state drive (SSD) may be used as necessary. Also, at least a part of the functions of each functional unit may be realized as a dedicated electronic circuit or the like rather than a program. The functions of each unit constituting the sentence number estimation device 320 will be described below.

The number of sentences estimation device 320 can operate in either a learning mode or an estimation execution mode. When operating in the learning mode, the number of sentences estimation device 320 performs machine learning of an internal model. When operating in the estimation execution mode, the translation device 1 performs a process of estimating the number of sentences in an output sentence that matches an input sentence, based on the model in the current state.

Note that the number of sentences estimation device 320 has a function of learning a model as described above. Therefore, the number of sentences estimation device 320 may be called a learning device. Furthermore, the number of sentences estimation device 320 has a function of processing natural language data. Therefore, the number of sentences estimation device 320 may be called a natural language processing device.

The estimation unit 321 outputs data of the target language sentence (translation result) based on data of the input source language sentence. However, the estimation unit 321 outputs information on an estimated value regarding the appropriate number of sentences in the translation result together with the target language sentence. The estimation unit 321 has an internal model capable of machine learning. This model is realized, for example, using a neural network. For example, a Transformer or the like can be used as this model. The estimation unit 321 has a storage means for storing internal parameters of the neural network. The values of the internal parameters are updatable.

In other words, the estimation unit 321 is equipped with a model capable of machine learning, receives a source language sentence as input, and outputs at least sentence count information indicating the number of target language sentences corresponding to the source language sentence.

When the sentence number estimation device 320 operates in the learning mode, the estimation unit 321 uses a large amount of learning data passed from the learning data supply unit 328 to learn the internal model. That is, the estimation unit 321 adjusts the values of the internal parameters of the model based on the learning data. The learning data is a pair of a source language sentence and a target language sentence (translation result). However, information on the number of output sentences is added to the data of the target language sentence in this pair. The information on the number of output sentences is, for example, tag information added to the beginning of the target language sentence. That is, the estimation unit 321 is trained by machine learning to output information on the appropriate number of output sentences corresponding to the input source language sentence. In other words, the estimation unit 321 adjusts the values of the internal parameters in this way by machine learning. The learning data used by the estimation unit 321 for learning will be described later with reference to FIG. 3.

In other words, in the learning mode, the estimation unit 321 performs machine learning of the model based on the learning data supplied by the learning data supply unit 328.

When the sentence number estimation device 320 operates in the estimation execution mode, the estimation unit 321 performs a process of estimating an appropriate number of sentences of a target language sentence corresponding to a source language sentence, based on the internal parameters of the trained model. Specifically, the estimation unit 321 outputs information on the number of sentences of the corresponding target language sentence (translation result) based on the data of the source language sentence passed from the source language sentence supply unit 322. The estimation unit 321 passes the output data to the sentence number information extraction unit 323. Note that the estimation unit 321 may pass information on the output number of sentences corresponding to the input source language sentence and information on the target language sentence that is the translation result together to the sentence number information extraction unit 323. The structure of the data that the estimation unit 321 receives from the source language sentence supply unit 322 will be described later with reference to FIG. 4. The structure of the data that the estimation unit 321 passes to the sentence number information extraction unit 323 will be described later with reference to FIG. 5.

In other words, in the estimation execution mode, the estimation unit 321 outputs sentence count information corresponding to a source language sentence based on the source language sentence supplied by the source language sentence supply unit 322.

When the sentence number estimation device 320 operates in the estimation execution mode, the source language sentence supply unit 322 supplies data of the source language sentence that is the subject of the translation process to the estimation unit 321. This source language sentence data is data of one sentence expressed as a string of words. The structure of the data supplied by the source language sentence supply unit 322 will be described later with reference to FIG. 4.

When the number of sentences estimation device 320 operates in the estimation execution mode, the number of sentences information extraction unit 323 extracts only the number of sentences information of the target language sentence (translation result) from the data output by the estimation unit 321. The number of sentences information extraction unit 323 outputs the obtained number of sentences information (numerical value) to the outside. The data on the number of sentences extracted by the number of sentences information extraction unit 323 can be used, for example, to control the number of output sentences during the translation process (see the second embodiment).

When the sentence number estimation device 320 operates in the learning mode, the sentence pair supplying unit 326 supplies pairs of source language sentences and target language sentences. Here, the target language sentences correspond to the translation results of the source language sentences. In other words, these sentence pairs are bilingual sentence pairs. The sentence pair supplying unit 326 can supply a large number of sentence pairs for learning by the estimation unit 321. An example of the configuration of the data supplied by the sentence pair supplying unit 326 will be described later with reference to FIG. 2.

The sentence number information adding unit 327 adds information on the number of sentences in the target language sentence to the data supplied by the sentence pair supplying unit 326. The sentence number information adding unit 327 counts the number of sentences included in the object sentence data in the sentence pair passed from the sentence pair supplying unit 326. The sentence number information adding unit 327 adds numerical information resulting from the count as a tag to the sentence pair data. Specifically, the sentence number information adding unit 327 adds a tag indicating the number of sentences to, for example, the beginning of the object sentence. The sentence number information adding unit 327 passes the sentence pair data after the sentence number information has been added to the learning data supplying unit 328 so that it can be used for learning by the estimation unit 321.

The sentence count information adding unit 327 counts the number of sentences, for example, by counting sentence separators in the target language sentence data. In the case of English sentences, sentence separators are characters such as periods, question marks, and exclamation marks. The sentence count information adding unit 327 may determine the number of sentences included in the target language sentence data by other methods. As one example, the sentence count information adding unit 327 may identify each sentence included in the target language sentence data and determine the number of sentences by performing a syntactic analysis process on the target language sentence.

The sentence count information adding unit 327 may add a tag representing the number of sentences to the data based on a manually counted number of sentences.

An example of the structure of the data that the sentence number information assignment unit 327 passes to the learning data supply unit 328 will be described later with reference to Figure 3.

The learning data supply unit 328 supplies the data received from the sentence number information attachment unit 327 to the estimation unit 321 as learning data. The learning data supply unit 328 passes both the source language sentence data and the target language sentence data to which the sentence number information has been added to the estimation unit 321. The source language sentence data is used as input data to a model (such as a neural network) held by the estimation unit 321. The target language sentence data to which the sentence number information has been added is passed to the estimation unit 321 to calculate an error (loss) for executing, for example, an error backpropagation method. Note that the method of performing machine learning of a model using an error backpropagation method or the like is itself an existing technology.

In other words, the learning data supply unit 328 supplies learning data including learning source language sentences, learning target language sentences corresponding to the learning source language sentences, and learning sentence number information indicating the number of sentences in the learning target language sentences.

The processing performed by the sentence count estimation device 320, which has the components described above, is described further below.

First, source language sentence data (Japanese text) and target language sentence data (English text) are prepared. Here, the target language sentence data (English text) is a manual translation of each sentence in the source language sentence data (Japanese text). All source language sentence data (Japanese text) and target language sentence data (English text) are associated with each other. Note that one sentence in the source language sentence data (Japanese text) may correspond to multiple sentences in the target language sentence data (English text).

The above data is provided by the text pair supply unit 326.

The sentence count information adding unit 327 estimates how many sentences the target language sentence data (English text) contains. In other words, the sentence count information adding unit 327 estimates (counts) the number of sentences in the English text that correspond to one sentence in the source language sentence data (Japanese text). The sentence count information adding unit 327 counts, for example, the number of periods (sentence separators) contained in the target language sentence data (English text). However, periods are also used in English to express abbreviations such as "Mr." In order to avoid erroneous estimation of the number of sentences due to such examples, the sentence count information adding unit 327 may divide the English sentence using a method such as byte-pair-encoding and count the number of sentences divided by a single period.

The sentence count information adding unit 327 adds a tag indicating the number of sentences, which is the estimation result, to the beginning of the target language sentence data (English text) of each sentence pair. The sentence count information adding unit 327 outputs the data with this tag as learning data for the estimation unit 321.

The estimation unit 321 operates in the learning mode and performs machine learning of the model using the above learning data. Information on the trained model is stored in the internal storage means of the estimation unit 321.

In this manner, the sentence number estimation device 320 becomes able to estimate the appropriate number of target language sentences that correspond to the input source language sentence.

Next, we will explain the data processed by the sentence count estimation device 320 with reference to Figures 2, 3, 4, and 5.

FIG. 2 is a schematic diagram showing an example of the configuration of data supplied by the sentence pair supply unit 326 when the sentence number estimation device 320 operates in the learning mode. As shown in the figure, the data supplied by the sentence pair supply unit 326 is data of a pair of a source language sentence (Japanese sentence) and a target language sentence (English sentence). This pair of a source language sentence and a target language sentence is a bilingual sentence pair. In this example, the source language sentence (Japanese sentence) is "Hiroshima prefecture has developed new oysters for eating raw so that tourists can enjoy oysters, a specialty of Hiroshima, which boasts the highest production volume in the country, even in the summer. Officials held a tasting event on July 16, inviting people concerned." The target language sentence (English sentence) is "Hiroshima is the number one producer of oysters in the country. The prefecture has newly developed oysters for eating raw so that tourists can enjoy oysters, a specialty of Hiroshima, also in summer. Officials held a tasting event on July 16, inviting people concerned."

When the data in FIG. 2 is received from the sentence pair supply unit 326, the sentence number information adding unit 327 adds information on the number of object sentences in the sentence pair data.

Figure 3 is a schematic diagram showing an example of the configuration of data that the sentence number information adding unit 327 passes to the learning data supply unit 328 when the sentence number estimation device 320 operates in learning mode. As shown in the figure, the data output by the sentence number information adding unit 327 has information on the number of sentences in the target language sentence in the sentence pair added to it. Specifically, this data has tag information <3> at the beginning of the target language sentence. This "3" represents the number of sentences included in the target language sentence "Hiroshima is the number one... (omitted)".

The data shown in FIG. 3 is supplied to the estimation unit 321 by the learning data supply unit 328 when the model held by the estimation unit 321 is trained. In other words, the model held by the estimation unit 321 performs machine learning using a bilingual sentence pair with information on the number of sentences added to the beginning of the target language sentence. In other words, information on the number of sentences in the target language sentence, together with information on the target language sentence itself, affects the adjustment of the internal parameters of the model. In other words, when the model held by the estimation unit 321 is constructed using a neural network, the number of sentences in the target language sentence ("3" in the case of the data in FIG. 3) affects the update of the values of the internal parameters of the neural network when the backpropagation method is used.

Figure 4 is a schematic diagram showing an example of the configuration of data that the source language sentence supply unit 322 passes to the estimation unit 321 when the sentence number estimation device 320 operates in the estimation execution mode. As shown in the figure, this data is data on the source language sentence to be translated. The data shown in Figure 4 itself does not have information regarding the number of sentences in the target language sentence, which is the translation result. In this example, the source language sentence is the same as the source language sentence in Figure 2, "Oysters, a specialty of Hiroshima, boast the highest production volume in the country... (omitted)." However, the sentence that the source language sentence supply unit 322 passes to the estimation unit 321 may be any sentence written in the source language (here, Japanese).

Figure 5 is a schematic diagram showing an example of data output by the estimation unit 321 when the sentence number estimation device 320 operates in the estimation execution mode. The data in Figure 5 is an example of data output by the estimation unit 321 when the data in Figure 4 is input. As shown, the data in Figure 5 includes a sentence in the target language (English in this case) that is the translation result, and has a tag of numerical information added to the beginning of the sentence. In this example, the tag is <3>. This tag represents a number that indicates the estimation result of the number of sentences in the target language sentence. The information on the number of sentences is generated by a trained model based on a large amount of training data.

When the estimation unit 321 outputs the data in FIG. 5, the sentence number information extraction unit 323 extracts sentence number information from this data. Specifically, the sentence number information extraction unit 323 extracts the sentence number "3" based on the tag <3> at the beginning of the data in FIG. 5. The sentence number information extraction unit 323 externally extracts the extracted sentence number information (for example, 3).

As described above, the sentence number estimation device 320 of this embodiment makes it possible to estimate the appropriate number of sentences in a target language sentence (the translation result of the source language sentence) according to the input source language sentence. Furthermore, by using information on the estimated number of sentences, the translation device 1 of the second embodiment described next becomes able to output a translation result of that number of sentences (or a number of sentences close to that number).

As a variation of the first embodiment, the number of sentences estimation device 320 may use a trained model to estimate the number of output sentences of a target language sentence corresponding to a source language sentence. In this case, the number of sentences estimation device 320 does not need a function for training the model. As another variation, the number of sentences estimation device 320 may only train the model. The model trained in this way can be ported to other devices. In this case, it is sufficient for the number of sentences estimation device 320 to have only the function for training the model, and does not need a function for actually estimating the number of output sentences.

It is possible to realize the number-of-sentences estimation device 320 using a computer and a program. A specific method for realizing the number-of-sentences estimation device 320 using a computer and a program will be described in the following second embodiment. The number-of-sentences estimation device 320 of this embodiment is a device equivalent to the number-of-sentences estimation unit 32 in the second embodiment.

[Second embodiment]
Next, a second embodiment of the present invention will be described. Note that the description of the matters already described in the previous embodiment may be omitted. Here, the description will be centered on matters unique to this embodiment.

This embodiment relates to a translation device that performs translation processing of a sentence written in a natural language. The sentence on the input side of the translation processing is written in a source language. The sentence on the output side of the translation processing is written in a target language. The source language and the target language are, for example, English, German, Japanese, French, Russian, Spanish, Italian, Chinese, Korean, or other languages. Basically, the type of source language and the type of target language in the translation processing are different. However, the type of source language and the type of target language may be the same.

FIG. 6 is a block diagram showing the schematic functional configuration of the translation device according to this embodiment. As shown in the figure, the translation device 1 includes a source language sentence input unit 21, a sentence number designation unit 22, a translation processing unit 23, a translation result output unit 24, a sentence pair supply unit 31, a sentence number estimation unit 32, a sentence number information assignment unit 33, and a learning data supply unit 34. Each of these functional units can be realized, for example, by a computer and a program. Each functional unit also has a storage unit as necessary. The storage unit is, for example, a variable in the program or a memory allocated by the execution of the program. Also, non-volatile storage units such as a magnetic hard disk device or a solid state drive (SSD) may be used as necessary. Also, at least a part of the functions of each functional unit may be realized as a dedicated electronic circuit or the like rather than a program.

The translation device 1 can operate in either a learning mode or a translation execution mode. When operating in the learning mode, the translation device 1 performs machine learning of an internal model. When operating in the translation execution mode, the translation device 1 performs natural language translation processing based on the model in the current state.

Note that translation device 1 has a function for learning a model as described above. Therefore, translation device 1 may be called a learning device. Translation device 1 also has a function for processing natural language data. Therefore, translation device 1 may be called a natural language processing device.

The serial connection of the source language sentence input unit 21, the sentence number designation unit 22, the translation processing unit 23, and the translation result output unit 24 corresponds to the processing flow when the translation device 1 operates in the translation execution mode. Below, the functions of each of the source language sentence input unit 21, the sentence number designation unit 22, the translation processing unit 23, and the translation result output unit 24 will be explained in relation to their operation in the translation execution mode.

The source language sentence input unit 21 acquires the source language sentence to be translated and passes it to the sentence number designation unit 22. The source language sentence input unit 21 acquires the source language sentence, for example, from an external source, or reads it from a storage unit accessible to the translation device 1. The source language sentence exists, for example, as text data within the translation device 1. The source language sentence input here is called the input source language sentence. Also, the target language sentence for output (translation result) corresponding to this input source language sentence is called the output target language sentence.

The number of sentences designation unit 22 designates the number of sentences in the target language sentence resulting from translating the source language sentence received from the source language sentence input unit 21. The number of sentences designation unit 22 designates the number of sentences in the target language sentence based on, for example, a numerical value acquired from an external source or a numerical value read from a predetermined storage unit in the translation device 1. The number of sentences designation unit 22 adds information on the number of sentences in the target language sentence to the source language sentence. The number of sentences designation unit 22 passes the source language sentence with the information on the number of sentences in the target language sentence added to the translation processing unit 23. An example of a specific method in which the number of sentences designation unit 22 adds the information on the number of sentences in the target language sentence to the source language sentence will be described later. In other words, the number of sentences designation unit 22 adds information on the number of sentences in the output target language sentence for output corresponding to the input source language sentence to the input source language sentence, and passes the result to the translation processing unit 23.

As exemplified later, the sentence number designation unit 22 may add information about the number of sentences in the output target language sentence to the beginning of the input source language sentence, which is text data. In this case, information about the number of sentences in the target language sentence is also added to the beginning of the source language sentence in the learning data. By adding information about the number of sentences to the beginning of the source language sentence, the information about the number of sentences is always added to a fixed position even if the text of the source language sentence is of variable length. This is advantageous when performing machine learning based on information about the number of sentences.

The translation processing unit 23 performs translation processing based on the input source language sentence, and outputs a target language sentence. The translation processing unit 23 has an internal translation model. This model can be trained using training data. The model is realized, for example, using a neural network. More specifically, the model may be realized using an RNN (Recurrent Neural Network) or a Transformer. However, the specific configuration for realizing the model is not limited to the example shown here. The technologies themselves, such as the neural network, RNN, and Transformer, listed here, are existing technologies that can be used. If the internal model of the translation processing unit 23 has been properly trained, the translation processing unit 23 is expected to generate and output a target language sentence (which may be multiple sentences) that is a translation of the input source language sentence. Furthermore, if the model inside the translation processing unit 23 has been properly trained, the translation processing unit 23 is expected to generate and output the number of target language sentences specified by the sentence number designation unit 22 (or at least a number close to the specified number). In other words, the translation processing unit 23 is provided with a model, trains the model using training data, and outputs the output target language sentence determined by the model by inputting an input source language sentence to which information on the number of sentences in the output target language sentence has been added into the model. Furthermore, the translation processing unit 23 trains the model as described above, and may be called a "training processing unit."

The operation of the translation processing unit 23 in the learning mode and in the translation execution mode is as follows:

In the learning mode, the translation processing unit 23 performs a learning process for the model based on the learning data provided by the learning data supply unit 34. The learning data is a set of pairs of input data and output data to the model. In this embodiment, the input data to the model is text data of the source language sentence to which the number of sentences (estimated value) of the target language sentence, which is the output data, is added. The output data included in the learning data is the correct text data of the target language sentence. The model realized by the neural network performs calculations at each node of the network based on this input data to obtain output data. The output data obtained by the model corresponds to the target language sentence. The translation processing unit 23 calculates the difference (loss) between the output data (estimated data) calculated by the model based on the input data and the output data in the learning data. Then, the translation processing unit 23 updates the internal parameters of the model by the backpropagation method based on the calculated difference. Updating the internal parameters of the neural network by the backpropagation method can be performed by existing technology. The translation processing unit 23 writes the updated internal parameter values of the model to a storage unit (not shown) as appropriate. The translation processing unit 23 learns the model by repeating this process for each of the many sentence pairs included in the training data.

When the translation processing unit 23 operates in the translation execution mode, it is assumed that at least a predetermined amount of learning processing has been completed. In the translation execution mode, the translation processing unit 23 receives input data passed from the sentence number designation unit 22. This input data includes text data of the source language sentence and data specifying the number of sentences (for example, a numerical value expressed as a tag). The translation processing unit 23 performs calculations within the model based on the input data, and obtains output data from the model. Note that the translation processing unit 23 may read parameters within the model (values of a learned parameter set) from a storage unit (not shown) or the like as necessary. The output data obtained by the translation processing unit 23 is expected to be the text of the target language sentence as the translation result. The translation processing unit 23 passes the output data (target language sentence) to the translation result output unit 24.

The translation result output unit 24 outputs the target language sentence output from the translation processing unit 23 to the outside.
The translation result output unit 24 outputs the target language sentence as the translation result, for example, by transmitting it to an external device, displaying it on a display device, recording it on a recording medium (such as a hard disk or semiconductor memory), etc. The translation result output unit 24 may also output the target language sentence in other forms.

The functional group consisting of the sentence pair supply unit 31, the sentence number estimation unit 32, the sentence number information assignment unit 33, and the learning data supply unit 34 is a functional group for generating learning data to be used in the learning mode of the translation device 1 and supplying it to the model in the translation processing unit 23. Below, the functions of each of the sentence pair supply unit 31, the sentence number estimation unit 32, the sentence number information assignment unit 33, and the learning data supply unit 34 will be explained in relation to the operation of generating learning data.

The sentence pair supplying unit 31 supplies pairs (sentence pairs) of source language sentences and target language sentences. The source language sentences here are referred to as learning language sentences. The target language sentences here are referred to as learning target language sentences. In other words, the sentence pair supplying unit 31 supplies data of pairs of learning source language sentences and learning target language sentences. The sentence pair supplying unit 31 processes sentence pairs acquired from the outside, or sentence pairs read from a storage unit (not shown) in the translation device 1, one pair at a time, and passes them to the sentence number estimation unit 32, sentence number information assignment unit 33, and learning data supplying unit 34. Specifically, the sentence pair supplying unit 31 passes the source language sentences in the sentence pair to the sentence number information assignment unit 33. The sentence pair supplying unit 31 also passes the target language sentences in the sentence pair to the sentence number estimation unit 32 and learning data supplying unit 34.

The sentence pairs supplied by the sentence pair supply unit 31 are pairs of source language sentences and target language sentences that have the same meaning. In other words, within a sentence pair, the result of translating a source language sentence is the target language sentence. Basically, in most cases, the result of translating a target language sentence within a sentence pair can be the source language sentence. The source language sentence within a sentence pair may be one or more sentences. The target language sentence within a sentence pair may be one or more sentences. Such sentence pairs can be obtained based on an existing language corpus. Alternatively, a target language sentence can be obtained by manually translating a source language sentence, and the two can be used as a sentence pair.

When constructing a sentence pair, the pair of source language and target language is assumed to match the model to be learned. The type of source language and the type of target language are arbitrary as long as they match the model for translation (the model inside the translation processing unit 23). As an example, to have the translation device 1 perform Japanese-English translation processing, the source language is Japanese and the target language is English.

The sentence number estimation unit 32 estimates and outputs the number of sentences included in the target language sentences supplied from the sentence pair supply unit 31. That is, the sentence number estimation unit 32 obtains the number of sentences included in the learning target language sentence in a certain sentence pair. The sentence number estimation unit 32 estimates the number of sentences using a method according to the type of target language. For example, for Western languages (English, French, German, etc.), the sentence number estimation unit 32 estimates the number of sentences by counting the number of periods and question marks based on the characteristic that a sentence ends with a period or a question mark. However, periods and question marks may appear other than at the end of a sentence. For example, an expression such as "Mr." (an abbreviation of "mister") in English includes a period, but the period in this example does not indicate that it is the end of a sentence. For this reason, the sentence number estimation unit 32 may use other methods to improve the estimation accuracy.

For example, the sentence count estimation unit 32 divides a target language sentence (e.g., an English sentence) using a byte pair encoding (BPE) technique. The byte pair encoding technique itself is an existing technology. The sentence count estimation unit 32 estimates the number of sentences, for example, by counting the number of periods when periods are divided individually by byte pair encoding.

For example, the sentence count estimation unit 32 may estimate the number of sentences by performing a syntactic analysis process on the target language sentence. For example, when the target language is English, one target language sentence has a basic structure such as SV, SVO, SVC, SVOC, etc. Here, S is the subject, V is the predicate, O is the object, and C is the complement. The sentence count estimation unit 32 counts up the number of sentences based on such a syntactic analysis result, and the result is the estimated number of sentences. In each language other than English, the estimation of the number of sentences by the syntactic analysis process is basically effective. Note that, in order to further improve the estimation accuracy, the sentence count estimation unit 32 may estimate the number of sentences based on both the result of the syntactic analysis and the number of periods and question marks.

For example, the number of sentences estimation unit 32 may take a numerical value provided from outside in correspondence with the target language sentence as the estimated result of the number of sentences. For example, the number of sentences for each target language sentence may be input by a person through judgment, and the number of sentences estimation unit 32 may take that numerical value as the estimated result of the number of sentences. Alternatively, the number of sentences estimation unit 32 may read out a numerical value stored in a memory unit (not shown) or the like in correspondence with the target language sentence, and use that numerical value as the estimated result of the number of sentences.

For example, the aforementioned periods and question marks in English are symbols that indicate the end of a sentence. Symbols that indicate the end of a sentence like this are called sentence-ending symbols. For example, in Japanese, full stops and periods are sentence-ending symbols. Also, in other languages, the sentence number estimation unit 32 may determine the number of sentences based on sentence-ending symbols. In other words, the sentence number estimation unit 32 may determine the number of sentences included in the target language sentence for training by counting the number of sentence-ending symbols included in the target language sentence for training that is supplied.

The number of sentences information adding unit 33 receives the source language sentence from the sentence pair supplying unit 31, and receives the number of sentences of the target language sentence (the above-mentioned estimated value) from the sentence number estimating unit 32. The sentence number information adding unit 33 adds the number of sentences information received from the sentence number estimating unit 32 to the data of the source language sentence received from the sentence pair supplying unit 31, and outputs the data. In other words, the sentence number information adding unit 33 adds the number of sentences information calculated by the sentence number estimating unit 32 to the learning source language sentence, and outputs the data of the learning source language sentence to which the information of the number of sentences of the learning target language sentence has been added. The sentence number information adding unit 33 passes the data of the source language sentence to which the information of the number of sentences of the target language sentence (estimated value) has been added to the learning data supplying unit 34. One example of a method in which the sentence number information adding unit 33 adds the information on the number of sentences is to add a tag indicating the number of sentences of the target language sentence to the text of the source language sentence. The sentence number information adding unit 33 may add information about the number of sentences in the target language sentence for training to the beginning of the source language sentence for training, which is text data. The position where the sentence number information adding unit 33 adds the sentence number information is aligned with that of the sentence number designation unit 22. The advantages of adding sentence number information to the beginning of a source language sentence have already been explained.

An example of source language sentence data with information on the number of target language sentences (estimated value) will be explained later with reference to another figure.

The learning data supply unit 34 passes the target language sentence data passed from the sentence pair supply unit 31 and the source language sentence data passed from the sentence number information attachment unit 33 (with an estimated value for the number of sentences in the target language sentence) as a pair to the translation processing unit 23. This pair can be used as learning data for the model held by the translation processing unit 23. In other words, the learning data supply unit 34 supplies the pair of the training source language sentence data with information on the number of sentences of the training target language sentence added, and the training target language sentence data, as learning data.

The sentence pair supplying unit 31 may supply a large number of sentence pairs. Corresponding to each sentence pair supplied by the sentence pair supplying unit 31, the learning data supplying unit 34 passes a pair of a source language sentence (with an estimated value for the number of sentences) and a target language sentence to the translation processing unit 23. Each of these pairs can be used to perform machine learning processing of a model inside the translation processing unit 23. The number of sentence pairs supplied by the learning data supplying unit 34 as learning data is arbitrary. The learning data supplying unit 34 may supply a large number of sentence pairs. The number of sentence pairs supplied as learning data may reach hundreds of thousands or millions of pairs, for example.

Next, we will explain how to configure the learning data.

Fig. 7 is a schematic diagram showing an example of data of a sentence pair supplied by the sentence pair supply unit 31. In the illustrated sentence pair, the source language sentence is a Japanese sentence, and the target language sentence is an English sentence. The source language sentence is text data of "Hiroshima prefecture has developed new oysters for eating raw so that tourists can enjoy oysters, a specialty of Hiroshima, which boasts the highest production volume in the country, even in the summer, and held a tasting event on the 16th, gathering related parties." The target language sentence corresponding to this source language sentence is text data including three sentences: "Hiroshima is the number one producer of oysters in the country.", "The prefecture has newly developed oysters for eating raw so that tourists can enjoy oysters, a specialty of Hiroshima, also in summer.", and "Officials held a tasting event on July 16, inviting people concerned."

8 is a schematic diagram showing an example of learning data (sentence pair) that the learning data supply unit 34 passes to the translation processing unit 23. The data shown in FIG. 8 is created based on the data shown in FIG. 7. The data shown in FIG. 8 differs from the data shown in FIG. 7 in that a tag <3> is added to the beginning of the text on the source language side. This tag corresponds to the number of sentences on the target language side being 3. In other words, this tag is a tag that represents the number of sentences on the target language side. As described above, the number of sentences on the target language side being 3 is based on the estimation result by the sentence number estimation unit 32. In addition, it is the sentence number information attachment unit 33 that added the tag <3> to the source language sentence data in accordance with the estimation result. In other words, in the data shown in FIG. 8, the source language sentence data is the text data "<3> Hiroshima Prefecture has developed a new oyster for eating raw, so that tourists can enjoy oysters, a specialty of Hiroshima, which boasts the highest production volume in the country, in the summer, and held a tasting event for related parties on the 16th." The target language sentence data shown in Figure 8 is the same text data as that in Figure 7.

By performing machine learning using learning data such as that shown in Figure 8, the translation model in the translation processing unit 23 not only learns the correspondence between the source language sentence itself and the target language sentence, but also learns the correspondence between the number "3" (the estimated number of sentences) and the target language sentence.

Figure 9 is a flowchart showing the steps of a process for generating learning data for machine learning by the translation processing unit 23 of the translation device 1. This flowchart shows the steps for generating a pair of input and output data. The process of this flowchart can be repeatedly executed for each pair of input and output data. Below, the operation steps are explained according to this flowchart.

First, in step S11, the sentence pair supplying unit 31 acquires a language pair consisting of a source language sentence and a target language sentence, and supplies it to the translation device 1. As described above, the sentence pair supplying unit 31 supplies the source language sentence included in the sentence pair to the sentence number information adding unit 33. In addition, the sentence pair supplying unit 31 supplies the target language sentence included in the sentence pair to the sentence number estimation unit 32 and the learning data supplying unit 34.

Next, in step S12, the sentence count estimation unit 32 estimates the number of sentences included in the target language sentence passed from the sentence pair supply unit 31 in step S11. An example of the estimation method specifically used by the sentence count estimation unit 32 has already been described. The sentence count estimation unit 32 notifies the sentence count information assignment unit 33 of the estimated number of sentences.

Next, in step S13, the sentence number information adding unit 33 adds information on the number of sentences of the target language sentence estimated in step S12 to the source language sentence passed from the sentence pair supplying unit 31 in step S11. As a specific example, the sentence number information adding unit 33 adds a tag indicating the number of sentences of the target language sentence to the data of the source language sentence. An example of the data of the source language sentence to which the estimated value of the number of sentences of the witness language sentence has been added has already been described (see FIG. 8).

Next, in step S14, the sentence number information adding unit 33 passes the data of the source language sentence to which the estimated value of the number of sentences of the witnessed language sentence has been added to the learning data supplying unit 34. The learning data supplying unit 34 sets the data of the source language sentence to which the estimated value of the number of sentences of the witnessed language sentence has been added as input data for the learning data.

Next, in step S15, the learning data supply unit 34 sets the target language sentence data passed from the sentence pair supply unit 31 in step S11 as the output data of the learning data.

Next, in step S16, the learning data supply unit 34 outputs a pair of the above-mentioned input data (data of the source language sentence to which an estimated value of the number of sentences in the target language sentence has been added) and output data (data of the target language sentence). This pair of data is used for the learning process in the translation processing unit 23.

Figure 10 is a block diagram showing an example of the internal configuration of the translation device 1. The translation device 1 can be realized using a computer. As shown in the figure, the computer is configured to include a central processing unit 901, a RAM 902, an input/output port 903, input/output devices 904 and 905, etc., and a bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions included in a program read from the RAM 902, etc. According to each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. Note that RAM is an abbreviation for "random access memory." The input/output port 903 is a port through which the central processing unit 901 exchanges data with external input/output devices, etc. The input/output devices 904 and 905 are input/output devices. Input/output devices 904 and 905 exchange data with the central processing unit 901 via the input/output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input/output port via the bus 906.

At least some of the functions of the translation device 1 (learning device) can be realized by a computer. In this case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to realize the function. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB memories, and storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include devices that temporarily and dynamically hold a program, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and devices that hold a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case. Furthermore, the above program may be for realizing some of the functions described above, and may further be capable of realizing the functions described above in combination with a program already recorded in the computer system.

Although the embodiment has been described above, the following modified examples may be implemented. Note that multiple modified examples may be implemented in combination to the extent that they can be combined.

[First Modification]
In the second embodiment, the translation device 1 can operate in both the learning mode and the translation execution mode. In the first modified example, the translation device 1 does not operate in the learning mode, but operates only in the translation execution mode. In this case, the translation model in the translation processing unit 23 has completed machine learning processing based on the learning data. Alternatively, the translation model may be configured to import the learned internal parameter values of the translation model from the outside. The translation device 1 of the first modified example includes at least the source language sentence input unit 21, the sentence number designation unit 22, the translation processing unit 23, and the translation result output unit 24 of the functional configuration shown in FIG. 6. In addition, the translation device 1 may be configured not to include some or all of the sentence pair supply unit 31, the sentence number estimation unit 32, the sentence number information assignment unit 33, and the learning data supply unit 34. It is expected that the translation device 1 of the first modified example performs processing to output a target language sentence consisting of a specified number of sentences using a learned model.

The translation device 1 of the first modified example as described above comprises a sentence number designation unit 22 that adds information on the number of sentences of an output target language sentence for output corresponding to an input source language sentence to the input source language sentence, and a translation processing unit 23 that has a model and outputs an output target language sentence determined by the model by inputting the input source language sentence to which the information on the number of sentences of the output target language sentence has been added, and the model has been trained using training data that is a pair of data of a training source language sentence to which information on the number of sentences of the training target language sentence has been added, and data of a training target language sentence.

[Second Modification]
In the second embodiment, the translation device 1 can operate in both the learning mode and the translation execution mode. In the second modified example, the translation device 1 does not operate in the translation execution mode, but operates only in the learning mode. That is, the translation device 1 of the second modified example performs only the learning process, and may be called a "learning device". In this case, the translation model in the translation processing unit 23 updates the internal parameters by the learning process. The learned model can be exported to, for example, an external device. In this case, if the translation device 1 and the external device have the same network structure in advance, only a set of values of the internal parameters of the learned model may be exported. This allows the external device to perform a translation process based on the results of learning by the translation device 1 (learning device) of the second modified example. It is expected that the external device will perform a translation process to output a target language sentence consisting of a specified number of sentences. The translation device 1 (learning device) of the second modified example includes at least all of the sentence pair supply unit 31, the sentence number estimation unit 32, the sentence number information assignment unit 33, the learning data supply unit 34, and the translation processing unit 23 among the functional configurations shown in FIG. 6. Furthermore, the translation device 1 (learning device) may be configured without some or all of the source language sentence input unit 21, the sentence number designation unit 22, and the translation result output unit 24.

The translation device 1 of the second modified example as described above includes a sentence pair supplying unit 31 that supplies data pairs of training source language sentences and training target language sentences, a sentence number estimating unit 32 that determines the number of sentences included in the supplied training target language sentences, a sentence number information adding unit 33 that adds the information on the number of sentences determined by the sentence number estimating unit to the training source language sentences and outputs data of the training source language sentences to which the information on the number of sentences of the training target language sentences has been added, a training data supplying unit 34 that supplies pairs of the data of the training source language sentences to which the information on the number of sentences of the training target language sentences has been added and the data of the training target language sentences as training data, and a translation processing unit 23 (learning processing unit) that uses the training data to train a model.

[Third Modification]
In the above-described second embodiment, the learning data is generated using information on the number of sentences estimated by the number-of-sentences estimation unit 32. In the third modified example, definite information on the number of sentences may be provided to the number-of-sentences estimation unit 32 instead of an estimated value. In that case, the number-of-sentences estimation unit 32 uses the provided number of sentences as an estimated value. The learning data supply unit 34 of the third modified example provides the learning data created in this way to the translation processing unit 23. The translation processing unit 23 performs a model learning process using the learning data created in this way.

[Fourth Modification]
In the second embodiment, the sentence count information adding unit 33 added the estimated sentence count information to the beginning of the source language sentence as data in the form of a tag surrounded by square brackets <>. In the fourth embodiment, the form of the added sentence count information is not limited to this. The sentence count information adding unit 33 in the fourth modified example adds sentence count information to the source language sentence in a form other than the above tag. In addition, the position where the sentence count information adding unit 33 in the fourth modified example adds the sentence count information is not limited to the beginning of the source language sentence, and may be another position. In either case, the sentence count information adding unit 33 passes data including the text information of the source language sentence and information corresponding to the sentence count value to the learning data supply unit 34 as input data for the learning model. The translation processing unit 23 performs a learning process for the model using the learning data created in this way.

The above describes in detail an embodiment of the present invention (including modified examples) with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs that do not deviate from the gist of the present invention.

[Example and evaluation of a translation device that controls the number of sentences (second embodiment)]
In the following, an example of implementing the translation device 1 of the above embodiment and the evaluation results thereof will be described.

Figure 11 is a schematic diagram showing an example of the results of a translation process performed using the translation device 1. In this example, the source language is Japanese and the target language is English. The figure shows an input sentence in Japanese and five types of output sentences in English. The number of sentences in the input sentence is 1. Target language sentence A is the subject of comparison and is the result of manual translation. The number of sentences in the translation result of A is 3. Output sentence B is the translation result when 1 is specified as the number of sentences in the target language sentence in the translation device 1. The number of sentences in the translation result of B is 1, as specified. Output sentence C is the translation result when 2 is specified as the number of sentences in the target language sentence. The number of sentences in the translation result of C is 2, as specified. Output sentence D is the translation result when 3 is specified as the number of sentences in the target language sentence. The number of sentences in the translation result of D is 3, as specified. Output sentence E is the result of a machine translation process performed without specifying the number of sentences in the target language sentence, as the subject of comparison. The number of sentences in the translation result for E was 2. On the other hand, when the number of output sentences was specified as 1, 2, or 3 in the translation device 1 of this embodiment, the result with the specified number of sentences was obtained.

In addition, this experiment showed that if the number of output sentences is not specified, the number of output sentences will be less than desired (the length of each sentence will tend to be long). In other words, being able to specify the number of output sentences in the translation device 1 of this embodiment is useful.

The effectiveness of the translation device 1 of this embodiment was quantitatively evaluated by the following method. Specifically, the translation results were compared between the case where information specifying the number of output sentences (number of sentences in the target language sentence) was provided (this embodiment) and the case where it was not provided (comparison). In both cases, approximately 100,000 sentence pairs of manual English translations (target language sentences) corresponding to each sentence in Japanese (source language sentence) were used as learning data. Furthermore, the development data and evaluation data were similar datasets, with 3,000 sentence pairs and 5,001 sentence pairs, respectively.

(1) When output sentence number information was added, a sentence number tag was added to the source language sentence data, and a translation model (network) trained by the translation processing unit 23 of this embodiment was used. During translation in this case, the sentence number estimation unit 32 of this embodiment estimated the number of sentences based on the English text data corresponding to the evaluation Japanese text data. Then, a tag indicating the estimated number of sentences was added to the beginning of the sentence of the evaluation Japanese text data, and the translation process was performed.

(2) When output sentence count information was not added, a sentence count tag was not added to the source language sentence data, and the translation model (network) trained by the translation processing unit 23 of this embodiment was used. During translation, the translation process was also performed without adding a sentence count tag to the source language sentence data.

The experimental results for both cases are as follows. The evaluation values used were the rate of agreement between the number of output sentences and the number of sentences in the original evaluation English sentences (target language sentences), and the translation performance based on the BLEU score. The evaluation values are as follows:

With output sentence count information: Sentence count match rate 96.6%
Without output sentence count information: Sentence count match rate 60.4%

With output sentence count information: BLEU 19.14
No output sentence count information: BLEU 18.38

As described above, when the number of output sentences was not specified, the match rate of the number of sentences was 60.4%, whereas when the number of output sentences was specified, the match rate of the number of sentences was 96.6%. In other words, it was confirmed that the translation device 1 of this embodiment can output the specified number of target language sentences as the translation result with a very high probability.

As described above, the BLEU value was 18.38 when the number of output sentences was not specified, whereas the BLEU value was 19.14 when the number of output sentences was specified and translation was performed. In other words, it was confirmed that even when the number of output sentences was specified and translation processing was performed as in the translation device 1 of this embodiment, the translation accuracy at least did not deteriorate compared to when it was not specified. As described above, the BLEU value when the number of output sentences was specified and translation was performed was 0.76 higher than when the number of output sentences was not specified. Whether this difference is significant or not requires further verification in the future, but one possibility is that specifying the number of output sentences acted to make the patterns of the output sentences closer to the patterns of the target language sentences in the original dataset.

As described above, this embodiment makes it possible to perform translation processing that outputs a specified number of sentences without degrading translation performance. In other words, this embodiment makes it possible to control the number of sentences to be output without degrading translation performance. Because a neural network was used to verify this embodiment, the reason why such an advantage is obtained was not necessarily explicitly stated. However, one possibility is that the model controlled the translation result so that the number of periods matched, or that the model controlled the syntactic structure at a deeper level so that the specified number of sentences was obtained.

[Example and Evaluation of the Sentence Number Estimation Device]
An example was evaluated when the source language was Japanese and the target language was English. Specifically, the following two methods were compared, and the matching rate of the number of output sentences with the evaluation English sentences and the translation performance were compared. As training data for learning the model, bilingual data of about 96,000 sentence pairs of Japanese sentences and their corresponding English translations by hand were used. In addition, from the same bilingual data, development data of 5,001 sentence pairs was used. As evaluation data, multi-reference data of 773 sentences in which one Japanese sentence was translated by two translators each were used.

The first method is a method that uses output sentence number estimation. The number of output sentences corresponding to the Japanese sentences for evaluation was estimated using a sentence number estimation device 320 (first embodiment) trained with the above-mentioned training data. A tag for the estimated number of sentences was added, and translation was performed using a translation device 1 (second embodiment). The translation model of the translation device 1 has also been trained with the same training data.

The second method does not use output sentence number estimation. Translation was performed using the same translation model as the first method, without using output sentence number estimation.

The experimental results are as follows. The number of output sentences in each of the first and second methods was evaluated based on the degree of agreement with the English text for evaluation. The degree of agreement was calculated as a match if the number of English sentences in the translation results of either of the two translators matched.
First method (with estimation of the number of output sentences, this embodiment): 87.0%
Second method (no estimation of the number of output sentences, comparison): 81.8%

As described above, it has been found that by using the sentence number estimation device 320 of the first embodiment, it is possible to estimate an appropriate number of output sentences. It has also been found that by using the translation device 1 of the second embodiment, it is possible to output a translation result with an appropriate number of sentences.

Although several embodiments and variations of the present invention have been described in detail above with reference to the drawings, the specific configurations are not limited to these embodiments and include designs that do not deviate from the gist of the present invention.

The present invention can be used, for example, in industries that use natural language translation processing. As an example, it can be used in the business of automatically translating news and other written content in the news reporting or content provision industry. However, the scope of use of the present invention is not limited to the examples given here.

1. Translation device (learning device, natural language processing device)
21 source language sentence input unit 22 number of sentences designation unit 23 translation processing unit (learning processing unit)
24 Translation result output unit 31 Sentence pair supply unit 32 Sentence number estimation unit 33 Sentence number information assignment unit 34 Learning data supply unit 320 Sentence number estimation device (learning device, natural language processing device)
321 Estimation unit 322 Source language sentence supply unit 323 Sentence number information extraction unit 326 Sentence pair supply unit 327 Sentence number information assignment unit 328 Learning data supply unit 901 Central processing unit 902 RAM
903 Input/Output Port 904, 905 Input/Output Device 906 Bus

Claims (5)

a sentence pair providing unit for providing data pairs of source language sentences for training and target language sentences for training;
a sentence number estimation unit for determining the number of sentences contained in the supplied target language sentence for training;
a sentence number information adding unit that adds the information on the number of sentences determined by the sentence number estimation unit to the training source language sentence, and outputs data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added;
a training data supplying unit that supplies, as training data, pairs of data on the training source language sentences to which information on the number of sentences in the training target language sentences has been added and data on the training target language sentences;
a sentence number specification unit for adding information on the number of sentences of an output target language sentence for output corresponding to an input source language sentence inputted;
a translation processing unit including a model, learning the model using the learning data, and inputting the input source language sentence to which information on the number of sentences in the output target language sentence has been added into the model, thereby outputting an output target language sentence determined by the model;
A natural language processing device;
Equipped with
The natural language processing device comprises:
a training data supplying unit that supplies training data including training source sentences, training target language sentences corresponding to the training source sentences, and training sentence number information indicating the number of sentences in the training target language sentences;
an estimation unit that is equipped with a machine learning model, inputs a source language sentence, and outputs at least sentence number information indicating the number of sentences of a target language sentence corresponding to the source language sentence;
a source language sentence supply unit that supplies the source language sentence to the estimation unit;
Equipped with
When operating in a learning mode, the estimation unit performs machine learning of the model based on the learning data supplied by the learning data supply unit;
When operating in an estimation execution mode, the estimation unit outputs the sentence number information corresponding to the source language sentence based on the source language sentence supplied by the source language sentence supply unit.
It is something that
a source language sentence supply unit of the natural language processing apparatus supplies the input source language sentence to the estimation unit of the natural language processing apparatus;
the sentence number designation unit adds information on the number of sentences of the output target language sentence for output based on the sentence number information output by the estimation unit of the natural language processing device.
Translation device. a sentence pair providing unit for providing data pairs of source language sentences for training and target language sentences for training;
a sentence number estimation unit for determining the number of sentences contained in the supplied target language sentence for training;
a sentence number information adding unit that adds the information on the number of sentences determined by the sentence number estimation unit to the training source language sentence, and outputs data of the training source language sentence to which the information on the number of sentences of the training target language sentence has been added;
a training data supplying unit that supplies, as training data, pairs of data on the training source language sentences to which information on the number of sentences in the training target language sentences has been added and data on the training target language sentences;
a sentence number specification unit for adding information on the number of sentences of an output target language sentence for output corresponding to an input source language sentence inputted;
a translation processing unit including a model, learning the model using the learning data, and inputting the input source language sentence to which information on the number of sentences in the output target language sentence has been added into the model, thereby outputting an output target language sentence determined by the model;
A natural language processing device;
Equipped with
The natural language processing device comprises:
an estimation unit that is equipped with a machine learning model, inputs a source language sentence, and outputs at least sentence number information indicating the number of sentences of a target language sentence corresponding to the source language sentence;
a source language sentence supply unit that supplies the source language sentence to the estimation unit;
Equipped with
the model has been machine-trained based on training data including training source language sentences, training target language sentences corresponding to the training source language sentences, and training sentence number information indicating the number of sentences in the training target language sentences;
the estimation unit outputs the sentence number information corresponding to the source language sentence based on the source language sentence supplied by the source language sentence supply unit.
It is something that
a source language sentence supply unit of the natural language processing apparatus supplies the input source language sentence to the estimation unit of the natural language processing apparatus;
the sentence number designation unit adds information on the number of sentences of the output target language sentence for output based on the sentence number information output by the estimation unit of the natural language processing device.
Translation device. a sentence number specification unit for adding information on the number of sentences of an output target language sentence for output corresponding to an input source language sentence inputted;
a translation processing unit including a model, the input source sentence having information about the number of sentences of the output target language sentence added thereto being input to the model, and outputting an output target language sentence determined by the model;
A natural language processing device;
Equipped with
The natural language processing device comprises:
a training data supplying unit that supplies training data including training source sentences, training target language sentences corresponding to the training source sentences, and training sentence number information indicating the number of sentences in the training target language sentences;
an estimation unit that is equipped with a machine learning model, inputs a source language sentence, and outputs at least sentence number information indicating the number of sentences of a target language sentence corresponding to the source language sentence;
a source language sentence supply unit that supplies the source language sentence to the estimation unit;
Equipped with
When operating in a learning mode, the estimation unit performs machine learning of the model based on the learning data supplied by the learning data supply unit;
When operating in an estimation execution mode, the estimation unit outputs the sentence number information corresponding to the source language sentence based on the source language sentence supplied by the source language sentence supply unit.
It is something that
the model has been trained using training data that is a pair of data of training source language sentences to which information on the number of sentences in the training target language sentences has been added, and data of the training target language sentences;
a source language sentence supply unit of the natural language processing apparatus supplies the input source language sentence to the estimation unit of the natural language processing apparatus;
the sentence number designation unit adds information on the number of sentences of the output target language sentence for output based on the sentence number information output by the estimation unit of the natural language processing device.
Translation device. a sentence number specification unit for adding information on the number of sentences of an output target language sentence for output corresponding to an input source language sentence inputted;
a translation processing unit including a model, the input source sentence having information about the number of sentences of the output target language sentence added thereto being input to the model, and outputting an output target language sentence determined by the model;
A natural language processing device;
Equipped with
The natural language processing device comprises:
an estimation unit that is equipped with a machine learning model, inputs a source language sentence, and outputs at least sentence number information indicating the number of sentences of a target language sentence corresponding to the source language sentence;
a source language sentence supply unit that supplies the source language sentence to the estimation unit;
Equipped with
the model has been machine-trained based on training data including training source language sentences, training target language sentences corresponding to the training source language sentences, and training sentence number information indicating the number of sentences in the training target language sentences;
the estimation unit outputs the sentence number information corresponding to the source language sentence based on the source language sentence supplied by the source language sentence supply unit.
It is something that
the model has been trained using training data that is a pair of data of training source language sentences to which information on the number of sentences in the training target language sentences has been added, and data of the training target language sentences;
a source language sentence supply unit of the natural language processing apparatus supplies the input source language sentence to the estimation unit of the natural language processing apparatus;
the sentence number designation unit adds information on the number of sentences of the output target language sentence for output based on the sentence number information output by the estimation unit of the natural language processing device.
Translation device. A program for causing a computer to function as the translation device according to any one of claims 1 to 4 .

Images