JP7845470B2 - Generation apparatus, generation method, and program - Google Patents

Description

This invention relates to the technical field of machine translation.

When translating text from one domain to another domain (e.g., another language), machine translation with vocabulary constraints imposes constraints aimed at including all specified words or phrases (constraint words). Because machine translation with vocabulary constraints can unify the translation of specific words, it is a particularly important technology in the translation of documents such as patents, legal documents, and technical documents where consistency is required.

Chen, G., Chen, Y., and Li, V. O. (2021). "Lexically Constrained Neural Machine Translation with Explicit Alignment Guidance." Proceedings of the AAAI Conference on Artificial Intelligence. Matt Post and David Vilar. 2018. Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics

In vocabulary-constrained machine translation methods, the translated text is generated to include the given constraint words. However, whether constraint words are extracted automatically or manually, the selection of constraint words may include inappropriate constraint words (noise).

If a lexically constrained machine translation method is applied using all constraint phrases containing noise as lexical constraints, it is expected that incorrect phrases will be included in the translated text, leading to a decrease in translation accuracy. This issue is not limited to machine translation; it can occur in any field that uses constraint information for sequence transformation.

This invention has been made in view of the above points, and aims to provide a technology for performing sequence transformations using constraint information with high accuracy.

According to the disclosed technology, a generation device for generating a second series of information from constraint information and a first series of information, which is another series of information,
An input generation unit that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint,
A sequence generation unit generates one or more candidates for the second sequence using the first sequence and the lexical constraints for each of the multiple lexical constraints output from the input generation unit ,
A generation device is provided, comprising a reranking unit that calculates a score indicating the appropriateness of the second series for each of the one or more candidates obtained for each of the multiple lexical constraints .

According to the disclosed technology, a technique is provided for performing sequence transformations with high accuracy using constraint information.

This figure shows an example of machine translation with vocabulary constraints. This is a diagram showing an example of the configuration of the generating device 100. This is a flowchart illustrating the operation of the generation device 100. This figure shows an example of the configuration of the extraction unit 120. This figure shows an example of the configuration of the extraction unit 120. This is a diagram showing an example of the configuration of the generating device 100. This figure shows an example of the configuration of the sequence generation unit 140. This diagram shows an example of a machine translation model configuration. This figure shows an example of the configuration of the sequence generation unit 140. This figure shows the display image provided by the display unit 500. This is a diagram showing an example of the configuration of the generating device 100. This diagram shows the base detailed settings and hyperparameters for each setting used in the experiment. This is a diagram showing the evaluation results. This figure shows an example of the device's hardware configuration.

The embodiments of the present invention (this embodiment) will be described below with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the embodiments described below.

The embodiments described below illustrate an example of applying the present invention to machine translation, but the present invention is applicable to sequence transformations in any field as long as they use constraint information. For example, the present invention can be used for summarization tasks, speech generation tasks, and tasks that add descriptive text to images.

Furthermore, in the embodiments described below, the unit of translation is defined as a sentence, but the unit of translation may be any unit.

The generation device 100 described below offers specific improvements to conventional technologies that perform constrained sequence transformations, and represents an improvement in the technical field related to constrained sequence transformations. Furthermore, the extraction device described below offers specific improvements to conventional technologies in the extraction of constraint information, and represents an improvement in the technical field related to the extraction of constraint information.

(Regarding the challenges)
Before describing the configuration and operation of this embodiment in detail, we will first describe the prior art and the problems associated with it. Note that the following description of problems is not publicly known. Furthermore, the problems described below relate to the technology of this embodiment.

As already explained, machine translation with vocabulary constraints is a method that imposes constraints on the translation of a sentence from one domain to another (e.g., another language) with the aim of including all specified words and phrases. For reference, Figure 1 shows an example of input and output in machine translation with vocabulary constraints.

In the example in Figure 1, the machine translation (MT Output), constraints, and lexical-constrained machine translation (Constrained MT Output) are shown for the original sentence, "A geometrical optical theory of standing waves based on ray coincidence was developed." The underlined parts indicate the constraints.

As a prior art technique for lexically constrained machine translation, Non-Patent Document 1, "Chen, G., Chen, Y., and Li, V. O. (2021). "Lexically Constrained Neural Machine Translation with Explicit Alignment Guidance." Proceedings of the AAAI Conference on Artificial Intelligence," discloses a lexically constrained machine translation method for constraint phrases created by humans. The method disclosed in Non-Patent Document 1 is also called the soft method. The method disclosed in Non-Patent Document 1 does not guarantee that the constraint phrases will always be included in the translated text.

Non-patent document 2, "Matt Post and David Vilar. 2018. Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics," and reference 1, "Chousa, K. and Morishita, M. (2021). "Input Augmentation Improves Constrained Beam Search for Neural Machine Translation: NTT at WAT 2021." In Proceedings of the 8th Workshop on Asian Translation (WAT), pp. 53-61, Online. Association for Computational Linguistics," also disclose a lexical-constrained machine translation method for manually created constraint phrases. This method guarantees that the constraint phrase will always be included in the translated text. This method is also known as the HARD method.

There are also use cases where constraint phrases are created automatically rather than manually. For example, in the translation of domain documents containing many proper nouns, such as patents and scientific and technical papers, translation memories and bilingual dictionaries created from past translation results are often used. Therefore, a use case can be conceivable where machine translation with vocabulary constraints is performed using constraint phrases automatically extracted from the bilingual dictionary.

On the other hand, when constraint terms are automatically extracted, it is possible that the extracted constraint terms may contain noisy terms. Furthermore, even when constraint terms are extracted manually, noise may be included.

Conventional vocabulary-constrained machine translation methods disclosed in Non-Patent Documents 1 and 2, etc., assume that the given constraint phrases are included in the reference translation. Therefore, when applying a vocabulary-constrained machine translation method using extracted constraint phrases as vocabulary constraints, it is anticipated that incorrect phrases may be included in the translated text, leading to a decrease in translation accuracy.

Based on the above points, the following describes techniques for reducing noise and appropriately extracting constraint phrases, as well as techniques for performing accurate lexical-constrained machine translation even when using sets of constraint phrases that may contain noise.

(Example of device configuration, overall operation)
Figure 2 shows an example of the configuration of the generation device 100 in this embodiment. As shown in Figure 2, the generation device 100 has an input unit 110, an extraction unit 120, an input generation unit 130, a series generation unit 140, a reranking unit 150, and an output unit 160.

Furthermore, a bilingual dictionary DB 200 and a model DB 300 are provided. The bilingual dictionary DB 200 stores bilingual dictionaries, and the model DB 300 stores trained machine translation models. The bilingual dictionary DB 200 and the model DB 300 may be located outside the generation device 100 (as in the example in Figure 2), or they may be located inside the generation device 100.

Referring to the flowchart in Figure 3, the overall operation flow of the generation device 100 will be explained. In S101, the source language text is input by the input unit 110. In S102, the extraction unit 120 automatically extracts constraint phrases based on the source language text (input text) input by the input unit 110 and the translation dictionary read from the translation dictionary DB 200.

In S103, the input generation unit 130 generates multiple inputs (lexical constraints) from any combination of constraint phrases. In S104, the sequence generation unit 140 performs translation on the input sentence using the multiple inputs generated in S103 and the machine translation model read from the model DB 300. Here, a translation result is obtained for each of the multiple inputs generated in S103. That is, the sequence generation unit 140 uses a certain sequence and lexical constraints to generate one or more candidates for another sequence based on a pre-trained sequence transformation model.

In S105, the reranking unit 150 predicts a reranking score for each translation result using the input sentence. In S106, the output unit 160 outputs the translation result (target language sentence) with the highest score. The configuration and operation of the main functional units will be described in detail below.

(Extraction unit 120)
First, let's explain the extraction unit 120. The extraction unit 120 receives the source language text and the translation dictionary as input and outputs the source language text and the constraint vocabulary list. Note that the source language text may not be output.

Figure 4 is a diagram showing the configuration of the extraction unit 120. As shown in Figure 4, the extraction unit 120 includes a filtering unit 121, a division unit 122, and a constraint phrase extraction unit 123. The extraction unit 120 also refers to the bilingual dictionary 200. Note that the extraction unit 120 may also be configured without the filtering unit 121.

The bilingual dictionary DB200 stores a set of pairs of two words that are used to convert sequences. Specifically, in this embodiment which targets translation, the bilingual dictionary DB200 stores a set of <source language phrase, target language phrase> pairs. The source language phrase and target language phrase may each consist of multiple words. In this embodiment, one <source language phrase, target language phrase> pair is referred to as a "translation". The source language phrase and target language phrase may also be referred to as the source language translation and the target language translation, respectively.

Furthermore, when using the DB200 bilingual dictionary for tasks other than translation, its contents are not limited to sets of <source language phrase, target language phrase> pairs.

The filtering unit 121 filters out noisy translations from the translation dictionary. The filtered translation dictionary is stored in the translation dictionary DB 200, and the splitting unit 122 and the constraint phrase extraction unit 123 refer to the filtered translation dictionary.

The splitting unit 122 performs morphological analysis on the source language sentence and the source language phrases in the translation dictionary. In other words, the splitting unit 122 divides the source language sentence and the source language phrases in the translation dictionary into unit information. The constraint phrase extraction unit 123 extracts translations corresponding to the phrases contained in the source language sentence (examples of unit information obtained by splitting) and creates a constraint phrase list. The processing of each unit will be explained in more detail below.

<Extraction unit 120: Filtering unit 121>
The filtering unit 121 removes translations that fall under (A) to (C) below, or words or phrases included in the translations, from the translation dictionary. However, it is not mandatory for the filtering unit 121 to perform all of (A) to (C), and it may perform at least one of (A) to (C). It may also perform filtering other than (A) to (C). In particular, in the modified examples 1 and 2 described later, the process of (C) below may be skipped.

(A) Translations that include words other than nouns/noun phrases (excluding verbs because they are conjugated)
(B) Translations consisting of phrases with a length of 1. Examples of (B) include single-character words such as units. For example, the translation of "Target language: C, Source language: degree" falls under (B).

(C) Cases where there is no unique correspondence between the source language and the target language (for example, cases where there are multiple translations for a single word or phrase in the source language).
For translations corresponding to (C), the translation shall be deleted. Alternatively, one translation shall be kept from multiple translations, and the others shall be deleted, so that there is a one-to-one correspondence between the source language phrase and the target language phrase. Any method may be used to keep one translation from multiple translations and delete the others; for example, one may keep the translation that is listed first, or one may keep the translation that appears most frequently.

For example, the translation of "Source language: computer, Target language: calculator, computer" falls under (C). In this case, for example, this translation should be deleted, or changed to "Target language: computer, Source language: calculator" so that there is a one-to-one correspondence between the source language phrase and the target language phrase.

<Extraction unit 120: Division unit 122>
The splitting unit 122 divides (tokenizes) the source language text and the source language translation from the bilingual dictionary into morphemes and inserts a predetermined symbol (e.g., a space, "/") at the morpheme boundaries. This splitting unit may differ from the splitting unit used in the subsequent translation process.

For example, if the original language sentence is "not limited to that," the original language sentence after processing by the splitting unit 122 becomes "not limited to that."

<Extraction unit 120: Constraint phrase extraction unit 123>
The constraint phrase extraction unit 123 extracts translations corresponding to words and phrases contained in the source language text and creates a constraint phrase list using the extracted translations. An example of a specific constraint phrase extraction method is described below. Note that the dictionary format and search method are not limited to the method described below, and other methods may be used as long as they can extract constraint phrases corresponding to words and phrases contained in the source language text.

In this example, we will use a translation dictionary that represents the original language translations character by character using a data structure called a Trie tree.

The constraint phrase extraction unit 123 performs a prefix matching search on the set of source language translations in the translation dictionary, starting from the beginning of the source language sentence. When a translation (pair) containing a source language translation that matches a phrase in the source language sentence is found, that object translation is extracted as a constraint phrase. During the prefix matching search, the translation with the longest source language translation phrase is selected.

For example, suppose that in the source language text, morphological analysis by the splitting unit 122 results in three phrases, namely "ABC/GHI/XYZ". Here, A, B, C, etc., are letters. When the constraint phrase extraction unit 123 searches the source language translation in the translation dictionary using "ABC/GHI/XYZ", it matches the beginning (or front) of "ABC/GHI/XYZ".

As a result of the above search, even if four phrases such as "AB," "ABC," "ABCG," and "ABC/GHI" match, as will be explained later, "AB" and "ABCG" can be excluded from the match because their morphemes do not match. In this case, of the remaining "ABC" and "ABC/GHI," the object translation paired with "ABC/GHI," which has the longest phrase length in the original language translation, is extracted as a constraint phrase. Then, the same process is performed using "XYZ," which is the part after "ABC/GHI."

As in this embodiment, the division unit 122 pre-divides the source language text and the source language translation in the bilingual dictionary into morphemes (examples of unit information), and by performing a search while considering the morpheme boundaries, it is possible to prevent the mis-extraction of words that do not match in the division units. This is particularly effective when the source language is a language that is not separated by spaces, such as Japanese. For example, it is possible to prevent the source language translation "hana" (flower) from matching "sono/kagiri/de/wa/nai" in the source language text. In other words, "hana" will no longer be able to match "ha/na".

The prefix matching, longest match, and word segmentation methods described here are just examples of techniques for achieving less noisy and less ambiguous restricted phrase extraction. Other methods for resolving ambiguity may also be used.

For example, when performing morphological analysis in the splitting unit 122, information necessary for resolving ambiguity, such as part of speech, base form, stem, conjugation form, and reading (pronunciation), is attached to the split words, and matching is performed using this attached information. In other words, by using not only the string but also its accompanying information such as part of speech during matching, ambiguity can be resolved in situations where, for example, the character "in" in the original language sentence matches both the preposition "in" and the noun "inn" (inn), which are the original language translations. Resolving ambiguity during matching is an important factor in improving translation accuracy.

<Other examples of the configuration of the extraction unit 120>
Note that the extraction unit 120 may be configured as shown in Figure 5 instead of the configuration shown in Figure 4. In the configuration shown in Figure 5, instead of filtering the bilingual dictionary, the filtering unit 121 performs filtering on the constraint phrases extracted by the constraint phrase extraction unit 121.

The filtering process is the same as that performed by the filtering unit 121 described above. However, "translation" is read as "constraining phrase". Specifically, the filtering unit 121 deletes constraint phrases that fall under (A) to (C) below from the extraction results of the constraint phrase extraction unit 123. However, it is not mandatory for the filtering unit 121 to perform all of (A) to (C), and it may perform at least one of (A) to (C). Also, rules other than (A) to (C) may be used. In particular, when performing the modified examples 1 and 2 described later, the process of (C) below may be skipped.

(A) Restrictive phrases that include words other than nouns/noun phrases (verbs are removed because they have conjugations)
(B) Constraint phrases consisting of words of length 1 (C) The correspondence between the source language and the target language is not unique (for example, there are multiple constraint phrases for a single word in the source language)
When implementing (C), if there are multiple constraint clauses for a single word in the source language, for example, the multiple constraint clauses may be deleted, or one constraint clause may be kept and the others deleted, so that there is a one-to-one correspondence between the source language word and the target language word.

(Regarding other configuration examples of the extraction unit 120 and the generation device 100)
The extraction unit 120 may be a standalone device independent of the generation device 100. This standalone device may be called an extraction device. Furthermore, an extraction unit 120 included in the generation device 100 may also be called an extraction device. Additionally, the generation device 100 having an extraction unit 120 may be called an extraction device. Both the extraction unit 120 and the extraction device may also include either or both of the display information generation unit 170 and the modification unit 180 as described in the embodiment later.

When the extraction unit 120 is configured as a separate device independent of the generation device 100, the generation device 100 may not include the extraction unit 120. The configuration of the generation device 100 in this case is shown in Figure 6. In the configuration of Figure 6, the constraint phrase list generated by the extraction device is input to the generation device 100. However, in the configuration of Figure 6, a constraint phrase list other than the constraint phrase list generated by the extraction device (e.g., a constraint phrase list containing a lot of noise) may be input to the generation device 100.

The operation of the input generation unit 130, the sequence generation unit 140, and the reranking unit 150 in Figure 6 is the same as the operation of the input generation unit 130, the sequence generation unit 140, and the reranking unit 150 in Figure 2.

(Input generation unit 130)
Next, the input generation unit 130 will be described. The input generation unit 130 receives a constraint phrase list as input and sets all elements of a subset of the words included in the constraint phrase list as lexical constraints. However, it is also possible to set only some of the elements out of all as lexical constraints.

Finally, the input generation unit 130 outputs the above vocabulary constraints as vocabulary constraints corresponding to the source language sentence input to the extraction unit 120. A specific example is shown below.

Assume that the input generation unit 130 receives {A, B, C} as a constraint phrase list. Here, A, B, and C are constraint phrases.

The input generation unit 130 extracts {}, {A}, {B}, {C}, {A,B}, {A,C}, {B,C}, {A,B,C} as elements of a subset of {A,B,C}, and outputs each of these as a lexical constraint.

Note that {{}, {A}, {B}, {C}, {A,B}, {A,C}, {B,C}, {A,B,C}} are constraint lexical sets, and each {...} represents one lexical constraint.

The lexical constraints created from a subset of the constraint phrase list C are 2 ^|C| , and as will be described later, multiple candidate translations can be obtained from each of these lexical constraints.

(Sequence generation unit 140)
Next, the sequence generation unit 140 will be described. Assume that the sequence generation unit 140 holds a trained machine translation model read from the model DB 300. The sequence generation unit 140 repeats the following process a number of times equal to the number of lexical constraints (the number of elements in the lexical constraint set). For example, if the lexical constraint set is {{}, {A}, {B}, {C}, {A,B}, {A,C}, {B,C}, {A,B,C}}, then it will be repeated 8 times.

The sequence generation unit 140 receives an input sentence (source language sentence) and lexical constraints as input. The sequence generation unit 140 generates a translated sentence (target language sentence) using a machine translation model by applying existing methods for machine translation with lexical constraints. Here, multiple translated sentences are generated as candidate translations (target language sentence candidates). Each candidate translation is also assigned a score as a translated sentence.

Any existing method can be used for vocabulary-constrained machine translation, but for example, LeCA or LeCA+LCD can be used. LeCA is disclosed in Non-Patent Document 1 and is also called the soft method. LeCA+LCD is disclosed in the aforementioned Reference Document 1 and is also called the hard method.

The sequence generation unit 140 outputs multiple generated translation candidate sentences. For example, the sequence generation unit 140 outputs a predetermined number of translation candidate sentences in descending order of score. The "determined number" may be 1. In other words, only the translation sentence with the highest score may be output. Here, for example, 30 translation candidate sentences are output for each vocabulary constraint.

<Example of the configuration of the sequence generation unit 140>
Figure 7 shows an example of the configuration of the sequence generation unit 140. As shown in Figure 7, the sequence generation unit 140 has a sequence conversion unit 141 and a search unit 142.

Furthermore, when using the soft method for generating translated text, the sequence conversion unit 141 uses vocabulary constraint information. When using the hard method, depending on the type of hard method, the sequence conversion unit 141 may or may not use vocabulary constraints. The input arrows for vocabulary constraints to the sequence conversion unit 142 are indicated by dotted lines. Among the hard methods, the LeCA+LCD mentioned above uses vocabulary constraint information in the sequence conversion unit 141. The following describes the configuration/operation assuming LeCA+LCD.

In the sequence conversion unit 141, as shown in Figure 8, a model based on a general encoder-decoder model (e.g., Transformer) having an encoder and a decoder can be used as the machine translation model. However, the present invention can also be implemented using a model other than the encoder-decoder model.

The sequence conversion unit 141 receives the source language sentence and lexical constraints as input. First, it expands the source language sentence using the lexical constraints to create an input sequence with added lexical constraint information, and then uses this as input to the machine translation model.

More specifically, in the above extension, the sequence conversion unit 141 creates a vocabulary-constrained input sequence by combining (concatenating) the input sequence, the original language sentence X, and each constraint phrase C _i via a special delimiter string, <sep>, as shown below. <eos> is a string that indicates the end of a sentence.

[X, <sep>, C ₁ , <sep>, C ₂ ,..., C _N , <eos>]
The sequence conversion unit 141 takes the expanded input sequence as input to the machine translation model and generates a sentence. More specifically, it outputs the probability of each word in the set of words that can constitute the output sequence.

The search unit 142 uses the output probabilities of the decoder in the machine translation model to search for an output sequence (or an approximate solution thereof) that maximizes the generation probability given an input sequence. The search unit 142 uses a grid beam search method based on beam search, which ensures that the output sequence satisfies all constraint vocabulary.

Note that the use of grid beam search by the search unit 142 is just one example. Any processing method that performs a lexical-constrained search to include the constraint phrases may be used.

(Re-ranking section 150)
Next, the reranking unit 150 will be explained. The reranking unit 150 receives one or more translation candidate sentences generated by the sequence generation unit 140 as input. For example, if the sequence generation unit 140 generates 30 translation candidate sentences for each vocabulary constraint, and there are eight vocabulary constraints, the reranking unit 150 receives 30 translation candidate sentences as input for each vocabulary constraint among those eight vocabulary constraints.

Next, the re-ranking unit 150 calculates a score for each translation candidate using the input sentence (source language sentence), and outputs the translation candidate with the best score as the final translation. At this point, it is also possible to output all (or some) translations and their scores, rather than narrowing it down to the highest score. This allows the output unit 160 to present the translations to the user in a ranked format using the scores.

Regarding the method for calculating the score by the reranking unit 150, any method that can calculate the score of the translated text may be used, but for example, the methods in Examples 1 and 2 below can be used.

Example 1:
The reranking unit 150 uses the likelihood of the translated sentence candidates output by the machine translation model used for translation in the sequence generation unit 140 as a score.

Example 2:
The reranking unit 150 uses a machine translation model trained on an Encoder-Decoder model called a Transformer for a Right-to-Left translation task that generates translated sentences from the end of a sentence to the beginning of a sentence as its reranking model, and uses the likelihood of forcing the output of translated sentence candidates with this reranking model as its score. Forcing the output of translated sentence candidates can also be rephrased as performing forced decoding using translated sentence candidates.

In other words, the source language sentence is input to the encoder of the reranking model, and the words of the translation candidate for which you want to evaluate the score (likelihood) are sequentially input to the decoder of the reranking model.

In Example 1 and Example 2, the likelihood output by the machine translation model can be any value that indicates plausibility. The likelihood output by the machine translation model can be a probability or a value other than a probability.

Furthermore, the reranking unit 150 may calculate the reranking score using both the likelihood of Example 1 and the likelihood of Example 2. For example, the average of the likelihoods of Example 1 and Example 2 may be used as the reranking score.

(Variation 1)
Next, Modification 1 will be described. In Modification 1, the constraint phrase list generated by the extraction unit 120 can be one in which multiple target language phrases correspond to one source language phrase. Such a constraint phrase list may be called a constraint phrase list that allows multiple translations. For example, such a constraint phrase list may be generated if the filtering unit of the extraction unit 120 does not perform procedure (C).

For example, suppose there are multiple target language phrases A and A' for a given source language phrase, and the extraction unit 120 generates "A, A', B, C" including these two and B and C as multiple elements of the constraint phrase list. For example, if the source language phrase is "computer" and the target language phrases are "computer" and "computer", then A and A' correspond to "computer" and "computer".

Here, a list of constraint phrases with multiple elements like this is represented as {{A, A'}, {B}, {C}}.

The input generation unit 130, having received {{A, A'}, {B}, {C}} from the extraction unit 120, generates {}, {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C} as well as {A'}, {A', B}, {A', C}, {A', B, C} as vocabulary constraints.

The input generation unit 130 inputs each of the generated vocabulary constraints to the sequence generation unit 140.

The sequence generation unit 140 uses each of the 12 lexical constraints {}, {A}, {B}, {C}, {A,B}, {A,C}, {B,C}, {A,B,C}, {A'}, {A',B}, {A',C}, {A',B,C} as a lexical constraint to perform 12 lexical-constrained machine translations and obtain candidate translations. For example, if one candidate translation is generated for each lexical constraint, 12 candidate translations will be obtained.

After performing machine translation with vocabulary constraints, the re-ranking unit 150 performs a re-ranking process using the method described above, and outputs, for example, the translation candidate with the highest score as the final translation.

(Variation 2)
Next, a second modification will be described. In the second modification as well, the constraint phrase list generated by the extraction unit 120 can be one in which multiple target language phrases correspond to one source language phrase.

In the second modification, the search process for translated text in the search unit 142 of the sequence generation unit 140 may be modified to allow for multiple phenotypes of a single constraint phrase. In other words, the search may be performed to satisfy one element from each candidate constraint phrase. Specifically, this is as follows.

In the second variation, let's assume that there are multiple target language phrases, A and A', for a given source language phrase, and that "A, A', B, C", including B and C, is generated as multiple elements of the constraint phrase list by the extraction unit 120. Here, the constraint phrase list {A, B, C} is generated, and information indicating that A can also be A' is input from the extraction unit 120 to the input generation unit 130. Alternatively, the constraint phrase list {A, A', B, C} may be generated, and information indicating that A and A' can be either may be input from the extraction unit 120 to the input generation unit 130. Note that the above example shows a format that allows two ambiguities for one constraint phrase, but a format that allows three or more ambiguities for one constraint phrase may also be used.

For example, if three ambiguities are allowed for A, the constraint phrase list {A, B, C} is generated, and information indicating that A can be A' or A'' is input from the extraction unit 120 to the input generation unit 130. Alternatively, the constraint phrase list {A, A', A'', B, C} may be generated, and information indicating that A, A', and A'' can be any of them may be input from the extraction unit 120 to the input generation unit 130.

For the constraint phrase list {A, B, C}, if A can also be A', the input generation unit 130 generates seven lexical candidate constraints: {}, {{A, A'}}, {{B}}, {{C}}, {{A, A'}, {B}}, {{A, A'}, {C}}, {{A, A'}, {B}, {C}}. In the modified example 2, since a certain source language phrase may correspond to multiple target language phrases (e.g., A, A'), there is ambiguity in the translation, and the vocabulary used as constraints is not fixed, so it is called a "lexical candidate constraint" instead of a lexical constraint. In other words, a "lexical candidate constraint" is a lexical constraint that retains ambiguity. Note that the above expression format for lexical candidate constraints is just one example. Any expression format other than the above format may be used as long as it can express that it can be either A or A'.

As shown in Figure 9, the sequence generation unit 140 receives the source language sentence along with the vocabulary candidate constraints as input. The sequence generation unit 140 uses each of the seven vocabulary candidate constraints—{}, {{A, A'}}, {{B}}, {{C}}, {{A, A'}, {B}}, {{A, A'}, {C}}, {{A, A'}, {B}, {C}}—as a vocabulary candidate constraint and performs seven vocabulary-constrained machine translations to obtain candidate translations. For example, if one candidate translation is generated for each vocabulary candidate constraint, seven candidate translations will be obtained.

After performing machine translation with vocabulary constraints, the re-ranking unit 150 performs a re-ranking process using the method described above, and outputs, for example, the translation candidate with the highest score as the final translation.

When using a lexical candidate constraint that includes {A, A'}, the search unit 142 of the sequence generation unit 140 performs the search assuming that the word A may also be A'. In other words, it performs a search that takes ambiguity into account. For the search, for example, the method described in reference 2, "Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2017. Guided Open Vocabulary Image Captioning with Constrained Beam Search. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 936-945, Copenhagen, Denmark. Association for Computational Linguistics," disclosed at https://aclanthology.org/D17-1098/, can be used. This method is an example of a "search that takes ambiguity into account."

The method disclosed in Reference 2 performs a lexical-constrained beam search that takes into account the ambiguity of the translation, where either A or A' is acceptable. In other words, the ambiguity between A and A' is resolved during the beam search.

Furthermore, the method disclosed in Reference 2 is a language generation method, but not a translation technique. There are no prior techniques that apply this method to the search process during translation decoding.

Furthermore, in the descriptions of the embodiments described above, and in the descriptions of modifications 1 and 2, the multiple target language terms (e.g., A, A') for a given source language term may not only be synonyms such as "calculator" and "computer," but may also be terms other than synonyms, such as "trunk," "elephant's trunk," "trunk," and "main road" for "trunk." Since the meaning of words is not considered during the search in the search unit 142, A and A' may be completely unrelated terms. Also, when using the technology according to the present invention for tasks other than translation, the criteria for determining which multiple words in the converted sequence correspond to the words in the original sequence can be set according to any arbitrary criteria.

Furthermore, in the descriptions of the embodiments described above, and in the descriptions of modifications 1 and 2, it is also possible to convert non-base forms of words into their base forms during morphological analysis, taking into account word variations (plural forms, tense changes, etc.).

As an example, suppose the translation dictionary is English-Japanese and has the dictionary entry "corn - corn, fish eye". In this case, assume that the source language sentence "We roasted corns over the charcoal." is input to the generation device 100. In this case, when the extraction unit 120 performs morphological matching, "corns" matches the translation dictionary because "corn" is included as a morpheme. However, if, for example, the entry in the translation dictionary is "feet" and the morpheme in the input sentence is "foot", then it will not match. This problem can be solved by reverting "foot" back to its base form before performing the matching.

(Examples)
Next, as a more specific example, an embodiment using the technology described above will be explained. In this embodiment, it is possible to edit (modify, add) constraint phrases in the display unit 500 (a device capable of display and input operations), which will be described later, and to check the target language sentence (translated sentence) for each constraint phrase each time.

<Display Image>
First, the display image on the display unit 500 will be explained with reference to Figure 10. In the example shown in Figure 10, the user enters "We have prototyped a superconducting single-phase autotransformer with auxiliary windings only on the shunt winding" as the source language text and presses "Send".

The display unit 500 displays multiple constraint phrases (a list of constraint phrases) for the input source language sentence. The phrases displayed here as constraint phrases are those filtered by the filtering unit 121. To the right of these, the filtered constraint phrases are displayed in the form of "Do you want to add them?".

Users can select constraint terms they wish to modify (or delete) or add by checking checkboxes, and then modify (or delete)/add the selected constraint terms by pressing the corresponding buttons. Users can also add constraint terms they have created themselves.

By pressing "Update" in the displayed image, you can view the target language sentence using the constraint phrases at that time.

<Device configuration and operation>
Figure 11 shows an example configuration of a generation device 100 for realizing the above-described display. As shown in Figure 11, the generation device 100 in this embodiment includes an extraction unit 120, a display information generation unit 170, a modification unit 180, a generation unit 190, a bilingual dictionary DB 200, and a constraint word list DB 400. Note that the modification unit 180 may be included in the display information generation unit 170.

Furthermore, the bilingual dictionary DB 200 and the constraint term list DB 400 may be provided outside the generation device 100. Similarly, the generation unit 190 may also be provided outside the generation device 100 (for example, on a separate server). The generation device 100 may also be used for the purpose of displaying the constraint term list on the display unit 500. In that case, the generation device 100 may consist only of the extraction unit 120 and the display information generation unit 170 from the functional units shown in Figure 11. The generation device 100 may also be called the extraction device. The functions of each unit are as follows.

The extraction unit 120 is the extraction unit 120 shown in Figure 4 or Figure 5. It takes the source language sentence as input and outputs a list of constraint words. The output list of constraint words is stored in the constraint word list DB 400 and is also input to the display information generation unit 170. The extraction unit 120 may also output the filtered constraint words as a filtered word list. The output filtered word list is also input to the display information generation unit 170.

The display information generation unit 170 generates information (referred to as information for presenting the constraint phrase list) for displaying the constraint phrase list on the display unit 500. The information for presenting the constraint phrase list includes the constraint phrase list. Furthermore, the information for presenting the constraint phrase list may also include information from the filter phrase list, such as deleted information, filter candidate phrases, or additional candidate phrases. The information for presenting the constraint phrase list is transmitted from the display information generation unit 170 to the display unit 500 and input to the display unit 500. The display information generation unit 170 may also generate display information for displaying the constraint phrases in a format that allows for modification, along with the target language sentence (translated text) generated using the constraint phrases. Additionally, when the generation device 100 receives added or modified constraint phrases from the display unit 500, the display information generation unit 170 may acquire the target language sentence (translated text) generated based on the received constraint phrases and generate display information for displaying the target language sentence (translated text).

Furthermore, the display information generation unit 170 may generate "correction support information" for the user to use when reviewing the constraint word list, and transmit it to the display unit 500. The correction support information includes at least one of the source language sentence entered by the user, the extracted constraint word list, and the target language sentence generated based on the extracted constraint word list.

The modification unit 180 receives from the display unit 500 at least one of the additional constraint phrases or modified constraint phrases as information indicating that the user has modified the presented list of constraint phrases.

The modification unit 180 modifies the information stored in the constraint word list DB 400 based on the received information. Furthermore, when the constraint word list is modified, the target language sentence may be regenerated using vocabulary-constrained machine translation based on the modified constraint word list. The display information generation unit 170 then generates modification support information containing the target language sentence, which is transmitted to the display unit 500 for display.

The generation unit 190 includes an input generation unit 130, a sequence generation unit 140, and a reranking unit 150. As described above, the generation unit 190 uses these functional units to generate a target language sentence (translated sentence) that takes vocabulary constraints into account, based on the constraint phrase list read from the constraint phrase list DB 400 and the source language sentence received from the display unit 500, and inputs the generated target language sentence to the display information generation unit 170.

The display unit 500 is, for example, a computer (terminal) having a display. The display unit 500 is connected to the generation device 100 via a network. As explained with reference to Figure 10, the display unit 500 receives the source language text from the user and displays the constraint word list, etc. The display unit 500 also receives instructions to add or modify constraint words and the source language text. The display unit 500 can also output the source language text, the final target language text, and the final constraint word list as a set.

The generation device 100 in the above embodiment allows for interactively modifying the list of constraint phrases while checking the results of machine translation with vocabulary constraints, thereby generating a target language sentence (translated sentence) that is closer to the user's intended image.

(Experimental results)
In the following explanation of the experimental results, "the generation apparatus 100 in this embodiment" will be referred to as the proposed method or proposed system.

To confirm the effectiveness of the proposed machine translation method with vocabulary constraints, which involves re-ranking translation candidates against automatically extracted vocabulary constraints, we evaluated the accuracy of machine translation with vocabulary constraints using vocabulary constraints automatically extracted from a parallel dictionary, focusing on Japanese-English translation.

<About the Bilingual Dictionary>
To extract vocabulary constraints, we used the general-purpose EDR Japanese-English Bilingual Dictionary (EDR-JE) and the Japanese-English translation system ALT-J/E as bilingual dictionaries.

<Model>
The following translation models were used for evaluation.

Transformer
• LeCA + {EDR-JE, ALT-J/E}
・LeCA+LCD + {EDR-JE, ALT-J/E}
ASPEC was used as the parallel corpus for training and evaluating the translation models. The detailed settings and hyperparameters for each model are shown in Figure 12.

For constraints extracted from the dictionary, we used the top 30 generated sentences for each of the 2 ^|C| lexical constraints. For the reranking of translation candidates, we used a score calculated by a reranking model using the source language sentences and translation candidates via Reranker.

The reranking model used was a model trained on a Right-to-Left translation task that generates translated text from the end of a sentence to the beginning of a sentence, using a Transformer (big) model. The reranking score was calculated using the likelihood obtained from forced decoding of the input translation candidates. BLEU, an automated evaluation metric for translation accuracy, was used to evaluate each method.

<About the experimental results>
Figure 13 shows the translation accuracy of each method when using vocabulary constraints automatically extracted from a bilingual dictionary. In the Reranker, which uses a score from a reranking model, it can be seen that LeCA and LeCA+LCD are able to improve translation accuracy compared to the baseline (Transformer). Furthermore, Figure 13 shows that the translation accuracy is high regardless of the type of dictionary used.

(Example hardware configuration)
Any of the devices described in this embodiment (generation device 100, extraction device) can be realized, for example, by having a computer run a program. This computer may be a physical computer or a virtual machine on the cloud.

In other words, the device can be realized by using hardware resources such as the CPU and memory built into a computer to execute a program corresponding to the processing performed by the device. This program can be recorded on a computer-readable recording medium (such as portable memory), saved, and distributed. Furthermore, this program can also be provided via a network, such as the internet or email.

Figure 14 shows an example of the hardware configuration of the computer described above. The computer in Figure 14 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., all of which are interconnected by a bus BS. The computer may also be equipped with a GPU.

The program that enables processing on the computer is provided on a recording medium 1001, such as a CD-ROM or memory card. When the recording medium 1001 containing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may also be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files and data.

The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when a program startup command is received. The CPU 1004 implements the functions related to the light touch maintenance device 100 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. based on the program. The input device 1007 consists of a keyboard and mouse, buttons, or a touch panel etc., and is used to input various operation commands. The output device 1008 outputs the calculation results.

(Summary of the embodiment, effects, etc.)
As described above, the technology described in this embodiment makes it possible to automatically extract constraint phrases used in vocabulary-constrained machine translation with low noise. Furthermore, the technology described in this embodiment enables accurate translation in vocabulary-constrained machine translation.

Further details regarding the above embodiments are disclosed below, specifically Appendix 1 and Appendix 2.

<Note 1>
(Additional note 1)
Memory and
At least one processor connected to the memory,
Includes,
The aforementioned processor,
The dictionary is a collection of pairs of first and second pieces of information. The first piece of information and each of the first sequence are divided into unit pieces of information.
An extraction device that extracts second information corresponding to the first information that matches the unit information of the first series from the dictionary as constraint information used to generate the second series based on the first series.
(Additional note 2)
The extraction device according to Appendix 1, wherein the processor deletes pairs from the dictionary that match predetermined rules and uses the dictionary from which the deletion process has been performed.
(Additional note 3)
The extraction device described in Appendix 2, wherein a pair that conforms to the predetermined rule is at least one of the following: a pair containing a word or phrase other than a noun, a pair consisting of a word or phrase of length 1, and a pair in which the correspondence between the first information and the second information is not unique.
(Additional note 4)
The extraction device according to any one of the appendices 1 to 3, wherein the processor performs matching of the first series of unit information with the first information in order to resolve ambiguity.
(Additional note 5)
The extraction device according to any one of the appendices 1 to 4, wherein the processor generates display information for transmitting the constraint information to the display unit, and receives constraint information that has been added to or modified from the constraint information displayed on the display unit.
(Additional note 6)
Memory and
At least one processor connected to the memory,
Includes,
The aforementioned processor,
Taking the first sequence as input, constraint information is extracted based on the first sequence and a dictionary which is a set of pairs of first and second pieces of information.
A second sequence is generated based on the aforementioned constraint information and the first sequence.
A generating device that generates display information for displaying the aforementioned constraint information together with the second series in a format that allows for modification of the constraint information.
(Supplementary Note 7)
The generation device according to Appendix 6, wherein, upon receiving constraint information that has been added or modified, the processor obtains a series generated based on the received constraint information and generates display information for displaying the series.
(Supplementary Note 8)
The processor generates display information for displaying constraint information filtered based on predetermined rules as additional candidates, as described in Appendix 6 or 7.
(Supplementary Note 9)
A computer-based extraction method,
The first information in a dictionary is a set of pairs of first and second information, and each of the first sequences is divided into unit information in a division step,
An extraction method comprising: a constraint information extraction step of extracting from the dictionary the second information corresponding to the first information that matches the unit information of the first series, as constraint information used to generate the second series based on the first series.
(Supplementary Note 10)
A generation method performed by a computer,
An extraction step in which constraint information is extracted based on the first sequence as input and a dictionary which is a set of pairs of first and second pieces of information,
A generation step of generating a second sequence based on the constraint information and the first sequence,
A generation method comprising: a display information generation step of generating display information for displaying the aforementioned constraint information together with the second series in a format that can be modified.
(Supplementary Note 11)
A non-temporary storage medium storing a program that causes a computer to function as an extraction device as described in any one of the appendices 1 through 5.

<Note 2>
(Additional note 1)
A generation device for generating a second sequence of information from constraint information and a first sequence of information, which is another sequence of information,
Memory and
At least one processor connected to the memory,
Includes,
The aforementioned processor,
A constraint information list is input, and each element of one or more subsets of constraint information included in the constraint information list is output as a lexical constraint.
Using the first sequence and the lexical constraints, one or more candidates for the second sequence are generated.
A generating device that calculates a score indicating the suitability of each of the one or more candidates as the second series.
(Additional note 2)
The generation apparatus according to Appendix 1, wherein the processor calculates the score based on at least one of the likelihood output by the model used to generate the candidates in the sequence generation unit and the likelihood obtained from the candidates by the reranking model.
(Additional note 3)
The generation apparatus according to Appendix 1 or 2, wherein, when the constraint information list includes constraint information with ambiguity, the processor generates one or more candidates by performing a beam search with vocabulary constraints that take ambiguity into account.
(Additional note 4)
The generation apparatus according to any one of the appendices 1 to 3, wherein at least one constraint information is input to the processor in a format that allows for two or more ambiguities, and the processor generates lexical constraints while retaining the ambiguities.
(Additional note 5)
A computer generation method for generating a second sequence of information from constraint information and a first sequence of information, wherein
An input generation step that inputs a constraint information list and outputs each element of one or more subsets of constraint information included in the constraint information list as a lexical constraint,
A sequence generation step that generates one or more candidates for the second sequence using the first sequence and the lexical constraints,
A generation method comprising: a reranking step of calculating a score for each of the one or more candidates indicating their suitability as the second series.
(Additional note 6)
A non-temporary storage medium storing a program for causing a computer to function as a component in any one of the generating devices described in any one of the appendices 1 to 4.

Although this embodiment has been described above, the present invention is not limited to this specific embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims.

100 Generator 110 Input unit 120 Extraction unit 121 Filtering unit 122 Splitting unit 123 Constraint phrase extraction unit 130 Input generation unit 140 Sequence generation unit 141 Sequence conversion unit 142 Search unit 150 Reranking unit 160 Output unit 170 Display information generation unit 180 Correction unit 190 Generation unit 200 Bilingual dictionary DB
300 Model DB
400 Constraint Phrase List DB
500 Display unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims (8)

A generation device for generating a second sequence of information from constraint information and a first sequence of information, which is another sequence of information,
An input generation unit that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint,
A sequence generation unit generates one or more candidates for the second sequence using the first sequence and the lexical constraints for each of the multiple lexical constraints output from the input generation unit ,
A generating apparatus comprising: a reranking unit that calculates a score indicating the appropriateness of the second series for each of the one or more candidates obtained for each of the multiple lexical constraints . The generation apparatus according to claim 1, wherein the reranking unit calculates the score based on at least one of the likelihood output by the model used to generate the candidates in the sequence generation unit and the likelihood obtained from the candidates by the reranking model. The generation apparatus according to claim 1, wherein, when the constraint information list includes constraint information with ambiguity, the sequence generation unit generates one or more candidates by performing a beam search with vocabulary constraints that take ambiguity into consideration. The generation apparatus according to claim 1, wherein at least one constraint information is input to the input generation unit in a format that allows for two or more ambiguities, and the input generation unit generates lexical constraints while retaining the ambiguities. The aforementioned constraint information is a constraint phrase, and the aforementioned constraint information list is a constraint phrase list.
The generating apparatus according to claim 1. The aforementioned constraint information list includes a constraint phrase list in which multiple target language phrases correspond to a given source language phrase.
The generating apparatus according to claim 5. A generation method performed by a generation device for generating a second sequence of information from constraint information and a first sequence of information, the second sequence of information being another sequence of information,
An input generation step that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint,
A sequence generation step in which, for each of the multiple lexical constraints output by the input generation step , one or more candidates for the second sequence are generated using the first sequence and the lexical constraints,
A generation method comprising: a reranking step of calculating a score indicating the appropriateness of the second series for each of the one or more candidates obtained for each of the multiple lexical constraints . A program for causing a computer to function as a component in the generating apparatus described in any one of claims 1 to 6 .