JP6413659B2 - Fuzzy word determination apparatus, fluctuation word determination method, fluctuation word determination program, and document analysis apparatus - Google Patents

Description

  The present invention relates to a shake word determination device that extracts a written shake from a document written using a natural language, for example.

  A system for extracting notation fluctuations contained in a document by analyzing a document described using a natural language by an information processing apparatus has been developed.

  The notation fluctuation detection device disclosed in Patent Literature 1 includes a term extraction unit, a similarity calculation unit, a notation shake candidate determination unit, and a group classification unit. The term extraction unit extracts a term from a document composed of electronic data according to a procedure such as morphological analysis and character surface analysis. Next, the similarity calculation unit selects any two terms from the extracted terms, calculates the edit distance between the two selected terms, and the degree of similarity between the terms. The degree of similarity representing is calculated. Next, the notation fluctuation candidate determination unit determines a notation fluctuation candidate based on the calculated similarity. Thereafter, the group classification unit groups the notation fluctuation candidates with reference to a character string common among the notation fluctuation candidates and a similarity between the notation fluctuation candidates.

  The different notation acquisition device disclosed in Patent Literature 2 includes a term pair storage unit, a learning data storage unit, a feature acquisition unit, a machine learning unit, and an output unit. First, the feature acquisition unit, for each term pair (pair) stored in the term pair storage unit, if the feature of the term pair matches the feature feature stored in the learning data storage unit, To get. The features stored in the learning data storage unit constitute the character type related features related to the character type of the edited portion, which is a character with different term pairs, the dictionary related features that are the features acquired using the term dictionary, and the term pairs. It consists of one or more of similarity features indicating the similarity of two terms. The feature is associated with positive / negative information indicating whether the term pair is different notation. Next, in accordance with the supervised learning method, the machine learning unit determines whether each term pair in the term pair storage unit is a differently expressed term pair, and outputs a determination result to the output unit.

  The notation fluctuation detection device disclosed in Non-Patent Document 1 is a notation fluctuation among pairs of compound word pairs included in a document that have close similarity in character strings and are biased in appearance frequency in the document. A pair excluding a pattern that does not apply to a pattern that is not is detected as a pair of fluctuations.

JP 2012-256197 A Japanese Patent No. 5470620

Prototype and evaluation of a tool that detects notation fluctuations in specification-specific expressions, Ayako Kuno, Eiji Hirao, Kazuki Tamura, Daihiro Yoshikawa, Takeshi Furuhashi, 76th National Convention of Information Processing Society of Japan, 2014

  However, the notation fluctuation detection device disclosed in Patent Document 1 detects a pair of terms having similar meanings but different meanings of character strings. As a result, there is a problem that the accuracy of the notation fluctuation detection device is lowered. For example, a pair of terms “input information” and “output information” are similar to each other as a character string, but are not distorted.

  Regarding the different notation acquisition device disclosed in Patent Literature 2, there is a problem that positive / negative judgments are often different depending on the context, particularly for the dictionary-related features among the features stored in the learning data storage unit. In other words, whether or not a pair of terms has similar meanings in the dictionary and whether or not they have different notations have low relevance. For example, the pair of terms “retained data” and “stored data” are similar in terms of the dictionary, but depending on the setting of the document term, there may be cases where the notation is distorted or not. is there. Even in the same document, even if the meanings are similar in the dictionary, there are cases where the notation is distorted or not. There is usually no consistent tendency between the spelling and the meaning in the dictionary. Therefore, the process in the different notation acquisition apparatus is highly likely to be substantially the same as the process of determining the different notation using only the character type related feature and the term related feature.

  The notation fluctuation detection tool disclosed in Non-Patent Document 1 refers to a dictionary storing idiom pairs that are not notation fluctuations when removing patterns that are not notation fluctuations. For this reason, there is a problem that it takes time to create the dictionary.

  A main object of the present invention is to provide a shake word determination device and the like that can accurately extract an expression that causes a shake in a document.

  In order to achieve the above-described object, in one aspect of the present invention, a swing word determination apparatus includes the following configuration.

That is, the shake word determination device
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string Differential extraction means for creating a second set by associating the third character string with the fourth character string, and associating the second set with the first set;
Selection means for determining whether or not the second set is a fuzzy word representing a fluctuation in notation based on the number of the first sets associated with the second set.

Further, as another aspect of the present invention, the swing word determination method is:
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string And creating a second set by associating the third character string with the fourth character string, associating the second set with the first set,
Based on the number of the first set associated with the second set, it is determined whether or not the second set is a fuzzy word representing a fluctuation in notation.

  Furthermore, this object is also realized by such a shake word determination program and a computer-readable recording medium for recording the program.

  According to the shake word determination device and the like according to the present invention, there is an effect that it is possible to accurately extract an expression that becomes a notation shake in a document.

It is a block diagram which shows an example of a structure which the document analyzer which concerns on the 1st Embodiment of this invention has. It is a flowchart which shows the operation | movement in the document analyzer which concerns on the 1st Embodiment of this invention. It is a figure which represents conceptually an example of notation fluctuation candidate pair information, and an example of difference pair information. It is a figure which represents notionally the example of notation fluctuation candidate pair information to which the frequency | count of appearance in the input document of the notation fluctuation candidate pair was provided to the list of notation fluctuation candidate pairs, and an example of difference pair information. It is a figure which expresses an example of difference pair information notionally. It is a figure showing notionally an example of the written fluctuation word pair which a written fluctuation word extraction part extracts. It is a figure showing an example of the user interface which displays the information which an output part outputs. It is a figure showing an example of the user interface which displays the information which an output part outputs. It is a figure showing an example of the user interface which displays the information which an output part outputs. It is a block diagram which shows an example of the structure which the fluctuation word determination apparatus which concerns on the 2nd Embodiment of this invention has. It is a flowchart which shows the flow of the process in the fluctuation word determination apparatus which concerns on 2nd Embodiment. It is a block diagram which shows roughly the hardware structural example of the calculation processing apparatus which can implement | achieve the fluctuation word determination apparatus which concerns on each embodiment of this invention.

  Next, embodiments for carrying out the present invention will be described in detail with reference to the drawings. Note that the document analysis apparatus according to the first embodiment can be realized using the shake word determination apparatus according to the second embodiment.

<First Embodiment>
FIG. 1 is a block diagram showing an example of the configuration of the document analysis apparatus 10 according to the first embodiment of the present invention. In FIG. 1, the constituent elements indicated by solid lines represent constituent elements of the document analysis apparatus 10 according to the first embodiment. In addition, the constituent elements indicated by dotted lines represent constituent elements that the document analysis apparatus 10 according to the first embodiment may have.

  The document analysis apparatus 10 includes an input unit 101, a morpheme dictionary 102, a written fluctuation candidate extraction unit 103, a difference extraction unit 104, a non-vibrated word selection unit 105, a written fluctuation word extraction unit 106, and an output unit 107.

  The input unit 101 receives a document (document data, input document) that is a target for extracting a notation fluctuation.

  The morpheme dictionary 102 is a dictionary (information) referred to in the morpheme analysis process, and is a dictionary in which at least a word and its part of speech are associated with each other. In the morpheme dictionary 102, the word and reading information indicating the reading of the word may be associated with each other.

  The notation fluctuation candidate extraction unit 103 follows a pair of candidates (that is, one or more words, character strings, compound words) that are presumed to be notation fluctuation from the input document, for example, according to an extraction procedure described later. Hereinafter, this is expressed as “notation fluctuation candidate pair”). Hereinafter, the candidate is referred to as “notation fluctuation candidate”.

  The extraction procedure for extracting the notation fluctuation candidate pair may be, for example, the procedure described in Non-Patent Document 1.

  In the extraction procedure for extracting the notation fluctuation candidate pair, for example, by referring to the morpheme dictionary 102, the input document is subjected to morphological analysis processing, whereby the sentence in the input document is converted into a word (morpheme, that is, the smallest character string that makes sense). And the part of speech of the word is specified. Next, in the extraction procedure, a compound word is extracted by connecting words obtained by consecutively specified parts of speech among the obtained words and the parts of speech of the words. When there are a plurality of designated parts of speech, compound words are extracted by connecting words in which any of the designated parts of speech are connected. Then, two arbitrary compound words are taken out from the compound word list as a pair, and expressed for each pair based on the edit distance, the similarity of the character string, or the degree of deviation of the appearance frequency of the compound word, etc. Calculate the degree of shaking. Next, in the extraction procedure, compound word pairs whose degree of notation fluctuation is greater than or equal to the specified threshold are extracted as notation fluctuation candidate pairs.

  The difference extraction unit 104 extracts a constituent word representing a difference or a character string representing a difference in the notation fluctuation candidate pair extracted by the notation fluctuation candidate extraction unit 103, and associates the extracted constituent words (character strings) with each other. To create a difference pair. Next, the difference extraction unit 104 associates and holds the difference pair and a notation fluctuation candidate pair that is a base from which the difference pair is extracted. Here, the constituent word represents a morpheme constituting the compound word.

  The non-swaying word selection unit 105 counts the number of types of notation-swaying candidate pairs associated with a certain difference pair in the input document. Next, when the number of types is equal to or greater than a predetermined threshold value, the non-swaying word selection unit 105 selects the difference pair as a pair of non-swaying words (hereinafter referred to as “non-swaying word pair”). Here, the non-swaying word pair represents a pair of words not corresponding to the notation shaking. Further, the non-blurred word selection unit 105 has a function of not selecting as a non-blurred word pair when a specific condition is satisfied even if the number of types of written variation candidate pairs associated with the difference pair is equal to or greater than a threshold value. May be. The specific condition is, for example, when one word in the written variation candidate pair includes the other word, when the readings of the two words in the written variation candidate pair match, and in the two variations in the written variation candidate pair. This is the case when words in the dictionary have similar meanings. In addition, the specific condition may be, for example, a case where there is a large deviation in the number of appearances in the document regarding two words in the notation fluctuation candidate pair associated with the difference pair.

  The written fluctuation word extraction unit 106, for example, excludes a non-fluctuated word pair from the written fluctuation candidate pairs extracted by the written fluctuation candidate extraction unit 103, thereby presuming a written fluctuation word pair (hereinafter referred to as “written fluctuation”). "Word pair").

  The output unit 107 outputs the written fluctuation word pair extracted by the written fluctuation word extraction unit 106 according to, for example, a mode (user interface, UI) that can be identified by the user. The UI includes, for example, a list display that can be recognized by the user, information provision to an external device, and the like.

  Processing in the document analysis apparatus 10 according to the first embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the document analysis apparatus 10 according to the first embodiment of the present invention.

  The input unit 101 receives a document (input document) from which notation fluctuation is extracted (step S101).

  The notation fluctuation candidate extraction unit 103 extracts a notation fluctuation candidate pair from the input document in accordance with the extraction procedure of the notation fluctuation candidate pair as described above (step S102). The difference extraction unit 104 extracts a constituent word representing a difference or a character string representing a difference for each notation fluctuation candidate pair extracted by the notation fluctuation candidate extraction unit 103, and extracts the extracted constituent words (character strings) from each other. By creating a difference pair, a difference pair is created (step S103). Next, the difference extraction unit 104 associates and holds the difference pair and the notation fluctuation candidate pair that is a base from which the difference pair is extracted.

  The non-swaying word selection unit 105 selects, as a non-swaying word pair, a notation-swaying candidate pair in which the number of types of the associated notation-swaying candidate pairs is equal to or greater than a specified threshold among the difference pairs. (Step S104). In this case, the non-blurred word selection unit 105 does not select a non-blurred word pair when a specific condition is met even if the number of types of written variation candidate pairs associated with a certain difference pair is equal to or greater than a threshold value. May be.

  For example, the written fluctuation word extraction unit 106 extracts a written fluctuation word pair by excluding the non-vibration word pair from the written fluctuation candidate pair extracted by the written fluctuation candidate extraction unit 103 (step S105).

  The output unit 107 outputs the extracted written fluctuation word pair (step S106). The output mode in step S106 may be, for example, a list display that can be recognized by the user, information provision to an external device, or the like.

  Next, processing according to the first embodiment of the present invention will be described with reference to specific examples shown in FIGS. 3, 4, 5, 6, 7, and 8.

  FIG. 3 is a diagram conceptually illustrating an example of notation fluctuation candidate pair information and an example of difference pair information. In the notation fluctuation candidate pair information T1 in FIG. 3, the notation fluctuation candidate pair extracted by the notation fluctuation candidate extraction unit 103 is stored. The difference extraction unit 104 creates a constituent word by performing a morphological analysis process on the notation fluctuation candidate pair. Next, the difference extraction unit 104 compares character strings in units of constituent words, and extracts different constituent words as differences. For example, in the first line of the notation fluctuation candidate pair information T1, notation fluctuation candidate pairs of “return processing” and “return processing” are stored.

  In this case, the difference extraction unit 104 extracts the difference between “return” and “return” by comparing the constituent words included in the notation fluctuation candidate pair. Alternatively, for example, the difference extraction unit 104 may extract the difference between “payment” and “product” by comparing the notation fluctuation candidate pairs in character string units based on character codes. Then, the difference extraction unit 104 creates a difference pair by associating the constituent words (character strings) extracted as differences with each other. Further, the difference extraction unit 104 associates the difference pair with a notation fluctuation candidate pair that is a base from which the difference pair is extracted. The difference extraction unit 104 similarly extracts difference pairs for other notation fluctuation candidates.

  The non-sway word selection unit 105 selects, as a non-sway word pair, a pair in which the number of types of notation swing candidate pairs associated with the difference pair is equal to or greater than a specified threshold. In the example shown in FIG. 3, the notation fluctuation candidate pair associated with the “return / return” difference pair includes “return process / return process”, “return information / return information”, and “partial return / one return”. There are three types of "part return". The non-swaying word selection unit 105 counts the number of types of notation-swaying candidate pairs that are similarly associated with other difference pairs. For example, when the threshold is 2 and the number of types is 2 or more, the non-sway word selection unit 105 selects a non-sway word pair. In the example shown in FIG. 3, the non-blurred word selection unit 105 selects a difference pair of “return / return”, “design / installation”, and “actual / result” as a non-blurred word pair.

  In addition, the non-blurred word selection unit 105 may not select a non-blurred word pair when a specific condition is met even if the number of types of written variation candidate pairs associated with the difference pair is equal to or greater than a threshold value. . 4 shows an example of notation fluctuation candidate pair information T2 in which the number of appearance fluctuation candidate pairs in the input document is added to the list of notation fluctuation candidate pairs similar to the notation fluctuation candidate pair information T1 shown in FIG. It is a figure which expresses an example of pair information notionally. For example, if the specific condition is “when one word includes the other word in a difference pair, it is not selected as a non-flicker word pair”, among the difference pairs in FIG. “Money” is not selected as a non-blurred word pair because it satisfies the specific condition.

  For example, if the specific condition is “not selected as a non-swaying word pair when the readings of two words match in a difference pair”, among the difference pairs illustrated in FIG. The “decision” and “settlement / clearing” are not selected as non-blurred word pairs. Note that reading is defined as reading information in a morpheme dictionary, for example.

  A case will be described in which the specific condition is “when a difference pair has similar meanings in a dictionary of two words, it is not selected as a non-blurred word pair”. In this case, further, general concept information representing a relationship between a plurality of words based on upper / lower relations, partial / whole relations, synonym relations (synonyms), synonym relations (synonyms), usages, and the like regarding words. Based on the stored concept dictionary, it may be determined whether or not it is a non-blurred word pair. For example, in the concept dictionary, when “use / use” is related to each other based on the meanings of synonyms and synonyms, the non-swaying word selection unit 105 sets “use / use” as non-use. Not selected as a twisted word pair.

When the specific condition is “when the number of occurrences of the notation-swing candidate pair associated with the difference pair is biased, it is not selected as a non-swaying word pair”, and each notation shake included in the notation-swaying candidate pair Evaluate the bias in the number of appearances of candidates. For example, when only one notation fluctuation candidate pair is associated with a difference pair, the non-sway word selection unit 105 evaluates the deviation of the number of appearances of each notation fluctuation candidate included in the notation fluctuation candidate pair. That's fine. When a plurality of notation fluctuation candidate pairs are associated with the difference pair, the process for evaluating the deviation of the number of appearances is, for example, the process shown in (1) or (2) described later. That is,
(1) Evaluate the bias based on the cumulative number of occurrences of the notation fluctuation candidate included in the “notation fluctuation candidate pair associated with a certain difference pair”.
(2) Evaluating the deviation of the number of appearances for each type of “notation fluctuation candidate pair associated with a certain difference pair”.

The procedure for evaluating the deviation of the number of appearances is, for example, the processes shown in (3) to (5) described later. That is,
(3) Among the notation fluctuation candidate pairs associated with the difference pair, when the number of appearances of one notation fluctuation candidate is equal to or less than a predetermined threshold and the number of appearances of the other notation fluctuation candidate is equal to or more than a predetermined threshold To determine that there is a bias,
(4) In the notation fluctuation candidate pair associated with the difference pair, the ratio of the number of occurrences of the notation fluctuation candidate with the smaller number of appearances to the appearance frequency of the notation fluctuation candidate with the larger number of appearances is equal to or less than the threshold value To determine that there is a bias,
(5) According to a method in which the determination methods shown in (3) and (4) above are combined (for example, at least one of them is satisfied), it is determined whether there is a bias.

  In a notation fluctuation candidate pair associated with a certain difference pair, when the number of appearances of one notation fluctuation candidate is 2 or less and the number of appearances of the other notation fluctuation candidate is 20 or more, there is a deviation in the number of appearances. A specific calculation example in the case where it is determined that there is one will be described.

  When a plurality of notation fluctuation candidates are associated with a difference pair, when the deviation of the number of appearances is calculated according to the process shown in (1) above, first, there are a plurality of notation fluctuation candidates for the difference pair. Find the number of occurrences of the case.

  The difference pair information T3 illustrated in FIG. 5 shows the result of accumulating the number of appearances of the variation candidates for each difference pair of the notation variation candidate pair information T2 illustrated in FIG. FIG. 5 is a diagram conceptually illustrating an example of difference pair information. When the difference pair information T3 is referred to, the number of occurrences of the notation fluctuation candidate included in the notation fluctuation candidate pair information that is the basis for extracting the difference pair is biased. The difference pair “clearing / settlement” indicated by R24 And a difference pair “result / result” indicated by R1. Therefore, “clearing / settlement” and “actual result / score” are excluded from the non-blurred word pair.

  When a plurality of notation fluctuation candidate pairs are associated with the difference pair, it is determined that there is a deviation in the notation fluctuation candidate pair information T2 when the appearance frequency deviation is calculated according to the process shown in (2) above. The notation variation candidate pair R5, R9, R10, or R11. The notation fluctuation candidate pair R5 is associated with the difference pair “settlement / clearing”. The notation fluctuation candidate pair R9 is associated with the difference pair “actual result / score”. The notation fluctuation candidate pair R10 is associated with the difference pair “settlement / decision”. The notation fluctuation candidate pair R11 is associated with the difference pair “actual result / score”.

  The notation fluctuation candidate pairs associated with the difference pair “settlement / decision” are the notation fluctuation candidate pair R2 and the notation fluctuation candidate pair R10. Among these notation fluctuation candidate pairs, the number of occurrences of the notation fluctuation candidates included in the notation fluctuation candidate pair R10 is biased.

  In addition, the notation fluctuation candidate pairs associated with the difference pair “actual / score” are the notation fluctuation candidate pair R9 and the notation fluctuation candidate pair R11. Both of these notation fluctuation candidate pairs are biased in the number of appearances of the notation fluctuation candidates included in the notation fluctuation candidate pair.

  When there is a bias of all the notation fluctuation candidate pairs associated with a certain difference pair, the difference pair may not be selected as a non-vibration word pair. Alternatively, when the deviation of the number of occurrences of the notation fluctuation candidate in the notation fluctuation candidate pair is equal to or greater than the specified threshold value, the difference pair associated with the notation fluctuation candidate may not be selected as the non-shake word pair.

  In the former case, the difference pair “settlement / decision” may be selected as a non-blurred word pair. Further, the difference pair “actual result / score” is not selected as a non-blurred word pair.

  In the latter case, when the threshold value is 1, if the deviation of the number of occurrences of the notation fluctuation candidate in the notation fluctuation candidate pair associated with the difference pair is even one, it is not selected as a non-vibration word pair. Therefore, in the above-described example, the difference pair “settlement / decision” and the difference pair “actual result / score” are not selected as the non-blurred word pair.

  FIG. 6 is a diagram conceptually illustrating an example of a written fluctuation word pair extracted by the written fluctuation word extraction unit 106. For example, the notation fluctuation word extraction unit 106 excludes a notation fluctuation candidate pair including a non-sway word pair from the notation fluctuation candidates included in the notation fluctuation candidate pair information T1 illustrated in FIG. 3 as a notation fluctuation word pair. Extract. For example, as illustrated in FIG. 6, the output unit 107 displays a list of the written fluctuation word pairs.

  FIG. 7 is a diagram illustrating an example of a user interface (UI) that displays information output by the output unit 107. In the UI illustrated in FIG. 7, the output unit 107 displays a list of notation fluctuation word pairs, and the notation fluctuation words included in the notation fluctuation word pair are sentences (or sentence sentences) used in the input document. , Part of the sentence, and a sentence including a part of the sentence). The output unit 107 may highlight the character string according to a mode in which the character string corresponding to the written fluctuation included in the written fluctuation word pair in the sentence is bolded or set to a different color from others. Good. In addition, the output unit 107 may display the number of appearances in the input document together with the highlighted display for each written fluctuation word.

  By referring to the UI as illustrated in FIG. 7, the user can easily check the written fluctuation word. The reason for this is that, in the UI, the written fluctuation word and a sentence including the written fluctuation word are displayed in association with each other. By highlighting the written fluctuation word on the UI, the user can more easily check the written fluctuation word.

  Another example relating to the output unit 107 is shown in FIG. FIG. 8 is a diagram illustrating an example of a user interface that displays information output by the output unit 107. In the UI example illustrated in FIG. 8, the output unit 107 displays information in which a non-swaying word pair and a notation-swaying candidate word are associated with each other. The output unit 107 further displays a check box that allows the user to select a pair that is not selected as a non-blurred word pair from the non-blurred word pairs. The non-swaying word selection unit 105 does not select a non-swaying word pair for the non-swaying word pair checked by the check box shown in the “exclusion” column.

  Another example of the output unit 107 is shown in FIG. FIG. 9 is a diagram illustrating an example of a user interface that displays information output by the output unit 107. In the example shown in FIG. 9, the output unit 107 displays an input document, and further highlights a written fluctuation word in the input document according to a mode such as changing the font thickness. Further, when there is a swing word pair including the written swing word, the output unit 107 uses a form such as a speech bubble associated with the swing word to use the other speech language pair included in the speech word pair. The written fluctuation word and the number of appearances of the written fluctuation word are displayed. In the UI illustrated in FIG. 9, the output unit 107 displays a speech bubble only for a shake word with a small number of appearances, but may display a speech bubble for a shake word with a large number of appearances.

  By referring to the UI as illustrated in FIG. 9, the user can easily check the written fluctuation word. This is because, with respect to the written fluctuation word included in the document, a phrase that is paired with the written fluctuation word is displayed in a form such as a balloon.

  As described above, according to the document analysis apparatus 10 according to the present embodiment, it is possible to accurately extract an expression that causes a notation. This is because the document analysis apparatus 10 selects a non-swaying word pair from the notation-swaying candidate pairs based on the number of types of the notation-swaying candidates.

  The document analysis apparatus 10 extracts a pair of notation fluctuation candidate words that is a combination of two compound words that are similar to each other, and calculates a difference between the notation fluctuation candidate words included in the pair from the notation fluctuation candidate word pair. Are extracted, and the extracted difference is associated as a difference pair. Next, when the number of types of notation fluctuation candidates attached to the difference pair is equal to or greater than a predetermined threshold, the document analysis apparatus 10 extracts the difference pair as a non-blurred word pair.

  Therefore, according to the document analysis apparatus and the like according to the present embodiment, it is possible to accurately extract an expression that causes a notation fluctuation in a document by a process of excluding a compound word including a non-waving word from a notation fluctuation candidate word. There is an effect.

<Second Embodiment>
Next, the shake word determination apparatus 201 according to the second embodiment will be described with reference to FIGS. 10 and 11. FIG. 10 is a block diagram illustrating an example of the configuration of the shake word determining apparatus 201 according to the second embodiment of the present invention. FIG. 11 is a flowchart showing the flow of processing in the shake word determining apparatus 201 according to the second embodiment.

  The shake word determination apparatus 201 according to the second embodiment includes a difference extraction unit 202 and a selection unit 203.

  The fluctuation word determination apparatus 201 receives the first set 301.

  In the first group 301, for example, among character strings in which words of a specific part of speech are connected, character strings similar to each other are associated with each other. For example, as exemplified in the first embodiment, a character string is obtained by obtaining a word by performing morphological analysis processing on an input document and extracting a word having a specific part of speech. For character strings that are similar to each other, for example, for a plurality of character strings, a degree of similarity that represents the degree of similarity between character strings is obtained using an edit distance or the like, and depending on whether the degree of similarity is a predetermined value or more. Can be obtained.

  Hereinafter, for convenience of explanation, it is assumed that the first character string and the second character string are associated with each other in the first group 301.

  The difference extraction unit 202 reads the first character string and the second character string included in the first set 301, and obtains a difference between the first character string and the second character string (step S201). Here, the character strings representing the differences are represented as a third character string and a fourth character string. The third character string is a character string that is included in the first character string but not included in the second character string. The fourth character string is a character string that is included in the second character string but not included in the first character string. That is, the difference extraction unit 202 extracts the third character string and the fourth character string from the first character string and the second character string.

  Next, the difference extraction unit 202 creates a second set by associating the third character string and the fourth character string (step S202). Next, the difference extraction unit 202 associates the second set with the first set 301 (step S203).

  The difference extraction unit 202 performs the above-described processing regarding the plurality of first sets 301.

  Next, the selection unit 203 obtains the number of first sets associated with the second set. The selection unit 203 determines whether the second set is a non-blurred word pair based on the obtained number (step S204). For example, when the number of first sets associated with the second set is equal to or greater than a predetermined number (for example, 2), the selection unit 203 determines that the second set is a non-blurred word pair (ie, , It is determined that it is not a spelling word pair). In addition, when the number of the first sets associated with the second set is less than the predetermined number, the selection unit 203 does not select the second set as a non-swing word pair (that is, a written swing word pair). ).

  That is, the selection unit 203 determines that the third character string and the fourth character string included in the second group determined to be a non-blurred word pair are non-blurred words. In addition, the selection unit 203 determines that the third character string and the fourth character string included in the second set determined to be a written fluctuation word pair are written fluctuation words. The selection unit 203 outputs the determined result as the determination result 302.

  According to the shake word determination device 201 according to the present embodiment, it is possible to accurately extract an expression that becomes a notation shake. The reason for this is that when the number of types of character strings including written fluctuation words is less than the predetermined number, there is a high possibility that they are written fluctuation words, and the number of types of character strings including written fluctuation words is more than the predetermined number. This is because, in many cases, there is a high possibility that it is not a written fluctuation word.

  The fluctuation word determination apparatus 201 determines that the second set is not a non-fluctuation word pair when the number of types of character strings including the fluctuation word is smaller than a predetermined number. Further, the swing word determination device 201 determines that the second set is a non-sway word pair when the number of types of character strings including the swing word is greater than a predetermined number. Therefore, according to the shake word determination device 201 according to the present embodiment, it is possible to accurately extract an expression that becomes a notation shake.

  Note that the difference extraction unit 202 can be realized by using the function of the difference extraction unit 202. Further, the selection unit 203 can be realized by using the function of the non-sway word selection unit 105.

(Hardware configuration example)
A configuration example of hardware resources that implements the document analysis apparatus according to each embodiment of the present invention described above using one calculation processing apparatus (information processing apparatus, computer) will be described. However, the document analysis apparatus may be realized using at least two calculation processing apparatuses physically or functionally. The document analysis apparatus may be realized as a dedicated apparatus.

  FIG. 12 is a diagram schematically illustrating a hardware configuration example of a calculation processing device capable of realizing the shake word determination device according to the first embodiment or the second embodiment. The computing device 20 includes a central processing unit (Central Processing Unit, hereinafter referred to as “CPU”) 21, a memory 22, a disk 23, and a nonvolatile recording medium 24. The calculation processing device 20 further includes an input device 25, an output device 26, a communication interface (hereinafter referred to as “communication IF”) 27, and a display 28. The calculation processing device 20 can transmit / receive information to / from other calculation processing devices and communication devices via the communication IF 27.

  The nonvolatile recording medium 24 is, for example, a compact disc (Compact Disc) or a digital versatile disc (Digital_Versatile_Disc) that can be read by a computer. The nonvolatile recording medium 24 may be a universal serial bus memory (USB memory), a solid state drive (Solid_State_Drive), or the like. The non-volatile recording medium 24 retains such a program without being supplied with power, and can be carried. The nonvolatile recording medium 24 is not limited to the above-described medium. Further, the program may be carried via the communication network via the communication IF 27 instead of the nonvolatile recording medium 24.

  That is, the CPU 21 copies a software program (computer program: hereinafter simply referred to as “program”) stored in the disk 23 to the memory 22 when executing it, and executes arithmetic processing. The CPU 21 reads data necessary for program execution from the memory 22. When the display is necessary, the CPU 21 displays the output result on the display 28. When output to the outside is necessary, the CPU 21 outputs an output result to the output device 26. When inputting a program from the outside, the CPU 21 reads the program from the input device 25. The CPU 21 interprets and executes the fluctuation word determination program (FIG. 2 or FIG. 11) in the memory 22 corresponding to the function (process) represented by each unit shown in FIG. 1 or FIG. The CPU 21 sequentially performs the processes described in the above-described embodiments of the present invention.

  That is, in such a case, it can be understood that the present invention can also be achieved by such a shake word determination program. Furthermore, it can be understood that the present invention can also be realized by a computer-readable non-volatile recording medium in which such a shake word determination program is recorded.

  The present invention has been described above using the above-described embodiment as an exemplary example. However, the present invention is not limited to the above-described embodiment. That is, the present invention can apply various modes that can be understood by those skilled in the art within the scope of the present invention.

In addition, a part or all of each embodiment mentioned above can be described also as the following additional remarks. However, the present invention described by way of example with the above-described embodiments is not limited to the following. That is,
(Appendix 1)
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string Differential extraction means for creating a second set by associating the third character string with the fourth character string, and associating the second set with the first set;
A swing word determination device comprising: selection means for determining whether the second pair is a swing word representing a swing in notation based on the number of the first pair associated with the second pair. .

(Appendix 2)
Even if the selection means determines that the second set is not the fuzzy word, the third character string includes the fourth character string, or the fourth character string is the third character string. The fluctuation word determination device according to appendix 1, wherein when the character string is included, the fluctuation word is determined.

(Appendix 3)
Even if the selection means determines that the second set is not the fuzzy word, a reading calculated by performing a morphological analysis process on the third character string and a morphological analysis on the fourth character string The fluctuation word determination apparatus according to Supplementary Note 1 or Supplementary Note 2, wherein when the reading calculated by the processing matches, the fluctuation word is determined.

(Appendix 4)
The selection means associates the third character string and the fourth character string in related information associating synonyms and synonyms with each other even when the second set is determined not to be the swing word. If it is, the swing word determination device according to any one of supplementary notes 1 to 3 is determined to be the swing word.

(Appendix 5)
A calculation means for obtaining a first number of times the first character string appears in the document and a second number of times the second character string appears in the document;
The selection function determines the third character string associated with the second set with respect to the third character string included in the second set, even when the second set is determined not to be the swing word. The third number is obtained by accumulating the first number of first character strings included in one set, and the fourth character string included in the second set is associated with the second set. The fourth number is obtained by accumulating the second number of second character strings included in the first group, and the second group is further determined based on the third number and the fourth number. The swing word determination device according to any one of appendix 1 to appendix 4, wherein whether or not is a swing word is determined.

(Appendix 6)
The selection means determines that the number of the first set is not a fluctuation word when the number of the first set is equal to or greater than a predetermined number, and the number of the fluctuations when the number of the first set is less than the predetermined number. The fluctuation word determination device according to any one of Supplementary Note 1 to Supplementary Note 5, which is determined to be a word.

(Appendix 7)
The fluctuation word judging device according to any one of appendices 1 to 6;
Based on the morpheme information in which the word and the part of speech are associated, the document is subjected to morphological analysis processing, based on the calculated part of speech, the character string including the words of the specific part of speech is extracted, and between the obtained character strings And extracting the first character string and the second character string on the basis of the degree of similarity representing the degree of similarity and associating the first character string with the second character string. Creating means to create
And a display unit for displaying at least a part of the document.

(Appendix 8)
The display means includes a portion of the first character string and the second character string included in the first group that is associated with the second group that is determined by the swing word determination device to be the swing word. The document analysis apparatus according to appendix 7, wherein the document is displayed in a manner that emphasizes.

(Appendix 9)
The display means includes the first character string and the second character string included in the first group associated with the second group determined by the fluctuation word determination device as not being the fluctuation word, The document analysis apparatus according to appendix 7 or appendix 8, wherein the third character string and the fourth character string included in the second set are displayed in association with each other.

(Appendix 10)
The display means includes the first character string included in the first set associated with the second set determined by the shake word determination device as the shake word, and the first set. The document analysis device according to any one of appendix 7 to appendix 9, wherein the document is displayed by adding the second character string included therein.

(Appendix 11)
Information processing device
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string And creating a second set by associating the third character string with the fourth character string, associating the second set with the first set,
A variation word determination method for determining whether or not the second set is a variation word representing a variation in notation based on the number of the first group associated with the second group.

(Appendix 12)
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string A difference extraction function for creating a second set by associating the third character string with the fourth character string, and associating the second set with the first set;
A selection function for determining whether or not the second set is a fuzzy word representing a fluctuation in notation based on the number of the first sets associated with the second set. Word determination program.

DESCRIPTION OF SYMBOLS 10 Document analyzer 101 Input part 102 Morphological dictionary 103 Notation fluctuation candidate extraction part 104 Difference extraction part 105 Non-sway word selection part 106 Notation shake word extraction part 107 Output part 201 Sway word judgment apparatus 202 Difference extraction part 203 Selection part 301 1st Group 302 determination result 20 calculation processing device 21 CPU
22 Memory 23 Disk 24 Non-volatile recording medium 25 Input device 26 Output device 27 Communication IF
28 Display

Claims (10)

With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string Differential extraction means for creating a second set by associating the third character string with the fourth character string, and associating the second set with the first set;
A swing word determination device comprising: selection means for determining whether the second pair is a swing word representing a swing in notation based on the number of the first pair associated with the second pair. . Even if the selection means determines that the second set is not the fuzzy word, the third character string includes the fourth character string, or the fourth character string is the third character string. The fluctuation word determination device according to claim 1, wherein when it includes a character string, the fluctuation word is determined to be the fluctuation word. Even if the selection means determines that the second set is not the fuzzy word, a reading calculated by performing a morphological analysis process on the third character string and a morphological analysis on the fourth character string The fluctuation word determination device according to claim 1, wherein the word is determined to be the fluctuation word when the reading calculated by processing matches. The selection means associates the third character string and the fourth character string in related information associating synonyms and synonyms with each other even when the second set is determined not to be the swing word. The swing word determination apparatus according to claim 1, wherein if it is determined, the swing word is determined to be the swing word. The fluctuation word judging device according to any one of claims 1 to 4,
Based on the morpheme information in which the word and the part of speech are associated, the document is subjected to morphological analysis processing, based on the calculated part of speech, the character string including the words of the specific part of speech is extracted, and between the obtained character strings And extracting the first character string and the second character string on the basis of the degree of similarity representing the degree of similarity and associating the first character string with the second character string. Creating means to create
And a display unit for displaying at least a part of the document. The display means includes a portion of the first character string and the second character string included in the first group that is associated with the second group that is determined by the swing word determination device to be the swing word. The document analysis apparatus according to claim 5, wherein the document is displayed in a manner that emphasizes. The display means includes the first character string and the second character string included in the first group associated with the second group determined by the fluctuation word determination device as not being the fluctuation word, The document analysis apparatus according to claim 5, wherein the third character string and the fourth character string included in the second set are displayed in association with each other. The display means includes the first character string included in the first set associated with the second set determined by the shake word determination device as the shake word, and the first set. The document analysis apparatus according to any one of claims 5 to 7, wherein the document is displayed by a mode in which the second character string included is added. Information processing device
With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string And creating a second set by associating the third character string with the fourth character string, associating the second set with the first set,
A variation word determination method for determining whether or not the second set is a variation word representing a variation in notation based on the number of the first group associated with the second group. With respect to a first set in which a first character string in which words of a specific part of speech are connected and a second character string similar to the first character string among character strings in which the words of a specific part of speech are connected, A third character string included in the first character string and not included in the second character string, and a fourth character string included in the second character string and not included in the first character string A difference extraction function for creating a second set by associating the third character string with the fourth character string, and associating the second set with the first set;
A selection function for determining whether or not the second set is a fuzzy word representing a fluctuation in notation based on the number of the first sets associated with the second set. Word determination program.

Images