JP4772233B2 - Document data analysis program, computer-based document data analysis method, and document data analysis system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a document data analysis program for analyzing one or more input document data,ComputerThe present invention relates to a document data analysis method and a document data analysis system.
[0002]
[Prior art]
Japanese Patent Application No. 11-332114 describes an AND search after extracting individual concepts.
[0003]
In Example 1 of Japanese Patent Application No. 11-332114, a method is proposed in which after extracting concepts belonging to a plurality of classification axes, a composite concept is determined by AND search of concepts belonging to different axes.
[0004]
In Example 2 of Japanese Patent Application No. 11-332114, there is proposed a method for determining connection relations of concepts using information on conjunctions, concept information, and position information in the document.
[0005]
[Problems to be solved by the invention]
The invention of the above Japanese Patent Application No. 11-332114 mainly determines a composite concept of a concept having a causal relationship such as an action and a result.
[0006]
In order to make a correct combination of a specific product and a concept related to the product from a document whose collation is colloquial and difficult to analyze, in the first embodiment, the reproduction rate (the information that should be extracted is actually extracted). However, in the second embodiment, the reproducibility is low but the reproducibility is low.
[0007]
In addition, it is difficult to distinguish between expressions that are prone to obsolescence such as product names and expressions that can be used for a long time such as concepts and treat them as compound concepts.
[0008]
  The present invention has been made in view of the above circumstances, and document data analysis program for correctly linking a product name and an action or result to colloquial and difficult to analyze document data,ComputerAn object of the present invention is to provide a document data analysis method and a document data analysis system.
[0009]
[Means for Solving the Problems]
Specific means taken for realizing the present invention will be described below.
[0010]
  The first invention provides a computer,Stored in the databaseRefers to dictionary information that associates the document elements that make up the document and their attribute data,Acquired on the computerIncluded in the document data to be analyzed and included in the dictionary informationpluralAn extraction function that extracts document elements and their attribute data;Make a combinationMultiple attribute dataTheDefinedClassification axisWhen,Extracted by the extraction functionMultiple document elements and theirBased on attribute data andAccording to the classification axisExtracted by the extraction functionFirstDocument element orIts firstAttribute dataAnd the second document element or its second attribute dataThe combinationA summary dataThis is a document data analysis program that realizes an analysis function.The analysis function includes a plurality of document elements extracted by the extraction function and their attribute data, and the document elements and their attribute data followed by an exclusion connection expression that means comparison in the analysis target document data acquired by the computer. Except for this, the first document element or first attribute data that does not continue with the excluded connection expression is combined with the second document element or second attribute data that does not continue with the excluded connection expression according to the classification axis.
[0011]
For example, the combination method includes AND search that combines all extracted document elements or attribute data.
[0012]
  In the first inventionThe relationship of attribute data to be combined is determined in advance as a classification axis, and the document elements or attribute data extracted according to the classification axis are combined.Is supposed to.
[0013]
The analysis function may combine the extracted document elements, may combine the extracted attribute data, or may combine the extracted document elements and the extracted attribute data.
[0014]
The document element includes various words, phrases, sections, expressions included in the document, and the like.
[0015]
In addition, the attribute data can use the concept of document elements such as the name of a product (product), the distinction between a company's product and another company's product, an action, and a result.
[0016]
If this 1st invention is utilized, it can distinguish easily whether the result of the combination is a document about a company product, a document about another company's product, or a document about other concepts. For example, a document including the concept of “selling” can be determined by distinguishing whether the company's products are sold, whether the products of other companies are sold, or sold at a specific store.
[0017]
  The second invention is a document data analysis program similar to the first invention. According to the second aspect of the present invention, the computer further realizes a display function for displaying the summary data created by the analysis function.
  First3The invention of the first aspectOr secondThis is a document data analysis program similar to that of the present invention. This first3In the present invention, the analysis function isSubject to analysisThe document data to be analyzed is divided according to a predetermined document division rule for dividing the document data.,Delimited rangeOr the first existing in the delimited range and the delimited range immediately preceding it.Document element orFirstAttribute dataAnd a second document element or second attribute dataCombine. Thereby, the relevance rate of the combination result can be improved.
[0018]
  First4The invention of the above3The document division rule of the invention ofAt least one type of punctuationBased onSubject to analysisThe rule is to separate document data.
[0019]
  This first4If the invention of (2) is used, the document data is delimited by “.”. And within a delimited rangesoA combination of the extracted document elements or attribute data is performed.
[0020]
The document data may be divided based on other punctuation marks such as “,” “.”, “,” And the like without being limited to the punctuation marks.
[0021]
  A certain breakWasA range and its immediately preceding delimiterWasrangeAt the innerDocument elements or attribute data may be combined.
[0022]
  First5The invention of the above3The document division rule of the invention ofThe document data to be analyzedIt is assumed that it is a rule that separates in front of a document element associated with attribute data indicating a proper noun.
[0023]
  First6The invention of the above4The document division rule of the invention ofIn the document data to be analyzed, the excluded connection expression follows the proper noun.In case,Proper nounsIt includes rules that are not used as delimiters.
[0024]
  First7The invention of the above3The document division rule of the invention ofIn the document data to be analyzed,It is assumed that the rule separates the document element associated with the attribute data indicating the result.
[0025]
  This number7In the invention, the document division rule may be divided after the document element associated with the attribute data indicating the action.
[0026]
  Above5To the second7By using the document division rules of the inventions up to the present invention, the relevance ratio and the recall ratio can be improved.
[0027]
  the above1st to 1st7In the above inventions, the analysis function excludes the document element from the combination candidates when the document element that means comparison is attached to the document element extracted by the extraction function..
[0028]
For example, document elements that mean comparison include “to more”, “compared to”, and “vs. to ratio”.
[0029]
As a result, for example, when analyzing document data such as “Our products are selling more than other companies”, the combined result will not be “Other products—selling”, but “Own products—selling”. " Such analysis is called dependency analysis.
[0030]
The precision can be improved by using this dependency analysis.
[0031]
  In an eighth aspect based on the first to seventh aspects, the analysis function is a document element having a comparative relationship.And other document elementsRefer to the correspondence information associated withIn the document data to be analyzed,Document elements extracted by the extraction functionIf an excluded connection expression follows,thisFollowed by the excluded connection expressionDocument elementOther comparisons withDocument elements,Determined based on the correspondence information,Based on the first document element or first attribute data, the second document element or second attribute data, and the correspondence informationIt has been determinedotherDocument elementAndcombinationRuThis is a document data analysis program.
[0032]
For example, it is assumed that the correspondence information associates the company product with the other company product. Assume that an analysis is performed on document data that “sells more than other companies' products”. In this case, “other company products” with “more” meaning comparison are extracted, but since there is no comparison target of this “other company products”, “own product” is determined from the corresponding information, The result is a combination of “product-selling”. Thereby, the precision and the recall can be improved.
[0042]
  First9The invention of the first to the second8In the invention ofOn the computer to be analyzedOut of document dataDictionary informationThe function to display the part that is not includedfurtherThis is a document data analysis program to be realized.
[0043]
For example, if the official name of a product is registered in the dictionary information, but the abbreviation of the product is not registered, the user can recognize the abbreviation by presenting the user with a part that does not include the official name. . Therefore, it can be used to update document elements and attribute data.
[0044]
  First10The invention of the first to the second9In the invention ofOn the computer,A registration function that updates dictionary information according to user-specified contentsfurtherThis is a document data analysis program to be realized.
[0045]
This enables analysis that meets the needs of the user.
[0046]
By using the program as described above and a recording medium storing the program, the above-described function can be easily added to a computer system, server, client, or other computer that does not have the above-described function. it can.
[0047]
  The first to the above10The document data analysis method implemented by the program of the invention may be the subject of the invention.
[0048]
  Also, from the first to the first10The document data analysis system having means for realizing the functions realized by the program of the invention may be the subject of the invention.
[0049]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings shown below, the same portions are denoted by the same reference numerals, description thereof is omitted, and only different portions will be described in detail.
[0050]
(First embodiment)
In the present embodiment, for example, “A snack” and “B snack” for document data such as “B snack sold out at a tasting party. A snack information. Sold at 120% of C snacks.” A description will be given of a document data analysis program that correctly links expressions (document elements) of product names such as “C snack” and expressions related to actions or results such as “tasting party”, “sold out”, and “sold”.
[0051]
Further, the document data analysis program according to the present embodiment can switch the determination rule for linking expressions.
[0052]
FIG. 1 is a block diagram showing functions realized on a computer by a document data analysis program according to the present embodiment.
[0053]
The document data analysis program 1 according to the present embodiment realizes an acquisition function 2, a concept extraction function 3, an analysis function 4, a selection function 5, and a display function 6 on a computer or a computer system.
[0054]
Further, the document data analysis program 1 refers to the concept definition dictionary (database) 7 and the association table 8.
[0055]
The acquisition function 2 acquires data to be analyzed. The acquisition method includes a method of acquiring document data designated by the user from a group of document data registered in advance, a method of acquiring document data input by the user, and the like.
[0056]
The document data includes report data such as a manufacturer's business daily report. Specifically, a document such as “B snack sold out at a tasting party. A snack information. Sold to 120% of C snacks” is acquired by the acquisition function 2 as document data.
[0057]
The concept definition dictionary 7 is created based on the characteristics of the document data to be analyzed according to the user's needs.
[0058]
The concept definition dictionary 7 includes an in-house (MA company) product dictionary 9, another company (MB company to MD company) product dictionary 10, and an action / result dictionary 11.
[0059]
The in-house product dictionary 9 has a structure in which a concept (attribute data) and a concept ID are associated with an expression of the in-house product. When different expressions exist in the same product, such as when the official name “A snack” is expressed as an abbreviation “A”, the same concept and concept ID are assigned to each expression of the same product. Is attached. The concept is defined so as to specify all the hierarchies to which the expression belongs, such as “product: own product: product type: product name”.
[0060]
The same applies to the other company's product dictionary 10, but here, other companies' products are defined instead of their own products.
[0061]
The action / result dictionary 11 has a structure in which a concept is associated with a concept ID. The concept is defined for each expression to specify “designation of action or result: semantic content of the expression”.
[0062]
The concept definition dictionary 7 may be configured as a single file or may be divided into a plurality of files for each content. It is desirable that the concept definition dictionary 7 be in a state in which a large classification of concepts (in-house product dictionary 9, other company product dictionary 10, action / result dictionary 11) can be easily grasped.
[0063]
The concept extraction function 3 receives the document data acquired by the acquisition function 2 and refers to the concept definition dictionary 7. The concept extraction function 3 compares each expression in the concept definition dictionary 7 with the document data. If the same expression as the expression in the concept definition dictionary 7 is found in the document data, the concept extraction function 3 Record the concept ID.
[0064]
FIG. 2 is a diagram illustrating an extraction result obtained by the concept extraction function 3. Expressions included in the document data and registered in the concept definition dictionary 7, their positions, and their concept IDs are extracted.
[0065]
The analysis function 4 extracts a composite concept by combining expressions or concepts such as products, actions, and results based on a plurality of classification axes. The classification axes constituting the composite concept are determined in advance, such as “product-action”, “product-result”, “product-action-result”, and the like.
[0066]
For the determination process for determining whether or not to perform the combination, a process selected by the selection function 5 among the AND search process 4a, the document separation process 4b, the dependency analysis process 4c, and the association analysis process 4d is used.
[0067]
The selection function 5 outputs an instruction to the analysis function 4 so as to perform analysis using the process set by the user among the processes 4a to 4d in consideration of the analysis performance required by the user.
[0068]
For example, the selection function 5 presents the user with options such as “prioritize the recall rate”, “prioritize the matching rate and the recall rate”, and “prioritize the matching rate”.
[0069]
When the user desires to prioritize the recall rate, the selection function 5 selects the AND search process 4a. When the user desires to give priority to the matching rate and the recall rate to the same extent, the selection function 5 selects the document separation process 4b. When the user desires to prioritize the precision, the selection function 5 selects the dependency analysis process 4c or the association analysis process 4d.
[0070]
As another example, the selection function 5 may present a selection screen for the determination processes 4a to 4d to the user. The selection function 5 may set the determination process for all concepts together, or may set the determination process for each composite concept. The selection function 5 may set a determination process for each classification axis, or may set the same determination process for all the classification axes.
[0071]
The selection function 5 may present the user with a list of the determination processes 4a to 4d to be selected, or present an abstract display to the user such as “extract all-increase the accuracy rate”. The analysis performance requested by the user may be specified by the user.
[0072]
In the present embodiment, the selection process 5 allows the determination processes 4a to 4d to be freely selected, but a predetermined determination process may be used even if the selection process 5 is not selected by the user.
[0073]
The display function 6 presents the composite concept extracted by the analysis function 4 to the user in a ranking table format including the document elements for each composite concept.
[0074]
The association table 8 is a table referred to by the association analysis process 4d.
In the association table 8, product names “A snack”, “B snack”, and “C snack” in a comparison relationship (for example, a competitive relationship) are registered in association with each other.
[0075]
Hereinafter, the determination processes 4a to 4d will be described in detail.
[0076]
The AND search process 4a combines all the extracted expressions or concepts.
[0077]
The document delimiter 4b delimits the document data with “.” And performs an AND search within each range delimited by “.”. If there is a concept that is included in the classification axis but not within the delimited range, and the concept is in the previous delimiter range, copy and use the concept in the previous delimiter range. May be.
[0078]
In the dependency analysis process 4c, when the corresponding product is identified with respect to the action or concept axis, the position closest to the action or concept axis is within or before the range delimited by “.”. Look for products that are not followed by excluded connection expressions such as "~ more", "compared to", and "~ ratio". Then, the dependency analysis process 4c determines that the found product is a corresponding product with respect to an action or concept axis, and performs a combination.
[0079]
The association process 4d refers to the association table 8 in which the same type of products of the company and other companies (competitive products) are registered, and if the corresponding product cannot be determined even if the dependency analysis process 4c is executed, The company's products that correspond to products of other companies with an excluded connection expression are judged as compatible products and combined.
[0080]
In the case of the above document data “B snack sold out at the tasting party. A snack information. Sold to 120% of C snacks.”, The correct combination of “product” and “action / result” There are three types: “tasting party”, “B snack-sold out” and “A snack-sold”.
[0081]
When the analysis accuracy of the above-described various determination processes 4a to 4d is evaluated based on the relevance ratio and the recall ratio, it is as follows. The classification axes used for the combination are “product-action” and “product-result”.
[0082]
In the AND search process 4a, all the expressions or concepts extracted as shown in FIG. 2 are combined according to the classification axis. Therefore, the combined concept extracted by the AND search process 4a is "B snack-tasting party", "B snack-sold out", "B snack-sold", "A snack-tasting party", "A snack-sold out", "A “Snack—Selling”, “C Snack—Tasting Party”, “C Snack—Sold Out”, “C Snack—Selling”. The precision of this result is about 33% and the recall is 100%.
[0083]
In the document separation process 4b, document data is separated by “.”, And AND search is performed within the range. Therefore, there are three compound concepts extracted by the document separation process 4b: “B snack-tasting party”, “B snack—sold out”, and “C snack—sold”. As a result, the matching rate is about 66%, and the recall rate is about 66%.
[0084]
In the dependency analysis process 4c, when the corresponding product is specified with respect to the action or concept axis, the dependency analysis process 4c is within the range delimited by "." Search for product expressions that do not continue to connect and combine them. Accordingly, there are three composite concepts extracted in the dependency analysis process 4c: “B snack-tasting party”, “B snack—sold out”, “A snack—sold”. As a result, the precision is 100%, and the recall is 100%.
[0085]
In the correspondence analysis process 4d, when the corresponding product cannot be determined by the dependency analysis process 4c, the corresponding product corresponding to the other company's product with the excluded connection expression is obtained from the correspondence table 8 and combined as the corresponding product. . Therefore, there are three composite concepts created by the correspondence analysis process 4d: “B snack-tasting party”, “B snack—sold out”, “A snack—sold”. The precision of this result is 100%, and the recall is 100%.
[0086]
As described above, when the document data analysis program 1 according to the present embodiment is used, for example, the same result “selling” indicates whether “in-house product” is sold or “other company product” is sold. And whether it is sold “at the tasting party”. Therefore, the analysis accuracy of document data can be improved.
[0087]
In the document data analysis program 1 according to the present embodiment, the analysis function 4 for extracting a composite concept can execute a plurality of determination processes 4a to 4d, and the determination processes 4a to 4d can be freely selected. . Therefore, it is possible to provide flexible analysis performance that matches the quality of document data to be analyzed and the needs of users.
[0088]
(Second Embodiment)
In the present embodiment, a modified example of the document data analysis program 1 according to the first embodiment will be described.
[0089]
FIG. 3 is a block diagram showing functions realized on the computer by the document data analysis program 1 according to the present embodiment.
[0090]
The analysis function 4 in this embodiment uses a proper noun delimiter process 4e, a proper noun determination process 4f, a result delimiter process 4g, and a result determination process 4h.
[0091]
In the action / result dictionary 11 according to the present embodiment, it is registered that the concept of “tasting party” is “action”, and the concepts of “sold out”, “immediately”, and “popular” are “results”. The contents of the action / result dictionary 11 are determined from user needs and document characteristic analysis according to the document data to be analyzed.
[0092]
The selection function 5 outputs an instruction to the analysis function 4 so as to perform analysis using the process set by the user among the processes 4e to 4h in consideration of the analysis performance required by the user.
[0093]
Hereinafter, the processes 4e to 4h will be described in detail.
[0094]
The proper noun delimiter 4e performs analysis according to a document division rule in which one section extends from an expression associated with a concept indicating a proper noun to a next expression associated with the same concept as this expression. The document data is divided, and an AND search is performed within each range divided for the expression or concept extracted by the concept extraction function 3.
[0095]
The proper noun determination process 4f includes a rule that takes a section from an expression associated with a concept indicating a proper noun to a next expression associated with the same concept as this expression, and a concept indicating a proper noun. Even if it is an expression associated with, a document division rule including a rule that it is not used as a delimiter criterion when an excluded connection expression such as “to more”, “compared to”, or “to ratio” is attached. The document data to be analyzed is divided according to Then, the proper noun determination process 4f excludes the expression or concept to which the excluded connection expression such as “to more”, “compared to”, and “to ratio” is added from the expression or concept extracted by the concept extraction function 3. Then, AND search is performed within the divided ranges.
[0096]
The result delimiter 4g delimits the document data to be analyzed according to the document division rule delimited after the expression associated with the concept indicating the result, and performs an AND search within each delimited range.
[0097]
The result determination process 4h delimits the document data to be analyzed in accordance with the document division rule delimiting after the expression associated with the concept indicating the result. Then, the result determination process 4h excludes an expression or concept to which an excluded connection expression such as “to more”, “compared to”, and “to ratio” is added from the expression or concept extracted by the concept extraction function 3. , AND search is performed within the divided ranges.
[0098]
When the analysis accuracy of the above processes 4e to 4g is evaluated based on the relevance ratio and the recall ratio, it is as follows. The classification axes used for the combination are “product-result” and “product-action-result”.
[0099]
In the proper noun separation process 4e, the document data to be analyzed is associated with a concept belonging to the classification axis and starts with an expression that is a proper noun such as a product name, and an expression associated with the same concept as the original concept It is separated by a section that ends just before the next occurrence. In the proper noun delimiter processing 4e, an AND search is performed within the respective delimited ranges for the expression or concept extracted by the concept extraction function 3.
[0100]
For example, suppose you want to analyze the document data of the manufacturer's daily report “B snacks sold out at tasting party, C snacks are one more. A snack is also held at tasting party.
[0101]
The concept extraction function 3 compares each expression in the concept definition dictionary 7 with the document data, and if the same expression as the expression in the concept definition dictionary 7 is found in the document data, the position in the document data and the concept ID Record.
[0102]
FIG. 4 is a diagram illustrating an extraction result by the concept extraction function 3. Expressions included in the document data and registered in the concept definition dictionary 7, their positions, and their concept IDs are extracted.
[0103]
The concept of “B snack”, “C snack”, and “A snack” is a product, the concept of “tasting party” is an action, and the concepts of “sold out”, “now” and “popular” are results.
[0104]
In the proper noun delimiter processing 4e, the analysis target is analyzed according to a document division rule in which one section is from the expression associated with the concept indicating the proper noun to the next expression associated with the same concept as this expression. Document data is separated.
[0105]
FIG. 5 is a diagram exemplifying a result of dividing document data in accordance with a document division rule that divides immediately before the proper noun. Expressions associated with a concept indicating a proper noun include “B snack”, “C snack”, and “A snack”. Therefore, in FIG. 5, the document data is divided immediately before “B snack”, “C snack”, and “A snack”.
[0106]
Then, all expressions or concepts belonging to each classification axis within the delimited range are AND-searched.
[0107]
"B snacks are sold out at the tasting party, and C snacks are one more. A snack is also held at the tasting party. It is well received." The three are "A snack-tasting party-popular".
[0108]
The compound concept obtained by the proper noun delimiter processing 4e is "B snack-tasting party-sold out", "C snack-immediately", and "A snack-tasting party-popular".
[0109]
As a result, the precision is 100% and the recall is 100%.
[0110]
In the proper noun determination process 4f, an expression that is associated with a concept indicating a proper noun and that is not attached with an excluded connection expression such as “to more”, “compared to”, and “to ratio” is a document to be analyzed. It is obtained from the data, and the document data to be analyzed is divided immediately before the obtained expression. In the proper noun determination process 4f, an AND search is performed within a range delimited with respect to expressions or concepts to which the excluded connection expression is not attached among the expressions or concepts extracted by the concept extraction function 3.
[0111]
For example, it is assumed that document data “A snack sells more than B snack” is analyzed.
[0112]
From this document data, “A snack”, “B snack”, and “Sold” are extracted by the concept extraction function 3.
[0113]
Here, when the document data “A snack is sold in comparison with B snack” is separated using the document division rule that simply separates the data before the proper noun, “A snack is” and “B snack is compared. It is sold. " In this case, the composite concept obtained by the proper noun delimiter processing 4e is “B snack-sold”, and the precision and recall are both 0%.
[0114]
On the other hand, in the proper noun determination process 4f, the expression with the excluded connection expression is not used as a delimiter. Therefore, in the above document data, “compared with” is added to “B snack”, so this expression “B snack” is not used as a delimiter. In addition, the expression “B snack” to which the excluded connection expression is attached is excluded from the combination target when the composite concept is obtained.
[0115]
As a result, the composite element obtained by the proper noun determination process 4f is “A snack—selling”, and the relevance ratio and the recall ratio are 100%.
[0116]
In addition, using this proper noun judgment processing 4f, when analyzing the above document data, “B snacks are sold out at the tasting party, C snacks are only one. A snack is also held at the tasting party. In addition, the combined concept of “B snack—tasting party—sold out”, “C snack—immediately” and “A snack—tasting party—popular” is obtained, and the relevance rate and the recall rate are 100%.
[0117]
In the result separation process 4g, the document data to be analyzed starts immediately after the expression associated with the concept indicating the result that is a concept belonging to the sentence head or the classification axis, and is associated with the concept indicating the next appearing result. It is separated by a section that ends immediately after the expression. In the result delimiting process 4g, an AND search is performed within the range delimited for the expression or concept extracted by the concept extraction function 3.
[0118]
In the proper noun separation process 4e and the proper noun determination process 4f, for example, an expression associated with a concept indicating a proper noun such as a product name is used as a reference for separation. However, since product names are frequently changed, the concept extraction function 3 may not be able to extract some product names when maintenance of the concept definition dictionary 7 is delayed.
[0119]
For example, if the expression “C snack” is not extracted from the document data, “B snacks are sold out at the tasting party, and C snacks are one more. A snack is also held. The combined concept obtained by 4e and proper noun judgment processing 4f is "B snack-tasting party-sold out", "B snack-tasting party-now one", and "A snack-tasting party-popular". Is about 66%.
[0120]
On the other hand, the expression associated with the concept indicating the result or the action is not changed easily, and the expression can be stably extracted from the document data to be analyzed.
[0121]
Using this result delimiter 4g, the document data “B snacks sold out at the tasting party, C snacks are still there. A snack is also held at the tasting party. The results separated by are “B snacks sold out at the tasting party”, “C snacks are just one”, “It seems. A snacks are also held at the tasting party.
[0122]
Therefore, even if “C snack” is not registered in the concept definition dictionary 14, the composite concept obtained by the result separation process 4g is “B snack-tasting party-sold out” and “A snack-tasting party-popular”. The matching rate is 100% and the recall rate is about 66%.
[0123]
In the result determination process 4g, the document data to be analyzed starts immediately after the expression that is a concept belonging to the sentence head or the classification axis and is associated with the concept indicating the result, and is associated with the concept indicating the next appearing result. It is separated by a section that ends immediately after the expression. Then, in the result determination process 4g, an AND search is performed within a range delimited with respect to expressions or concepts to which the excluded connection expression is not attached among the expressions or concepts extracted by the concept extraction function 3.
[0124]
Thereby, for example, even when the document data to be analyzed is “A snack is sold in comparison with B snack”, the relevance ratio and the reproduction ratio can be improved.
[0125]
By making it possible to use the various processes 4e to 4g described above, it is possible to perform analysis in accordance with the quality of document data to be analyzed and the needs of users.
[0126]
(Third embodiment)
In the present embodiment, a dictionary for registering expressions (terms) to be switched like a product name and a dictionary for registering expressions that can be used for a long period of time such as “tasting party”, “sold out”, “sold” A document data analysis program to be distinguished will be described.
[0127]
FIG. 6 is a block diagram showing functions realized on a computer by the document data analysis program according to the present embodiment.
[0128]
The mask concept extraction function 14 executed by the document data analysis program 13 refers to the mask term definition dictionary 15.
[0129]
The mask term definition dictionary 15 has a structure in which a mask term (mask data) that can be used for a long period of time, a concept, and a concept ID are assigned to an expression whose usage period is shorter than a predetermined reference. In this example, since a specific product name has a usage period shorter than a predetermined standard (because it is easily updated), a mask term “own product” or “other company product” is assigned. As the mask term, a term indicating the nature of the expression that can be easily updated can be used.
[0130]
The mask concept extraction function 14 extracts an expression registered in the mask term definition dictionary 15 from the document data to be analyzed, and replaces it with a mask term corresponding to the document element.
[0131]
The processing contents of the document data analysis program 13 according to this embodiment will be described below.
[0132]
For example, the document data “B snack is sold out at the tasting party. A snack information. It sells at 120% of C snacks” is “A snack”, which is our company ’s product, “C snack” "Is sold at 120%.
[0133]
In order to make it possible to determine this meaning, the following problems arise when an expression such as “to C snack ratio” including a product name of another company that is easily changed is registered in the dictionary.
[0134]
First, dictionary registration must be performed in accordance with the number of products of other companies such as “vs. B snack ratio” and “vs. C snack ratio” for the concept definition dictionary, which increases the load of dictionary construction.
[0135]
Secondly, when a product name that is easily changed is registered in the concept definition dictionary, it is necessary to maintain an expression that can be used for a long time according to the product cycle.
[0136]
Therefore, as shown in FIG. 7, the mask concept extraction function 14 extracts the expression registered in the mask term definition dictionary 15 and included in the document data to be analyzed, and sets the position and the concept ID of the mask term. Extract and record.
[0137]
Next, the mask concept extraction function 14 converts the extracted expression into a mask term defined in the mask term definition dictionary 15 as shown in FIG.
[0138]
Then, as shown in FIG. 9, the concept extraction function 3 extracts the expression registered in the concept definition dictionary 16 from the document data changed by the mask concept extraction function 14, and acquires the concept and meaning content. . This makes it possible to determine what meaning is included in the original document data. In this example, it can be grasped that the original document data includes the meaning of “winning the products of other companies”.
[0139]
In the present embodiment, the analysis function 4 and the display function 6 perform the same processing as in the first embodiment.
[0140]
As described above, when the document data analysis program 13 according to the present embodiment is used, the mask term definition dictionary 15 that defines proper names and the concept definition dictionary 16 that can be used for a long time are separated. Thereby, the concept definition dictionary 16 can be constructed in a compact manner. In addition, even expressions including in-house product names and other company product names can be effectively used for concept extraction. Furthermore, since the dictionaries are divided according to the easiness of updating, dictionary maintenance can be easily performed.
[0141]
(Fourth embodiment)
In the present embodiment, a document data analysis program according to the third embodiment when mask terms are set hierarchically will be described.
[0142]
For example, the mask term is not single, such as “in-house product” or “other company product”, but “product: own product: snack: product name” or “product: other company product: snack: product name”, etc. Define so that all the multiple hierarchies can be specified.
[0143]
Then, the mask term replacement is executed in a hierarchy that meets the needs of the user among a plurality of hierarchies such as “product”, “in-house product”, and “snack”.
[0144]
Thereby, the classification power of the document data to be analyzed can be improved.
[0145]
(Fifth embodiment)
In the present embodiment, a document data analysis program for realizing a function of displaying document data when a user-specified expression or concept is not extracted from the document data will be described.
[0146]
FIG. 10 is a block diagram showing functions realized on a computer by the document data analysis program according to the present embodiment.
[0147]
The display function 18 realized by the document data analysis program 17 collectively displays the document data from which the company product name designated by the user has not been extracted or the document data from which the other company product name designated by the user has not been extracted. To do.
[0148]
If there is a product name that the user wants to newly register in the dictionary in the display contents by the display function 18, the user inputs the expression and the item (concept, mask term, etc.) to be registered in the dictionary registration function 19. As an input method of items to be registered, a method of presenting a list to the user and selecting it can be used.
[0149]
The dictionary registration function 19 registers the contents input by the user in the mask term definition dictionary 15. The dictionary registration function 19 can also register the contents input by the user in the concept definition dictionary 16.
[0150]
When the document data analysis program 17 according to the present embodiment described above is used, the user can update a dictionary relating to unique names that are easy to maintain. Therefore, the user can also maintain the dictionary effectively.
[0151]
For example, if there is an abbreviation of the company product in the document data from which the company product has not been extracted, this abbreviation can be handled as the company product.
[0152]
Note that the inventions described in the above embodiments can be freely combined. In addition, each function and each element described in each of the above embodiments may be rearranged as long as the same action and function can be realized, and each function and each element may be freely combined, It may be decomposed.
[0153]
The document data analysis programs 1, 13, and 17 described in the above embodiments are written in a recording medium 12 such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), and a semiconductor memory. Applicable to computers. In addition, this program can be transmitted by a communication medium and applied to a computer.
[0154]
The computer reads the document data analysis programs 1, 13, and 17 recorded in the recording medium 12 from the recording medium 12, and the operation is controlled by the program, thereby realizing the above-described functions.
[0155]
(Sixth embodiment)
In the present embodiment, usage modes of the document data analysis programs 1, 13, and 17 according to the above embodiments will be described.
[0156]
For example, the number of document data such as company daily report data, monthly report data, and business report data becomes enormous.
[0157]
When the user wants to read out and refer to document data having the contents requested by the user from the enormous amount of document data, or to obtain the summary data of the document data, the user uses the document data analysis programs 1, 13, and 17 on the computer. The document data analysis program 1, 13, 17 is read and the contents are analyzed.
[0158]
As a result, the user can create and output summary data by combining expressions included in the document data in a predetermined format.
[0159]
In addition, the user can output only document data such as “the company's products are sold”, for example.
[0160]
FIG. 11 is a block diagram exemplifying a form in which an ASP (Application Service Provider) provides services implemented by the document data analysis programs 1, 13, and 17 according to the above embodiments.
[0161]
The user 20 can easily analyze the document data by using the document data analysis programs 1, 13, and 17 managed by the ASP 23 from his / her terminal 21 via the network 22.
[0162]
Further, by receiving the service provided by the ASP 23, the user 20 can efficiently use the analysis service without maintaining and operating the document data analysis programs 1, 13, and 17 by himself / herself.
[0163]
【The invention's effect】
As described above in detail, in the present invention, by referring to the dictionary information that associates the document elements constituting the document and the attribute data thereof, the contents included in the document data are correctly identified from the document data to be analyzed. Can be extracted.
[0164]
In the present invention, document elements can be easily organized by distinguishing the registration of dictionary information according to the period of use. Moreover, dictionary information can be constructed in a compact manner. Also, dictionary information can be easily maintained.
[0165]
By utilizing the present invention, for example, daily reports, monthly reports, business reports, and the like are document data, and even when a large amount of document data is collected, information summarized in a predetermined format can be extracted from the large amount of document data. . Thereby, the user can grasp the contents described in the document data without reading all the large amount of document data.
[Brief description of the drawings]
FIG. 1 is a block diagram showing functions realized on a computer by a document data analysis program according to a first embodiment of the invention.
FIG. 2 is a diagram illustrating an extraction result by a concept extraction function in the embodiment.
FIG. 3 is a block diagram showing functions realized on a computer by a document data analysis program according to a second embodiment of the present invention.
FIG. 4 is a diagram illustrating an extraction result obtained by a concept extraction function according to the embodiment.
FIG. 5 is a diagram exemplifying a result of dividing document data in accordance with a document division rule that divides a proper noun immediately before.
FIG. 6 is a block diagram showing functions realized on a computer by a document data analysis program according to a third embodiment of the present invention.
FIG. 7 is a block diagram illustrating contents extracted by a mask concept extraction function.
FIG. 8 is a block diagram illustrating document data converted by a mask concept extraction function.
FIG. 9 is a diagram illustrating an extraction result of a concept extraction function for document data converted into mask terms.
FIG. 10 is a block diagram showing functions realized on a computer by a document data analysis program according to a fifth embodiment of the invention.
FIG. 11 is a block diagram illustrating a form in which an ASP provides a service implemented by a document data analysis program.
[Explanation of symbols]
1, 13, 17 ... Document data analysis program
2 ... Acquisition function
3. Concept extraction function
4. Analysis function
4a ... AND search function
4b ... Document separation processing
4c: Dependency analysis processing
4d ... Association analysis processing
4e ... proper noun delimiter processing
4f ... proper noun judgment processing
4g ... Result separation processing
4h ... Result judgment processing
5 ... Selection function
6, 18 ... Display function
7, 16 ... Concept definition dictionary
8 ... Association table
14 ... Mask concept extraction function
15 ... Mask term definition dictionary
19 ... Dictionary registration function

Claims (17)

On the computer,
Reference is made to dictionary information stored in the database, which associates document elements constituting the document with their attribute data, and is included in the document data to be analyzed obtained by the computer and included in the dictionary information An extraction function for extracting a plurality of document elements and their attribute data;
A classification axis that defines a plurality of attribute data to be combined with, based on the plurality of document elements extracted and the attribute data by the extraction function, the first document extracted by the extraction function according to the classification axis Realizing an analysis function for creating summary data combining the element or the first attribute data thereof and the second document element or the second attribute data thereof ;
The analysis function is
Excluding the document element and its attribute data that are followed by an excluded connection expression meaning comparison in the document data to be analyzed obtained by the computer from the plurality of document elements and its attribute data extracted by the extraction function The first document element or the first attribute data that does not follow the excluded connection expression according to the classification axis, and the second document element or the second attribute data that does not follow the excluded connection expression combine
Document data analysis program characterized by that . The document data analysis program according to claim 1,
  A document data analysis program for further realizing a display function for displaying summary data created by the analysis function on the computer. In the document data analysis program according to claim 1 or 2 ,
The analysis function is
The document data to be analyzed is divided according to a predetermined document division rule for dividing the document data to be analyzed , and within the delimited range , or within the delimited range and the immediately preceding delimited range A document data analysis program characterized by combining the existing first document element or the first attribute data with the second document element or the second attribute data . In the document data analysis program according to claim 3 ,
The document data analysis program characterized in that the document division rule is a rule for separating the document data to be analyzed on the basis of at least one of punctuation marks . In the document data analysis program according to claim 3 ,
The document division rule is a document data analysis program for dividing the document data to be analyzed before a document element associated with attribute data indicating a proper noun. The document data analysis program according to claim 5 ,
The document division rule includes a rule that does not use the proper noun as a delimiter when the excluded connection expression follows the proper noun in the document data to be analyzed. In the document data analysis program according to claim 3 ,
The document divider rule, the document data of the analysis target, the document data analysis program, which is a rule that delimit behind the document element that is associated with the attribute data indicating the result. The document data analysis program according to any one of claims 1 to 7,
The analysis function is
Refer to the correspondence information that associates the document element in the comparison relationship with other document elements ,
In the document data of the analysis target, when the exclusion connected representation to the document elements extracted by the extraction function continues, the other document elements in comparison relationship between document elements this the negative connection expression continues, the correspondence information Based on
Said first document element or the first attribute data, and the second document element or the second attribute data, the Ru combine with other document elements determined based on the correspondence information <br /> A document data analysis program characterized by the above. The document data analysis program according to any one of claims 1 to 8 ,
Wherein the computer, the document data analysis program for causing said further a function of displaying not included portions in the dictionary information realization of document data of the analyte. The document data analysis program according to any one of claims 1 to 9 ,
The computer, the document data analysis program for causing a registration function further implemented to update the dictionary information according to the specification contents of the user. In the document data analysis method by a computer,
The computer refers to dictionary information that is stored in a database and associates document elements constituting the document with attribute data thereof, is included in the document data to be analyzed obtained by the computer, and the dictionary information An extraction step for extracting a plurality of document elements and their attribute data included in
The computer is extracted by the extraction step according to the classification axis based on a classification axis that defines a plurality of attribute data to be combined, a plurality of document elements extracted by the extraction step, and its attribute data An analysis step of creating summary data combining the first document element or the first attribute data thereof and the second document element or the second attribute data thereof ;
The analysis step includes
From among the plurality of document elements extracted by the extraction step and their attribute data, excluding the document element followed by an excluded connection expression meaning comparison in the document data to be analyzed acquired by the computer and its attribute data The first document element or the first attribute data that does not follow the excluded connection expression according to the classification axis, and the second document element or the second attribute data that does not follow the excluded connection expression combine
Document data analysis method characterized by the above . The document data analysis method according to claim 11,
  The document data analysis method, further comprising a display step in which the computer displays summary data created by the analysis function. The document data analysis method according to claim 11 or 12,
  The analysis step includes
  The document data to be analyzed is divided according to a predetermined document division rule for dividing the document data to be analyzed, and within the delimited range, or within the delimited range and the immediately preceding delimited range The existing first document element or the first attribute data is combined with the second document element or the second attribute data.
Document data analysis method characterized by the above. The document data analysis method according to claim 13,
  2. The document data analysis method according to claim 1, wherein the document division rule is a rule for separating the document data to be analyzed on the basis of at least one of punctuation marks. The document data analysis method according to claim 13,
  The document data analysis method, wherein the document division rule is a rule for dividing the document data to be analyzed before a document element associated with attribute data indicating a proper noun. Reference is made to dictionary information stored in the database, which associates document elements constituting the document with their attribute data, and is included in the document data to be analyzed acquired by the computer and included in the dictionary information. Extracting means of the computer for extracting a plurality of document elements and their attribute data;
A classification axis that defines a plurality of attribute data to be combined with, based on the plurality of document elements extracted and the attribute data by the extracting means, a first document extracted by said extraction means in accordance with the classification axis An analysis means of the computer for generating summary data combining the element or the first attribute data thereof and the second document element or the second attribute data thereof ,
The analysis means includes
From among the plurality of document elements extracted by the extraction means and their attribute data, excluding the document element followed by an excluded connection expression meaning comparison in the analysis target document data acquired by the computer and its attribute data The first document element or the first attribute data that does not follow the excluded connection expression according to the classification axis, and the second document element or the second attribute data that does not follow the excluded connection expression Document data analysis system characterized by combining . The document data analysis system according to claim 16, wherein
  The document data analysis system further comprising display means of the computer for displaying the summary data created by the analysis means.

Images