JP3620968B2 - Document retrieval method, apparatus for implementing the same, and medium on which processing program is recorded - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document search system that searches for a desired document from a large-scale document database that increases daily, such as newspaper articles and patent specifications, and in particular searches for the entire text of a document for the presence of a specified character string. The present invention relates to a technique effective when applied to a document search system for full text search.
[0002]
[Prior art]
There is an n-gram index method (hereinafter referred to as method 1) as a method for quickly searching a document including a specified search character string (hereinafter referred to as a search term) from a large-scale document database.
[0003]
In the n-gram index method, the appearance position information of all n-grams (character strings consisting of consecutive n characters) in a document is stored as an index when the document is registered, and n in the search term is retrieved. A method of searching for a document in which a search term appears by referring to the index with respect to -gram and determining whether the positional relationship in the search term is equal to the positional relationship in the index (hereinafter referred to as adjacency determination) It is.
[0004]
FIG. 20 is a diagram showing an example of a conventional 1-gram index method. As shown in the figure, in the method 1, at the time of document registration, the appearance position information is stored as an index for all n-grams in the document (n = 1 in the example of FIG. 20). For example, since 1-gram “ni” appears at the “3” character of the document “001”, the document number “001” and the character position “3” are stored in the index 200 corresponding to “ni”.
[0005]
At the time of a search, the appearance position information of a search term is acquired by performing adjacency determination of the appearance position information between n-gram indexes extracted from a specified search term. For example, when “microorganism” is designated as a search term, 1-gram “fine”, “raw”, and “thing” are first extracted from this search term. Then, the appearance position information of “microorganism” is obtained by performing adjacent determination of the appearance position information among the index 201 corresponding to “fine”, the index 202 corresponding to “raw”, and the index 203 corresponding to “thing”. get. In the example of this figure, “Fine”, “Raw” and “Object” are adjacent to each other from the “9” character of the document number “001” and the “5” character of the document number “056”. Therefore, it turns out that "microorganisms" exist in those appearance positions.
[0006]
As described above, method 1 can perform a search only by reading an index and determining the adjacency determination of appearance position information without scanning the document. Therefore, even when applied to a large-scale document database, high-speed full-text search can be realized. there is a possibility.
[0007]
However, in Method 1, when n is 1, that is, when an index of 1-gram is used, since 1-gram has a high appearance frequency in all documents (in the database), appearance position information for one 1-gram increases. , The size of each index increases. For this reason, not only does it take time to read an index, but also the number of adjacent determinations based on the appearance position information increases, which causes a problem that it takes time to search.
[0008]
In order to speed up the search, it is necessary to create a small index with a larger value of n. However, make sure that all the small indexes of n are created so that the search can be performed even when a short search term is specified. There must be. As a result, the total index size becomes enormous.
[0009]
In addition, in the index type document retrieval method such as method 1, “Key search technique—IV try and its application” (Junichi Aoe, Information Processing Vol.34, No.2, 1993, pp.244-251). It is necessary to manage character strings (n-gram in the case of method 1) with a tree structure like the trie described.
[0010]
A trie is a tree structure created by grouping forward substrings common to each keyword (hereinafter referred to as a key) in a character string to be searched, that is, a set of keywords (hereinafter referred to as a key set). It is.
[0011]
This trie is used for registration and search, and by searching for the trie by tracing the key with the character string to be registered or the character string in the search term, pointer information indicating the index corresponding to the character string is obtained. Can do. Since the time required to search for a trie does not depend on the number of keys, it is characterized in that keywords can be searched at high speed even when applied to a large database.
[0012]
FIG. 21 is a diagram showing the structure of a trie corresponding to a conventional key set {baby, badge, badger, jar}. In this trie, branch label b (302) is defined from clause 1 (300) to clause 2 (301), and pointer information to the index for the key is set in the clause corresponding to the end of the key indicated by a double circle. .
[0013]
For example, when “baby” is designated as a search term, the pointer information Pt1 set in clause 5 (303) is obtained by searching for a trie in this figure using the character string “baby”. The index corresponding to the search term “baby” is stored at the destination indicated by the pointer information Pt1.
[0014]
In Method 1, when managing an n-gram using this trie, if an index with a longer n-gram is created in order to reduce the size of each index and increase the search speed, the type of n-gram is changed. As the number of trials increases and the number of trials increases, there is a problem that the scale of the whole trial becomes enormous.
[0015]
In order to solve the contradictory problems described above, that is, an increase in search time and an increase in the total index size and the tree structure size for managing the index, the index size is a certain reference value (hereinafter referred to as the reference index size). Only for n-grams exceeding), by increasing the value of n and creating a small index, it is always possible to read a small index and determine the adjacency of appearance position information. A document retrieval method and apparatus that realizes retrieval and prevents the enlargement of the size of the total index size and the tree structure that manages the index (which will be described in the following) will be disclosed in Japanese Patent Laid-Open No. 8-194718 (hereinafter referred to as known example 1). Are disclosed).
[0016]
FIG. 22 is a diagram showing the processing contents of the conventional example 1 of the related art. In this method, an index corresponding to n-gram is created when a document is registered, and connection information of two characters in the document is registered in the trie 401. Then, when the document is registered and the index size exceeds the reference index size, an index of an n-gram (hereinafter referred to as an extended n-gram) obtained by adding a character string to the corresponding n-gram is used. create.
[0017]
Hereinafter, the index creation method will be described in detail with reference to FIG. In order to create an extended n-gram in which a character string is added to an n-gram, an n-gram that may follow the n-gram is first acquired by referring to the trie 401.
[0018]
And between the index of the detected n-gram (hereinafter referred to as connection n-gram) and the index of the n-gram (hereinafter referred to as reference index excess n-gram) whose size is larger than the reference index size. An extension n-gram index is created by performing adjacency determination on appearance position information.
[0019]
In the example of this figure, the index corresponding to 1-gram “raw” is larger than the reference index size, and “raw” is the reference index excess n-gram. Here, first, the trie 401 is searched with the characters “raw”, and the connection n-gram following “raw” is acquired. In the example shown in the figure, by searching for the try 401, it can be seen that “raw” and “breath” follow “raw”. Therefore, character strings were added to “live” such as “living” and “inhabiting” by performing adjacent determination of appearance position information between the indexes of “raw” and “things” and “raw” and “breath”. An extended n-gram index 400 is created.
[0020]
As described above, by using the document search method and apparatus disclosed in the known example 1, for an index that is large in size and takes a long time to search, an extended n− that is obtained by adding a character string to the n−gram. Since an index for a gram is created, it is only necessary to read a small-sized index and determine the adjacency of appearance position information, thereby realizing a high-speed search. In addition, for other indexes, since an index with an increased n-gram length is not created, it is possible to prevent an increase in the total index size and the tree structure (tri) size for managing the n-gram. .
[0021]
[Problems to be solved by the invention]
According to the document search method and apparatus disclosed in the known example 1 described above, for an index that is larger than the reference index size and takes a long time to search, a character string is added to the key n-gram. Since a small-sized index is created, reading of a small-sized index and adjacency determination of appearance position information can be completed, and high-speed search can be realized. In addition, for an index having a size smaller than the reference index size, since an index with a longer n-gram is not created, the tree structure (tri) size for managing the total index size and the n-gram is prevented from being enlarged. It becomes possible.
[0022]
However, in the document search method and apparatus disclosed in the known example 1, an extended n-gram index is created by adding a character string to the n-gram based only on the size of the index. As a result, when the index size becomes large even for n-grams (for example, hiragana used as particles) that are rarely used in search, an extended n-gram index is created. Will be.
[0023]
Since these n-grams are rarely used for searching, there is a problem that even if the cost of the index size is paid, the merit of speeding up the search performance cannot be obtained, resulting in a useless index. is there.
[0024]
An object of the present invention is to provide a technique capable of solving the above-described problems, speeding up search performance, and minimizing an increase in the total index size.
[0025]
[Means for Solving the Problems]
According to the present invention, in a document search system for searching text data including a specific search term, the size of an index corresponding to a partial character string created from the text data is larger than a reference index size, and the partial character string corresponding to the index is When the possibility of searching is high, an extended partial character string obtained by adding a character string to the partial character string and an index corresponding to the extended partial character string are created.
[0026]
In the document search system of the present invention, first, in the index creation / registration step, for a text database in which document information is stored as character code data, a predetermined partial character string from the registration target text data and the target text of the partial character string Text identification information in the data and position information including appearance position information are extracted, the position information is stored as an index, and the partial character string and the partial character string table for managing the storage position of the index are updated.
[0027]
When searching for text data including a specific search term, an index search step refers to the partial character string table, extracts a predetermined partial character string from the search term, reads the corresponding index, Extracts position information of text data having the same positional relationship as the partial character string.
[0028]
On the other hand, in the index size comparison step, the size of the index corresponding to the partial character string created from the text data is compared with a predetermined reference index size, and when the size of the index is larger than the reference index size, In the search possibility determination step, it is determined whether or not the partial character string corresponding to the index is highly likely to be searched in the index search step.
[0029]
If the partial character string is likely to be searched in the index search step, the character string increment step corresponds to the extended partial character string obtained by adding the character string to the partial character string and the extended partial character string. An index is created and the partial character string table is updated.
[0030]
The search possibility determination step determines whether or not a partial character string corresponding to a specific index is highly likely to be searched in the index search step, for example, according to a reference frequency corresponding to the index.
[0031]
That is, when the index is read in the index search step, the reference frequency table that manages the number of times the index is read is updated by the reference frequency table update step, and the reference frequency comparison step that is the search possibility determination step includes: The reference frequency corresponding to the index in the reference frequency table is compared with a predetermined standard reference frequency, and when the reference frequency corresponding to the index in the reference frequency table is larger than the standard reference frequency, An extended partial character string obtained by adding a character string to the partial character string and an index corresponding to the extended partial character string are created by a character string increment step.
[0032]
The searchability determination step may determine whether or not there is a high possibility that a partial character string corresponding to a specific index is searched in the index search step according to an important word input by the user.
[0033]
That is, the important word input by the user is stored in the important word table by the important word registration step, and the important character string search step, which is the search possibility determination step, includes the partial character string corresponding to the index as the important word. Check whether it is included in a key word stored in the table. When the partial character string corresponding to the index is included in the important word of the important word table, an extended partial character string obtained by adding a character string to the partial character string and an index corresponding to the extended partial character string are displayed. Created by the string increment step.
[0034]
Further, the search possibility determination step determines whether or not there is a high possibility that a partial character string corresponding to a specific index is searched in the index search step according to a related word in which words that are related to each other are related in advance. You may judge.
[0035]
That is, a related word file in which related words are stored in association with each other in advance is created, and the expanded word search step, which is the searchability determination step, includes a partial character string corresponding to the index in the related word table. To see if it is included in the related terms stored in. When a partial character string corresponding to the index is included in the related word of the related word table, an extended partial character string obtained by adding a character string to the partial character string and an index corresponding to the extended partial character string are displayed. Created by the string increment step.
[0036]
The related word file is a file used by the search term expansion step when extracting the position information of the text data having the same positional relationship as the search term and the partial character string in the related word in the index search step.
[0037]
As described above, according to the present invention, when a partial character string having an index size larger than the standard index size is compared with the reference frequency of the index and a predetermined standard reference frequency, the partial character string is larger than the standard reference frequency. By creating an index with a small size by adding a character string to the substring, search is performed for substrings that have a large index and take a long time to search and are likely to be used for the search. It is possible to increase the speed.
[0038]
In addition, for substrings that are large in index but are not often used in search (for example, hiragana used as particles), an index with an increased string is not created, so the total index size Can be suppressed. As a result, it is possible to realize a document search system capable of improving the search performance with a minimum increase in the total index size.
[0039]
As described above, according to the document search system of the present invention, a small index with a large number of characters is added only to a partial character string that has a large index size and takes a long time to search and is likely to be used for the search. Therefore, the search performance can be increased and the increase in the total index size can be minimized.
[0040]
DETAILED DESCRIPTION OF THE INVENTION
(Embodiment 1)
The document search system according to the first embodiment that creates an extended n-gram obtained by adding a character string to an n-gram corresponding to the index and an index corresponding to the extended n-gram for an index having a large size and a high reference frequency. explain.
[0041]
FIG. 1 is a diagram showing a schematic configuration of a document search system according to the present embodiment. The document retrieval system of this embodiment includes a display 100, a keyboard 101, a central processing unit (CPU) 102, a magnetic disk device 106 as a secondary storage device that stores text 107 and an index 108, and a floppy disk drive (FDD) 104. The main storage device 109 is configured. Further, the CPU 103, the magnetic disk device 106, the FDD 104, and the main storage device 109 are connected to the bus 103.
[0042]
Information stored in the floppy disk 105 is accessed by the FDD 104. The main storage device 109 stores a system control program 110, a document registration control program 111, an increment control program 112, a search control program 113, an index creation registration program 114, an increment program 115, a search program 119 and a tri storage table 122. A reference frequency table 123 and a work area 124 are secured.
[0043]
The increment program 115 includes an index size comparison program 116, a reference frequency comparison program 117, and an n-gram increment program 118. The search program 119 includes an index search program 120 and a reference frequency table update program 121.
[0044]
The above program can be stored in a recording medium that can be read and written by a computer, such as the magnetic disk device 106, the floppy disk 105, and a CD-ROM.
[0045]
The index search program 120 performs an index search step of extracting position information of text data having the same positional relationship as the partial character string in the search term by using an index corresponding to the partial character string of the search term.
[0046]
The index size comparison program 116 performs an index size comparison step process for comparing the size of the index corresponding to the partial character string created from the text data with a predetermined reference index size.
[0047]
The reference frequency comparison program 117 compares the reference frequency corresponding to the index in the reference frequency table 123 with a predetermined standard reference frequency when the size of the index is larger than the standard index size, and The reference frequency comparison step is performed to determine whether or not the partial character string corresponding to is likely to be searched by the index search program 120.
[0048]
The n-gram increment program 118 corresponds to an extended partial character string obtained by adding a character string to the partial character string and the extended partial character string when there is a high possibility that the partial character string will be searched by the index search program 120. The character string increment step for creating the index to be performed is performed.
[0049]
The reference frequency table update program 121 performs a process of a reference frequency table update step for updating the reference frequency table 123 that manages the number of times the index is read.
[0050]
The system control program 110 is activated in response to an instruction from the keyboard 101. The document registration control program 111 is activated by the system control program 110 in response to a document registration instruction from the keyboard 101 and controls the index creation / registration program 114.
[0051]
The increment control program 112 is activated by the system control program 110 in response to an increment instruction from the keyboard 101 and controls the increment program 115. The search control program 113 is activated by the system control program 110 in response to a search instruction from the keyboard 101 and controls the search program 119.
[0052]
Hereinafter, an outline of the processing content in the present embodiment will be described. At the time of registration, the system control program 110 activates the document registration control program 111 in response to a document registration instruction from the keyboard 101. Then, the document registration control program 111 activates the index creation / registration program 114, reads text data from the floppy disk 105 and stores it as text 107 in the magnetic disk device 106, and information on the occurrence position of a predetermined n-gram in the text 107. Is stored in the magnetic disk device 106 as an index 108.
[0053]
The processing content of the index creation / registration program 114 is the same as the method disclosed in the known example 1. The text data is not limited to be input using the floppy disk 105, but can be configured to be input from another device using a communication line, a CD-ROM device (not shown), or the like.
[0054]
In the search, the system control program 110 starts the search control program 113 and the search control program 113 starts the search program 119 in response to a search instruction from the keyboard 101.
[0055]
FIG. 2 is a diagram showing the processing contents of the search program 119 of this embodiment. As shown in a PAD (Problem Analysis Diagram) diagram of FIG. 2, the search program 119 starts the index search program 120 in step 600, and the appearance position information in the text 107 of the search term input from the keyboard 101 is retrieved from the index 108. get. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0056]
In step 601, the reference frequency table update program 121 is activated to update the corresponding reference frequency in the reference frequency table 123 corresponding to the index used in the index search program 120.
[0057]
At the time of increment, the system control program 110 activates the increment control program 112 and the increment control program 112 activates the increment program 115 according to the increment instruction from the keyboard 101.
[0058]
FIG. 3 is a diagram showing the processing contents of the increment program 115 of this embodiment. As shown in FIG. 3, the increment program 115 repeats a series of processing from step 701 to step 705 for all indexes in step 700.
[0059]
During this repetitive process, the increment program 115 first activates the index size comparison program 116 in step 701, and compares the size of the index to be processed with a predetermined reference index size.
[0060]
If it is determined in step 702 that the processing target index is larger than the reference index size, the reference frequency comparison program 117 is started in step 703 and the reference frequency of the processing target index stored in the reference frequency table 123 is set. A predetermined reference frequency is compared.
[0061]
If it is determined in step 704 that the reference frequency of the processing target index is higher than the reference reference frequency, the n-gram increment program 118 is started in step 705 and characters are added to the n-gram that is the key of the processing target index. Create a small index with columns added. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1.
[0062]
FIG. 4 is a diagram showing a processing example of the document search system of this embodiment. Hereinafter, an outline of processing according to the present embodiment will be described with reference to FIG. Since the process of the index creation / registration program 114 is the same as the method disclosed in the known example 1, it is omitted.
[0063]
First, the processing outline of the search program 119 will be described. The index search program 120 extracts a predetermined n-gram from a search term input by the user with reference to an n-gram table such as a trie, and acquires the storage location of the index corresponding to the n-gram. Then, the index is read from the storage position of the index corresponding to the acquired n-gram, and the position information having the same positional relationship as the n-gram in the search term is extracted. This position information acquisition step is the same processing as the method disclosed in the method 1 and the known example 1.
[0064]
The reference frequency table update program 121 updates the index reference frequency (number of times read) read in the position information acquisition step. In the example of FIG. 4, “microorganism” and “organism” are used as search terms, so the reference frequency corresponding to “Fine” is “1”, and the reference frequency corresponding to “Raw” is “2” “thing”. The reference frequency corresponding to is updated to “2”. Here, since the index corresponding to “ni” is not used in the search, the reference frequency remains “0”.
[0065]
Next, an outline of processing of the increment program 115 will be described. The index size comparison program 116 compares the size of the index corresponding to the n-gram created from the text data by the index creation registration program 114 with a predetermined reference index size.
[0066]
When the index size comparison program 116 determines that the size of the index corresponding to n-gram is larger than a predetermined reference index size, the reference frequency comparison program 117 sets the reference frequency corresponding to the index to the reference frequency table. 123 and is compared with a predetermined reference frequency.
[0067]
When the reference frequency comparison program 117 determines that the reference frequency of the index corresponding to n-gram is higher than a predetermined standard reference frequency, the n-gram increment program 118 uses the n−gram corresponding to the corresponding index. An extended n-gram with a character string added to gram and an index corresponding to the extended n-gram are created.
[0068]
The index size comparison program 116 and the n-gram increment program 118 are the same as the method disclosed in the known example 1. In the case of the example shown in FIG. 4, the indexes “ni” and “raw” are large, and the index size comparison program 116 determines that the index is larger than the reference index size. Therefore, the reference frequency comparison program 117 is executed for “ni” and “raw”.
[0069]
In the reference frequency comparison program 117, “Ni” and “Raw” reference frequencies are acquired from the reference frequency table 123 and compared with a predetermined reference frequency. In the case of the example shown in FIG. 4, the standard reference frequency is “1”. As a result, it is determined that only the reference frequency corresponding to “raw” is higher than the standard reference frequency, and the n-gram increment program 118 is executed, so that an extended n-gram in which a character string is added to “raw” is supported. The index to be created is created. Here, since the reference frequency corresponding to “ni” is less than the standard reference frequency, an extended n-gram index is not created even though the index is large.
[0070]
As described above, in the present embodiment, for an n-gram having an index larger than the standard index size, the reference frequency to the index is compared with a predetermined standard reference frequency, and the n-gram is larger than the standard reference frequency. Creates a small index with a character string added to the n-gram. When the reference frequency is lower than the standard reference frequency, even if the index is large, a small-sized index obtained by increasing the character string in the n-gram is not created.
[0071]
By doing so, it is possible to speed up the search for an n-gram that has a large index and takes a long time to search and is likely to be used for the search. For n-grams (for example, hiragana characters used as particles) that are large in index but are not often used in search, an index with an increased character string is not created, so the total index size Can be suppressed. As a result, it is possible to improve search performance with a minimum increase in the total index size.
[0072]
Hereinafter, the processing contents of the search program 119 and the increment program 115 different from the method disclosed in the known example 1 among the processing contents in the present embodiment will be described in detail using specific examples.
[0073]
The processing contents of the search program 119 are as shown in FIG. First, in step 600, the index search program 120 is activated, and the appearance position information in the text 107 of the search term input from the keyboard 101 is acquired from the index 108. The contents of this processing are the same as the method disclosed in Method 1 and Known Example 1 as shown in FIG. 20 (n = 1 in the example of FIG. 20).
[0074]
For example, when “microorganism” is designated as a search term, 1-gram “fine”, “raw” and “thing” are first extracted from this search term. Then, the appearance position information of “microorganism” is acquired by performing adjacency determination of the appearance position information among the index 201 corresponding to “fine”, the index 202 corresponding to “raw”, and the index 203 corresponding to “thing”. .
[0075]
In the example of this figure, “Fine”, “Raw” and “Object” are adjacent from the “9” character of the document number “001” and from the “5” character of the document number “056”. It can be seen that “microorganism” exists at the position indicated by.
[0076]
Next, in step 601, the reference frequency table update program 121 is activated to update the corresponding reference frequency in the reference frequency table 123 corresponding to the index used in the index search program 120.
[0077]
FIG. 5 is a diagram illustrating an initial state of the reference frequency table 123 before update according to the present embodiment. As shown in the figure, the reference frequency table 123 includes entries that store reference frequencies corresponding to individual indexes.
[0078]
FIG. 6 is a diagram showing an update result of the reference frequency table 123 when “microorganism” is designated as the search term of this embodiment. The index search program 120 acquires the appearance position information of “microorganisms” using the 1-gram “fine”, “raw”, and “thing” indexes. Therefore, the reference frequency table update program 121 increases the corresponding reference frequency of the reference frequency table 123 corresponding to the indexes “fine”, “raw”, and “thing” to “1”. At this time, the location corresponding to the index “ni” remains “0”.
[0079]
Next, it is assumed that “organism” is designated as the search term. At this time, the index search program 120 uses the 1-gram “raw” and “thing” indexes to obtain the appearance position information of “living organism”. Therefore, in the reference frequency table update program 121, the corresponding reference frequency in the reference frequency table 123 corresponding to the “raw” and “thing” indexes is increased by “1”.
[0080]
FIG. 7 is a diagram showing the processing contents of the reference frequency table update program 121 in the search term “living organism” of the present embodiment. The reference frequency corresponding to the indexes of “raw” and “thing” is “2”, the reference frequency of “ni” remains “0”, and the reference frequency of “fine” remains “1”. The above is the detailed processing content of the search program 119.
[0081]
The processing contents of the increment program 115 are as shown in FIG. First, in step 700, a series of processing from step 701 to step 705 is repeated for all indexes created from the text 107.
[0082]
In this repetitive processing, first, in step 701, the index size comparison program 116 is started, and the size of the processing target index is compared with a predetermined reference index size. In the example shown in FIG. 4, the “ni” and “raw” indexes are larger than the reference index size.
[0083]
Next, if it is determined in step 702 that the index is larger than the reference index size, the reference frequency comparison program 117 is started in step 703, and the reference frequency of the processing target index stored in the reference frequency table 123 is stored in advance. Compare established standard reference frequencies.
[0084]
If it is determined in step 704 that the reference frequency is higher than the standard reference frequency, the n-gram increment program 118 is started in step 705 and a character string is added to the n-gram that is the key of the index to be processed. Create a small index.
[0085]
In the example of FIG. 4, since it is determined that the indexes “ni” and “raw” are larger than the standard index size, the reference frequencies corresponding to these indexes are compared with a predetermined standard reference frequency.
[0086]
In the example of FIG. 4, the reference reference frequency is 1. Therefore, a “raw” index with a reference frequency of “2” is a processing target of the n-gram increment program 118, but an index of “ni” with a reference frequency of “0” is an object of the n-gram increment program 118. Not subject to processing.
[0087]
Finally, a small index with a character string added to “raw” is created, and the process of the increment program 115 is terminated. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1 shown in FIG.
[0088]
As described above, in the present embodiment, for an n-gram having an index larger than the standard index size, the reference frequency to the index is compared with a predetermined standard reference frequency, and is greater than the standard reference frequency. In this case, a small index is created by increasing the character string in the n-gram.
[0089]
When the reference frequency is lower than the standard reference frequency, even if the index is large, a small-sized index obtained by increasing the character string in the n-gram is not created. By doing so, it is possible to speed up the search for an n-gram that has a large index and takes a long time to search and is likely to be used for the search.
[0090]
In addition, for n-grams (eg hiragana used as particles) that are large in index but are not often used in searches, the total index size is not created because an index with an increased character string is not created. Can be suppressed.
[0091]
As described above, even if a search term with a high frequency of occurrence, that is, a large index is specified in a large-scale document database whose number of articles increases every day, such as newspaper articles and patent specifications, the total index size is minimized. As a result, the high-speed search can be realized.
[0092]
As described above, according to the document search system of the present embodiment, only a partial character string that has a large index size and takes a long time to search and is likely to be used for the search has a small size with an increased number of characters. Since the index is created, it is possible to speed up the search performance and minimize the increase in the total index size.
[0093]
(Embodiment 2)
The document search according to the second embodiment for creating an extended n-gram in which a character string is added to an n-gram corresponding to the index and an index corresponding to the extended n-gram for an index having a large index size and included in an important word. The system will be described.
[0094]
In the present embodiment, when an index with a small size obtained by adding a character string to an n-gram having an index larger than the reference index size is created, it is searched whether the n-gram is included in the keyword specified by the user. However, this is a method of creating a small index with a character string added only when it is included.
[0095]
By having the user specify key words that are likely to be used frequently in searches, it is possible to speed up searches for n-grams that have a large index and take a long time to search. This can be achieved with a minimum increase in the total index size.
[0096]
This embodiment basically has the same configuration as that of the first embodiment (FIG. 1), but the configuration in the main storage device 109 is different.
[0097]
FIG. 8 is a diagram showing a schematic configuration of the main storage device 109a of this embodiment. The main storage device 109a includes a system control program 110a, a document registration control program 111, an increment control program 112, a search control program 113, a keyword registration control program 1100, an index creation registration program 114, an increment program 115a, a search program 119a, an important A word registration program 1102 is stored, and a trie storage table 122, an important word table 1103, and a work area 124 are secured.
[0098]
The increment program 115a includes an index size comparison program 116, an important character string search program 1101, and an n-gram increment program 118. The search program 119a is composed of an index search program 120. The above program can be stored in a recording medium that can be read and written by a computer, such as the magnetic disk device 106, the floppy disk 105, and a CD-ROM.
[0099]
The important character string search program 1101 checks whether or not the partial character string corresponding to the index is included in the important word stored in the important word table 1103, and the partial character string corresponding to the index is the index search program 120. The important character string search step is performed to determine whether or not there is a high possibility of being searched. The keyword registration program 1102 performs a keyword registration step for storing the keyword input by the user in the keyword table 1103.
[0100]
The system control program 110a is activated in response to an instruction from the keyboard 101. The document registration control program 111 is activated by the system control program 110a in response to a document registration instruction from the keyboard 101, and controls the index creation / registration program 114.
[0101]
The increment control program 112 is activated by the system control program 110a in response to an increment instruction from the keyboard 101, and controls the increment program 115a. The search control program 113 is activated by the system control program 110a in response to a search instruction from the keyboard 101, and controls the search program 119a. The keyword registration control program 1100 is activated by the system control program 110 a in response to a search instruction from the keyboard 101, and controls the keyword registration program 1102.
[0102]
Hereinafter, processing outlines of the search program 119a, the keyword registration program 1102, and the increment program 115a different from the first embodiment will be described.
[0103]
When searching, the system control program 110a starts the search control program 113 and the search control program 113 starts the search program 119a in response to a search instruction from the keyboard 101.
[0104]
The search program 119 a activates the index search program 120 and obtains the appearance position information in the text 107 of the search term input from the keyboard 101 from the index 108. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0105]
When registering a key word, the system control program 110a starts the key word registration control program 1100 according to a key word registration instruction from the keyboard 101, and the key word registration control program 1100 starts the key word registration program 1102. The important word registration program 1102 stores the important words input from the keyboard 101 in the important word table 1103.
[0106]
At the time of increment, the system control program 110a activates the increment control program 112 according to an increment instruction from the keyboard 101, and the increment control program 112 activates the increment program 115a.
[0107]
FIG. 9 is a diagram showing the processing contents of the increment program 115a of this embodiment. As shown in FIG. 9, the increment program 115 a repeats a series of processing from step 1201 to step 1205 for all indexes created from the text 107 in step 1200.
[0108]
In this repetitive processing, first, in step 1201, the index size comparison program 116 is started, and the size of the processing target index is compared with a predetermined reference index size.
[0109]
If it is determined in step 1202 that the processing target index is larger than the reference index size, the important character string search program 1101 is started in step 1203 and stored in the important word table 1103 as the processing target n-gram. Search for important words.
[0110]
If it is determined in step 1204 that the processing target n-gram constitutes a part of the important word stored in the important word table 1103, the n-gram increment program 118 is started in step 1205. A small-sized index is created by adding a character string to the n-gram that is the key of the processing target index. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1. The above is the outline of the processing content in the present embodiment.
[0111]
Hereinafter, among the processing contents in the present embodiment, the processing contents of the method 1, the method disclosed in the known example 1, and the processing contents of the keyword registration program 1102 and the increment program 115a different from the first embodiment will be described in detail using specific examples. To do.
[0112]
The important word registration program 1102 stores the important words input from the keyboard 101 in the important word table 1103.
[0113]
FIG. 10 is a diagram showing an example of the important word table 1103 of this embodiment. In the example shown in this figure, the user builds a database with documents related to “microorganisms”, and the words “virus”, “bacteria”, “mold”, “yeast”, “bacteria” “Single-cell organism”, “microorganism”,... The important word registration program 1102 stores these input character strings in the important word table 1103 as important words.
[0114]
The processing contents of the increment program 115a are as shown in FIG. First, in step 1200, a series of processing from step 1201 to step 1205 is repeated for all indexes created from the text 107.
[0115]
In this repetitive processing, first, in step 1201, the index size comparison program 116 is started, and the size of the processing target index is compared with a predetermined reference index size. The processing contents of the index size comparison program 116 are the same as those in the first embodiment.
[0116]
FIG. 11 is a diagram showing an example of processing contents of the increment program 115a of the present embodiment. In the example shown in the figure, the “ni” and “raw” indexes are larger than the reference index size.
[0117]
Next, when it is determined in step 1202 that the processing target index is larger than the reference index size, the important character string search program 1101 is started in step 1203 and stored in the important word table 1103 as the processing target n-gram. Search for important words.
[0118]
If it is determined in step 1204 that the processing target n-gram constitutes a part of the important word stored in the important word table 1103, the n-gram increment program 118 is started in step 1205. A small-sized index is created by adding a character string to the n-gram that is the key of the processing target index.
[0119]
In the example of FIG. 11, since it is determined that the indexes “ni” and “raw” are larger than the reference index size, the important words stored in the important word table 1103 are searched using “ni” and “raw”. .
[0120]
In the example of FIG. 11, “living” is included in the important words “single cell organism” and “microorganism”, but there is no important word including “ni”. Therefore, the “raw” index is the processing target of the n-gram increment program 118, but the “ni” index is not the processing target of the n-gram increment program 118.
[0121]
Finally, a small index with a character string added to “raw” is created, and the process of the increment program 115a is terminated. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1 shown in FIG.
[0122]
As described above, in the present embodiment, for an n-gram having an index larger than the reference index size, whether or not the n-gram is included in the keyword specified by the user is included. In this case, a small index with a character string added to the n-gram is created.
[0123]
If n-gram is not included in the important word designated by the user, even if the index is large, an index with a small size in which the character string is increased is not created in the n-gram.
[0124]
For n-grams included in important words that have a large index and are likely to be used for searching by having the user specify important words that are often used for searching It becomes possible to speed up the search.
[0125]
In addition, for n-grams that are large in index but are not frequently used by the user for searching, an index with an increased character string is not created, so that the total index size can be suppressed.
[0126]
As described above, even if a search term with a high frequency of occurrence, that is, a large index is specified in a large-scale document database whose number of articles increases every day, such as newspaper articles and patent specifications, the total index size is minimized. As a result, the high-speed search can be realized.
[0127]
(Embodiment 3)
The document search according to the third embodiment that creates an extended n-gram in which a character string is added to an n-gram corresponding to the index and an index corresponding to the extended n-gram for an index that has a large index size and is included in a related word. The system will be described.
[0128]
If a user specifies a search term and specifies that a search is also performed for a word having the same meaning as the search term (hereinafter referred to as a synonym), the search term specified by the user is assumed to be one. However, the number of search terms that are actually searched increases, and the search takes time.
[0129]
Therefore, in this embodiment, when creating a small index by adding a character string to an n-gram having an index larger than the reference index size, the index is expanded into a plurality by referring to a dictionary and rules for synonym expansion. This is a method of searching for n-grams included in a word and creating a small index with a character string added only if included.
[0130]
This makes it possible to search at high speed with a minimum increase in the total index size, even if the index is large and it takes a long time to search, and multiple words must be searched like a synonym search. It will be possible.
[0131]
This embodiment basically has the same configuration as that of the first embodiment (FIG. 1), but the configuration in the main storage device 109 is different.
[0132]
FIG. 12 is a diagram showing a schematic configuration of the main storage device 109b of the present embodiment. The main storage device 109b stores a system control program 110, a document registration control program 111, an increment control program 112, a search control program 113, an index creation registration program 114, an increment program 115b, a search program 119b, and a tri storage table 122. The synonym expansion dictionary 1502 and the work area 124 are secured.
[0133]
The increment program 115b includes an index size comparison program 116, an expanded word search program 1500, and an n-gram increment program 118. The search program 119b includes a search term expansion program 1501 and an index search program 120.
[0134]
The above program can be stored in a recording medium that can be read and written by a computer, such as the magnetic disk device 106, the floppy disk 105, and a CD-ROM.
[0135]
The expanded word search program 1500 checks whether the partial character string corresponding to the index is included in the synonym stored in the synonym expansion dictionary 1502, and the partial character string corresponding to the index is the index search program 120. The expanded word search step is performed to determine whether or not there is a high possibility of being searched.
[0136]
Further, the search term expansion program 1501 searches the synonym expansion dictionary 1502 corresponding to the related word file in which the words related to each other are stored in advance in association with the search term, and one or more related to the search term. The index search program 120 performs a process of extracting position information of text data having the same positional relationship as the partial character string in the search term and the synonym.
[0137]
The system control program 110 is activated in response to an instruction from the keyboard 101. The document registration control program 111 is activated by the system control program 110 in response to a document registration instruction from the keyboard 101 and controls the index creation / registration program 114.
[0138]
The increment control program 112 is activated by the system control program 110 in response to an increment instruction from the keyboard 101, and controls the increment program 115b. The search control program 113 is activated by the system control program 110 in response to a search instruction from the keyboard 101, and controls the search program 119b.
[0139]
Hereinafter, processing outlines of the search program 119b and the increment program 115b different from the first embodiment will be described.
[0140]
When searching, the system control program 110 starts the search control program 113 and the search control program 113 starts the search program 119b in response to a search instruction from the keyboard 101.
[0141]
FIG. 13 is a diagram showing the processing contents of the search program 119b of this embodiment. As shown in FIG. 13, the search program 119b starts the search term expansion program 1501 in step 1600, searches the synonym expansion dictionary 1502 using the search terms input from the keyboard 101, and converts the search terms into a plurality of synonyms. expand.
[0142]
Next, in step 1601, the index search program 120 is started, and appearance position information in the text 107 of all synonyms expanded by the search term expansion program 1501 is acquired from the index 108. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0143]
At the time of increment, the system control program 110 activates the increment control program 112 and the increment control program 112 activates the increment program 115b according to the increment instruction from the keyboard 101.
[0144]
FIG. 14 is a diagram showing the processing contents of the increment program 115b of this embodiment. As shown in FIG. 14, the increment program 115 b repeats a series of processing from step 1701 to step 1705 for all indexes created from the text 107 in step 1700.
[0145]
In this iterative process, first, in step 1701, the index size comparison program 116 is started, and the size of the processing target index is compared with a predetermined reference index size.
[0146]
If it is determined in step 1702 that the processing target index is larger than the reference index size, the expanded word search program 1500 is started in step 1703 and stored in the synonym expansion dictionary 1502 as the processing target n-gram. Search for synonyms.
[0147]
If it is determined in step 1704 that the processing target n-gram is included in a plurality of expanded words, the n-gram increment program 118 is started in step 1705, and the processing target index key n- Create a small index with a character string added to gram. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1. The above is the outline of the processing content in the present embodiment.
[0148]
Hereinafter, the processing contents of the search program 119b and the increment program 115b which are different from the processing contents in the first embodiment will be described in detail using specific examples.
[0149]
The processing contents of the search program 119b are as shown in FIG. First, in step 1600, the search term expansion program 1501 is started, the synonym expansion dictionary 1502 is searched with the search term input from the keyboard 101, and the search term is expanded into a plurality of synonyms.
[0150]
FIG. 15 is a diagram showing an example of search term synonym expansion by the synonym expansion dictionary 1502 of this embodiment. For example, assume that “computer” is designated as the search term. The search term expansion program 1501 searches the synonym expansion dictionary 1502 with this search term “computer” and acquires synonyms that are words having the same meaning as “computer”.
[0151]
In the example of the synonym expansion dictionary 1502 shown in FIG. 15, a set of synonyms is described in one line with a comma (,) as a delimiter. When “computer” is expanded into a set of synonyms using this synonym expansion dictionary 1502, “computer”, “electronic computer”, “computer”, “COMPUTER”, and “computer” are searched as expanded search terms 1800. can get.
[0152]
Next, in step 1601, the index search program 120 is started, and appearance position information in the text 107 of all synonyms expanded by the search term expansion program 1501 is acquired from the index 108. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0153]
The processing contents of the increment program 115b are as shown in FIG. First, in step 1700, a series of processing from step 1701 to step 1705 is repeated for all indexes created from the text 107.
[0154]
In this iterative process, first, in step 1701, the index size comparison program 116 is started, and the size of the processing target index is compared with a predetermined reference index size. The processing contents of the index size comparison program 116 are the same as those in the first embodiment.
[0155]
FIG. 16 is a diagram showing the processing contents of the increment program 115b of this embodiment. In the example shown in the figure, the “ni” and “raw” indexes are larger than the reference index size.
[0156]
Next, if it is determined in step 1702 that the processing target index is larger than the reference index size, the expanded word search program 1500 is started in step 1703 and stored in the synonym expansion dictionary 1502 as the processing target n-gram. Search for synonyms that are being used.
[0157]
If it is determined in step 1704 that the processing target n-gram is included in a plurality of expanded words, the n-gram increment program 118 is started in step 1705, and the processing target index key n- Create a small index with a character string added to gram.
[0158]
In the example of FIG. 16, since it is determined that the indexes “ni” and “raw” are larger than the reference index size, the important words stored in the synonym expansion dictionary 1502 are “ni” and “raw”. Search for.
[0159]
In the example of FIG. 16, “raw” is included in “marine life” and “marine life”, but there is no word including “ni”. Therefore, the “raw” index is the processing target of the n-gram increment program 118, but the “ni” index is not the processing target of the n-gram increment program 118.
[0160]
Finally, a small index with a character string added to “raw” is created, and the process of the increment program 115b is terminated. The processing content of the n-gram increment program 118 is the same as the method disclosed in the known example 1 shown in FIG.
[0161]
As described above, in the present embodiment, with respect to an n-gram having an index larger than the reference index size, the n-gram included in a plurality of words is referred to by referring to a dictionary and rules for synonym expansion. If it is included, an index with a small size is created by increasing the character string in the n-gram. If n-gram is not included in a plurality of expanded words, even if the index is large, an index with a small size obtained by adding a character string to the n-gram is not created.
[0162]
This makes it possible to speed up the search even when the index is large and it takes a long time to search, and a search must be performed for a plurality of words like a search for synonyms. In addition, for n-grams that are large but not included in a plurality of expanded words, an index with an increased character string is not created, so that the total index size can be suppressed.
[0163]
As described above, a search term having a high frequency of occurrence, that is, a large index, is specified in a large-scale document database whose number of cases is increasing every day, such as newspaper articles and patent specifications, and synonyms of the search terms are also searched. Even if it is specified to be performed simultaneously, a high-speed search can be realized with a minimum increase in the total index size.
[0164]
(Embodiment 4)
The document search system according to the fourth embodiment that creates an extended n-gram obtained by adding a character string to an n-gram corresponding to the index and an index corresponding to the extended n-gram for an index having a large index size and a long processing time. Will be described.
[0165]
This embodiment measures the processing time when acquiring the search term appearance position information by reading the index and determining the adjacency, and if the processing time exceeds a predetermined target search time, In this method, the obtained appearance position information is indexed and stored in the magnetic disk device.
[0166]
For search terms that take a long time to search, once the search is performed, the appearance position information of the search term can be acquired only by reading the index created the first time, so the index is large and the search takes time. It is possible to increase the search speed for an n-gram that is highly likely to be used for a search with a minimum increase in the total index size. In addition, since the appearance position information acquired at the first search can be used as an index as it is, high-speed index creation can be performed.
[0167]
This embodiment basically has the same configuration as that of the first embodiment (FIG. 1), but the configuration in the main storage device 109 is different.
[0168]
FIG. 17 is a diagram showing a schematic configuration of the main storage device 109c of this embodiment. The main storage device 109c stores a system control program 110c, a document registration control program 111, a search control program 113c, an index creation registration program 114, a search and increment program 2000, and a trie storage table 122 and a work area 124. The The search and increment program 2000 includes an index search program 120 and an index creation program 2001.
[0169]
The above program can be stored in a recording medium that can be read and written by a computer, such as the magnetic disk device 106, the floppy disk 105, and a CD-ROM.
[0170]
The index creation program 2001 measures the processing time in the index search program 120, and registers the position information obtained in the index search program 120 as an index when the processing time is longer than a predetermined reference processing time. Process the creation step.
[0171]
The system control program 110c is activated in response to an instruction from the keyboard 101. The document registration control program 111 is activated by the system control program 110c in response to a document registration instruction from the keyboard 101, and controls the index creation / registration program 114. The search control program 113c is activated by the system control program 110c in response to a search instruction from the keyboard 101, and controls the search and increment program 2000.
[0172]
Hereinafter, a processing outline of the search and increment program 2000 different from the first embodiment will be described. When searching, the system control program 110 c starts the search control program 113 c and the search control program 113 c starts the search and increment program 2000 according to a search instruction from the keyboard 101.
[0173]
FIG. 18 is a diagram showing the processing contents of the increment program 2000 of this embodiment. As shown in FIG. 18, the search and increment program 2000 first starts the index search program 120 in step 2100, acquires the appearance position information in the text 107 of the search term input from the keyboard 101 from the index 108, and searches the index. The processing time of the program 120 is measured. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0174]
If it is determined in step 2101 that the measured processing time has exceeded a predetermined target search time, the index creation program 2001 is started in step 2102 and the obtained appearance position information is set as the index 108 to the magnetic disk. Store in device 106.
[0175]
In step 2103, the search term is registered as a key in the trie storage table 122, and the pointer information to the index is stored. The method of adding a key to the trie storage table 122 is the same as the method disclosed in the known example 1. The above is the outline of the processing content in the present embodiment.
[0176]
Hereinafter, processing contents of the search and increment program 2000 different from those in the first embodiment will be described in detail using specific examples.
[0177]
The processing contents of the search and increment program 2000 are as shown in FIG. First, in step 2100, the index search program 120 is started, appearance position information in the text 107 of the search term input from the keyboard 101 is acquired from the index 108, and the processing time of the index search program 120 is measured. The processing content of the index search program 120 is the same as the method disclosed in Method 1 and Known Example 1.
[0178]
FIG. 19 is a diagram showing a processing example of the increment program 2000 of the present embodiment. In the example shown in this figure, “organism” is designated as the search term. For this reason, the index search program 120 reads the “raw” index 202 and the “thing” index 203 from the magnetic disk device 106 into the work area 124, and makes an adjacency determination between these indexes, so that “living organism” appears. The position information 2200 is acquired.
[0179]
The search and increment program 2000 measures the processing time of the index search program 120. If it is determined in step 2101 that the measured processing time exceeds a predetermined target search time, the index creation program 2001 is started in step 2102, and the acquired appearance position information 2200 is set as the index 108. It is stored in the magnetic disk device 106.
[0180]
In the example shown in FIG. 19, since the processing time of the index search program 120 exceeds a predetermined target search time, the appearance position information 2200 of “living organism” is stored in the magnetic disk device 106 as an index 2201 of “living organism”. To do.
[0181]
Finally, in step 2103, the search term is registered as a key in the trie storage table 122, and the pointer information to the index is stored. The method of adding a key to the trie storage table 122 is the same as the method disclosed in the known example 1.
[0182]
As described above, in the present embodiment, when the search term appearance position information is obtained by reading the index and determining the adjacency, the processing time is measured, and the processing time is determined in advance as the target search time. Is exceeded, the acquired appearance position information is indexed and stored in the magnetic disk device 106.
[0183]
For search terms that take a long time to search, once the search is performed, the appearance position information of the search term can be acquired only by reading the index created the first time, so the index is large and the search takes time. For n-grams that are likely to be used for searching, it is possible to speed up searching.
[0184]
In addition, for n-grams that are large in index but are not used for searching, an index with an increased character string is not created, so that the total index size can be suppressed. In addition, since the appearance position information acquired at the time of the first search can be used as it is as an index, high-speed index creation is possible.
[0185]
As described above, even if a search term with a high frequency of occurrence, that is, a large index is specified in a large-scale document database whose number of articles increases every day, such as newspaper articles and patent specifications, the total index size is minimized. As a result, the high-speed search can be realized.
[0186]
【The invention's effect】
According to the present invention, a small-sized index with an increased number of characters is created only for a partial character string that has a large index size and takes a long time to search, and is likely to be used for searching. And an increase in the total index size can be minimized.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a schematic configuration of a document search system according to a first embodiment.
FIG. 2 is a diagram illustrating processing contents of a search program 119 according to the first embodiment.
FIG. 3 is a diagram illustrating processing contents of an increment program 115 according to the first embodiment.
FIG. 4 is a diagram illustrating a processing example of the document search system according to the first embodiment.
FIG. 5 is a diagram illustrating an initial state of a reference frequency table 123 before update according to the first exemplary embodiment.
6 is a diagram showing an update result of a reference frequency table 123 when “microorganism” is designated as a search term in the first embodiment. FIG.
7 is a diagram showing the processing contents of a reference frequency table update program 121 in the search term “organism” of the first embodiment. FIG.
FIG. 8 is a diagram illustrating a schematic configuration of a main storage device 109a according to the second embodiment.
FIG. 9 is a diagram illustrating processing contents of an increment program 115a according to the second embodiment.
FIG. 10 is a diagram illustrating an example of an important word table 1103 according to the second embodiment.
FIG. 11 is a diagram illustrating an example of processing contents of an increment program 115a according to the second embodiment.
FIG. 12 is a diagram illustrating a schematic configuration of a main storage device 109b according to the third embodiment.
FIG. 13 is a diagram illustrating processing contents of a search program 119b according to the third embodiment.
FIG. 14 is a diagram illustrating processing contents of an increment program 115b according to the third embodiment.
FIG. 15 is a diagram showing an example of synonym expansion of search terms by the synonym expansion dictionary 1502 of the third embodiment.
FIG. 16 is a diagram illustrating processing contents of an increment program 115b according to the third embodiment.
FIG. 17 is a diagram illustrating a schematic configuration of a main storage device 109c according to the fourth embodiment.
FIG. 18 is a diagram illustrating processing contents of an increment program 2000 according to the fourth embodiment.
FIG. 19 is a diagram illustrating a processing example of an increment program 2000 according to the fourth embodiment.
FIG. 20 is a diagram illustrating an example of a conventional 1-gram index method.
FIG. 21 is a diagram showing a configuration of a trie corresponding to a conventional key set {baby, badge, badger, jar}.
FIG. 22 is a diagram showing the processing contents of a known example 1 in the related art.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... Display, 101 ... Keyboard, 102 ... CPU, 103 ... Bus, 104 ... FDD, 105 ... Floppy disk, 106 ... Magnetic disk apparatus, 107 ... Text, 108 ... Index, 109 ... Main storage device, 110 ... System control program 111 ... Document registration control program, 112 ... Increment control program, 113 ... Search control program, 114 ... Index creation registration program, 115 ... Increment program, 116 ... Index size comparison program, 117 ... Reference frequency comparison program, 118 ... n- Gram increment program, 119 ... search program, 120 ... index search program, 121 ... reference frequency table update program, 122 ... trie storage table, 123 ... reference frequency table , 124 ... work area, 1100 ... important word registration control program, 1101 ... important character string search program, 1102 ... important word registration program, 1103 ... important word table, 1500 ... expanded word search program, 1501 ... search term expansion program, 1502 ... Synonym expansion dictionary, 1800 ... Search term after expansion, 2000 ... Increment program, 2001 ... Index creation program, 2200 ... Appearance position information, 2201 ... Index, 200 to 203 ... Index, 400 ... Index, 401 ... Trie.

Claims (7)

In a document search method for searching text data including a specific search term,
An index search step for extracting position information of text data having the same positional relationship as the partial character string of the search term by an index corresponding to the partial character string of the search term;
An index size comparison step for comparing the size of the index corresponding to the partial character string of the text data with a predetermined reference index size;
A searchability determination step for determining whether or not a partial character string corresponding to the index is likely to be searched in the index search step when the size of the index is larger than the reference index size;
A character string increment that creates an extended partial character string obtained by adding a character string to the partial character string and an index corresponding to the extended partial character string when the partial character string is likely to be searched in the index search step And a document search method. A reference frequency table update step for updating a reference frequency table for managing the number of times the index is read;
The search possibility determination step compares a reference frequency corresponding to the index in the reference frequency table with a predetermined reference frequency, and searches for a partial character string corresponding to the index in the index search step. The document search method according to claim 1, wherein the document search method is a reference frequency comparison step for determining whether or not there is a high possibility of being performed. A key word registration step of storing key words input by the user in the key word table;
The searchability determination step checks whether or not the partial character string corresponding to the index is included in the important word stored in the important word table, and the partial character string corresponding to the index is determined in the index search step. 2. The document search method according to claim 1, wherein the document search method is an important character string search step for determining whether or not there is a high possibility of being searched. A search term expansion step of searching for a related word file in which words related to each other in advance are stored with the search term and acquiring one or more related words related to the search term;
The index search step extracts position information of text data having the same positional relationship as a partial character string in the search term or related word,
The searchability determination step checks whether or not the partial character string corresponding to the index is included in the related word stored in the related word table, and the partial character string corresponding to the index is determined by the index search step. 2. The document search method according to claim 1, wherein the document search method is a expanded word search step for determining whether or not the search possibility is high. In a document search method for searching text data including a specific search term,
An index search step for extracting position information of text data having the same positional relationship as the partial character string of the search term by an index corresponding to the partial character string of the search term;
An index creation step of measuring the processing time in the index retrieval step and registering the position information obtained in the index retrieval step as an index when the processing time is longer than a predetermined reference processing time. A document retrieval method as a feature. In a document search apparatus for searching text data including a specific search term,
An index search processing unit for extracting position information of text data having the same positional relationship as the partial character string of the search term by an index corresponding to the partial character string of the search term;
An index size comparison processing unit that compares the size of the index corresponding to the partial character string of the text data with a predetermined reference index size;
A search possibility determination processing unit that determines whether or not a partial character string corresponding to the index is highly likely to be searched by the index search processing unit when the size of the index is larger than the reference index size;
When the partial character string is highly likely to be searched by the index search processing unit, an extended partial character string obtained by adding a character string to the partial character string and a character string for creating an index corresponding to the extended partial character string A document search apparatus comprising an increment processing unit. The same position as the partial character string in the search term by an index corresponding to the partial character string of the search term in the medium recording the program for causing the computer to function as a document search device for searching text data including a specific search term An index search processing unit for extracting positional information of text data having a relationship;
An index size comparison processing unit that compares the size of an index corresponding to a partial character string created from text data with a predetermined reference index size, and when the index size is larger than the reference index size, A searchability determination processing unit that determines whether or not a corresponding partial character string is highly likely to be searched by the index search processing unit;
When the partial character string is highly likely to be searched by the index search processing unit, an extended partial character string obtained by adding a character string to the partial character string and a character string for creating an index corresponding to the extended partial character string A medium in which a program for causing a computer to function as an increment processing unit is recorded.

Images