JP7519795B2 - Natural language processing device and program - Google Patents

Description

The present invention relates to a technology for searching text data.

In a wide variety of fields, a common method of collecting information is to search a database by entering keywords as a search query. Keywords are generally selected based on the user's judgment, so the search queries set vary depending on the user's taste. Also, when there are many synonyms or similar words, the search results will differ depending on which words are used.

JP 2002-297636 A

When trying to extract text that matches a particular purpose using conventional keyword searches, there is an uncertainty involved because it relies on human factors. In other words, there is a problem of incompleteness, which means that text that matches a particular purpose may be missed.

The present invention was completed based on the recognition of the above problems, and its main purpose is to make up for the lack of comprehensiveness in keyword searches.

A natural language processing device according to one aspect of the present invention is characterized in that it includes a first reception unit that receives a search query for a database that manages a plurality of sentences linked to classification terms, and a search unit that searches for sentences that meet the conditions of the search query or the conditions of classification terms whose variance vectors are similar to the variance vector of the search query.

The present invention makes it possible to compensate for the lack of comprehensiveness in keyword searches.

FIG. 2 is a diagram showing a network configuration of the search system according to the present embodiment. FIG. 13 is a data structure diagram of a company table. FIG. 13 is a data structure diagram of a business characteristic word table. FIG. 13 is a data structure diagram of an industry table. FIG. 4 is a data structure diagram of a word table. FIG. 2 is a data structure diagram of a company vector storage unit. FIG. 13 is a data structure diagram of a business characteristic word vector storage unit. FIG. 13 is a data configuration diagram of an industry vector storage unit. FIG. 13 is a diagram illustrating a screen for accepting a search query. 13 is a diagram of a company information box included in a search result screen. FIG. 13 is a diagram illustrating a screen for accepting a search query. FIG. 13 is a diagram illustrating a screen for accepting a search query. FIG. 4 is a data structure diagram of a synonym list. FIG. 13 is a data configuration diagram of a business characteristic word list. FIG. 13 is a data configuration diagram of an industry list. 13 is a diagram of a company information box included in a search result screen. 13 is a diagram of a company information box included in a search result screen. FIG. 2 is a functional block diagram of a server. FIG. 4 is a functional block diagram of a data storage unit. FIG. 13 is a flowchart showing the process of a preparation phase process. FIG. 11 is a flowchart showing the process of generating a distributed representation model. FIG. 13 is a flowchart showing the process of the company vector calculation process. FIG. 13 is a flowchart showing the process of business characteristic word vector calculation processing. FIG. 13 is a flowchart showing the process of an industry vector calculation process. FIG. 13 is a flowchart showing the process of utilization phase processing. FIG. 11 is a flowchart showing the process of a list generation process. FIG. 11 is a flowchart showing the process of screen generation processing. 13 is a diagram of a company information box included in a search result screen.

FIG. 1 is a diagram showing the configuration of a search system network in this embodiment.
The search system in this embodiment includes a server 100 and a user terminal 200. The user terminal 200 is connected to the server 100 via a network (for example, the Internet, a LAN, or a dedicated line). The server 100 has a company database 120.

When a user searches for company information managed in the company database 120, a search query is input by user operation on the user terminal 200, and the search query is sent to the server 100. Then, company information extracted based on the search query is returned to the user terminal 200 and displayed on the user terminal 200.

In this embodiment, a distributed representation model is used to improve search functionality. In a distributed representation model, words are represented as high-dimensional real vectors. A distributed representation model is obtained by a learning process based on a database called a corpus, which is a large-scale collection of structured natural language sentences. This technology is well known in the field of natural language processing. The real vector of words obtained as a distributed representation model is sometimes called a distributed vector or word vector. Hereinafter, it will be referred to as a word vector.

A well-known example is Word2vec. Word2vec works on the premise that "words that appear in close proximity are similar in meaning," and solves the problem of searching for other words that appear in the vicinity of a target word in a sentence (for example, five words before or ten words before it). The five words before and after here refer to a range of 10 words in total, including the word five before the target word and the word one after the target word and the word five after the target word. A neural network is used as a learning method. During the learning process using a large corpus, the word vectors of words that appear in close proximity and are similar change to show similar values.

In a distributed representation model, word vectors can indicate not only similarity between words, but also similarity between sentences. For a sentence containing multiple words, a sentence vector for the sentence is generated from the word vectors of these words. If the sentence vector of one sentence is similar to the sentence vector of another sentence, this indicates that the sentences are similar in content. In this example, the distributed representation model is stored in the distributed representation model storage unit 130.

In this embodiment, tags that are highly relevant to the search query are selected based on the tag vectors of tags attached to the business content sentences of the company information. Then, character strings representing the tags are used as conditions that supplement the search query.

Roughly speaking, the procedure is divided into a preparation phase and an utilization phase. In the preparation phase, a distributed representation model based on the corporate database 120 is generated using conventional technology. In other words, word vectors for words in the corporate database 120 are generated. Next, tag vectors are generated for tags used in the corporate database 120 and added to the distributed representation model. This extends the distributed representation model to include tag representations.

In the utilization phase, a specific search query is received, and tags related to the search query are selected by referring to the distributed representation model. The textual expressions of the tags are then used as search conditions. At this time, the textual expressions of the tags are selected as related phrases and stored in the related phrase storage unit 140. Specific processing is described in detail below.

FIG. 2 is a data structure diagram of the company table.
The company table is managed in the company database 120. The company table has a record for each company. The company table records include a company ID, company name, business content sentence, business characteristic word ID, main industry ID, and sub-industry ID. Only one sentence or multiple sentences may be set in the business content sentence field. The business characteristic word and industry correspond to tags. In this example, up to two industry tags can be set for each company. If only one industry tag is set, the main industry tag is used. The company table is prepared before the preparation phase.

FIG. 3 is a data structure diagram of the business characteristic word table.
The business characteristic word table is managed in the enterprise database 120. The business characteristic word table has a record for each business characteristic word. A business characteristic word ID and a business characteristic word are set in the record of the business characteristic word table. The business characteristic word table is prepared before the preparation phase.

FIG. 4 is a data structure diagram of the business type table.
The business type table is managed in the company database 120. The business type table has a record for each business type. An business type ID and a business type name are set in each record of the business type table. The business type table is prepared before the preparation phase.

FIG. 5 is a data structure diagram of the word table.
The word table is included in the distributed representation model. The word table has a record for each word. A word and a word vector are set in the record of the word table. The word table is generated in the preparation phase.

FIG. 6 is a data configuration diagram of the company vector storage unit.
The company vector corresponds to a distributed representation vector of the entire business description sentence of the company. The company vector is generated from the word vector of the words included in the business description sentence. The company vector storage unit is included in the distributed representation model storage unit 130. The company vector is calculated in the preparation phase. The company vector is the basis for calculating the business characteristic word vector and the industry vector.

FIG. 7 is a data configuration diagram of the business characteristic word vector storage unit.
The business characteristic word vector indicates the practical meaning of the business characteristic word. The business characteristic word vector is generated from the company vector of the company in which the business characteristic word is set, and is different from the word vector of the word corresponding to the business characteristic word (e.g., "region"). The business characteristic word vector quantifies the operational characteristic that the meaning of the business characteristic word is determined by the type of company to which the business characteristic word is likely to be set. The business characteristic word vector is calculated in the preparation phase.

The business characteristic word vector storage unit stores business characteristic word IDs and business characteristic word vectors in association with each other. The business characteristic word vector storage unit is included in the distributed representation model storage unit 130. The business characteristic word IDs are examples of tag IDs, and the business characteristic word vectors are examples of tag vectors.

FIG. 8 is a data configuration diagram of the business type vector storage unit.
The industry vector indicates the practical significance of the industry tag. The industry vector is generated from the company vector of the company to which the industry tag is set, and is different from the semantic vector of a phrase equivalent to the industry name (e.g., "automobile sales"). The industry vector quantifies the operational characteristic that the significance of an industry tag is determined by the type of company to which it is likely to be set in practice. The industry vector is calculated in the preparation phase.

The industry vector storage unit stores an industry ID and an industry vector in association with each other. The industry vector storage unit is included in the distributed representation model storage unit 130. The industry ID is an example of a tag ID, and the industry vector is an example of a tag vector.

FIG. 9 is a diagram showing a screen for accepting a search query.
The search query reception screen is generated by the server 100 and displayed on the user terminal 200. In addition to a search query input area 300, this reception screen includes a slider 302, a display area 304 for synonyms, a display area 306 for business characteristic words, and a display area 308 for industry.

The similar words display area 304 displays similar words whose similarity to the term input as the search query is equal to or greater than a reference value. The similar words are selected from all words included in the business content sentence. This reference value is called the first reference value. The business characteristic words display area 306 displays business characteristic words whose similarity to the term input as the search query is equal to or greater than a reference value. This reference value is called the second reference value. The business characteristic words are set in advance. Similar words are selected from among them. The industry display area 308 displays industries whose similarity to the term input as the search query is equal to or greater than a reference value. This reference value is called the third reference value. The industries are also set in advance. Similar words are selected from among them.

The slider value indicated by the slider 302 indicates a first reference value, a second reference value, and a third reference value. In this example, the slider value is at the maximum value, and the first reference value, the second reference value, and the third reference value are also at their maximum values. There are no synonyms whose similarity to "automobile" is equal to or greater than the first reference value, no business characteristic words whose similarity is equal to or greater than the second reference value, and no industries whose similarity is equal to or greater than the third reference value, and none of them are displayed. In this case, a search is performed using the search query entered in the input area 300.

FIG. 10 is a diagram of a company information box included in the search result screen.
The figure shows a company information box displayed as a search result in the state of FIG. 9. Company information containing "automobile" in the text of the business content sentence, business characteristic word name, main business name, and sub-business name is extracted from the company database 120. The company name included in the extracted company information is displayed in the display area 322. Similarly, the business content sentence is displayed in the display area 324. Similarly, the business characteristic word is displayed in the display area 326. Similarly, the main business is displayed in the display area 328. Similarly, the sub-business is displayed in the display area 330. If there is company information containing "automobile" other than this example, it will be displayed as a box list on the search result screen. The search result screen is displayed in a row below the search query reception screen. When a search query is entered, a search is automatically performed and a search result screen is displayed.

FIG. 11 is a diagram showing the search query reception screen when the slider value is decreased.
When the user operates the slider 302 to decrease the slider value, the first reference value, the second reference value, and the third reference value decrease, and synonyms, business characteristic words, and industries that satisfy the conditions appear.

For example, the similarity between the word vector of "automobile" and the word vector of "motorcycle" is equal to or greater than the first reference value indicated by the slider value at this time. The similarity between the word vector of "motorcycle" and the business characteristic word vector of "used" is equal to or greater than the second reference value indicated by the slider value at this time. Furthermore, the similarity between the word vector of "motorcycle" and the industry vector of "automobile sales" is equal to or greater than the third reference value indicated by the slider value at this time.

The synonyms, business characteristic words, and industry names that appear here are added as OR conditions in the search for company information. Company information that includes terms such as "bikes," "used," or "automobile sales" in the text of the business description sentence, business characteristic word name, main industry name, and sub-industry name is extracted from the company database 120.

FIG. 12 is a diagram showing the search query reception screen when the slider value is further reduced.
When the user operates the slider 302 to further reduce the slider value, the first reference value, the second reference value, and the third reference value become further smaller, and the number of similar words, business characteristic words, and industries that satisfy the conditions increases.

The resulting increased search terms, such as "trucks," "quality," and "automobile manufacturing," can now be added as OR conditions when searching for company information.

In this way, by operating the slider 302, the strength of association with the search query can be adjusted, and the number of similar words, business characteristic words, and industry types added to the search criteria can be increased or decreased. This allows the comprehensiveness of the search for company information to be adjusted. In the above example, the comprehensiveness increases in order. This operation will be explained.

FIG. 13 is a data structure diagram of the synonym list.
When a search query is input in the utilization phase, a synonym list is generated accordingly. The synonym list is stored in the related phrase storage unit 140. The synonym list stores, for each word, the degree of similarity between the word and the phrase of the search query. The synonym list is sorted in descending order of the degree of similarity.

In the example of FIG. 11, the value of the slider 302 is 0.7 and the first reference value is also 0.7, so the similar words "bike," "taxi," and "bus" are selected. In the example of FIG. 12, the value of the slider 302 is 0.6 and the first reference value is also 0.6, so the similar words "truck," "traffic," and "railroad" are added.

FIG. 14 is a data configuration diagram of the business characteristic word list.
When a search query is input in the utilization phase, a business characteristic word list is generated accordingly. The business characteristic word list is stored in the related phrase storage unit 140. The business characteristic word list stores, in association with each business characteristic word, the similarity between the business characteristic word and the words of the search query. The business characteristic word list is sorted in descending order of similarity.

In the example of FIG. 11, the first reference value is also 0.7, so the business characteristic words "used," "repaired," and "luxury" are selected. In the example of FIG. 12, the first reference value is 0.6, so the business characteristic words "quality," "popular," and "new" are added.

FIG. 15 is a data configuration diagram of the business category list.
When a search query is input in the utilization phase, an industry list is generated in response to the input. The industry list is stored in the related phrase storage unit 140. The industry list stores the similarity between each industry and the words in the search query in association with the industry. The industry list is sorted in descending order of similarity.

In the example of FIG. 11, the first reference value is 0.7, so the industries "automobile sales," "automobile maintenance," and "automobile rental" are selected. In the example of FIG. 12, the first reference value is 0.6, so the industries "automobile manufacturing," "transportation," and "home sales" are added.

FIG. 16 is a diagram showing the company information box included in the search result screen in the state of FIG.
In this example, the character string of the business characteristic word "luxury", which is similar to the search query, matches the business characteristic word name, so this company information is displayed as a search result. Note that a hit will also be made if the character string "luxury" is included in the business content sentence, main industry name, or sub-industry name. In other words, the business content sentence, main industry name, and sub-industry name are also within the search range. However, it is not necessary to include the business content sentence, main industry name, or sub-industry name in the search range.

FIG. 17 is a diagram showing the company information box included in the search result screen in the state of FIG.
In this example, the character string "home sales" for the business type, which is similar to the search query term, matches the main business type name, so this company information is displayed as a search result. Note that the character string "home sales" will also be hit if it is included in the business content sentence, business characteristic word, or main business type name. In other words, the business content sentence, business characteristic word, and sub-business name are also within the search range. However, it is not necessary to include the business content sentence, business characteristic word, or sub-business name in the search range.

An example of operation based on these screen examples will be given. For example, assume that an insurance company wishing to find new contract candidates for automobile insurance uses this search system. The insurance company first inputs "automobile" as a search query and obtains the search results shown in Figure 10. The automobile dealers shown in Figure 10 are unlikely to be a new transaction route. This is because it has long been common to recommend automobile insurance through automobile dealers.

Therefore, the slider 302 is operated as shown in FIG. 11 to increase comprehensiveness, and the re-searched company information shown in FIG. 16 is obtained. Although the company information of "Red Rose" does not contain the character string "automobile", it is substantially related to "automobile" through the business characteristic word "luxury" which is highly related to "automobile". Looking at the business content of "Red Rose", the insurance company thinks that there is a possibility that "Red Rose" will be prepared to cover high compensation when a helper dispatched by "Red Rose" has an accident while driving a luxury car of a wealthy customer. The insurance company then realizes that it can propose an insurance contract to "Red Rose".

Next, the search is further improved by manipulating slider 302 as shown in FIG. 12, and the re-searched company information in FIG. 17 is obtained. Although the company information for "Yellow Lemon" does not contain the character string "automobile," it is substantially related to "automobile" through the business of "housing sales," which is highly related to "automobile." Looking at this business description, the insurance company assumes that since the tenant is purchasing a house with a parking space, the tenant will definitely have a car. Also, choosing an insurance company that is familiar with the local conditions means that the tenant will not be familiar with the local traffic conditions and will have few acquaintances. Therefore, if the tenant is worried about getting into trouble with other residents if they have a traffic accident in an unfamiliar area, it is realized that this will be a motivation for them to take out insurance. In this way, clues for business expansion are found one after another.

In this example, it is unlikely that an insurance company would realize that "luxury" or "home sales" would be key to expanding their business. However, with this embodiment, the common word "automobile" can be used to find the desired company information.

Since human descriptions often vary in spelling and manually added tag information often does not guarantee comprehensiveness, this embodiment can supplement comprehensiveness by supporting searches. The details of the process are explained below.

FIG. 18 is a functional block diagram of the server 100.
Each component of the server 100 is realized by hardware including computing units such as a CPU (Central Processing Unit) and various coprocessors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, and software stored in the storage devices and supplying processing instructions to the computing units. The computer programs may be configured by device drivers, an operating system, various application programs located at higher layers than those, and libraries that provide common functions to these programs. Each block shown in the figure shows a functional block, not a hardware-based configuration.

The server 100 includes a data storage unit 110, a data processing unit 150, and a communication unit 190. The communication unit 190 is responsible for communication processing via the network. The data storage unit 110 stores various data. The data processing unit 150 executes various processes based on the data acquired by the communication unit 190 and the data stored in the data storage unit 110. The data processing unit 150 also functions as an interface between the communication unit 190 and the data storage unit 110.

The communication unit 190 includes a transmission unit 180 that transmits data and a reception unit 170 that receives data.
The transmitting unit 180 includes a screen data transmitting unit 182 that transmits various screen data. The receiving unit 170 includes a search query receiving unit 172 and a slider value receiving unit 174. The search query receiving unit 172 receives a search query. The slider value receiving unit 174 receives a slider value.

The data processing unit 150 includes a model generation unit 152 , a company vector calculation unit 154 , a business characteristic word vector calculation unit 156 , an industry vector calculation unit 158 , a screen data generation unit 160 , and a search unit 162 .
The model generation unit 152 executes a distributed representation model generation process to calculate word vectors. The company vector calculation unit 154 executes a company vector calculation process. The business characteristic word vector calculation unit 156 executes a business characteristic word vector calculation process. The industry vector calculation unit 158 executes an industry vector calculation process. The screen data generation unit 160 generates various screen data. The search unit 162 searches for company information.

The data storage unit 110 includes a corporate database 120, a distributed representation model storage unit 130, and a related phrase storage unit 140.

FIG. 19 is a functional block diagram of the data storage unit 110.
The company database 120 includes a company table storage section 122, a business characteristic word table storage section 124, and an industry table storage section 126. The company table storage section 122 stores the company table (FIG. 2). The business characteristic word table storage section 124 stores the business characteristic word table (FIG. 3). The industry table storage section 126 stores the industry table (FIG. 4).

The distributed representation model storage unit 130 includes a word table storage unit 132 , a company vector storage unit 134 , a business characteristic word vector storage unit 136 , and an industry vector storage unit 138 .
The word table storage unit 132 stores the word table (FIG. 5). The company vector storage unit 134 was described in relation to FIG. 6. The business characteristic word vector storage unit 136 was described in relation to FIG. 7. The industry type vector storage unit 138 was described in relation to FIG. 8.

The related phrase storage unit 140 includes a similar word list storage unit 142 , a business characteristic word list storage unit 144 , and an industry list storage unit 146 .
The synonymous word list storage unit 142 stores a synonymous word list (FIG. 13). The business characteristic word list storage unit 144 stores a business characteristic word list (FIG. 14). The business category list storage unit 146 stores a business category list (FIG. 15).

FIG. 20 is a flow chart showing the process of the preparation phase processing.
First, the model generation unit 152 executes a distributed representation model generation process (S20). In the distributed representation model generation process, a word table (FIG. 5) is generated as a distributed representation model. The distributed representation model generation process will be described later with reference to FIG. 21.

Next, the company vector calculation unit 154 executes the company vector calculation process (S22). The company vector calculation process will be described later with reference to FIG. 22.

Next, the business characteristic word vector calculation unit 156 executes a business characteristic word vector calculation process (S24). The business characteristic word vector calculation process will be described later with reference to FIG. 23.

Finally, the industry vector calculation unit 158 executes an industry vector calculation process (S26). The industry vector calculation process will be described later with reference to FIG. 24.

FIG. 21 is a flow chart showing the process of generating a distributed representation model.
The model generation unit 152 generates a corpus from the company table (S30). A corpus is large-scale data in which sentences in natural language are structured. The corpus is stored in the data storage unit 110. Specifically, the model generation unit 152 performs morphological analysis on each sentence extracted from the business content sentence of each company, and converts it into a word-spaced format. Word-spaced text is a character string in which words are separated by spaces. The word-spaced data becomes the corpus.

The model generation unit 152 generates word vectors using the corpus (S32). Specifically, the model generation unit 152 obtains word vectors for each word included in the corpus, for example, by a Word2vec learning process. The generated word vectors are stored in the word table storage unit 132. Then, the process returns to S22.

FIG. 22 is a flow chart showing the process of the company vector calculation process.
The company vector calculation unit 154 repeats the following process for each company (S40). The company vector calculation unit 154 calculates the sentence vector of each sentence included in the business content sentence of this company (S42). A sentence vector is generated from the word vector of the words included in the sentence. The generation method is, for example, averaging. In other words, the sentence vector is the average vector of the word vectors.

The company vector calculation unit 154 calculates the average of the sentence vectors of each sentence included in the business content sentence, and sets the calculated average vector as the company vector (S44). The company vector calculation unit 154 may generate the company vector from the sentence vectors by a method other than averaging. The calculated company vector is stored in the company vector storage unit 134.

If there are companies that have not yet been processed (N in S46), the company vector calculation unit 154 returns to processing in S40. When processing has been completed for all companies (Y in S46), it returns to processing in S24.

FIG. 23 is a flow chart showing the process of calculating the business characteristic word vector.
The business characteristic word vector calculation unit 156 repeats the following process for each business characteristic word (S50): The business characteristic word vector calculation unit 156 refers to the company table and selects companies in which this business characteristic word is set (S52).

The business characteristic word vector calculation unit 156 calculates the average vector of the selected company vectors, and sets the calculated average vector as the business characteristic word vector of this business characteristic word (S54). The business characteristic word vector calculation unit 156 may generate the business characteristic word vector from the company vector by a method other than averaging. The calculated business characteristic word vector is stored in the business characteristic word vector storage unit 136.

If there are any business characteristic words that have not been processed yet (N of S56), the business characteristic word vector calculation unit 156 returns to the process of S50. If the process has been completed for all business characteristic words (Y of S56), the process returns to the process of S26.

FIG. 24 is a flow chart showing the process of the business type vector calculation process.
The business type vector calculation unit 158 repeats the following process for each business type (S60): The business type vector calculation unit 158 refers to the company table and selects companies for which this business type is set (S62).

The industry vector calculation unit 158 calculates the average vector of the selected company vectors, and sets the calculated average vector as the industry vector for this industry. The industry vector calculation unit 158 may generate an industry vector from the company vectors by a method other than averaging. The calculated industry vector is stored in the industry vector storage unit 138.

If there are industries that have not yet been processed (N in S66), the industry vector calculation unit 158 returns to the process of S60. When processing has been completed for all industries (Y in S66), the preparation phase processing ends.

FIG. 25 is a flow chart showing the process of the utilization phase processing.
When the search query receiving unit 172 receives a search query from the user terminal 200 that is displaying a search query reception screen (Y in S70), the screen data generating unit 160 executes a list generating process (S72). In the list generating process, a synonym list (FIG. 13), a business characteristic word list (FIG. 14), and an industry list (FIG. 15) are generated. The list generating process will be described later with reference to FIG. 26.

Next, the screen data generation unit 160 executes a screen generation process (S74). In the screen generation process, the synonymous words, business characteristic words, and industry are redisplayed on the search query reception screen. A search for company information is then performed, and a screen showing the search results is displayed. The screen generation process will be described later in relation to FIG. 27.

Then, in the screen data transmission process, the screen data transmission unit 182 transmits the screen data to the user terminal 200 (S76). Then, the process returns to S70. The user terminal 200 displays the search query reception screen and the search result screen based on the received screen data.

When the slider value receiving unit 174 receives a slider value from the user terminal 200 displaying the search query reception screen (Y in S78), the first reference value, the second reference value, and the third reference value are changed based on the slider value. In this example, the slider value is used as it is as the new first reference value, the new second reference value, and the new third reference value.

When the first, second, and third reference values are changed, the screen data generating unit 160 executes a screen generation process based on the changed first, second, and third reference values (S82). Furthermore, the screen data transmitting unit 182 transmits the regenerated screen data in the screen data transmitting process (S84). The user terminal 200 displays the search query reception screen and the search result screen based on the received screen data. Then, the process returns to S70.

If an end instruction is received from the user terminal 200 (Y in S86), the usage phase processing ends. If an end instruction is not received from the user terminal 200, the processing returns to S70.

FIG. 26 is a flow chart showing the process of list generation.
The screen data generation unit 160 calculates the similarity between all words and the search query (S90). If the search query is a word, the word vector of that word is set as the semantic vector of the search query. If the search query is a phrase, the semantic vector of the search query is generated from the word vectors of the words included in the phrase. The generation method is, for example, averaging. Then, the similarity between the word vector of each word and the semantic vector of the search query is obtained. The screen data generation unit 160 calculates, for example, the cosine similarity. The same applies to the following similarities.

The screen data generating unit 160 sorts the words in descending order of similarity to generate a similar word list (S92). The generated similar word list is stored in the similar word list storage unit 142.

The screen data generation unit 160 calculates the similarity between all business characteristic words and the search query (S94). Specifically, the screen data generation unit 160 calculates the similarity between the business characteristic word vector of each business characteristic word and the semantic vector of the search query.

The screen data generation unit 160 sorts the business characteristic words in descending order of similarity to generate a business characteristic word list (S96). The generated business characteristic word list is stored in the business characteristic word list storage unit 144.

The screen data generating unit 160 calculates the similarity with the search query for all industries (S98). Specifically, the screen data generating unit 160 calculates the similarity between the industry vector of each industry and the semantic vector of the search query.

The screen data generating unit 160 sorts the industries in descending order of similarity to generate an industry list (S100). The generated industry list is stored in the industry list storage unit 146. Then, the process returns to S74.

FIG. 27 is a flow chart showing the steps of the screen generation process.
The screen data generation unit 160 places the similar words having a similarity equal to or greater than a first reference value in the similar word display area 304 on the search query reception screen (S110). The screen data generation unit 160 places the business characteristic words having a similarity equal to or greater than a second reference value in the business characteristic word area 306 on the search query reception screen (S112). Furthermore, the screen data generation unit 160 places the industries having a similarity equal to or greater than a third reference value in the industry area 308 on the search query reception screen (S114).

The screen data generation unit 160 searches for company information using an OR condition of these synonyms, business characteristic words, and industry (S116). For example, a new search query is generated in which the synonyms, business characteristic words, and industry are used as character strings to search for, and this search query is input to a database management system. The database management system selects relevant company information by comparing each company's company information (a set of text representing business content sentences, business characteristic words, main industry, and sub-industry). The database management system is software that manages the company database 120 and performs operations on the company database 120.

The screen data generation unit 160 places a company information box that displays the hit company information on the search result screen (S118). Then, the process returns to S70.

[Modification]
It may be possible to delete unnecessary synonymous words, business characteristic words, and industries displayed on the search query reception screen. When an unnecessary synonymous word, business characteristic word, or industry is specified by a user operation, a deletion request for the synonymous word, business characteristic word, or industry is transmitted from the user terminal 200 to the server 100. When a request receiving unit (not shown) of the server 100 receives the deletion request, a deletion unit (not shown) deletes the unnecessary synonymous word, business characteristic word, or industry from the synonymous word list, business characteristic word list, or industry list. The server 100 then performs list generation processing, screen generation processing, and screen data transmission processing again.

An example has been shown in which a search for company information is performed automatically in conjunction with the reception of a search query, but the search for company information may be performed after a user operation to perform a search is received.

The screen data generating unit 160 may arrange the similarity between the search query and the similar words next to the similar words on the search query reception screen. The screen data generating unit 160 may arrange the similarity between the search query and the business characteristic words next to the business characteristic words on the search query reception screen. The screen data generating unit 160 may arrange the similarity between the search query and the business type next to the business type on the search query reception screen.

An example has been shown in which the search query reception screen and the search result screen are displayed side by side, but the search query reception screen and the search result screen may be displayed in a switched manner.

The server 100 may convert the slider value to obtain the first reference value, the second reference value, and the third reference value. The first reference value, the second reference value, and the third reference value may be different values. For example, the screen data generating unit 160 calculates the first reference value, the second reference value, and the third reference value so as to be correlated with the slider value. The screen data generating unit 160 may multiply the slider value by a first coefficient to obtain the first reference value. The screen data generating unit 160 may multiply the slider value by a second coefficient to obtain the second reference value. The screen data generating unit 160 may multiply the slider value by a third coefficient to obtain the third reference value.

When averaging vectors, weighting may be performed for each original word vector, phrase vector, or sentence vector, for example, using an evaluation index such as TF-IDF (Term Frequency-Inverse Document Frequency).

Although an example of searching for corporate information has been given, other types of information may also be searched for. For example, when searching for product information on an e-commerce site, product classification vectors and product characteristic vectors may be calculated for product classification tags and product characteristic tags associated with product introduction sentences. Then, the search conditions may be supplemented using product classification vectors and product characteristic vectors similar to the search query.

For example, when searching for patent documents on a patent document viewing site, patent classification vectors may be calculated for the patent classification tags associated with the patent documents. Then, patent classification vectors similar to the search query may be used to supplement the search criteria.

For example, when searching for posted articles on a posting site of a social networking service (SNS), an article classification vector may be calculated for an article classification tag such as a hashtag that is attached to the posted article. Then, the article classification vector similar to the search query may be used to supplement the search conditions.

The present invention is not limited to the above-described embodiment and modifications, and can be embodied by modifying the components without departing from the spirit of the invention. Various inventions may be formed by appropriately combining multiple components disclosed in the above-described embodiment and modifications. In addition, some components may be deleted from all the components shown in the above-described embodiment and modifications.

FIG. 28 is a diagram showing the company information box included in the search result screen in the state shown in FIG.
In this example, the business characteristic word is "quick", which does not match any of the business characteristic keywords "used", "repair", and "luxury" (FIG. 11) that are similar to the search query phrase. However, the character string of the business characteristic word "luxury" is included in the business content sentence, so this company information is displayed as a search result. As described above in relation to FIG. 16, a hit is also generated when the character string of "luxury" is included in the business content sentence.

100 Server, 110 Data storage unit, 120 Company database, 122 Company table storage unit, 124 Business characteristic word table storage unit, 126 Industry table storage unit, 130 Distributed representation model storage unit, 132 Word table storage unit, 134 Company vector storage unit, 136 Business characteristic word vector storage unit, 138 Industry vector storage unit, 140 Related phrase storage unit, 142 Synonym list storage unit, 144 Business characteristic word list storage unit, 146 Industry list storage unit, 150 Data processing unit, 152 Model generation unit, 154 Company vector calculation unit, 156 Business characteristic word vector calculation unit, 158 Industry vector calculation unit, 160 Screen data generation unit, 162 Search unit, 170 Reception unit, 172 Search query reception unit, 174 Slider value reception unit, 180 Transmission unit, 182 Screen data transmission unit, 190 Communication unit, 200 User terminal, 300 Input area, 302 Slider, 304 Display area, 306 Display area, 308 Display area, 320 Company information box, 324 Display area, 326 Display area, 328 Display area, 330 Display area

Claims (5)

A first reception unit that receives a search query for a database that manages a plurality of sentences linked to a classification term that is at least one of a business characteristic word and an industry type related to a company;
a first calculation unit that calculates variance vectors of words included in the plurality of sentences;
a second calculation unit that calculates a variance vector of each sentence by averaging variance vectors of words included in the sentence;
a third calculation unit that calculates a variance vector of the classification term by averaging the variance vectors of each sentence associated with the same classification term;
A search unit that searches for a sentence that satisfies a condition that a character string of a classification term of a variance vector similar to a variance vector of the search query is included in the character string of the sentence ,
The natural language processing apparatus according to claim 1, wherein the variance vector of the classification term is at least one of a business characteristic word vector representing the meaning of the business characteristic word and an industry vector representing the meaning of the industry . a second reception unit that receives an instruction to adjust a reference value related to the similarity between the variance vectors;
The natural language processing device according to claim 1 , further comprising an extraction unit that extracts the classification terms of the variance vectors similar to the variance vector of the search query based on the adjusted reference value. A first reception unit that receives a search query for a database that manages a plurality of sentences linked to a classification term that is at least one of a business characteristic word and an industry type related to a company;
A natural language processing apparatus comprising: a search unit that searches for the sentence that meets a condition that a character string of the search query is included in a character string of the classification term linked to the sentence . A function of accepting a search query for a database that manages a plurality of sentences linked to a classification term that is at least one of a business characteristic word and an industry type related to a company ;
A function of calculating variance vectors of words included in the plurality of sentences;
A function of calculating the variance vector of each sentence by averaging the variance vectors of the words contained in the sentence;
A function for calculating the variance vector of a classification term by averaging the variance vectors of each sentence associated with the same classification term;
a function of searching for a sentence that satisfies a condition that a character string of a classification term of a variance vector similar to a variance vector of the search query is included in the character string of the sentence ;
The program, characterized in that the variance vector of the classification word is at least one of a business characteristic word vector representing the meaning of the business characteristic word and an industry vector representing the meaning of the industry . A function of accepting a search query for a database that manages a plurality of sentences linked to a classification term that is at least one of a business characteristic word and an industry type related to a company ;
A program that causes an information processing device to perform a function of searching for the sentence that meets the condition that the string of the classification word linked to the sentence contains the string of the search query.

Images