JP5458058B2 - Product name identity determination device and product name identity determination program - Google Patents

Description

  The present invention relates to a marketing analysis technique for purchasing behavior on the web, and relates to a product name identity determination device and a program for determining what is the same product name from various product name descriptions.

  Along with the spread of the Internet environment, a lot of purchasing behavior has been carried out on websites that offer mail order. Since the purchase behavior on these webs is carried out via a web browser, by using the browsing history etc. of the web browser, what products are viewed and compared by consumers, and what products are purchased as a result It has become possible to conduct marketing analysis.

  However, since the product name displayed on the mail order site is variously written even if it is the same product, if the product name written on the web page is used as it is, the consumer sees the same product repeatedly, It is impossible to distinguish whether different products are being compared, and the number of products viewed or purchased by consumers cannot be analyzed correctly.

  On the other hand, it is considered that more accurate marketing analysis can be performed if the same product can be discriminated from the product description in different sites in consideration of such fluctuations in the description.

  Many methods have been proposed to determine whether or not they are the same by determining the shaking of the description on the web.

  As a conventional technique, there is a technique that mainly determines the identity of a person's proper noun (for example, Patent Document 1). This method constructs a graph structure in which websites are nodes and links are created between websites containing the same person name, and proper nouns appearing in a set of closely related websites are the same. Judge that it exists.

  In the present invention, a technique used when constructing category information is described in Patent Document 2, and a technique used when determining whether or not the product name notation information is in a wobble relation is described in Patent Document 3. .

JP 2006-107361 A JP 2010-12001 A Japanese Patent No. 4084515

  Since the technique disclosed in Patent Document 1 is a technique for eliminating the ambiguity of proper nouns having the same notation, the same product as seen on a mail order site is represented by different notations There was a problem that it was not possible to determine the identity of.

  The present invention has been made in view of the above points. The purpose of the present invention is to determine the identity of a product name in consideration of the sales promotion wording that is characteristic in the description of the product name. It is to provide an identity determination device and a program.

The product name identity determination apparatus of the present invention for solving the above-mentioned problem is based on a product description database in which product name information is stored, taking as input a set of two product name descriptions to be subjected to identity determination. In addition, for each word included in the product description database, a product score that is high for a word useful for identifying a specific product and low for words included in a plurality of products. The product score calculation means that accumulates in the product score database and the combination of the two input product name notations are analyzed, and the common part and the difference part of the words and phrases included in each product name notation are obtained. If the product score of the phrase is obtained with reference to the product score database, the product score of the commonly appearing phrase is high, and the product score of the phrase appearing only on one side is low In, and a identity determining means for determining a set of input trade name notation with high identity, wherein the trade designation database, trade name category information indicating a classification category notation information and product item The product score calculation means accumulates as information, and the distribution of the category belonging to the product group including the processing target word is analyzed with reference to the product notation database, so that the product score calculation means appears biased only to a specific category. A high product score is calculated as a useful phrase to uniquely identify a product for a phrase, and a low product score is calculated as an ineffective phrase to characterize a product for terms that appear evenly across multiple categories A means for summing up the category belonging to the product group including the processing target phrase, and let n be the number of differences in the total number of category belonging to each category. The number over data as xi (i takes 1 to n), determining the product score s of the processing target phrase in the formula, it is characterized in that.

  According to the above configuration, when the same product is represented in different notation, the identity can be determined, and the product taking into account the words for the purpose of promotion that appear characteristically in the description of the product name It is possible to determine the identity of names.

  According to the present invention, when the same product is expressed in different notations, the identity can be determined, and the product taking into account the words for the purpose of promotion that appear characteristically in the description of the product name It is possible to determine the identity of names.

  For example, it is possible to guarantee the identity of different product names on different websites, so if a consumer sees products with similar names on various sites, the same product is compared and sold on multiple sites. It is possible to determine whether a site with good conditions is being searched or whether a better product is being searched by comparing different products.

  As a result, the number of browsing and purchasing of each product across the sites is correctly counted, and the accuracy of marketing analysis such as the sales share of each product and the purchase frequency of the user's products can be improved.

The principle block diagram of this invention. The block diagram of the brand name identity determination apparatus in the 1st Embodiment of this invention. Explanatory drawing which shows the example of the information accumulate | stored in the merchandise description database in the 1st Embodiment of this invention. The flowchart which shows the flow of a process of the goods score calculation part in the 1st Embodiment of this invention. Explanatory drawing which shows the example of the information acquired from the product description database in the 1st Embodiment of this invention. Explanatory drawing which shows the example of the information accumulate | stored in the goods score database in the 1st Embodiment of this invention. The flowchart which shows the flow of a process of the identity determination part in the 1st Embodiment of this invention. The block diagram of the brand name identity determination apparatus in the 2nd Embodiment of this invention. The flowchart which shows the flow of a process of the identity determination part in the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. First, the principle of the present invention will be described with reference to the block diagram of FIG. In FIG. 1, reference numeral 100 is an input of a pair (product name notation pair 110) consisting of two product name notations to be subjected to identity determination, and whether or not the product name notation pair 110 represents the same product. A product name identity determination device that outputs a product identity determination result 130 representing the product name, and includes a product score calculation unit 140, a product score database 150, and an identity determination unit 160, and the product name identity The determination device 100 is connected to a product description database 120 in which a large amount of product description data is accumulated.

  The product score calculation means 140 analyzes a large number of product descriptions included in the product description database 120, and the product score becomes lower as the product name is inappropriate for each word / phrase included in the product description. And the result is stored in the product score database 150. The term inappropriate for the expression of the product name is a term related to the sales state of the product such as “free shipping”, “sale”, “2000 yen discount”, and is used in many product descriptions regardless of the type of product. Calculate a low score for the included words.

  When calculating a product score, a product name that represents a product that is registered as a trademark or a term that well describes the product, such as “rice cooker”, is a term that is used only in the category to which the product belongs. On the other hand, terms related to the sales status of a product use a characteristic that appears regardless of the category of the product, and a term that appears biased only in a specific category (a term useful for identifying a specific product) A high product score is calculated, and a low product score is calculated for words that appear evenly over a plurality of categories.

  The identity determination means 160 determines whether or not they represent the same product for the input product name notation pair 110 and outputs the result. The identity is determined by determining that the identity is high when a common phrase in the phrase included in each product name notation is a product phrase having a high product score and appropriate for the description of the product. It is determined that the identity is low when the product score is high and the word is appropriate for the product notation.

  FIG. 2 shows the configuration of the product name identity determination apparatus 200 according to the first embodiment of the present invention. The product name identity determination apparatus 200 includes a notation fluctuation replacement unit 240, a product score calculation unit 250 as a product score calculation unit, a product score database 260, and an identity determination unit 270 as an identity determination unit, and includes a product name description pair. An input device 210, a product description database 220, and an identity determination result display device 230 are connected.

  The product name identity determination device 200 is configured by a computer, for example, and includes hardware resources of a normal computer, such as a ROM, a RAM, a CPU, an input device, an output device, a communication interface, a hard disk, a recording medium, and a driving device thereof. Yes.

  As a result of the cooperation between the hardware resource and the software resource (OS, application, etc.), the product name identity determination device 200 has the same product name notation pair input device 210, product notation database 220, as shown in FIG. The sex determination result display device 230, the notation shaking replacement unit 240, the product score calculation unit 250, the product score database 260, and the identity determination unit 270 are implemented.

  The product list database 220, the product score database 260, and the product score calculation unit 250, which will be described later, store a phrase list buffer for acquiring and storing a list of terms included in the product description information. It is assumed that the storage means is constructed.

  The product name notation pair input device 210 is an input device such as a keyboard, and is used by the user to input two product name notations to be determined for identity.

  The merchandise notation database 220 is a database in which a large amount of merchandise information including a set of notation information of merchandise sold at a mail order site on the Internet and its category information is accumulated. An example of information stored in the product description database 220 is shown in FIG.

  In FIG. 3, the notation information is the description content of the product name. In the mail order site, not only the name of the product but also “free shipping” and “20% off” are often written. The category information is information representing a classification category of the product such as “one piece” or “rice cooker”.

  Many mail-order sites each have a product classification structure, and it is possible to construct category classification information by extracting the information from the page. Further, instead of extracting the classification structure of the mail order site, it may be constructed by using a phrase included in the page and assigning a predetermined category using a technique as disclosed in Patent Document 2, for example. .

  Further, when each product information is input, the product description database 220 obtains a group of words constituting the description information by performing morphological analysis on the description information of each product information, and totalizes them to obtain the product description database 220. Assume that all words contained within are retained. In general, an inverted index for making the data in the product notation database 220 searchable by a phrase included in the notation information is constructed in the database, so that the index list of the inverted index is all in the product notation database 220. It can be used as a list of words.

  The identity determination result display device 230 is an external output device such as a display or a printer, and displays the identity determination result of the product name by the product name identity determination device 200.

  The notation shaking replacement unit 240 starts operation when a set of product name notation information to be processed is input from the product name notation pair input device 210, and uses English / Japanese parallel translation technology as disclosed in Patent Document 3. It is used to determine whether the product name notation information to be processed has a notation swing relationship. As a result of the determination, when the combination of the product name description information is in a relationship of notation, one of the items is replaced with the other content to create a set of the same description information and output to the identity determination unit 270. If there is no relationship of notation shaking, the set of product name notation information is output to the identity determination unit 270 as it is.

  The merchandise score calculation unit 250 analyzes a large amount of notation information stored in the merchandise notation database 220, thereby including words that are used evenly in the merchandise notation of all product categories for promotional purposes. It is discriminated as a word and phrase, and a low product score is assigned and stored in the product score database 260.

  The processing flow of the product score calculation unit 250 is shown in FIG.

  When the process starts, the product score calculation unit 250 accesses the product notation database 220, acquires a list of words included in all the product notation information, and stores it in the word list buffer (step 401).

  Next, one word is extracted from the word list buffer and used as processing target word information (step 402).

Next, the product description database 220 is accessed, all product information including the processing target word / phrase in the description information is acquired, the category information included in each acquired product information is totaled, and how many data of each category exist and all The number of data is calculated (step 403).

For example, if there are 10 pieces of data including the phrase “shipping” and the contents are as shown in FIG. 5, the total result of the category information is 2 for “women's clothing” and 3 for “home appliances”. The number of “data” is 2, the number of “music” is 1, the number of “computers” is 2, and the total number of data is 10.

  Next, the product score of the processing target phrase is calculated using the counting result (step 404). The product score s of the processing target phrase is obtained by the following equation, where n is the number of categories that appear in the data including the processing target phrase, and xi is the number of data for each category (i takes 1 to n).

  For this reason, the product score s shows a low value for words used in many categories (n is large), and shows a high value for words that appear only in a small number of categories (n is small). In addition, the standard deviation δ that appears multiple times in a plurality of categories shows a low value for a phrase with a small standard deviation, and has a high value for a phrase with a large standard deviation δ that appears only in a specific category. The obtained product score s is stored in the product score database 260 together with the processing target phrase. FIG. 6 shows an example of data stored in the product score database 260.

  On the other hand, there are 10 items of product information including “One Piece” in the product notation database 220, and the number of category information included in those data is 9 for “Women's clothing” and 1 for “Handicraft”. In this case, the product score s of “One Piece” is 0.238. As a result, “one piece” can be said to be a more productive phrase than “shipping”.

Next, it is confirmed whether or not processing has been performed for all the phrases in the phrase list. If processing has been performed for all the phrases, the processing is terminated. If there is an unprocessed word / phrase, the process returns to step 402 to resume the process (step 405).
When the identity determination unit 270 receives a set of product name description information from the notation fluctuation replacement unit 240, the identity determination unit 270 starts processing, and whether or not the product name description information set is a description for the same product. It is discriminated based on whether the difference part is a word representing the product content.

  A processing flow of the identity determination unit 270 is shown in FIG.

  First, a set of inputted product name description information is subjected to morpheme analysis using an existing morpheme analysis technique, and is transformed into a string of words divided for each morpheme (step 701). For example, if the input product name notation information is “shareholder preferential ticket” and “[free shipping] shareholder preferential ticket [20% off]”, if the morpheme analysis technique is used to divide each morpheme, , Courtesy, ticket "and" [, shipping, free,], shareholders, courtesy, ticket, [, 20%, off,] ".

  Next, the two word strings obtained in step 701 are compared, and a word group that appears in both word strings and a word group that appears only in one word string are extracted (step 702).

  For example, consider the two word strings used in the example of step 701: “shareholder, special offer, ticket” and “[, shipping, free,], shareholder, special offer, ticket, [, 20%, off,]”. At this time, “Shareholder, Special Offer, Ticket” is a group of words that appear in common in the two word strings, and “[, Shipping, Free,], [, 20%, Off,]” appears in only one word. It becomes a group of phrases.

  Next, the product scores of the words included in the commonly appearing phrase group and the phrase group appearing only on one side are obtained by accessing the product score database 260, and the sum of the product scores of the commonly appearing phrase groups And the total of the product scores of the word groups that appear only on one side, a distance score S indicating a value that is so small that the set of two input product name description information is considered to be the same product (step) 703).

  Specifically, C is the sum of the product scores s of commonly appearing word groups, and D is the sum of the product scores s of word groups appearing only on one side, and the distance score S is calculated using the following equation.

  When the obtained distance score S is equal to or less than a predetermined threshold value, the set of input product name description information is determined to be the same product, and the result is output to the identity determination result display device 230. If the obtained distance score S is larger than a predetermined threshold value, it is determined as a different product and the result is output to the identity determination result display device 230 (step 704).

  At this time, instead of the binary result indicating whether or not they are identical, the distance score is output to the identity determination result display device 230 as it is, so that the identity of the set of product name description information may be output. .

  The first embodiment is a method for determining identity by using a common part and a difference part of words constituting each set of two input product name description information. However, in the product name, even if the two phrases “casual” and “coat” are common, such as a set of product name notation information such as “10% OFF casual coat” and “best for long coat casual use” There are cases where the order of appearance is meaningful in determining identity.

  The second embodiment of the present invention is a technique for improving the determination accuracy by determining the identity of a product name having the above-described characteristics in consideration of the appearance order of words.

  FIG. 8 shows the configuration of the product name identity determination apparatus 800 according to the second embodiment of the present invention.

  The product name identity determination device 800 includes a notation fluctuation replacement unit 240, a product score calculation unit 250, a product score database 260, and an identity determination unit 870 as identity determination means, and includes a product name description pair input device 210, a product description A database 220 and an identity determination result display device 230 are connected.

  Among these, the notation shaking replacement unit 240, the product score calculation unit 250, the product score database 260, the product name notation pair input device 210, the product notation database 220, and the identity determination result display device 230 perform the same operations as in the first embodiment. Therefore, only the identity determination unit 870 operates differently from the first embodiment.

  Hereinafter, only the process of the identity determination unit 870 that operates differently from the first embodiment will be described.

  When the identity determination unit 870 receives a set of product name description information from the notation fluctuation replacement unit 240, the identity determination unit 870 starts processing, and indicates whether or not the set of product name description information is a description of the same product. It is discriminated based on whether the difference part is a word representing the product content.

  FIG. 9 shows the flow of processing of the identity determination unit 870.

  First, morpheme analysis is performed on a set of inputted product name notation information using an existing morpheme analysis technique, and transformed into a string of words divided for each morpheme (step 901). For example, if the input product name notation information is “shareholder preferential ticket” and “[free shipping] shareholder preferential ticket [20% off]”, when the morpheme analysis technology is used to divide each morpheme, , Courtesy, ticket "and" [, shipping, free,], shareholders, courtesy, ticket, [, 20%, off,] ".

  Next, it is necessary to add / delete when transforming from one of the two word strings obtained in step 901 to the other using an algorithm using dynamic programming used for general edit distance calculation. A certain phrase is extracted (step 902).

  For example, considering the transformation between two word strings “shareholder, special offer, ticket” and “[, shipping, free,], shareholder, special offer, ticket, [, 20%, off,]”, the previous morpheme It can be seen that 8 characters “[, Shipping, Free,], [, 20%, Off,]” are inserted into the column.

  Next, the product score of the deleted or added word is acquired from the product score database 260, and the total value is calculated as the semantic distance score of the set of the inputted product name notation information (step 903). Since the product score is lower for words that do not represent the product content, the distance score will be smaller if the deleted or added word does not represent the product content, and the distance score will be smaller if the word represents the product content. Get higher.

  Next, when the obtained distance score is equal to or less than a predetermined threshold value, it is determined that the set of input product name description information is the same product, and the result is output to the identity determination result display device 230. (Step 904). When the obtained distance score is larger than a predetermined threshold value, it is determined as a different product and the result is output to the identity determination result display device 230.

  At this time, instead of the binary result indicating whether or not they are identical, the distance score is output to the identity determination result display device 230 as it is, so that the identity of the set of product name description information may be output. .

  In addition, it goes without saying that the present invention can be realized by configuring a part or all of the functions of each means in the product name identity determination apparatus of the present embodiment by a computer program and executing the program using the computer. In addition, a program for realizing the function by a computer is recorded on a computer-readable recording medium such as FD (Floppy (registered trademark) Disk), MO (Magneto-Optical disk), ROM (Read Only Memory), It can be recorded on a memory card, CD (Compact Disk) -ROM, DVD (Digital Versatile Disk) -ROM, CD-R, CD-RW, HDD, removable disk, etc., and can be stored or distributed. That. It is also possible to provide the above program through a network such as the Internet or electronic mail.

100, 200, 800 ... Product name identity determination device 120, 220 ... Product description database 140 ... Product score calculation means 150, 260 ... Product score database 160 ... Identity determination means 210 ... Product name description pair input device 230 ... Identity Judgment result display device 240 ... notation shaking replacement unit 250 ... product score calculation unit 270, 870 ... identity judgment unit

Claims (4)

The input consists of a set of two product names that are subject to identity determination.
Refers to the product notation database in which product name information is stored, and for each of the phrases included in the product notation database, a high value is obtained for a phrase useful for identifying a specific product. A product score calculating means for calculating a product score that is a low value for the included words and storing it in a product score database;
Analyzes a set consisting of two input product name notations, obtains the common part and difference part of the words included in each product name notation, and obtains the product score of the words by referring to the product score database And an identity determination means for determining that the set of input product name notations has high identity when the product score of the commonly appearing phrase is high and the product score of the phrase appearing only on one side is low. , equipped with a,
In the product description database, product description information and category information representing product classification categories are stored as product name information,
The product score calculation means includes:
By analyzing the distribution of the category belonging to the product group including the processing target phrase with reference to the product notation database, it is possible to uniquely identify the product with respect to a phrase that appears only in a specific category. A means of calculating a high product score as a useful word and a low product score as a word that is not effective in characterizing the product against words that appear evenly across multiple categories ,
The categorization category of the product group including the processing target phrase is totaled, the difference number of the totaled belonging categories is set to n, and the number of data for each category is xi (i takes 1 to n). The product score s is obtained by the following formula.
A product name identity determination device characterized by that.

The identity determination means includes
By dividing the set of two input product name notations into constituent words and referring to the product score database, the product score total C of commonly appearing words and products of words that appear only on one side Each of the total scores D is acquired, and a distance score S that is so low as to be considered to be the same pair of product names is calculated using the following formula, and the distance score S is lower than a predetermined threshold value. In the case where it is determined that the combination of the product name notation indicates the same product,
The product name identity determination device according to claim 1, wherein:

The identity determination means includes
Divide the two pairs of product name notation into the words that make up each, and transform it from one word string to the other using a dynamic programming algorithm that is used for general editing distance calculation A set of terms that need to be added / deleted at the time is acquired, and the total of the product scores of each of the constituent words of the set of terms that need to be added / deleted obtained by referring to the product score database is aggregated The total value of the product scores is calculated as a distance score that is so low that the pair of input product name notations is considered to be the same, and the product is calculated when the distance score is lower than a predetermined threshold value. It is determined that the set of name notation indicates the same product.
The product name identity determination device according to claim 1, wherein: A product name identity determination program for causing a computer to function as each means according to any one of claims 1 to 3 .

Images