JP7565077B2 - Information processing device, information processing method, and information processing system - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing system that provide information regarding a user's interest in a product.

Patent Document 1 discloses a system that uses an integrated circuit (IC) attached to products displayed in a store to detect whether a user has picked up a product, and identifies products that the user did not purchase by detecting the user's movement path within the store.

JP 2004-348681 A

For product planning and promotion purposes, product manufacturers and sellers are required to know what products users are interested in and to what extent. The system disclosed in Patent Document 1 can tell which products displayed in a store users have purchased and which products they have not purchased, but it cannot tell the user's level of interest in the products.

The present invention was made in consideration of these points, and aims to enable product manufacturers or sellers to understand the level of user interest in the product.

The information processing device of the first aspect of the present invention has an acquisition unit that acquires a captured image of an area including a user by a camera associated with a product or product information related to the product, an identification unit that identifies the behavior of the user in the captured image, and a calculation unit that calculates an interest level indicating the degree of interest the user has in the product based on the behavior.

The acquisition unit may acquire the captured image generated by capturing an image of the user with the camera installed in a facility for displaying the product.

The acquisition unit may acquire the captured image generated by the camera provided in a user terminal used by the user capturing an image of the user viewing content for displaying the product displayed on the user terminal.

The identification unit may identify as the behavior at least one of the following: the time or number of times that it is detected that the user looked at the product or the product information in the captured image, the number of times that it is detected that the user blinked while looking at the product or the product information, and the time or number of times that it is detected that the user touched the product.

The identification unit may identify the behavior for each part of the product or the product information that the user has looked at or touched in the captured image, and the calculation unit may calculate the interest level based on the behavior for each part of the product or the product information that the user has looked at or touched.

The information processing device may further include an output unit that notifies an information terminal associated with the product of information indicating the part of the product or the part of the product information in which the interest level satisfies a predetermined condition.

The information processing device may further include an output unit that associates information corresponding to the degree of interest in a first product and information corresponding to the degree of interest in a second product among the multiple products exhibited in the same facility or content with the facility or content, and notifies an information terminal associated with the products.

The information processing device may further include an extraction unit that extracts attributes of the products in which the user is interested based on the degree of interest in each of the multiple products.

The identification unit may identify the behavior that influences whether the user purchases the product as a purchase behavior based on purchase information indicating whether the user has purchased the product, and the calculation unit may calculate the degree of interest in a product different from the product based on the behavior and the purchase behavior.

The acquisition unit may acquire the captured image and a voice uttered by a user viewing the product or the product information, and the identification unit may identify the behavior in the captured image and the voice.

The information processing method of the second aspect of the present invention includes the steps of: acquiring an image of an area including a user by a camera associated with a product or product information related to the product, executed by a processor; identifying the behavior of the user in the captured image; and calculating an interest level indicating the degree of interest the user has in the product based on the behavior.

The information processing system of the third aspect of the present invention is an information processing system including an information processing device and an information terminal capable of communicating with the information processing device, the information processing device having an acquisition unit that acquires a captured image of an area including a user by a camera associated with a product or product information related to the product, an identification unit that identifies the behavior of the user in the captured image, and a calculation unit that calculates an interest level indicating the degree of interest the user has in the product based on the behavior, and the information terminal displays information corresponding to the interest level.

The information terminal may output payment information for paying a predetermined reward to a person associated with the facility or content that provided the captured image used to calculate the interest level.

The present invention has the effect of allowing product manufacturers or sellers to understand the level of interest users have in the product.

FIG. 1 is a diagram illustrating an overview of an information processing system. FIG. 1 is a block diagram of an information processing system. FIG. 2 illustrates an exemplary captured image. 10 is a schematic diagram for explaining a method for calculating a degree of interest in a product or product information for each part that the user has looked at or touched; FIG. FIG. 11 is a schematic diagram for explaining a method for extracting attributes of a product in which a user is interested. FIG. 2 is a schematic diagram of exemplary interest information displayed by a management terminal. FIG. 11 is a flowchart illustrating an information processing method executed by the information processing device. 11 is a schematic diagram for explaining a method of reflecting purchase information in rules for calculating an interest level. FIG.

[Outline of Information Processing System]
1 is a diagram showing an overview of an information processing system according to this embodiment. The information processing system includes an information processing device 1, a facility terminal 2, a user terminal 3, and a management terminal 4. The information processing system may also include other devices such as servers and terminals.

The information processing device 1 is a computer that identifies user behavior based on an image captured of an area including the user by a camera associated with the product or product information, and calculates a degree of interest that indicates the user's level of interest in the product based on the identified behavior. A user is a person who visits a facility for displaying products, or a person who browses content such as a website for displaying products. In the facility or content, products may not only be displayed, but also sold.

The facility terminal 2 is an information terminal such as a computer installed in a facility such as a store or space for displaying products, and is, for example, a smartphone, tablet terminal, or personal computer. The facility terminal 2 is installed near the products displayed at the facility, or carried by users within the facility. The facility terminal 2 displays product information related to the products, and captures images of users who are near the products or the product information. The facility terminal 2 has a display unit such as a liquid crystal display, an operation unit such as a touch panel, and an imaging unit such as a camera. The facility terminal 2 transmits and receives information to and from the information processing device 1 via communication.

The user terminal 3 is an information terminal such as a computer used by a user, and is, for example, a smartphone, a tablet terminal, or a personal computer. The user terminal 3 displays content such as a website for displaying products via a network such as the Internet, and captures an image of the user of the user terminal 3 displaying content including product information related to the products. The user terminal 3 has a display unit such as a liquid crystal display, an operation unit such as a touch panel, and an imaging unit such as a camera. The user terminal 3 transmits and receives information to and from the information processing device 1 through communication.

The management terminal 4 is an information terminal such as a computer used by a producer or seller of a product, and is, for example, a smartphone, tablet terminal, or personal computer. The management terminal 4 displays various information received from the information processing device 1. The management terminal 4 has a display unit such as a liquid crystal display, and an operation unit such as a touch panel. The management terminal 4 transmits and receives information to and from the information processing device 1 via communication.

An overview of the processing executed by the information processing system according to this embodiment is described below. The information processing device 1 acquires a captured image of an area including a user by a camera associated with a product or product information related to the product (1). The captured image is, for example, a captured image generated by a camera of a facility terminal 2 capturing an image of the user, or a captured image generated by a camera of a user terminal 3 capturing an image of the user.

The information processing device 1 identifies the user's behavior in the captured image (2). The user's behavior is, for example, at least one of the following: the time or number of times that the user looks at the product or product information in the captured image, the number of times the user blinks while looking at the product or product information, and the time or number of times that the user touches the product.

The information processing device 1 calculates a degree of interest indicating the degree of interest the user has in the product based on the identified user behavior (3). The information processing device 1 calculates the degree of interest corresponding to the user behavior using predetermined rules. For example, the information processing device 1 calculates the degree of interest so that the longer the user spends looking at the product or product information, the higher the degree of interest, and the shorter the user spends looking at the product or product information, the lower the degree of interest.

The information processing device 1 notifies the management terminal 4 of interest information corresponding to the calculated interest level (4). The interest information is, for example, information that associates a product with the calculated interest level for that product.

In this way, the information processing device 1 identifies the behavior of the user who is or is not looking at a product or product information in the user's captured image, and calculates the user's level of interest in the product based on the identified behavior. This allows the information processing device 1 to visualize the level of interest in a product by the user in the process of considering purchasing the product, and allows the product manufacturer or seller to understand the level of interest in the product by the user.

[Configuration of Information Processing System]
FIG. 2 is a block diagram of an information processing system according to this embodiment. In FIG. 2, the arrows indicate the main data flows, and there may be data flows other than those shown in FIG. 2. In FIG. 2, each block indicates a functional configuration, not a hardware (device) configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between blocks via any means, such as a data bus, a network, or a portable storage medium.

The information processing device 1 has a storage unit 11 and a control unit 12. The information processing device 1 may be configured by connecting two or more physically separate devices via a wired or wireless connection. The information processing device 1 may also be configured by a cloud, which is a collection of computer resources.

The storage unit 11 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, etc. The storage unit 11 pre-stores a program to be executed by the control unit 12. The storage unit 11 also pre-stores predetermined rules such as a formula for calculating the interest level from a value indicating the user's behavior. The storage unit 11 may be provided outside the information processing device 1, in which case data may be exchanged between the storage unit 11 and the control unit 12 via a network.

The control unit 12 has an acquisition unit 121, an identification unit 122, a calculation unit 123, an extraction unit 124, and an output unit 125. The control unit 12 is a processor such as a CPU (Central Processing Unit), and functions as the acquisition unit 121, the identification unit 122, the calculation unit 123, the extraction unit 124, and the output unit 125 by executing a program stored in the memory unit 11. At least some of the functions of the control unit 12 may be performed by an electric circuit. In addition, at least some of the functions of the control unit 12 may be realized by the control unit 12 executing a program executed via a network.

The process executed by the information processing device 1 will be described in detail below. The acquisition unit 121 acquires from the user terminal 3 a captured image of an area including a user captured by a camera associated with a product or product information related to the product. That is, the captured image acquired by the acquisition unit 121 includes users who may or may not be viewing the product or product information related to the product. A product is an item exhibited in the content of a facility or a website. Product information is information related to a product, and includes, for example, the product name, price, weight, place of origin, product description, etc. The captured image is a still image or a moving image.

When a user views a product or product information at a facility for displaying products, the product is displayed at the facility and the product information is posted near the product or displayed on the display unit of the facility terminal 2. The user looks at or touches the product or views the product information. In this case, the acquisition unit 121 acquires an image generated by a camera (e.g., the camera of the facility terminal 2) associated with the product or product information at the facility capturing an image of a user in the vicinity of the product or product information.

FIG. 3(a) is a diagram showing an exemplary captured image IM1 acquired by the acquisition unit 121 from the facility terminal 2. The acquisition unit 121 identifies a user appearing in the captured image IM1, and stores in the storage unit 11 user identification information capable of identifying the identified user in association with the captured image IM1. The user identification information is, for example, a user ID (Identification) that has been assigned to the user in advance. If a user ID has not been assigned to the user, the acquisition unit 121 may automatically generate a user ID and assign it to the user.

The acquisition unit 121 identifies the user appearing in the captured image, for example, by performing a known image recognition process on the captured image. The acquisition unit 121 may also acquire facility identification information from the user terminal 3, acquired by the camera of the user terminal 3 reading a code posted in a facility. The code is, for example, a one-dimensional code or a two-dimensional code generated by encoding the facility identification information according to a predetermined rule. In this case, the acquisition unit 121 identifies the user using the user terminal 3 that captured the code as the user appearing in the captured image captured at the facility corresponding to the facility identification information.

On the other hand, when a user views a product or product information in content such as a website for displaying products, the product image and product information are displayed within the content displayed on the display unit of the user terminal 3. In this case, the acquisition unit 121 acquires an image generated by the camera of the user terminal 3 displaying the product image and product information (i.e., the camera of the user terminal 3 associated with the product or product information) capturing an image of the user viewing the content displayed on the user terminal 3.

FIG. 3(b) is a diagram showing an exemplary captured image IM2 acquired by the acquisition unit 121 from the user terminal 3. The acquisition unit 121 identifies the user who is using the user terminal 3, and stores in the storage unit 11 user identification information capable of identifying the identified user, the content being viewed by the user, and the captured image IM2 in association with each other. The user identification information is, for example, a user ID that has been assigned to the user in advance. The user ID is, for example, set in advance in the user terminal 3 that has captured the image of the user. If a user ID has not been assigned to the user, the acquisition unit 121 may automatically generate a user ID and assign it to the user.

The identification unit 122 identifies the user's behavior in the captured image acquired by the acquisition unit 121. For example, the identification unit 122 identifies at least one of the following as the user's behavior in the captured image: the time or number of times the user looked at the product or product information, the number of times the user blinked while looking at the product or product information, and the time or number of times the user touched the product. The identification unit 122 may also identify the above-mentioned behavior for each part of the product or product information that the user looked at or touched.

In the example shown in FIG. 3(a), the identification unit 122 recognizes the user's eyes and body and the product P in the captured image IM1, for example, by executing a known image recognition process on the captured image IM1 acquired by the acquisition unit 121 from the facility terminal 2. The identification unit 122 identifies the line of sight L corresponding to the direction in which the user is looking, based on the user's eyes recognized in the captured image IM1. The identification unit 122 identifies, for example, the direction from the center of the user's eye to the pupil as the line of sight L.

The identification unit 122 determines whether the user is looking at the product P or product information related to the product P, based on the relationship between the line of sight L and the position of the product P or the facility terminal 2. The identification unit 122 may also identify the part of the product P or the product information that the user has looked at, based on the relationship between the line of sight L and the position of the product P or the facility terminal 2. The part of the product P or the product information that the user has looked at is any of the members or parts that make up the product P, or any of the information included in the product information.

The determination unit 122 determines the time or number of times the user has viewed the product P or product information by repeatedly determining whether the user has viewed the product P or product information in the captured image IM1, which is a plurality of still images or a moving image.

The identification unit 122 also identifies the number of times the user's eyes open and close in the captured image IM1, which is a plurality of still images or a moving image, as the number of blinks. The identification unit 122 also identifies whether the user is touching the product P based on the relationship between the position of the user's body recognized in the captured image IM1 and the position of the product P. The identification unit 122 may also identify the part of the product P that the user touched based on the relationship between the position of the user's body recognized in the captured image IM1 and the position of the product P. The part of the product P or the product information that the user touched is any of the members or parts that make up the product P.

The determination unit 122 determines the time or number of times the user touches the product P by repeatedly determining whether or not the user touches the product P in the captured image IM1, which is a plurality of still images or a moving image.

In the example shown in FIG. 3(b), the identification unit 122 recognizes the user's eyes in the captured image IM2, for example, by executing a known image recognition process on the captured image IM2 acquired by the acquisition unit 121 from the user terminal 3. The identification unit 122 identifies the line of sight L corresponding to the direction in which the user is looking, based on the user's eyes recognized in the captured image IM2. The identification unit 122 identifies, for example, the direction from the center of the user's eye to the pupil as the line of sight L.

The identification unit 122 also identifies the product and product information corresponding to the content displayed on the user terminal 3 at the time the captured image IM2 was captured as the product P and product information viewed by the user.

The identification unit 122 determines whether the user is looking at the product P or product information related to the product P, based on the relationship between the line of sight L and the position of the user terminal 3. The identification unit 122 may also identify the part of the product P or the product information that the user has looked at, based on the relationship between the line of sight L and the position of the user terminal 3. The part of the product P or the product information that the user has looked at is any of the members or parts that make up the product P, or any of the information included in the product information.

The identification unit 122 identifies the time or number of times the user viewed the product P or product information by repeatedly determining whether or not the user is viewing the product P or product information in the captured image IM2, which is a plurality of still images or a moving image.

The determination unit 122 also determines the number of times the user's eyes are opened and closed in the captured image IM2, which is a plurality of still images or a moving image, as the number of blinks.

The calculation unit 123 calculates the degree of interest that indicates the degree of interest the user has in the product based on the user behavior identified by the identification unit 122. The calculation unit 123 acquires predetermined rules for calculating the degree of interest based on the user behavior that are pre-stored in the storage unit 11. The rules for calculating the degree of interest include, for example, a formula for calculating the degree of interest from at least one value among the time or number of times the user looked at the product or product information, the number of times the user blinked while looking at the product or product information, and the time or number of times the user touched the product.

The calculation unit 123 calculates the degree of interest by applying the acquired rules to the user behavior identified by the identification unit 122. For example, the degree of interest is greater the greater the degree of interest the user has in the product, and is smaller the greater the degree of interest the user has in the product. The degree of interest is expressed, for example, as a numerical value in the range from 0 to 100. The degree of interest is not limited to the values shown here, and may be other values corresponding to the degree of interest the user has in the product.

In this way, the information processing device 1 can calculate the level of interest based on the user's behavior identified from the captured image, allowing the product manufacturer or seller to understand the user's level of interest in the product as an objective numerical value.

The calculation unit 123 may also calculate the degree of interest for each part of the product or product information that the user has viewed or touched. In this case, the calculation unit 123 calculates the degree of interest by applying rules to the user's behavior for each part of the product or product information that the user has viewed or touched, identified by the identification unit 122. The calculation unit 123 may also determine the maximum degree of interest among the degrees of interest calculated for each part of the product or product information that the user has viewed or touched, as the degree of interest in the entire product.

Figure 4 is a schematic diagram for explaining a method for calculating the degree of interest for each part of a product or product information that the user has seen or touched. In the example of Figure 4, the calculation unit 123 calculates the degree of interest for each of the sides and handles of the product. The calculation unit 123 also calculates the degree of interest for each of the product name, price, and product description of the product information. The calculation unit 123 may also calculate the degree of interest for other parts of the product or product information.

The extraction unit 124 extracts attributes of products in which the user is interested based on the degree of interest calculated by the calculation unit 123 for each of the multiple products. Product attributes are information on the characteristics of the product, such as weight, color, pattern, place of origin, and components (e.g., whether or not the product has a handle). The product attributes are associated with the product and stored in advance in the memory unit 11. The extraction unit 124 extracts, for example, attributes that are possessed by products with a relatively high degree of interest among the multiple products and that are not possessed by products with a relatively low degree of interest, as attributes of products in which the user is interested.

The extraction unit 124 may extract attributes of products in which a single user is interested, based on the degree of interest of the user in each of a plurality of products. The extraction unit 124 may also extract attributes of products in which general users are interested, based on the degree of interest of a plurality of users in each of a plurality of products.

Figure 5 is a schematic diagram for explaining a method for extracting attributes of a product in which a user is interested. In the example of Figure 5, the extraction unit 124 extracts the attributes of "handle" (i.e., having a handle) and "lightweight" that are possessed by a product that has a relatively high level of interest among a plurality of products, as attributes of the product in which the user is interested. In this way, the information processing device 1 can identify product attributes that affect the user's level of interest in the product by extracting product attributes based on the user's level of interest in the product.

The output unit 125 associates interest information corresponding to the interest level of the product calculated by the calculation unit 123 with the product and stores it as a database in the storage unit 11, and notifies the management terminal 4 associated with the product. The management terminal 4 is, for example, an information terminal used by the manufacturer or seller of the product, and is associated with the product in advance.

The interest information includes at least one of the following: the degree of interest in the entire product, the degree of interest for each part of the product or product information, and information indicating the part of the product or the part of the product information whose interest degree satisfies a predetermined condition. The part of the product or the part of the product information whose interest degree satisfies a predetermined condition is, for example, a part of the product or the product information whose interest degree is equal to or exceeds a predetermined threshold, or a predetermined number of parts in descending order of interest degree. The interest information may also include information indicating the behavior of the user with respect to the product identified by the identification unit 122.

Fig. 6(a) is a schematic diagram of exemplary interest information displayed by the management terminal 4. The management terminal 4 displays the interest information notified from the output unit 125 on the display unit. In the example of Fig. 6(a), the management terminal 4 displays, as interest information, information indicating the interest level for the entire product and two product parts or product information parts in descending order of interest level for the product.

In this way, the information processing device 1 can allow the product manufacturer or seller to understand the user's level of interest in the product by outputting the interest level calculated based on the captured image. In addition, the information processing device 1 can inform the product manufacturer or seller of the parts of the product or product information that are affecting the user's interest by notifying the parts of the product or product information whose interest level satisfies a predetermined condition.

The output unit 125 may also notify the management terminal 4 associated with a plurality of products, which are exhibited in the same facility or content, of interest information corresponding to the degree of interest in a first product and interest information corresponding to the degree of interest in a second product, by associating the interest information with the facility or content.

Figure 6 (b) is a schematic diagram of interest information for two exemplary products displayed by the management terminal 4. In the example of Figure 6 (b), the management terminal 4 displays the interest information for two products exhibited in one facility side by side. This allows the information processing device 1 to make it easier for product manufacturers or sellers to compare the degree of user interest in multiple products exhibited in the same facility or content.

The output unit 125 may also notify the management terminal 4 associated with a product of a captured image of the user when the user's interest in the product satisfies a predetermined condition (for example, when the interest is equal to or greater than a predetermined threshold) as interest information. This allows the information processing device 1 to allow the manufacturer or seller of the product to understand the user's behavior that indicates the user's interest.

The output unit 125 may also notify the user terminal 3 or the management terminal 4 of information indicating the attributes of the product extracted by the extraction unit 124 as interest information. For example, the output unit 125 causes the user terminal 3 displaying content such as a website for displaying products, or the management terminal 4 displaying interest information, to display tags indicating the product attributes extracted for the product (such as "light" or "cheap"). This allows the information processing device 1 to allow the user, producer, or seller to understand the attributes of products in which the user is interested that have been extracted in the past.

The output unit 125 may also notify the user terminal 3 used by a single user of one or more products having attributes of products of interest extracted for that user by the extraction unit 124 as interest information. This allows the information processing device 1 to suggest products having attributes of interest to the user and assist the user in selecting products.

[Flowchart of information processing method]
7 is a diagram showing a flowchart of an information processing method executed by the information processing device 1 according to the present embodiment. The acquisition unit 121 acquires a captured image of an area including a user by a camera associated with a product or product information related to the product (S11). The captured image is captured by a camera installed in a facility such as a store or space, or a camera of a user terminal 3 used by the user.

The identification unit 122 identifies the user's behavior in the captured image acquired by the acquisition unit 121 (S12). For example, the identification unit 122 identifies, as the user's behavior, at least one of the following: the time or number of times that the user looked at the product or product information in the captured image; the number of times that the user blinked while looking at the product or product information; and the time or number of times that the user touched the product.

The calculation unit 123 calculates the degree of interest that indicates the degree of interest the user has in the product based on the behavior of the user identified by the identification unit 122 (S13). The calculation unit 123 calculates the degree of interest using predetermined rules that are pre-stored in the memory unit 11.

The extraction unit 124 extracts attributes of products in which the user is interested based on the interest level calculated by the calculation unit 123 for each of the multiple products (S14). The output unit 125 associates interest information corresponding to the interest level of the product calculated by the calculation unit 123 with the product, stores the interest information as a database in the storage unit 11, and notifies the management terminal 4 associated with the product (S15).

[Effects of the embodiment]
In the information processing system according to the present embodiment, the information processing device 1 identifies the behavior of the user who is or is not looking at a product or product information in the captured image of the user, and calculates the user's level of interest in the product based on the identified behavior. In this way, the information processing device 1 visualizes the level of interest in the product by the user in the process of considering purchasing the product, and allows the manufacturer or seller of the product to understand the level of interest of the user in the product.

[First Modification]
In this modification, the information processing device 1 reflects purchase information indicating whether or not a user has purchased a product in the rules for calculating the interest level. Fig. 8 is a schematic diagram for explaining a method for reflecting purchase information in the rules for calculating the interest level in this modification.

After the identification unit 122 identifies the user's behavior with respect to the product, the acquisition unit 121 acquires purchase information indicating whether the user has purchased the product. The acquisition unit 121 acquires purchase information such as a purchase history indicating the products purchased by the user from, for example, a sales management system for managing the sales of products in content such as a facility or a website.

The acquisition unit 121 may acquire purchase information indicating whether or not the user purchased a product in content such as a website viewed after viewing the product at a facility. In this case, for example, when the user terminal 3 captures an image of a code posted at a facility, the acquisition unit 121 associates the facility identification information indicated by the code with user identification information previously set in the user terminal 3, and stores the associated information in the storage unit 11. The acquisition unit 121 determines that the user using the user terminal 3 was in the facility corresponding to the facility identification information associated with the user identification information used to access the content.

The acquisition unit 121 then acquires purchase information indicating whether the user purchased the product viewed by the user in the specified facility in the content. This enables the information processing device 1 to associate the user who was at the facility with the user who viewed the content, and to identify that the user purchased the product in the content after viewing it in the facility.

The acquisition unit 121 may also acquire purchase information based on a captured image of the user. In this case, the acquisition unit 121 determines whether or not the user has approached a sales location such as a cash register in the facility based on the captured image. If the user approaches a sales location after viewing a product in the facility, the acquisition unit 121 presumes that the user has purchased the product, and if the user has not approached the sales location, the acquisition unit 121 presumes that the user has not purchased the product. The acquisition unit 121 acquires the result of the estimation of whether or not the user has purchased the product as purchase information.

The identification unit 122 identifies user behavior that influences whether or not the user will purchase a product as a purchase behavior based on the purchase information acquired by the acquisition unit 121. In the example of FIG. 8, among the user's behaviors, the number of blinks when the user is looking at a product that the user has purchased is greater than the number of blinks when the user is looking at a product that the user has not purchased. Therefore, the identification unit 122 identifies the number of blinks as a purchase behavior that influences whether or not the user will purchase a product. The identification unit 122 stores information indicating the identified purchase behavior in the storage unit 11.

When a user views a second product that is different from the first product used to identify the purchasing behavior, the calculation unit 123 calculates the degree of interest in the second product based on the user's behavior identified for the second product by the identification unit 122 and the purchasing behavior identified for the first product by the identification unit 122. In other words, the calculation unit 123 reflects the purchasing behavior identified for the first product viewed by the user in the past in the rules for calculating the degree of interest, and then calculates the user's degree of interest in the second product.

For example, the calculation unit 123 increases the weight of the value corresponding to the purchasing behavior (the number of blinks in the example of FIG. 8) in a calculation formula included in the rule for calculating the interest level obtained from the storage unit 11. This increases the degree to which the value corresponding to the purchasing behavior influences the interest level.

In the information processing system according to this modified example, the information processing device 1 identifies purchase behavior that influences whether or not a user purchases a product based on whether or not the user has purchased the product, and calculates the interest level using the identified purchase behavior. This allows the information processing device 1 to reflect the user's behavior when actually purchasing a product in the rules for calculating the interest level, thereby improving the accuracy of the calculated interest level.

[Second Modification]
In this modification, the information processing device 1 calculates the interest level based on the captured image of the user as well as the voice uttered by the user. The acquisition unit 121 acquires a captured image of an area including the user captured by a camera associated with the product or product information, and the voice uttered by the user looking at the product or product information. While looking at products in the facility, the user talks to the store clerk or talks with other users. For example, the acquisition unit 121 acquires from the facility terminal 2 the voice collected from the user by a microphone installed in the facility (for example, a microphone provided in the facility terminal 2).

The identification unit 122 identifies the user's behavior in the captured image in the same manner as in the above-described embodiment, and also identifies the user's behavior in the voice. For example, in the voice, the identification unit 122 identifies at least one of the pitch (i.e., frequency) of the user's voice and the time or number of times the user utters the voice as the user's behavior.

The calculation unit 123 acquires a predetermined rule for calculating the degree of interest based on the user's behavior, which is stored in advance in the storage unit 11. The rule for calculating the degree of interest includes, for example, a formula for calculating the degree of interest from a value indicating the user's behavior identified based on the captured image and a value indicating the user's behavior identified based on the audio. The calculation unit 123 calculates the degree of interest by applying the acquired rule to the user's behavior identified by the identification unit 122 based on the captured image and audio.

In the information processing system according to this modified example, the information processing device 1 calculates the interest level based on the voice uttered by the user as well as on the captured image of the user, and therefore can provide product manufacturers or sellers with interest levels that reflect a wider variety of user behaviors.

[Third Modification]
In this modification, the information processing device 1 or the management terminal 4 has a function related to charging and payment. The information processing device 1 or the management terminal 4 outputs payment information for paying a predetermined reward to a person (such as an owner or an exhibitor) associated with a facility or content that provides information (captured image) used to identify a user's behavior related to a product or calculate the interest level in the product. The payment information includes, for example, an amount calculated based on the number of times, data volume, or period that the store terminal 2 or the user terminal 3 transmits captured images to the information processing device 1. This allows the information processing device 1 or the management terminal 4 to provide a reward to the owner or exhibitor of the facility or content that provides information for calculating the interest level.

Similarly, when the information processing device 1 receives an inquiry from an information terminal via a network to obtain information including a user's behavior or interest in a product in the memory unit 11, the information processing device 1 may output billing information for billing the user of the information terminal. The billing information includes, for example, an amount calculated based on the number of inquiries made by the information terminal, the amount of data, or a period of time. This allows the information processing device 1 to obtain an information provision fee from a person who has obtained information including user behavior or interest, or a person who has used a curation service related to user behavior or interest.

Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination also has the effect of the original embodiment.

The processor of the information processing device 1 is responsible for each step (process) included in the information processing method shown in FIG. 7. That is, the processor of the information processing device 1 reads a program for executing the information processing method shown in FIG. 7 from the storage unit 11, and executes the program to control each part of the information processing system, thereby executing the information processing method shown in FIG. 7. Some of the steps included in the information processing method shown in FIG. 7 may be omitted, the order of the steps may be changed, and multiple steps may be performed in parallel.

REFERENCE SIGNS LIST 1 Information processing device 11 Storage unit 12 Control unit 121 Acquisition unit 122 Identification unit 123 Calculation unit 124 Extraction unit 125 Output unit

Claims (12)

an acquisition unit that acquires a captured image of an area including a user by a camera associated with a product or product information related to the product;
an identification unit that identifies a behavior of the user for each part that the user has looked at or touched in the captured image of the product or the product information ;
a calculation unit that calculates a degree of interest that indicates a degree of interest that the user has in the product based on the behavior for each part that the user has looked at or touched in the product or the product information ;
An information processing device having the above configuration. The acquisition unit acquires the captured image generated by the camera installed in a facility for exhibiting the product capturing an image of the user.
The information processing device according to claim 1 . The acquisition unit acquires the captured image generated by the camera of a user terminal used by the user capturing an image of the user viewing content for displaying the product displayed on the user terminal.
3. The information processing device according to claim 1 or 2. The identification unit identifies, as the behavior, at least one of a time or a number of times that the user has looked at the product or the product information in the captured image, a number of times that the user has blinked while looking at the product or the product information, and a time or a number of times that the user has touched the product.
The information processing device according to claim 1 . an output unit that notifies an information terminal associated with the product of information indicating a part of the product or a part of the product information in which the interest level satisfies a predetermined condition;
The information processing device according to claim 1 . an output unit that associates information corresponding to the degree of interest in a first product and information corresponding to the degree of interest in a second product among the multiple products exhibited in a same facility or content with the facility or content, and notifies an information terminal associated with the products;
The information processing device according to claim 1 . an extraction unit that extracts attributes of the products in which the user is interested based on the degree of interest in each of the plurality of products;
an output unit that outputs information corresponding to the attribute extracted by the extraction unit to an information terminal associated with the product;
Further comprising
The information processing device according to claim 1 . The identification unit identifies the behavior that influences whether or not the user purchases the product as a purchase behavior based on purchase information indicating whether or not the user has purchased the product;
the calculation unit calculates the degree of interest in a product different from the product based on the behavior and the purchase behavior.
The information processing device according to claim 1 . The acquisition unit acquires the captured image and a voice uttered by a user who is viewing the product or the product information,
The identification unit identifies the behavior in the captured image and the sound.
The information processing device according to claim 1 . The processor executes
acquiring a captured image of an area including the user by a camera associated with the product or product information related to the product;
Identifying a behavior of the user for each part of the product or the product information that the user has looked at or touched in the captured image;
calculating a degree of interest in the product based on the behavior of the user for each part of the product or the product information that the user has looked at or touched ;
An information processing method comprising the steps of: An information processing system including an information processing device and an information terminal capable of communicating with the information processing device,
The information processing device includes:
an acquisition unit that acquires a captured image of an area including a user by a camera associated with a product or product information related to the product;
an identification unit that identifies a behavior of the user for each part that the user has looked at or touched in the captured image of the product or the product information ;
a calculation unit that calculates a degree of interest that indicates a degree of interest that the user has in the product based on the behavior for each part that the user has looked at or touched in the product or the product information ;
having
The information terminal displays information corresponding to the interest level.
Information processing system. the information terminal outputs payment information for paying a predetermined reward to a person associated with the facility or content that provided the captured image used to calculate the interest level.
The information processing system according to claim 11 .

Images