JP7150688B2 - Apparatus, program and method for estimating user's attribute considering salience of content - Google Patents

Description

The present invention relates to technology for estimating user attributes.

In recent years, digital signage capable of presenting content including still images, moving images, and audio has become widespread in environments and facilities where many people come and go, such as streets, stations, and airports.

Digital signage can appropriately output various contents of electronic data. It is a very suitable means for presenting such as.

At present, in the field of digital signage use, studies are underway to improve presentation effects by presenting content according to the personal attributes of the user who is the presentation destination. In fact, if it is possible to estimate the user's personal attributes, for example, advertising content according to the personal attributes can be presented to increase the advertising effect, or guidance content according to the personal attributes can be presented more accurately or more effectively. It also becomes possible to quickly transmit necessary information.

Here, the personal attributes to be estimated include attributes that can be estimated based on appearance, such as age and gender, and attributes that are difficult to infer from appearance with a predetermined degree of accuracy, such as personality, personality, and primary attributes. There are also attributes such as language or nationality. It is well known that the former can be estimated with considerable accuracy from camera image data (for example, from a camera installed in a digital signage) using image recognition technology based on a deep learning model, for example.

On the other hand, several estimation methods have been proposed for the latter attribute, which is difficult to estimate from appearance alone (that is, from camera image data alone).

For example, in Patent Document 1, content corresponding to a specific psychographic attribute is presented to the user in order to estimate a psychographic attribute that indicates the inner characteristics of the user, and based on the user's reaction to it. Techniques are disclosed for determining the likelihood of having a particular psychographic attribute. Here, in this technology, when a user comes from an area where it is estimated that people with a specific psychographic attribute are concentrated, it is proposed to increase the probability that the user has a specific psychographic attribute. It is

In addition, in Non-Patent Document 1, in order to estimate the user's first language, while displaying words and sentences written in multiple languages on the display, the user's line of sight reading the display is read using a sensor. , discloses a technique for estimating the language corresponding to the word or sentence in which the line of sight stays for the longest time as the first language of the user.

JP 2017-204134 A

Wang, Asai, Yamana, ``First and second language estimation using gaze information for multilingual display'', DEIM Forum 2016 F4-1, 2016, [searched November 25, 2019], Internet <URL : https://db-event.jpn.org/deim2016/papers/179.pdf＞

However, with the conventional techniques described in Patent Document 1 and Non-Patent Document 1, it is still difficult to ensure sufficient accuracy in estimation results of human attributes.

For example, in the technique described in Patent Literature 1, a plurality of contents are prepared by changing some of the constituent elements such as sentences and patterns, and after investigating the reaction of the user to each content, specific psychographic attributes are estimated. We determine the content for the target and use it for estimation. Therefore, the content that can be presented to the user for attribute estimation is limited from the beginning. That is, content is determined by focusing on what kind of constituent elements the content is effective for a user with a specific attribute. Even if it does, it is difficult to achieve high accuracy of estimation results.

In addition, in the technique described in Non-Patent Document 1, the method of displaying the contents of words and sentences presented to the user is verified with several patterns, and the user is presented with one of these patterns. The first language (as a person's attribute) is estimated by displaying the first language. In other words, improving the accuracy of attribute estimation results is limited to selecting a more suitable content display method, and for example, no consideration is given to the effects of the characteristics of the content itself on the attribute estimation results.

On the other hand, the inventor of the present application has confirmed that the characteristics of the content itself, such as the size of constituent characters, have a great influence on the result of attribute estimation. It is understood that it is still difficult to ensure sufficient accuracy in the first language estimation result with the described technique.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an attribute estimation device, a program, and a method capable of further improving the accuracy of user attribute estimation results.

According to the present invention, an attribute estimation device for estimating attributes of a user presented with content,
The content has an attribute-related element related to the attribute whose accuracy should be determined, and the attribute estimation device
a saliency determination obtaining means for determining or obtaining salience information in the content, including salience regarding the degree of attracting attention in the attribute-related element;
reaction information determination and acquisition means for determining or acquiring information related to the reaction to the content, including the degree or presence or absence of the user's reaction to the attribute-related element;
an attribute accuracy determining means for determining a likelihood that the user has an attribute related to the attribute-related element based on the information about the salience and the information about the reaction. is provided.

In the attribute estimating apparatus according to the present invention, it is also preferable that the attribute accuracy determining means determines a higher accuracy as the salience of the attribute-related element is lower. Moreover, it is also preferable to determine a higher degree of certainty as the degree of reaction of the user to the attribute-related element is higher.

In addition, when the content also has unrelated elements unrelated to the attribute whose accuracy should be determined, the information related to the salience includes the saliency in the unrelated element, and the attribute accuracy determination means It is also preferable to determine the likelihood that the lower the salience in the attribute-related element is relative to the salience in the non-relevant element, the higher.

Furthermore, in the attribute estimation device according to the present invention, the reaction information determination acquisition means
(a) information related to the user's viewpoint position within the presentation range including the attribute-related element of the content;
(b) Information related to the movement of the user who was presented with the part or the whole of the content including the attribute-related element, or
(c) It is also preferable to determine the information about the voice uttered by the user who was presented with the part or the whole of the content including the attribute-related element as the information about the reaction.

Furthermore, in the attribute estimation device according to the present invention, the saliency determination obtaining means
(a) From the result of measuring or analyzing the physical or geometric quantity related to vision, recognition or display of the content,
(b) from the results of measuring or analyzing physical quantities related to hearing or sound of the content, or
(c) From the results of measuring or analyzing the reaction of the living body's sensory organs to the content,
It is also preferred to determine the salience information.

Further, as a modified aspect, the content is provided with information related to the salience determined in advance, and the saliency determination acquisition means is provided with information related to the saliency pre-given to the content presented to the user. Obtaining information is also preferred.

Furthermore, as a specific example of the attribute estimation device according to the present invention, the attribute whose accuracy should be determined is that the user has a certain language as the first language, or that the user belongs to a country related to a certain language. It is also preferable that the content has attribute-related elements including characters, words, phrases or sentences of the certain language, or sounds of the certain language.

According to the present invention, there is also provided an attribute estimation program that causes a computer to estimate attributes of a user presented with content,
The content has an attribute-related element related to the attribute whose accuracy should be determined, and the attribute estimation program
a saliency determination obtaining means for determining or obtaining salience information in the content, including salience regarding the degree of attracting attention in the attribute-related element;
reaction information determination and acquisition means for determining or acquiring information related to the reaction to the content, including the degree or presence or absence of the user's reaction to the attribute-related element;
An attribute characterized by causing a computer to function as attribute accuracy determination means for determining the likelihood that the user has an attribute related to the attribute-related element based on the information regarding the salience and the information regarding the reaction. An estimation program is provided.

According to the present invention, there is further provided an attribute estimation method in a computer for estimating attributes of a user presented with content,
The content has an attribute-related element related to the attribute whose accuracy should be determined, and the attribute estimation method includes:
Determining or obtaining salience information in the content, including the attention-grabbing salience of the attribute-related elements, and the content, including the degree or lack of reaction of the user to the attribute-related elements determining or obtaining information relating to response to
determining the likelihood that the user has the associated attribute of the attribute-related element based on the saliency information and the reaction information. be.

According to the attribute estimation device, program, and method of the present invention, it is possible to further improve the accuracy of user attribute estimation results.

1 is a schematic diagram showing an embodiment of an attribute estimation system including an attribute estimation device according to the present invention; FIG. 1 is a schematic diagram for explaining an embodiment of an attribute estimation method according to the present invention; FIG. FIG. 5 is a schematic diagram for explaining another embodiment of the attribute estimation method according to the present invention; FIG. 10 is a schematic diagram for explaining still another embodiment of the attribute estimation method according to the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Attribute estimation device/system]
FIG. 1 is a schematic diagram showing one embodiment of an attribute estimation system including an attribute estimation device according to the present invention.

The attribute estimation system of this embodiment shown in FIG.
(a) At least one digital signage 2 that is installed in environments and facilities where many people come and go, such as streets, stations, and airports, and that can present content to these people (hereinafter also referred to as users). When,
(b) a content distribution server 3 that distributes content to be presented to the digital signage 2;
(c) an attribute estimating device 1 as an embodiment of the attribute estimating device according to the present invention for estimating attributes of a user whose content is presented by the digital signage 2;

Of these, the digital signage 2 of (a) in this embodiment includes a screen for displaying the image (video) when the content distributed from the content distribution server 3 includes the image (video), and a screen for displaying the image (video). It also has a speaker 203 for outputting sound when the content includes sound, and can appropriately provide information related to advertisements, announcements, and guidance to users in various situations and environments.

In addition, the digital signage 2 in this embodiment,
(a1) A camera 201 capable of photographing a user who has been presented with content using a visible light sensor or a near-infrared light sensor, and also capable of measuring the line of sight of the user by a known eye movement measurement method. When,
(a2) a microphone 202 capable of capturing the voice uttered by the user;
(a3) A communication interface for communicating with the attribute estimation device 1 and the content distribution server 3 is further provided, and measurement information including the user's reaction to the presentation of the content (for example, the user's image data, line-of-sight data, etc.) and voice data) can be transmitted to the attribute estimation device 1 .

Furthermore, in the present embodiment, the content distribution server 3 of (b) above transmits content to be presented to the user to the digital signage 2, and transmits the content (and/or the content described later) to the attribute estimation device 1. saliency information) can be transmitted.

Here, the content (related to the saliency information) transmitted to the attribute estimation device 1 and used for user attribute estimation has an "attribute-related element" related to the "attribute" whose accuracy should be determined. ing. For example, if the "attribute" whose accuracy should be determined is "Spanish is the first language", the content includes Spanish sentences as "attribute-related elements". of.

In addition, the attribute estimation device 1 of (c) above specifically has the following features:
(A) Salience for determining or acquiring 'salience information', which is information relating to 'salience' in content presented to the user, including 'saliency' relating to the degree of attention-attracting in 'attribute-related elements' a decision acquisition unit 111;
(B) A reaction information determination/acquisition unit 112 that determines or acquires "reaction information" that is information related to the "reaction" to the content, including the user's "degree or presence/absence of reaction" to the "attribute-related element";
(C) An attribute accuracy determination unit 113 that determines the "accuracy" that the user has the attribute associated with the "attribute-related element" based on the "saliency information" and the "reaction information".

For example, as an easy-to-understand specific example, the attribute that should determine the "accuracy" of (C) above is the attribute that "the first language is Spanish", and the digital signage 2 installed in the lobby of the airport Suppose that guidance text content including five guidance texts of the same content in Japanese, English, Spanish, Korean, and Chinese is presented.

In this case, the "attribute-related element" of (A) above is a Spanish guidance text, and the "prominence" is, for example, the size (display size) of the characters that make up this Spanish guidance text. can be done. Furthermore, the user's "degree or presence/absence of reaction" to the "attribute-related element" in (B) above is a guideline content that is visually recognizable. (attribute-related element) The time spent on the top can be adopted.

Here, the attribute estimating device 1 simply determines, for example, a high value for the "probability" having the attribute "first language is Spanish" because the viewing point retention time (degree of reaction) is greater than or equal to a predetermined value. , etc., the "prominence" in the Spanish guide (attribute-related element), for example, the display size of characters, is also considered to determine the "probability".

In fact, when estimating attributes, not only the reaction of the user but also the "saliency", which is a feature of the content itself that is basically independent of the user, such as the size of the constituent characters, has a great impact on the estimation result. , and the attribute estimation device 1 can perform attribute estimation processing that incorporates such important knowledge.

More specifically, in the present embodiment, the attribute accuracy determination unit 113 of (C) above determines that the smaller (lower) the display size (salience) of the characters in the Spanish guidance text (attribute-related element), the higher the " determine the accuracy. In fact, despite the smaller (lower) characters (salience) of the Spanish guidance sentence (attribute-related element), there are more than a predetermined amount of gazes (reactions) to this guidance sentence (attribute-related element). If so, it is assumed that "the first language is Spanish" (having a predetermined "attribute") is more likely.

Thus, in the attribute estimation device 1, it is possible to further improve the accuracy of the user's attribute estimation result by also considering the "salience information" in the content.

Here, the "content" is of course not limited to information such as characters, phrases, sentences, symbols, and figures including symbols and marks displayed on the display, and still images such as photographs, moving images ( For example, it may include a 3D image (video) projected in the air. Also, the "contents" are not limited to those including visual information as described above, and may be, for example, those that output information such as voice, sound, or music. It is also possible to include information appealing to the sense of touch, such as wind pressure and vibration.

Also, for the "attributes" to be estimated, various attributes (personal attributes if the user is a person) such as the above-mentioned first language, nationality, character/personality, occupation, and hobbies are set. It is possible to Furthermore, "attributes" are not limited to such static attributes. can be set as an attribute. In any case, various attributes can be adopted as "attributes" as long as they can set "attribute-related elements" related to the content.

Furthermore, the "saliency", which indicates the degree of attracting attention in the target portion, is of course not limited to the display size of characters as described above. Various types of settings are possible for this "salience" as well, as will be described later. Further, the "degree of reaction or presence/absence of reaction" is not limited to the above-described viewpoint retention time, and various types can be set, which will also be described in detail later.

In FIG. 1, the attribute estimation device 1 acquires the "content" and/or its "salience information" from the content distribution server 3, but of course, from another device, for example, the digital signage 2 that received the distribution may be obtained. Here, in the attribute estimation device 1 of the present embodiment, acquisition of the "content" itself is not essential, and it is sufficient if the "salience information" of the content presented to the user can be acquired or calculated.

Also, it is possible to adopt a form in which the attribute estimation device 1 itself distributes "contents" as a content distribution server and further generates "salience information" of the distributed "contents". Furthermore, the digital signage 2 itself may have a function of attribute estimation processing according to the present invention and may operate as the attribute estimation device 1 . For example, when a plurality (a large number) of digital signages 2 are installed, each digital signage 2 may function as the attribute estimation device 1, or a specific digital signage 2 may measure from other digital signages 2 It is also possible to acquire necessary information including information and perform attribute accuracy determination processing in the entire digital signage 2 .

Furthermore, without using the digital signage 2, the attribute estimation device 1 itself may present the content to the user. For example, the attribute estimating device 1 is a smartphone, presents content through its own display and speakers, and furthermore, takes an embodiment in which the user viewing the presented content is photographed by a built-in camera. is.

Furthermore, at least one of the above (A) to (C), which are components of the attribute estimation device 1 according to the present invention, may not be included in a device separate from the other components. Not possible. For example, it is possible to implement the above functions (A) to (C) by a plurality of servers as a whole. Even in such a case, the entirety of these can be regarded as an attribute estimation device or system that implements the attribute estimation method according to the present invention.

[Equipment functional configuration, attribute estimation program/method]
Similarly, according to the functional block diagram of FIG. 1, the attribute estimation device 1 includes a communication interface unit 101, a content/saliency storage unit 102, a measurement/reaction information storage unit 103, a user attribute storage unit 104, and a keyboard ( KB) 105, display (DP) 106, and processor memory.

Here, the processor memory stores an attribute estimation program according to an embodiment of the present invention, and also has a computing function, and performs attribute estimation processing by executing the attribute estimation program. implement. For this reason, the attribute estimation device 1 may be a dedicated device for user attribute estimation, but may be a cloud server, a non-cloud server device, a personal computer (PC), a personal computer (PC), etc., which is equipped with the attribute estimation program according to the present invention. It can also be a notebook or tablet computer, or a smart phone or the like.

Furthermore, the processor memory includes a saliency determination acquisition unit 111 including a salience calculation unit 111a, a reaction information determination acquisition unit 112 including a reaction information calculation unit 112a, an attribute accuracy determination unit 113, a communication control unit 121, It has an input/output control unit 122 and an application unit 123 . These functional components can be regarded as functions of an attribute estimation program stored in the processor memory. Further, the flow of processing shown by connecting the functional components of the attribute estimating device 1 with arrows in FIG. 1 can also be understood as an embodiment of the attribute estimating method according to the present invention.

Also in the functional block diagram of FIG. 1, the content/saliency storage unit 102 acquires the content presented to the user by the digital signage 2 from the content distribution server 3 via the communication interface unit 101 and the communication control unit 121, Save and manage.

Here, the acquired content has an "attribute-related element" related to the attribute whose accuracy should be determined. It may also have "unrelated elements" that are not related to the attribute whose accuracy should be determined. For example, when the attribute to be estimated is the attribute that "the first language is Spanish", five guidance sentences with the same content in Japanese, English, Spanish, Korean, and Chinese are generated. When the user is presented with guidance text content including be.

By the way, if the attribute of the user whose accuracy should be determined is "having 'a certain language' as the first language" or "belonging to a country related to 'a certain language'", the The content stored (further stored and managed in the content/salience storage unit 102) is a character, word, phrase or sentence in a "certain language", or an "attribute-related element ” is also preferred. In addition, it is preferable to further have "non-relevant elements" including characters, words, phrases or sentences in languages other than "the certain language", or voices in languages other than the "certain language". As a result, subsequent attribute accuracy determination processing can be performed more appropriately as described later.

The attributes whose accuracy should be estimated can be set in advance by the administrator in the attribute estimation device 1, for example. In addition, the content stored and managed in the content/saliency storage unit 102 is attribute estimation content having an "attribute-related element" related to a predetermined attribute whose accuracy should be estimated, for example, the attribute estimation device 1 or It may be registered in advance in the content distribution server 3 . In this case, the content is preferably registered in association with the predetermined attribute, and its saliency information is also preferably registered in association with the content.

Furthermore, the content may have a plurality of "attribute-related elements" respectively associated with a plurality of attributes whose accuracy should be estimated. For example, in the above guidance text content, if the attributes to be estimated are the attribute "first language is Spanish" and the attribute "first language is Chinese", , the guidance text content has two “attribute-related elements” of the guidance text in Spanish and the guidance text in Chinese.

Also in the functional block diagram of FIG. 1, the saliency determination acquisition unit 111 receives content for attribute estimation stored and managed in the content/salience storage unit 102, and determines saliency information of the content. As mentioned above, this saliency information includes:
(a) contains prominence in "attribute-relevant elements," and if the content contains "non-relevant elements,"
(b) Salience in "irrelevant elements" is also included. Here, it is also preferable that the saliency information thus determined is stored and managed in the content/salience storage unit 102 after being linked with the content ID of the corresponding content.

Moreover, the saliency of the above (a) and (b) is calculated and determined by the saliency calculator 111a in this embodiment.

Specifically, the saliency calculation unit 111a can (a) measure or analyze a physical quantity or a geometric quantity related to vision, recognition, or display of the acquired content, and calculate saliency from the result.

For example, when the acquired content is an image (video) content containing visual information, in the image part of the "attribute-related element" (or the image part of the "unrelated element"),
(A1) Difference in brightness (contrast) from other parts,
(A2) Hue difference from other parts,
(A3) Brightness gradient at the boundary with the surrounding part, or brightness difference in the direction of the brightness gradient, or (A4) Structural units, such as characters, symbols, and figures such as symbols and marks (display size)
Based on such characteristics, a quantity representing these characteristics may be determined, and the saliency may be obtained by normalizing the quantity to a value between 0 and 1, for example.

It should be noted that the features as described above can be determined by analyzing the pixel values of the images that make up the image content, or by measuring the actually displayed screen image.

In addition, the salience calculation unit 111a may (a) measure or analyze physical quantities related to hearing or sound of the acquired content, and calculate salience from the result.

For example, if the acquired content is audio content that includes audio information, the audio part of "attribute-related elements (for example, guidance voice in Spanish)" (or the "non-related element (for example, guidance voice in languages other than Spanish)" voice part) in
(a) output volume (or amplitude of voice wave), or (b) pitch of sound (or frequency of voice wave)
Based on such characteristics, a quantity representing these characteristics may be determined, and the saliency may be obtained by normalizing the quantity to a value between 0 and 1, for example. These features can be determined by analyzing the waveform of the audio data that constitutes the audio content, or by measuring and analyzing the waveform of the audio that is actually output.

Furthermore, the saliency calculation unit 111a can also (c) measure or analyze the response of the living body's sensory organs to the acquired content, and calculate the salience from the result.

For example, (c1) When the acquired content is an image (video) content containing visual information, there is a well-known technique that simulates the response to visual stimulation by on-centred and off-centred receptive fields in retinal ganglion cells. The saliency may be obtained by extracting a feature by image processing and normalizing the amount representing the feature to a value between 0 and 1, for example. Incidentally, such image processing and feature extraction are, for example, non-patent literature: Laurent Itti, Christof Koch, and Ernst Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis", IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI), Vol. 20, No. 11, pp. 1254-1259, 1998.

Or, (c2) extracting features related to changes in the size of the pupil or the size of the microsaccade of the eyeball, and normalizing the amount representing the feature to a value from 0 to 1, for example, is remarkable It is also possible to Such feature extraction is described in, for example, Japanese Patent Laid-Open No. 2015-132783.

In any case, the salience calculation unit 111a can calculate the degree of attracting attention in the "attribute-related element" or the "non-related element" from the measurement/analysis results, and set the degree as saliency. . By the way, the saliency calculated here includes the so-called saliency, which is a technical concept that expresses the strength of the characteristic that attracts bottom-up attention depending on the spatial arrangement in the fields of brain science and image recognition. do.

In the present embodiment, it is also preferable that the saliency calculated by the saliency calculation unit 111a is basically determined based on the configuration and features of the content itself, which does not depend on the attributes of the user. For example, as disclosed in Japanese National Publication of International Patent Application No. 2012-532366, the visual attention of the user is generally divided into extrinsic attention governed by visual features of content such as local contrast and color, and display Intrinsic attention, which is related to the content being displayed and depends on the user's task and experience. Of these, the saliency calculated in the present embodiment is preferably used as an index related to extrinsic attention.

As a modification, the saliency determination acquisition unit 111 does not calculate the salience as described above by itself, but receives the salience of the content presented to the user from the content distribution server 3 or the digital signage 2, for example. Sexual information may be obtained in the form attached to the content or alone. In this case, the content distribution server 3 or the digital signage 2 has a function of calculating and determining the saliency of the content as described above, and the saliency information of the content including the calculated and determined saliency is Transmission to the attribute estimation device 1 is also preferable.

Furthermore, in the above modification, the content distribution server 3 calculates in advance the salience of each content to be distributed, associates the content ID of the content with the salience information of the content for each content, and It is also preferable to store and manage the content data set generated in the previous step in a content database provided by itself.

Also, in this case, the content distribution server 3 may transmit the content to which the saliency information is added to the attribute estimation device 1 . Alternatively, as another preferred embodiment, the digital signage 2 notifies the attribute estimation device 1 of the content ID of the content presented to the user, for example, along with measurement information, and the attribute estimation device 1 sends the content ID to the content distribution server 3. It is also preferable to query the saliency information and obtain the corresponding saliency information from the content distribution server 3 .

Similarly, in the functional block diagram of FIG. 1, the reaction information calculation unit 112a of the reaction information determination/acquisition unit 112 receives measurement information (for example, the user's image data , line-of-sight data and voice data), the reaction information for the content is calculated and determined. Incidentally, it is also preferable that these measurement information and reaction information are stored and managed in the measurement/reaction information storage unit 103 after being associated with the content ID of the corresponding content.

where, in this reaction information,
(a) "Level or presence/absence of reaction" of the user to the "attribute-related element" of the content
and the content contains "non-relevant elements",
(b) "Level or presence/absence of reaction" of the user to "unrelated elements"
is also included.

For example, the reaction information calculation unit 112a (a) calculates information related to the user's viewpoint position within the presentation range including the "attribute-related element" of the content presented to the user, and determines the information as the reaction information. may

Specifically, the line-of-sight data acquired as measurement information (from the camera 201) is converted into point data within the image coordinate system of the image of the content displayed on the screen of the digital signage 2, and the time-series viewpoint position (point-of-view coordinates) is determined, and the point-of-view residence time is the time during which the viewpoint stays on the image portion of the “attribute-related element” (or the image portion of the “non-related element”) as the “extent or presence of reaction” may be calculated.

In addition, the reaction information calculation unit 112a may (a) calculate information related to the movement of the user who was presented with the part or the whole of the content including the “attribute-related element”, and determine the information as the reaction information.

Specifically, from the image data acquired as measurement information (from the camera 201), time-series information related to the user's position and orientation, such as user and the screen of the digital signage 2 (the image part of the "attribute-related element" (or the image part of the "non-related element")), and the face of the user facing the screen of the digital signage 2 Time-series information of the azimuth angle indicating the orientation of the user's face is calculated with the orientation as the reference orientation, and information derived from this information, such as the user's staying time within a predetermined range in front of the screen of the digital signage 2, or The facing time facing the screen of the digital signage 2 can be determined as reaction information.

Furthermore, the reaction information calculation unit 112a can also determine (c) information related to the voice uttered by the user who was presented with the part or the whole of the content including the "attribute-related element", and determine the information as the reaction information. .

For example, from the voice data acquired as measurement information (from the microphone 202), the user's voiceprint is specified using a well-known voice analysis technique, the user's voice is identified, and the user's voice volume ( The time-series information of the amplitude of the audio wave) is calculated, and information derived from the information, for example, the staying time of the user within a predetermined range in front of the screen of the digital signage 2 may be determined as the reaction information.

Note that if the reaction information calculation unit 112a cannot calculate information related to the user's reaction from the measurement information received from the digital content 2 (the user does not exist within the measurement range of the digital content 2, or if the user exists but noise If the reaction of the user is not extracted due to such influence, it is also preferable to terminate the reaction information calculation process.

As a modification, it is preferable that the reaction information as described above is not calculated and determined by the reaction information calculation unit 112a, but is calculated and determined by the digital signage 2, which is the origin of the measurement information. In this case, the reaction information determination/acquisition unit 112 receives the reaction information transmitted from the digital signage 2 .

Similarly, in the functional block diagram of FIG. and reaction information associated with the same content ID), determine the likelihood that a user presented with the content has the associated attribute of the content's "attribute-related element".

The attribute accuracy determining unit 113 further estimates the attribute of the user based on the determined accuracy. For example, if the determined certainty exceeds a preset certain certainty threshold, it may be inferred that the user has the attribute associated with this certainty.

Here, in this embodiment, the attribute accuracy determining unit 113
(A) The lower the saliency in the "attribute-related element", the higher the certainty is determined,
In addition, when the content also has non-relevant elements,
(b) The lower the salience in the 'attribute-related elements' compared to the salience in the 'non-relevant elements', the higher the certainty is determined.

for example,
(a) The salience s1 of "attribute-related elements" (saliency s1_1, s1_2, ... when there are multiple attribute-related elements) is a monotonically decreasing function,
(b) The saliency S2 of "unrelated elements" (saliency s2_1, s2_2, ... when there are multiple unrelated elements) is a monotonically increasing function,
(c) "Reaction level (reaction information)" Δ1 of "attribute-related element"("reactionlevel" Δ1_1, Δ1_2, ... when there are multiple attribute-related elements) becomes a monotonically increasing function, and further,
(d) The “degree of reaction (reaction information)” Δ2 of “unrelated elements” (“degree of reaction” Δ2_1, Δ2_2, etc. when there are multiple unrelated elements) is a monotonically decreasing function The accuracy PL can be calculated using the accuracy function PL of the following equation (1).
(1) PL = PL(S1, S2, Δ1, Δ2)

In fact, when the salience s1 of an "attribute-related element" is small, it is unlikely that this "attribute-related element" will attract the user's attention given the characteristics of such content. If the user reacted higher to the attribute-related element, that is, if the 'degree of reaction' Δ1 takes a large value, the user may have an attribute that should pay attention to the 'attribute-related element' is high, that is, it is appropriate to calculate the accuracy PL as a larger value.

In addition, when the saliency S2 of the "non-relevant element" is large, the possibility that the other part, the "attribute-relevant element", will attract the user's attention is low, given the characteristics of such content. However, in spite of this, the user responded to the “attribute-related element” higher, that is, the “degree of reaction” Δ1 took a large value, and the “degree of reaction” Δ2 took a relatively small value. If so, it is highly likely that the user has an attribute to which attention should be paid to the "attribute-related element", that is, it is appropriate to calculate the probability PL as a larger value.

Furthermore, since the saliency s1 and the saliency S2 are considered to have a relative effect on the accuracy PL, the accuracy PL is an amount related to the degree of difference between the two, for example, an amount related to the ratio or difference (specifically For example, it is also preferable to set it so as to be a function of the reciprocal internal division ratio, such as C1 in equation (5) described later.

<Attribute Estimation Processing Based on Display Content>
FIG. 2 is a schematic diagram for explaining an embodiment of an attribute estimation method according to the present invention.

According to FIG. 2, the screen of the digital signage 2 displays guidance sentence content having attribute-related elements that are sentences in language L (Chinese in FIG. 2). By the way, in this embodiment, the attribute whose probability is to be estimated is set to the person attribute that "the first language is language L (Chinese)".

Here, the camera 201 of the digital signage 2 has a line-of-sight measurement function, generates line-of-sight data of the user with respect to the displayed guidance text content, and the digital signage 2 displays the line-of-sight data as measurement information. It is transmitted to the attribute estimation device 1 together with the content ID of the guidance message content.

Next, the reaction information calculation unit 112a of the reaction information determination/acquisition unit 112 of the attribute estimation device 1 analyzes the line-of-sight data, which is the received measurement information, and determines that the viewpoint of the user presented with the content is an attribute-related element (language L ( A point-of-view dwell time Δt, which is the dwell time on the Chinese sentence), is calculated.

On the other hand, the saliency calculation unit 111a of the saliency determination acquisition unit 111 of the attribute estimation device 1 calculates the attribute-related element (Language L (Chinese The saliency S_L (0<S_L≦1) of the word) sentence) is calculated. Here, the saliency S_L can be a value obtained by standardizing the display size of constituent characters, for example.

Next, the attribute accuracy determination unit 113 of the attribute estimation device 1 calculates the accuracy PL of the user "the first language is language L (Chinese)" by the following equation (2) PL={1+exp(-a*(Δt) -b)} ^-1 *(k*S_L) ^-1
Calculated using In addition, when the value of the left side exceeds 1, PL in the above formula (2) is
(3) PL = 1
is set to Alternatively, the attribute accuracy determining unit 113 calculates the accuracy PL by the following formula (4) PL={1+exp(-a*(Δt-b)/(k*S_L))} ^-1
may be calculated using

Here, the parameters a, b and k in the above equations (2) and (4) may be determined experimentally or empirically, or It is also possible to determine by well-known machine learning using learning data.

Finally, when the calculated PL exceeds a predetermined threshold value Th_PL (PL>Th_PL), the attribute accuracy determining unit 113 determines that the user has determined that "the first language is language L (Chinese)". It outputs an attribute estimation result indicating that the user has the attribute, that is, the user's first language is language L (Chinese).

FIG. 3 is a schematic diagram for explaining another embodiment of the attribute estimation method according to the present invention.

According to FIG. 3, on the screen of the digital signage 2, there are attribute-related elements that are guidance text in language L1 (Japanese in FIG. 3) and non-related elements that are guidance text in language L2 (Portuguese in FIG. 3). is displayed. By the way, in this embodiment, the attribute whose accuracy should be estimated is set to the person attribute that "the first language is language L1 (Japanese)".

Here, the camera 201 of the digital signage 2 has a line-of-sight measurement function, generates line-of-sight data of the user with respect to the displayed guidance text content, and the digital signage 2 displays the line-of-sight data as measurement information. It is transmitted to the attribute estimation device 1 together with the content ID of the guidance message content.

Next, the reaction information calculation unit 112a of the reaction information determination/acquisition unit 112 of the attribute estimation device 1 analyzes the line-of-sight data, which is the received measurement information. A viewpoint retention time Δt1, which is the time spent on an unrelated element (guidance text in language L2 (Portuguese)), and a viewpoint retention time Δt2, which is a time spent on an unrelated element (guidance text in language L2 (Portuguese)) are calculated.

On the other hand, the saliency calculation unit 111a of the saliency determination acquisition unit 111 of the attribute estimation device 1 calculates the attribute-related element (language L1 (Japanese The saliency S_L1 (0<S_L1≦1) of the guidance sentence of the language) and the saliency S_L2 (0<S_L2≦1) of the unrelated element (the guidance sentence of the language L2 (Portuguese)) are calculated. Here, both the saliency S_L1 and the saliency S_L2 can be values obtained by standardizing the display size of constituent characters, for example.

Next, the attribute accuracy determination unit 113 of the attribute estimation device 1 calculates the accuracy PL1 of "the first language is language L1 (Japanese)" of the user by the following equation (5) PL1=EX1/(EX1+EX2)
EX1=exp(-a1*(Δt1-b1)*C1) ^-1
EX2 = exp(-a2*(Δt2-b2)*C2) ^-1
C1=(1/S_L1)/{(1/S_L1)+(1/S_L2)}=S_L2/(S_L1+S_L2)
C2=(1/S_L2)/{(1/S_L1)+(1/S_L2)}=S_L1/(S_L1+S_L2)
Calculated using

Here, the parameters a1, a2, b1 and b2 in the above equation (5) may be determined experimentally or empirically, or learning data It is also possible to determine by well-known machine learning using

Finally, when the calculated PL1 exceeds a predetermined threshold value Th_PL (PL1>Th_PL), the attribute accuracy determining unit 113 determines that the user has determined that "the first language is language L1 (Japanese)." It outputs an estimation result indicating that the user has the attribute, that is, the user's first language is language L1 (Japanese).

An embodiment of estimating a person's attribute that "the first language is language L1 (Japanese)" has been described above. ) is daringly made smaller than the salience S_L1 (display size of constituent characters) of the guidance text of language L2 (Portuguese), which is an unrelated element. It is also preferable to leave As a result, even though the saliency S_L1 (the display size of the constituent characters) is smaller, when more attention is paid to the guidance text in the language L1 (Japanese), from equation (5), the actual possibility Therefore, it becomes possible to determine a higher accuracy PL that is suitable for the nature of the subject.

Further, in the present embodiment, the certainty PL2 that the user's "first language is language L2 (Portuguese)" can also be calculated together with the above certainty PL1. That is, the following formula (6) PL2=EX2/(EX1+EX2)
can be used to calculate PL2.

Furthermore, unlike the above embodiments, there are N (N≧2) non-related elements (guidance sentences in languages other than language L1) in the guidance sentence content, and each saliency S_L2_i (i=1 , …, N) are calculated, the following formula (7) EX2_i＝exp(－a2_i＊(Δt2_i－b2_i)＊C2_i) ^－1
where C2_i is the
The sum of i in EX2_i (i = 1, ..., N) calculated by the internal ratio of "reciprocal of saliency S_L2_i" is replaced with EX2 in the above formula (5), and the accuracy PL1 is can be calculated.

Furthermore, in the guidance text content, there are M (M≧2) attribute-related elements (guidance text in language L1), and the saliency S_L1_j (j=1, . . . , M) of each is calculated. (8) EX1_j＝exp(－a1_j＊(Δt1_j－b1_j)＊C1_j) ^－1
where C1_j is the
Summation of internal ratios of "inverse of saliency S_L1_j" with respect to j
The accuracy PL1 can be calculated by replacing the sum of j in EX1_j (j=1, . . . , M) calculated by Eq. (5) with EX1.

<Attribute Estimation Processing Using Audio Content>
FIG. 4 is a schematic diagram for explaining still another embodiment of the attribute estimation method according to the present invention.

According to FIG. 4, the guidance voice content having attribute-related elements that are guidance voices in language L1 (Chinese in FIG. 4) and non-related elements that are guidance voices in language L2 (Korean in FIG. 3) is The digital signage 2 and the digital signage 2' installed on the left and right sides of the aisle in the airport facility are presented to users traveling through the aisle.

Specifically, from the speaker 203 of the digital signage 2 located on the left front as seen from the user, guidance voice (attribute-related element) in language L1 (Chinese) is output, while the right front as seen from the user is output. The guidance voice (attribute-related element) in language L2 (Korean) is output from the speaker 203' of the digital signage 2' located at , at the same time as or slightly shifted from the guidance voice in language L1.

Incidentally, in the present embodiment, the attribute whose probability should be estimated is set to the person attribute that "the first language is language L1 (Chinese)".

Here, the cameras 201 and 201' of the digital signages 2 and 2' respectively generate image data that captures the user who changed the traveling direction in response to the displayed guidance voice content, and the digital signages 2 and 2' , the image data is transmitted to the attribute estimation device 1 as measurement information together with the content ID of the output guidance voice content.

Next, the reaction information calculation unit 112a of the reaction information determination acquisition unit 112 of the attribute estimation device 1 analyzes the received image data, which is the measurement information, and obtains time-series information of the distance between the user and the screen of the digital signage 2, and Determine the time-series information of the distance between the user and the screen of the digital signage 2', and from each, the stay time Δt1 during which the user stayed within a predetermined range in front of the screen of the digital signage 2, and the screen of the digital signage 2' A staying time Δt2 for staying within the previous predetermined range is calculated.

On the other hand, the saliency calculation unit 111a of the saliency determination acquisition unit 111 of the attribute estimation device 1 calculates the attribute-related element (language L1 (Chinese Calculate the saliency S_L1 (0<S_L1≦1) of the guidance voice of the language) and the saliency S_L2 (0<S_L2≦1) of the unrelated element (the guidance voice of the language L2 (Korean in FIG. 3)) . Here, both the saliency S_L1 and the saliency S_L2 can be values obtained by standardizing the output volume of the guidance voice, for example.

Next, the attribute accuracy determination unit 113 of the attribute estimation device 1 calculates the certainty PL1 that the user's "first language is language L1 (Chinese)" using the above-described formula (5), and the calculated PL1 is a value exceeding a predetermined threshold Th_PL (PL1>Th_PL), the user has the attribute that "the first language is language L1 (Chinese)", that is, the user's first language is language It outputs the estimation result that it is L1 (Chinese).

An embodiment of estimating a person's attribute that "the first language is language L1 (Chinese)" has been described above. ) is set lower than the saliency S_L2 (output volume) of the guidance voice of language L2 (Korean), which is an unrelated element. As a result, even though the saliency S_L1 (output volume) is smaller, when the movement of paying attention to the guidance voice in language L1 (Chinese) becomes clearer, from equation (5), It becomes possible to determine a greater accuracy PL1 that matches the actual possibility.

As described above, in the embodiments described with reference to FIGS. 3 and 4, not only reaction information but also saliency information are taken into account to determine the accuracy of user attribute estimation. Here, for example, by setting (k*S_L) to 1 in the above equation (4) or setting C1 and C2 to 1 in the above equation (5), the accuracy is calculated only from the reaction information. It is also possible. However, as described above, by also using the saliency information in the accuracy calculation, it is possible to determine the accuracy with higher precision that reflects the actual possibility.

In addition, as described above, by adjusting the salience of attribute-related elements, the salience of unrelated elements, and the degree of difference between both salience in advance in the content to be presented, It is also possible to carry out estimation processing with higher accuracy than that obtained by the method. That is, in the attribute estimation of the present invention, it is possible to perform the attribute estimation process by utilizing given content, or by adding some contrivance to the content to be presented, attribute It is also possible to realize estimation processing.

Returning to the functional block diagram of FIG. 1, information related to the attributes estimated (determined) by the attribute accuracy determination unit 113, such as the content ID, the presentation date and time of the content, and the presentation location (or signage ID), is estimated. Information that associates an attribute with its accuracy (and user ID if it can be acquired/determined) is sent as an attribute estimation result via the communication control unit 121 and the communication interface unit 101 to an external information processing device, It may be transmitted to a terminal of a company involved in content distribution.

The company that has acquired such an attribute estimation result can, for example, cause the digital signage 2 to present more suitable advertising content or guide content according to the estimated attribute at a predetermined timing. For example, a user whose first language is presumed to be Portuguese may be provided with advertising content or guidance content in Portuguese. It is also possible to present news content in Portugal.

Moreover, it is preferable that the result of the attribute estimation process described above is stored and managed by the user attribute storage unit 104 and used as appropriate. Furthermore, it is loaded into a predetermined application program stored in the application unit 123, and the processing progress and results of the program are displayed on the display 106 via the input/output control unit 122 under instructions from the keyboard 105. , various kinds of processing, for example, statistical processing related to user attributes, content evaluation processing, etc., are also preferably performed.

As described in detail above, according to the present invention, it is possible to further improve the accuracy of user attribute estimation results by also considering salience information in content.

In addition, this also makes it possible to present more appropriate information to the user according to the user's attributes estimated with higher accuracy. For example, if the content to be presented is advertisement content, the advertising effect can be further improved, and if the content to be presented is guidance content, more accurate information can be transmitted more smoothly. .

For the various embodiments of the present invention described above, various changes, modifications and omissions within the spirit and scope of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 attribute estimation device 101 communication interface unit 102 content/salience storage unit 103 measurement/reaction information storage unit 104 user attribute storage unit 105 keyboard (KB)
106 Display (DP)
111 saliency determination acquisition unit 111a saliency calculation unit 112 reaction information determination acquisition unit 112a reaction information calculation unit 113 attribute accuracy determination unit 121 communication control unit 122 input/output control unit 123 application unit 2 digital signage 201 camera 202 microphone 203 speaker 3 content Distribution server

Claims (10)

An attribute estimation device for estimating attributes of a user presented with content,
The content has an attribute-related element related to an attribute whose accuracy should be determined, and the attribute estimation device
a saliency determination obtaining means for determining or obtaining salience information in the content, including salience regarding the degree of attracting attention in the attribute-related element;
reaction information determination and acquisition means for determining or acquiring information related to the reaction to the content, including the degree or presence or absence of the user's reaction to the attribute-related element;
an attribute accuracy determining means for determining a likelihood that the user has an attribute related to the attribute-related element based on the information about the salience and the information about the reaction. . 2. The attribute estimating apparatus according to claim 1, wherein said attribute accuracy determining means determines a higher accuracy as the salience of said attribute-related element is lower. The content also has non-relevant elements that are not related to the attributes whose accuracy should be determined;
the salience information also includes salience in the non-relevant elements,
3. The attribute accuracy determination means according to claim 1, wherein the lower the salience of the attribute-related element compared to the saliency of the non-related element, the higher the attribute accuracy is determined. attribute estimator. The reaction information determination/acquisition means provides information relating to the viewpoint position of the user within a presentation range including the attribute-related element of the content, and movement of the user who was presented with a part or the whole of the content including the attribute-related element. or information related to the voice uttered by the user who was presented with the part or the whole of the content containing the attribute-related element is determined as the information related to the reaction. Attribute estimation device according to any one of the above. The saliency determination acquisition means is based on the result of measuring or analyzing the physical quantity or geometric quantity related to vision, visual recognition or display of the content, or from the result of measuring or analyzing the physical quantity related to hearing or sound of the content 5. The attribute estimation device according to any one of claims 1 to 4, wherein the information on the salience is determined from the result of measuring or analyzing the response of a living body sensory organ to the content. . The content is given information related to its own saliency determined in advance,
5. The attribute estimation device according to any one of claims 1 to 4, wherein said saliency determination acquisition means acquires information relating to saliency given in advance to said content presented to said user. . The attribute estimation device according to any one of claims 1 to 6, wherein the attribute accuracy determination means determines a higher accuracy as the degree of reaction of the user to the attribute-related element is higher. . The attribute that should determine the certainty is that the user speaks a language as a first language or that the user belongs to a country related to a language;
8. The content according to any one of claims 1 to 7, wherein the content has attribute-related elements including characters, words, phrases or sentences in the certain language, or sounds in the certain language. Attribute estimator as described. An attribute estimation program that causes a computer to estimate attributes of a user presented with content,
The content has an attribute-related element related to the attribute whose accuracy should be determined, and the attribute estimation program
a saliency determination obtaining means for determining or obtaining salience information in the content, including salience regarding the degree of attracting attention in the attribute-related element;
reaction information determination and acquisition means for determining or acquiring information related to the reaction to the content, including the degree or presence or absence of the user's reaction to the attribute-related element;
An attribute characterized by causing a computer to function as attribute accuracy determination means for determining the likelihood that the user has an attribute related to the attribute-related element based on the information regarding the salience and the information regarding the reaction. estimation program. An attribute estimation method in a computer for estimating attributes of a user presented with content,
The content has an attribute-related element related to the attribute whose accuracy should be determined, and the attribute estimation method includes:
Determining or obtaining salience information in the content, including the attention-grabbing salience of the attribute-related elements, and the content, including the degree or lack of reaction of the user to the attribute-related elements determining or obtaining information relating to response to
determining a likelihood that the user has the associated attribute of the attribute-related element based on the saliency information and the response information.

Images