JP6972526B2 - Content providing device, content providing method, and program - Google Patents

Description

The present invention relates to a content providing device, a content providing method, and a program, and more particularly to a content providing device capable of providing appropriate content according to a content viewer.

In recent years, research and development on interactive robots that interact with humans have been actively promoted. As a robot that provides content while communicating with a person, it has been proposed that the content that matches the emotion of the robot is selected and reproduced (for example, Patent Document 1). In the content reproduction system described in Patent Document 1, a robot device that changes emotions in response to an external action and operates autonomously transmits emotion information to the content reproduction device. The content reproduction device expresses the emotion of the robot device by comparing the metadata describing the information about the content with the emotion information of the robot device and selecting and reproducing the content matching the emotion of the robot device.

However, in Patent Document 1, only the content according to the emotion of the robot device is displayed, the user evaluates the content, and the user's reaction at the time of reproducing the content is reflected in the reproduced content. I can't.

On the other hand, Patent Document 2 proposes a content providing system capable of easily acquiring a user's reaction to a content and providing a content that suits the user's taste. Specifically, the output unit that outputs the content, the determination unit that determines the type of the user's behavior based on the information obtained from the sensor that detects the user's behavior, and the content output by the output unit. It is provided with a content selection unit that determines the content to be further output by the output unit based on the determination result by the corresponding determination unit.

Japanese Unexamined Patent Publication No. 2005-169567 Japanese Unexamined Patent Publication No. 2012-155616

By the way, when a robot installed in an area where many people gather, such as a product introduction area or an exhibition area, provides content such as products and exhibits, the viewer of the content is a single person (individual). It is not limited to this, but it may be a group consisting of multiple people such as a family with children, a couple, and so on.

In this respect, since the content providing system of Patent Document 2 determines the content that suits the user's preference from the operation information of a single user with respect to the content, the content providing system provides appropriate content to a single user. Although it is possible, if a group consisting of a plurality of people becomes a content viewer as described above, there is a possibility that appropriate content cannot be provided.

The present invention has been made from such a viewpoint, and the object thereof is.
Content providing devices, content providing methods, and programs that can provide appropriate content to content viewers in places where individuals and groups coexist as content viewers, such as product introduction areas and exhibition areas. To provide.

The first invention for solving the above-mentioned problem recognizes a plurality of people existing in the surroundings, acquires the position information, recognition time or attribute information of each recognized person, and identifies each person uniquely. A person recognition means that stores a person in association with information, and an individual / group recognition means that recognizes each recognized person separately as an individual or a group based on the position information, the recognition time, or the attribute information. The content providing means for providing the content to the individual and the group, the interest acquisition means for acquiring the interest level of the individual and the group for the content, and the interest level for the acquired content are described. The content providing means includes a storage means for storing the content in association with the attribute information of the individual and the group, and the content providing means refers to the storage means when providing the content to a new individual or the group. Alternatively, it is a content providing device characterized in that content having a high degree of interest is selected and provided with respect to the attributes of the group .

The second invention for solving the above-mentioned problem recognizes a plurality of people existing in the surroundings, acquires the position information, recognition time or attribute information of each recognized person, and identifies each person uniquely. A person recognition means that stores the person in association with the information, and an individual / group recognition means that recognizes each recognized person separately as an individual or a group based on the position information, the recognition time, or the attribute information. The content providing means for providing the content to the individual and the group, the degree of interest acquisition means for acquiring the degree of interest of the individual and the group for the content, and the degree of interest for the acquired content are described above. The means for acquiring the degree of interest includes a storage means for storing an individual and the attribute information of the group in association with the attribute information, and the representative of the group is selected from each person included in the group. It is a content providing device characterized by acquiring the degree of interest of a person as the degree of interest of the group. In this case, the degree of interest acquisition means, position information of each person included in the group, the attribute information, speech information, or based on the residence time, it is desirable to select a representative of the group.

Further, in the first or second invention, the individual / group recognition means recognizes people who are close to each other as the same group, or recognizes people who have similar recognition times as the same group. May be good. Further, the individual / group recognition means may recognize a person who is not recognized as a group as an individual.
Further, in the first or second invention, the gesture recognition means for recognizing the gestures of the individual and each person included in the group recognized by the individual / group recognition means is further provided, and the degree of interest acquisition means is provided. It is desirable to acquire the degree of interest based on the gesture recognized during the provision of the content.
Further, the interest degree acquisition means may acquire the interest degree based on the gesture recognized after the content is provided.

Further, in the first or second invention, the content providing means may change the content to be provided according to the degree of interest in the content acquired by the interest degree acquiring means while the content is being provided. ..

In the third invention, the control unit of the content providing device recognizes a plurality of people existing in the surroundings, acquires the position information, recognition time or attribute information of each recognized person, and identifies each person uniquely. A person recognition step that stores a person in association with information, and an individual / group recognition step that recognizes each recognized person separately as an individual or a group based on the position information, the recognition time, or the attribute information. The content providing step for providing the content to the individual and the group, the interest level acquisition step for acquiring the interest level of the individual and the group for the content, and the interest level for the acquired content are described. seen including a storage step of storing in the storage unit in association attribute information and straps of individuals and the group, wherein the content providing step, when providing content to a new individual or group, by referring to the storage unit The content providing method is characterized by selecting and providing content having a high degree of interest in the attributes of the individual or the group.
In the fourth invention, the control unit of the content providing device recognizes a plurality of people existing in the surroundings, acquires the position information, recognition time or attribute information of each recognized person, and identifies each person uniquely. A person recognition step that stores information in association with information, and an individual / group recognition step that recognizes each recognized person separately as an individual or a group based on the position information, the recognition time, or the attribute information. The content providing step for providing the content to the individual and the group, the interest level acquisition step for acquiring the interest level of the individual and the group for the content, and the interest level for the acquired content are described. In the interest degree acquisition step, the representative of the group is selected from each person included in the group, and the selected representative is included. It is a content providing method characterized by acquiring the degree of interest of a person as the degree of interest of the group.

A fifth invention is a program characterized in that it functions as the content providing device according to any one of claims 1 to 8.

INDUSTRIAL APPLICABILITY According to the present invention, a content providing device or the like capable of providing appropriate content to a content viewer is provided in a place where individuals and groups coexist as content viewers such as a product introduction area and an exhibition area. NS.

The figure which shows an example of the system configuration of the content provision system 1. Internal configuration diagram of robot 2 Internal configuration diagram of computer 3 Functional configuration diagram of content provision system 1 (robot 2) The figure which shows the person recognition information 30 The figure which shows the group recognition information 40 The figure which shows the group attribute table 50 The figure which shows the personal recognition information 45 The figure which shows the personal attribute table 55 The figure which shows the content list 60 The figure which shows the gesture table 70 The figure which shows the interest degree calculation table 80 The figure which shows the interest degree history table 90 The figure which shows the content interest degree table 100 Flow chart showing the operation of robot 2

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing an example of a system configuration of the content providing system 1. The content providing system 1 includes a robot 2, a computer 3, a camera 4, a microphone 5, and a display 6, which are one form of the content providing device according to the present invention. The robot 2, the camera 4, the microphone 5, and the display 6 are communicated with and connected to the computer 3, and the operation of each device is controlled by the computer 3 so as to cooperate with each other. The form of communication connection between the robot 2, the camera 4, the microphone 5, and the display 6 and the computer 3 may be wired or wireless.

The camera 4 acquires surrounding video data. The camera 4 is, for example, a single camera, a stereo camera, a distance sensor, or a composite sensor in which these are combined. It should be noted that the video data can be acquired by using the camera 210 built in the robot 2 instead of the camera 4. In this case, the camera 4 may be omitted.

The microphone 5 acquires ambient voice data. The microphone 5 is, for example, a single microphone or a microphone array composed of a plurality of microphones. As for the microphone 5, voice data can be acquired by using the microphone 211 built in the robot 2. In this case, the microphone 5 may be omitted.

The display 6 is controlled by the computer 3 and displays the content. The display 6 may be omitted when the display is controlled by using the display unit 204 mounted on the robot 2 or when the robot 2 provides the content only by voice.

The computer 3 acquires video data by the camera 4, acquires audio data by the microphone 5, controls the display of the display 6, and controls the operation of the robot 2.

The robot 2 is, for example, an interactive robot. The robot 2 is installed in, for example, a product introduction area or an exhibition area, and provides contents for introducing products and exhibits.
As shown in FIG. 2, the robot 2 includes a control unit 201, a storage unit 202, an input unit 203, a display unit 204, a media input / output unit 205, a communication I / F 206, a movable unit drive mechanism 207, and a peripheral device I / F unit. The 208, the voice processing unit 209, the camera 210, the microphone 211, the sensor 212, and the speaker 213 are connected to each other via the bus 220.

The control unit 201 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls and executes a program stored in the storage unit 202, ROM, recording medium, etc. in the work memory area on the RAM, and drives and controls each unit connected via the bus 220.

The CPU of the control unit 201 executes all the processing by the robot 2.
The ROM permanently holds the boot program of the robot 2, programs such as BIOS, data, and the like. The RAM temporarily holds the loaded program and data, and includes a work area used by the control unit 201 to perform various processes.

The storage unit 202 stores a program executed by the control unit 201, data necessary for program execution, an operating system, and the like. These program codes are read out by the control unit 201 as necessary, transferred to the RAM, read out by the CPU, and executed.

Further, the storage unit 202 includes a person recognition information 30 (FIG. 5), a group recognition information 40 (FIG. 6), a group attribute table 50 (FIG. 7), an individual recognition information 45 (FIG. 8), and an individual attribute table 55 (FIG. 6), which will be described later. 9), content list 60 (FIG. 10), gesture table 70 (FIG. 11), interest degree calculation table 80 (FIG. 12), interest degree history table 90 (FIG. 13), content interest degree table 100 (FIG. 14) and content. Various databases of DB 110, utterance DB 120, and action DB 130 are stored.

The input unit 203 includes, for example, an input device such as a touch panel, various operation buttons, and the like, and outputs the input data to the control unit 201.
The display unit 204 is composed of a display device such as a liquid crystal panel and a logic circuit for executing display processing in cooperation with the display device, and displays display information input under the control of the control unit 201 on the display device. Let me. The touch panel of the input unit 203 is integrally configured with the display of the display unit 204.

The media input / output unit 205 is, for example, a drive device for a recording medium (media) such as a magnetic disk, an optical disk, or a semiconductor memory, and inputs / outputs data.

The communication I / F 206 is an interface that mediates communication with the network 8. The network 8 includes a LAN (Local Area Network), a WAN (Wide Area Network) connected to a wider area, a public communication line such as the Internet, a base station, and the like. The communication connection with the network 8 may be wired or wireless. The robot 2 can access the Web server via the network 8 and send and receive various programs and data.

The movable portion drive mechanism 207 includes a mechanism and a drive device for driving the movable portion provided on the head, neck, shoulders, arms, hips, legs, and the like of the robot 2. The operation of the movable unit drive mechanism 207 is controlled by the control unit 201. The control unit 201 reads the movable unit control sequence from the action DB 130 of the storage unit 202, sends a control signal to each movable unit drive mechanism 207, and operates each drive unit.

The peripheral device I / F (interface) unit 208 is a port for connecting the peripheral device, and mediates the transmission / reception of data with the peripheral device. The connection form with peripheral devices may be wired or wireless. The robot 2 can communicate with the computer 3 and the like via the peripheral device I / F unit 208.

The voice processing unit 209 includes a voice synthesis unit, a language discrimination unit, a voice recognition unit, and the like. The voice synthesis unit generates synthetic voice data corresponding to character information (text data) based on a command from the control unit 201, and outputs the synthetic voice from the speaker 213. The synthesized voice is generated based on the correspondence between the character information and the voice data stored in the utterance DB 120 (see FIG. 4) of the storage unit 202 in advance.

The language discrimination unit uses a language discrimination model that statistically models, for example, differences in phonetic types, word phonetic arrangement patterns, prosodic patterns, vocabulary, etc. for the input person's voice data. By executing the voice language discrimination process, the spoken language of the person is discriminated.
The voice recognition unit executes voice recognition processing using the statistical model (acoustic model and language model) for voice recognition prepared for each language on the language-discriminated voice data, so that the speech content of the person Recognize.

The camera 210 is a single camera such as a CCD (Charge-Coupled Device) camera, a stereo camera, a distance sensor, or a composite sensor built in the robot 2, and acquires ambient video data. do. The acquired video data is sent to the control unit 201.

The microphone 211 is a microphone array built in the robot 2, for example, a single microphone or a plurality of microphones, and acquires ambient voice data. The acquired voice data is sent to the control unit 201.

The sensor 212 includes sensors such as a 3D sensor, a touch sensor, a gyro sensor, a sonar sensor, a laser sensor, a bumper sensor, and an infrared sensor. Each sensor inputs the detected signal to the control unit 201.

The speaker 213 outputs the synthesized voice generated by the voice synthesis unit of the voice processing unit 209.
The bus 220 is a path that mediates the transfer of control signals, data signals, and the like.

The computer 3 is a control device that controls the operation of the entire content providing system 1, and controls the operation of the robot 2 in addition to acquiring video data by the camera 4, acquiring audio data by the microphone 5, and controlling the display of the display 6. do.

As shown in FIG. 3, in the computer 3, the control unit 301, the storage unit 302, the media input / output unit 303, the peripheral device I / F unit 304, the input unit 306, the display unit 307, and the communication I / F 308 are via the bus 309. Connected and configured.

The control unit 301 is composed of a CPU, a ROM, a RAM, and the like. The CPU calls and executes a program stored in the storage unit 302, ROM, recording medium, etc. in the work memory area on the RAM, and drives and controls each unit connected via the bus 309.

The CPU of the control unit 301 performs shooting control by the camera 4, voice acquisition control of the microphone 5, and display control of the display 6 in the process executed by the robot 2 (FIG. 15).

The ROM permanently holds a boot program of the computer 3, a program such as a BIOS, data, and the like. The RAM temporarily holds the loaded program and data, and includes a work area used by the control unit 301 to perform various processes.

The storage unit 302 stores a program executed by the control unit 301, data necessary for program execution, an operating system, and the like. These program codes are read out by the control unit 301 as necessary, transferred to the RAM, read out by the CPU, and executed.

The media input / output unit 303 is, for example, a drive device for a recording medium (media) such as a magnetic disk, an optical disk, or a semiconductor memory, and inputs / outputs data.

The peripheral device I / F (interface) section 304 is a port for connecting peripheral devices, and data is transmitted / received to / from the peripheral device via the peripheral device I / F section 304. The connection form with peripheral devices may be wired or wireless. A robot 2, a camera 4, a microphone 5, a display 6, and the like can be communicated and connected to the computer 3 via the peripheral device I / F unit 304.

The input unit 306 is an input device such as a keyboard, a mouse, and a touch panel, and outputs the input data to the control unit 301.

The display unit 307 is composed of a display device such as a liquid crystal panel and a logic circuit for executing display processing in cooperation with the display device, and displays display information input under the control of the control unit 301 on the display device. Let me. The touch panel of the input unit 306 is integrally configured with the display of the display unit 307. Further, the display unit 307 may be an external display 6. The control unit 301 sends display information to the display unit 307 or the display 6 to display the display information.

The communication I / F 308 is an interface that mediates communication with the network 8. The computer 3 can send and receive various programs and data from the Web server on the network 8.
The bus 309 is a path that mediates the transfer of control signals, data signals, and the like.

Next, the functional configuration of the content providing system 1 (robot 2) will be described with reference to FIG.

The content providing system 1 (robot 2) has each function of a person recognition unit 21, an individual / group recognition unit 22, a content provision unit 23, a gesture recognition unit 24, an interest level acquisition unit 25, and an interest level reflection unit 26. The processing of each function is executed by the program. This program may be stored in advance in the storage unit 202 of the robot 2 or the storage unit 302 of the computer 3, or may be downloaded and installed via the network 8.

The person recognition unit 21 recognizes one or more people existing around the robot 2, and stores the recognized person information in the person recognition information 30 (FIG. 5).
As shown in FIG. 4, the person recognition unit 21 further includes each function of the face recognition unit 21A, the position acquisition unit 21B, the recognition time acquisition unit 21C, and the attribute acquisition unit 21D.

The face recognition unit 21A recognizes a face image of one or more persons in the video data acquired by the camera 210 built in the robot 2 or the external camera 4 (hereinafter referred to as face recognition). When face recognition is performed, an ID that uniquely identifies the face-recognized person is issued and stored in the person ID 31 of the person recognition information 30 (see FIG. 5).

The position acquisition unit 21B tracks (tracks) the face image recognized by the face recognition unit 21A, acquires the position information of the face, and stores it in the tracking position 32 of the person recognition information 30 in association with the person ID 31. When the person moves, the tracking position 32 is also constantly updated.

The recognition time acquisition unit 21C acquires the time (recognition time) for face recognition by the face recognition unit 21A, and stores the recognition time 33 of the person recognition information 30 in association with the person ID 31.

The attribute acquisition unit 21D acquires the attribute information of each person whose face is recognized by the face recognition unit 21A. The attribute information is, for example, the age (age), gender, etc. of a person. The attribute acquisition unit 21D acquires attribute information from the face image of each person by using a known image processing technique (for example, Japanese Patent Application Laid-Open No. 2012-053813), and associates the person attribute 34 of the person recognition information 30 with the person ID 31. Attach and memorize.

When the person whose face is recognized by the face recognition unit 21A is framed out from the video data acquired by the camera 210 built in the robot 2 or the external camera 4, the person recognition unit 21 causes the person recognition unit 21 to frame out. The recognition information of the person is deleted from the person recognition information 30. That is, the person recognition information 30 (FIG. 5) stores only the information of the surrounding people currently recognized by the robot 2.

The individual / group recognition unit 22 refers to the person recognition information 30 and recognizes one or more "individuals" and / or one or more "groups" that are the content provision targets (content viewers).
A "group" is defined as a group consisting of at least a plurality of people, and there is some kind of physical relationship (close position, etc.) or attribute relationship (family with children, couple, etc.) between the persons. say. An "individual" is a single person who does not form a group with any person.

The individual / group recognition unit 22 is a group of one or more based on (1) tracking position 32, (2) recognition time 33, (3) person attribute 34, etc. of the person recognition information 30 (FIG. 5) as follows. Recognize.

(1) Tracking position 32
The distance between the people is calculated based on the tracking position 32 of each person, and the people who are close to each other are recognized as a group.
(2) Recognition time 33
People whose recognition times 33 are close to each other are recognized as a group.
(3) Person attribute 34
Based on the person attribute 34 of each person, people with similar attributes such as age (age) and gender are recognized as a group.

The individual / group recognition unit 22 recognizes one or more groups based on any one of the above (1), (2), and (3) or a plurality of pieces of information, and uses the recognized group information as group recognition information 40 (FIG. 6). See).

FIG. 6 shows an example of the group recognition information 40. The group recognition information 40 includes a group ID 41 that uniquely identifies the recognized group, a person ID list 42 that holds the person ID 31 of each person belonging to the group, and a group attribute ID 43 that uniquely identifies the group attribute (group attribute). And are associated and held.

In the case of FIG. 6, it is shown that the individual / group recognition unit 22 has recognized two groups (groups whose group ID 41 is “G1” and “G2”) from the person recognition information 30 (FIG. 5).
The group of the group ID 41 (“G1”) is a group composed of persons whose person ID 31 is “P1”, “P2”, and “P3”, and the group attribute is the “family with children” of the group attribute ID “A1”. Is.
Further, the group of the group ID 41 (“G2”) is a group composed of persons whose person IDs are “P4” and “P5”, and the group attribute is a “couple in their twenties” of the group attribute ID “A2”.

The group attribute is determined by referring to the group attribute table 50 as shown below.
FIG. 7 shows a group attribute table 50 that is referred to when determining group attributes.
The group attribute table 50 holds the group attribute ID 43 that uniquely identifies the group attribute, the group attribute 52 that indicates the content of the group attribute, and the group condition 53 that indicates the conditions of the group attribute in association with each other.

The individual / group recognition unit 22 refers to the tracking position 32, the recognition time 33, the person attribute 34, and the like of each person included in the group, and determines whether or not the group condition 53 of each group attribute is satisfied. When the group condition 53 of any of the group attributes is satisfied, the group attribute ID 43 related to the group attribute is set in the group recognition information 40.

For example, in the case of the group of group ID 41 "G1" in FIG. 6, the person whose person ID 31 is "P1" is "male (42 years old)" and the person whose person ID 31 is "P3" is "female (42 years old)" (Fig. 6). 5), so the condition (1) “one male and / or one female” of the group condition 53 of the group attribute ID 43 “A1” (“family with children”) is satisfied.

Further, since the person whose person ID 31 is "P2" is a "boy (10 years old)" (see FIG. 5), the condition (2) "one or more boys older than the person in the condition (1) and / or" , More than one girl "satisfies.

Further, when it is determined that the persons are close to each other based on the tracking positions 32 of the persons of "P1" to "P3", the condition (3) is satisfied. In the case of FIG. 6, since all the conditions (1) to (3) are satisfied, "A1" ("family with children") is set in the group attribute ID 43 of the group recognition information 40.

On the other hand, the individual / group recognition unit 22 recognizes a person when only the recognition information of a single person exists in the person recognition information 30 (FIG. 5) (when only a single person is face-recognized by the person recognition unit 21). Among the persons in the information 30 (FIG. 5), the persons who are not recognized as a group are recognized as "individuals".

FIG. 8 shows an example of personal recognition information 45. The personal recognition information 45 holds a personal ID 46 that uniquely identifies a recognized individual, a person ID 31 that corresponds to the recognized individual, and a personal attribute ID 47 that uniquely identifies an individual attribute (personal attribute). do.

Personal attributes are information that categorizes recognized individuals (persons) by age (age), gender, and the like.
The individual / group recognition unit 22 determines the individual attribute by referring to the individual attribute table 55 (FIG. 9). The person attribute 34 in FIG. 5 may be used as the personal attribute.

The content providing unit 23 executes the provision of the content by the robot 2. The content providing unit 23 refers to the content list 60 stored in the storage unit 202, and reads and provides the content from the content DB 110 at random or in a predetermined order. Further, the content providing unit 23 acquires the attribute of the content viewer, refers to the content interest level table 100 (FIG. 14), selects and provides the content having a high degree of interest in the attribute of the content viewer. You may.

FIG. 10 shows a content list 60 stored in the storage unit 202. As shown in the figure, the content list 60 holds the content ID 61 that uniquely identifies the content and the content storage destination 62 that indicates the information of the storage destination of the content data in the storage unit 202 (content DB 110) in association with each other. ..

The content data is, for example, video data, audio data, text data, motion data, or any combination of these data.
When the content data is video data, the video is displayed on the display unit 204 or the display 6 of the robot 2. When the content data is voice data, the voice is output from the speaker 213 of the robot 2. When the content data is text data, the voice processing unit 209 refers to the utterance DB 120 to generate synthetic voice corresponding to the text data, and outputs the voice from the speaker 213. Alternatively, the text data is displayed on the display unit 204 or the display 6 of the robot 2. When the content data is operation data, the movable unit control sequence corresponding to each action included in the operation data is read from the action DB 130, a control signal is sent to the movable unit drive mechanism 207, and each drive unit is operated.

In the utterance DB 120, text data (character information) and voice data are stored in association with each other. The voice processing unit 209 generates synthetic voice based on the correspondence between the text data stored in the utterance DB 120 and the voice data, and outputs the synthesized voice from the speaker 213 to realize the utterance by the robot 2. The utterance DB 120 may store voice data of fixed phrases, utterance conversion data for changing utterance intonation and endings according to the emotional state of the robot 2.

The action DB 130 stores a control sequence table that describes the control sequence of the robot 2. The control sequence table stores data in which the action name of the action and the movable part control sequence of the robot 2 are associated with each other. The action name corresponds to the action name of the action of the content data, and is, for example, the robot action when the robot 2 executes a series of movable part control sequences such as "nodding", "raising one hand", and "raising both hands". It shows the situation. The movable portion control sequence is a list of drive points and values (for example, head tilt angle 20 degrees, right arm tilt angle 40 degrees, etc.) of each movable portion to be controlled.

The gesture recognition unit 24 recognizes the gestures of each person included in the "individual" and / or "group" (content viewer) recognized by the individual / group recognition unit 22.
FIG. 11 shows a gesture table 70 that holds the gesture content to be recognized. As shown in the figure, the gesture table 70 has a gesture ID 71 that uniquely identifies a gesture, a gesture content 72 that indicates the content of the gesture, and a gesture that indicates the gesture content 72 as features such as the positions and movements of facial organs and limbs. The gesture feature amount 73 represented as (operation pattern) is associated with and held.

The gesture recognition unit 24 analyzes the video data acquired by the camera 210 built in the robot 2 or the external camera 4, and recognizes the "individual" and / or "group" (contents) by the individual / group recognition unit 22. Recognize the gestures of each person included in the viewer) in real time. Specifically, the gesture recognition unit 24 extracts feature quantities (motion patterns) such as the positions and movements of facial organs and limbs from video data, and the extracted feature quantities (motion patterns) and gesture feature quantities 73 (FIG. 11). ) Is matched to recognize the gesture. When the gesture recognition unit 24 recognizes the gesture, the gesture recognition unit 24 notifies the interest level acquisition unit 25 of the person ID and the gesture ID of the person who recognized the gesture.

According to the present embodiment, since the gesture is recognized by analyzing the video data acquired from the camera, it is not necessary to attach a wearable device (posture detection device, etc.) for recognizing the gesture, and the video It is possible to recognize the gestures of multiple people reflected in the image at the same time.

In order to evaluate the reaction to the content, the gesture recognition unit 24 preferably recognizes the gesture of the person during the content provision, but may recognize the gesture of the person after the content is provided. For example, it is assumed that a person who has viewed the content with interest may show a favorable reaction to the robot 2 (for example, requesting the robot 2 to shake hands at the time of parting) after the content is provided.

The interest level acquisition unit 25 acquires a score of interest level for the content based on the gesture recognized by the gesture recognition unit 24.
Specifically, first, when the gesture ID recognized by the gesture recognition unit 24 and the person ID of the person who made the gesture are notified, the degree of interest increase / decrease value 82 for the content of the person is calculated in the degree of interest calculation table 80 (FIG. 12). ).
FIG. 12 shows the contents of the interest rate calculation table 80 referred to for calculating the interest rate. As shown in the figure, an interest degree increase / decrease value 82, which is a score of interest degree for each gesture, is set in the interest degree calculation table 80.

According to the interest degree calculation table 80 of FIG. 12, for example, when the acquired gesture is “nodding” with the gesture ID “J1”, it is considered that the interest in the content is high, so the interest degree increase / decrease value 82 is “+4”. To get. Further, when the acquired gesture is "tilt" with the gesture ID "J2", it is considered that the degree of interest in the content is low, so "-4" is acquired as the degree of interest increase / decrease value 82.

The interest degree acquisition unit 25 stores the acquired interest degree increase / decrease value 82 in the interest degree history table 90 as a history. The interest level history table 90 is the total value of the interest level increase / decrease value 82 and the interest level increase / decrease value 82 acquired during the content provision (including after the content is provided if the gesture after the content is provided). It is a table which stores a degree score 93 (see FIG. 13 (a)). The degree of interest history table 90 is prepared for each "individual" and / or "group" recognized by the individual / group recognition unit 22.

When the person who made the gesture is an "individual", the interest degree acquisition unit 25 stores the acquired interest degree increase / decrease value 82 in the interest degree history table 90 associated with the individual (person). When the person who made the gesture belongs to the "group", the acquired interest rate increase / decrease value 82 is stored in the interest rate history table 90 associated with the group.

FIG. 13A shows an interest rate history table 90 that stores the history of the interest rate increase / decrease value 82 of each person included in the group of the group ID 41 “G1” of FIG. As shown in the figure, the history of the interest rate increase / decrease value 82 of each person P1, P2, P3 included in the group is stored.

When the content is provided, the interest level acquisition unit 25 records the total value of the interest rate increase / decrease values 82 in the interest level score 93 of the interest rate history table 90. In the example of FIG. 13A, "+3", "+17", and "+7" are recorded as the total value of the interest rate increase / decrease values 82 of each person P1, P2, and P3 in the interest rate score 93.

In the case of a person recognized as an "individual" by the individual / group recognition unit 22, the interest level acquisition unit 25 has a history of the person's interest level history table 90 (history and interest level increase / decrease of the person's interest level increase / decrease value 82). The interest level score 93 is obtained from the table in which the interest level score 93, which is the total value of the values 82, is stored, and is used as the final interest level score for the content of the person.
On the other hand, when it is recognized as a "group" by the individual / group recognition unit 22, it is a score of the degree of interest of the group as a whole based on the degree of interest score 93 of each person in the degree of interest history table 90 of the group. The group interest score 95 is calculated and acquired.

As the group interest score 95, for example, an average score 95a, a maximum score 95b, and a representative score 95c can be adopted as shown in FIG. 13B.
Of these, the average score 95a (+9) in FIG. 13B is a group interest score 95 as an average value of interest scores 93 (+3, +17, +7) of each person included in the group.
Further, the maximum score 95b (+17) in FIG. 13B is the group interest score 95 as the maximum value of the interest score 93 (+3, +17, +7) of each person included in the group.

Further, in the representative score 95c (+17) of FIG. 13 (b), the group interest score 95 is the interest score 93 of the group representative (P2 in the case of FIG. 13 (b)). The representative of the group is selected based on (1) attribute information, (2) position information, (3) utterance information, (4) residence time, etc. of each person included in the group as follows.

(1) Attribute information For example, referring to the person attribute 34 of the person recognition information 30, the person with the oldest age among each person included in the group is selected as the representative of the group. Alternatively, if the content targets a specific attribute (age group, gender, etc.), the person closest to that attribute is selected as the representative of the group.
(2) Position information For example, a camera 210 (distance sensor) or a microphone 211 (microphone array) is used to measure the distance to a person, and the person closest to the robot 2 is selected as the representative of the group.
(3) Utterance information For example, the recorded voice of the microphone 211 is analyzed, and the person who speaks the most (speaks well) or the person who speaks the loudest (speaks loudly) in the group is selected as the group. Select as a representative.
(4) Resident time For example, with reference to the recognition time 33 of the person recognition information 30, the residence time, which is the time from the recognition time 33 to the current time, is calculated for each person included in the group, and the person with the longest residence time. Is selected as the representative of the group.

The interest level acquisition unit 25 selects a representative based on any one of the above (1), (2), (3), and (4) or a plurality of pieces of information, and sets the representative's interest level score 93 as the group interest rate score 95. Get as.

The interest level reflecting unit 26 reflects the degree of interest in the provided content in the content interest level table 100 (FIG. 14) based on the interest level score acquired by the interest level acquisition unit 25.

Specifically, in the case of "individual", the degree of interest in the content determined from the degree of interest score 93 of the individual (person) (degree of interest = "high", degree of interest = "medium", degree of interest = "Low") is associated with the provided content and the personal attribute, and is reflected in the content interest level table 100. In the case of "group", the degree of interest (interest = "high", interest = "medium", interest = "low" determined from the group interest score 95, which is the interest score of the entire group. ”) Is associated with the provided content and the group attribute, and is set to be reflected in the content interest level table 100.

In this embodiment, the degree of interest is defined in three stages (“high”, “medium”, and “low”) based on the degree of interest score 92 or the degree of group interest score 95. Is not limited to this. For example, the values of the interest rate score 93 and the group interest rate score 95 may be used as they are.

FIG. 14 shows a content interest level table 100 in the case of a “group”. As shown in the figure, the content interest table 100 includes a content ID (C1, C2, C3, ...), a group attribute ID (A1 (“family with children”), A2 (“couple in their 20s”), and A3 (“20s couple”). The degree of interest ("high", "medium", "low") is set in association with "old couple"), ...). In the example of the figure, when the group interest rate score 95 ≧ +10, the interest level is “high”, when 0 <group interest rate score 95 <+10, the interest level is “medium”, and the group interest rate score 95. When ≤0, the degree of interest is defined as "low".

Although not shown, in the case of "individual", the provided content and personal attributes ("male 40s", "female 40s", etc.) are linked to the content interest level table 100 as in FIG. The degree of interest (“high”, “medium”, “low”) is set.

By reflecting the degree of interest in the content provided in the content interest level table 100 in this way, it is possible to determine what kind of content an individual or group has high (or low) interest in. Be learned. As a result, the robot 2 can select and provide appropriate content according to the attributes of the content viewer.

Next, the flow of processing executed by the robot 2 will be described with reference to FIG.
The robot 2 is installed in, for example, a product introduction area or an exhibition area, and executes the following processing on a visitor.

First, the robot 2 recognizes one or more people around it in real time (step S0). The recognized person information is stored in the person recognition information 30 (FIG. 5) as described above.

Subsequently, the robot 2 recognizes one or more "individuals" and / or one or more "groups" that are the content provision targets (content viewers) with reference to the person recognition information 30 (FIG. 5). Step S1). As described above, the group is recognized based on the tracking position 32, the recognition time 33, and the person attribute 34 of the person recognition information 30. On the other hand, when only the information of a single person exists in the person recognition information 30 (when only a single person is recognized by the person recognition unit 21), or when the person in the person recognition information 30 is not recognized as a group. Recognize as "individual".

Subsequently, the robot 2 executes the provision of the content (step S2). Specifically, the robot 2 refers to the content list 60 stored in the storage unit 202, reads the content data from the content DB 110 at random or in a predetermined order, and executes the provision of the content.

Subsequently, the robot 2 recognizes the gestures of each person included in the "individual" and / or "group" (content viewer) recognized in step S1 in real time while providing the content (step S3).

When the robot 2 recognizes the gesture of the person (step S3; Yes), the robot 2 refers to the interest degree calculation table 80 (FIG. 12), acquires the interest degree increase / decrease value 82 corresponding to the gesture, and displays the interest degree history table 90. It is stored as a history (step S4). As described above, when the person who performed the gesture is an "individual", the acquired interest degree increase / decrease value 82 is stored in the interest degree history table 90 associated with the individual (person). When the person who made the gesture belongs to the "group", the acquired interest rate increase / decrease value 82 is stored in the interest rate history table 90 associated with the group.

The robot 2 repeatedly recognizes the gesture (step S3) described above and acquires the interest level increase / decrease value 82 (step S4) corresponding to the recognized gesture until the content provision is completed (step S5; Yes).
If it is desired to reflect the reaction after the content is provided in the degree of interest, steps S3 and S4 are repeated until a predetermined time elapses after the content is provided.

Then, when the provision of the content is completed (step S5; Yes), the robot 2 obtains the degree of interest for each "individual" and / or "group" recognized in step S1 and reflects it in the content interest degree table 100 (step). S6).
Specifically, as described above, in the case of an "individual", the interest level score 93 (total value of the interest level increase / decrease value 82) of the individual interest level history table 90 is acquired, and the interest level score 93 is used. The determined degree of interest (“high”, “medium”, “low”) is associated with the provided content and the personal attribute, and is reflected and set in the content interest degree table 100.

Further, in the case of a "group", a group interest score 95, which is a score of the interest of the group as a whole, is acquired based on the interest score 93 of each person in the interest history table 90 of the group, and the group is obtained. The degree of interest (“high”, “medium”, “low”) determined from the degree of interest score 95 is associated with the provided content and the group attribute, and is reflected and set in the content degree of interest table 100. As the group interest score 95, for example, an average score 95a, a maximum score 95b, and a representative score 95c can be adopted as described above (see FIG. 13B).

The processing executed by the robot 2 has been described above. According to the present embodiment, the robot 2 recognizes the content provision target (content viewer) in units of "individual" and "group" based on the position information, recognition time, person attribute, etc. of surrounding people, and "individual". Get the score of interest in the content in "group" units. Then, the acquired degree of interest is stored in association with the provided content, individual attributes, and group attributes. In this way, it is learned what kind of attribute individuals and groups are interested in what kind of content. As a result, the robot 2 can provide appropriate content to content viewers in places where individuals and groups coexist as content viewers, such as product introduction areas and exhibition areas where an unspecified number of people gather. It is possible to improve the interest / interest of content viewers, customer experience, purchasing motivation, etc.

Specifically, the robot 2 can provide appropriate content to the content viewer as follows.
For example, when the robot 2 provides the content in step S2, the robot 2 acquires the attribute of the content viewer (“individual” and / or “group”) recognized in step S1. Then, with reference to the content interest level table 100, the content having a high degree of interest in the acquired attributes is selected and provided. For example, it is assumed that the group attribute ID "A2" is acquired as the group attribute of the content viewer in step S2. In this case, the robot 2 refers to the content interest level table 100 of FIG. 14 and selects and provides the content of the content ID “C3” having a high degree of interest in the group attribute ID “A2”. When a plurality of "individuals" or a plurality of "groups" are recognized in step S1 (when there are a plurality of content viewers), for example, the content viewers closest to the robot 2 ("individuals"). Alternatively, the attribute of "group") may be acquired, and the content having a high degree of interest in the attribute may be provided.

Further, for example, the robot 2 may switch the content to be provided in real time according to the degree of interest. For example, when the interest level increase / decrease value 82 acquired in step S4 tends to decrease (for example, when the interest level increase / decrease value 82 continuously becomes a negative value) while the content is being provided, the robot In 2, it may be determined that the interest or interest in the content being provided is low, and the content may be switched to another content.

Although the preferred embodiments according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or modified examples within the scope of the technical idea disclosed in the present application, and these also naturally belong to the technical scope of the present invention. Understood.

1 …………………… Content provision system 2 …………………… Robot 3 …………………… Computer 4 …………………… Camera 5 …………………… … Mike 6 …………………… Display 8 …………………… Network 21 …………………… Person recognition unit 21A ……………… Face recognition unit 21B ……………… Position acquisition unit 21C ……………… Recognition time acquisition unit 21D ……………… Attribute acquisition unit 22 …………………… Group recognition unit 23 …………………… Content provision unit 24 ………… ………… Gesture recognition unit 25 …………………… Interest level acquisition unit 26 …………………… Interest level reflection unit 30 …………………… Person recognition information 32 …………………… Tracking position 33 …………………… Recognition time 34 …………………… Person attribute 40 …………………… Group recognition information 45 …………………… Individual recognition information 50 ……………… …… Group attribute table 55 …………………… Personal attribute table 60 …………………… Content list 70 …………………… Gesture table 80 …………………… Interest degree calculation table 90 …… ……………… Interest level history table 93 …………………… Interest level score 95 …………………… Group interest level score 95a ……………… Average score 95b ……………… Maximum Score 95c ……………… Representative score 100 ……………… Content interest level table 201 ……………… Control unit 202 ……………… Storage unit 203 ……………… Input unit 204 ……………… Display unit 205 ……………… Media input / output unit 206 ……………… Communication I / F
207 ……………… Movable part drive mechanism 208 ……………… Peripheral device I / F part 209 ……………… Voice processing unit 210 ……………… Camera 211 ……………… Microphone 212 ……………… Sensor 213 ……………… Speaker 220 ……………… Bus

Claims (11)

A person recognition means that recognizes multiple people existing in the vicinity, acquires the position information, recognition time or attribute information of each recognized person, and stores it in association with the identification information that uniquely identifies each person.
An individual / group recognition means for distinguishing and recognizing each recognized person into an individual and a group based on the position information, the recognition time, or the attribute information.
Content providing means for providing content to the recognized individual and the group, and
An interest degree acquisition means for acquiring the interest level of the individual and the group with respect to the content, and
A storage means for storing the acquired degree of interest in the content in association with the attribute information of the individual and the group.
Equipped with
When providing content to a new individual or group, the content providing means refers to the storage means and selects and provides content having a high degree of interest in the attributes of the individual or the group. A content providing device characterized by the fact that. A person recognition means that recognizes multiple people existing in the vicinity, acquires the position information, recognition time or attribute information of each recognized person, and stores it in association with the identification information that uniquely identifies each person.
An individual / group recognition means for distinguishing and recognizing each recognized person into an individual and a group based on the position information, the recognition time, or the attribute information.
Content providing means for providing content to the recognized individual and the group, and
An interest degree acquisition means for acquiring the interest level of the individual and the group with respect to the content, and
A storage means for storing the acquired degree of interest in the content in association with the attribute information of the individual and the group.
Equipped with
The degree of interest acquisition means selects a representative of the group from each person included in the group, and acquires the degree of interest of the selected representative as the degree of interest of the group. A featured content providing device. The degree of interest acquisition means selects a representative of the group based on the position information, attribute information, utterance information, or residence time of each person included in the group.
The content providing device according to claim 2, wherein the content providing device is characterized by the above. Claim 1 to claim 1, wherein the individual / group recognition means recognizes people who are close to each other as the same group, or recognizes people who have similar recognition times as the same group. The content providing device according to any one of 3. The content providing device according to claim 4 , wherein the individual / group recognition means recognizes a person who is not recognized as a group as an individual. Further provided with a gesture recognition means for recognizing the gestures of the individual and each person included in the group recognized by the individual / group recognition means.
The content providing device according to any one of claims 1 to 5 , wherein the degree of interest acquisition means acquires the degree of interest based on the gesture recognized during the provision of the content. The content providing device according to claim 6 , wherein the interest degree acquisition means further acquires the interest degree based on the gesture recognized after the content is provided. Any of claims 1 to 7 , wherein the content providing means changes the content to be provided according to the degree of interest in the content acquired by the degree of interest acquisition means while the content is being provided. The content providing device described in. The control unit of the content providing device
A person recognition step that recognizes multiple people existing in the vicinity, acquires the position information, recognition time or attribute information of each recognized person, and stores it in association with the identification information that uniquely identifies each person.
An individual / group recognition step in which each recognized person is recognized separately as an individual or a group based on the position information, the recognition time, or the attribute information.
A content provision step that provides content to the recognized individual and the group, and
An interest level acquisition step for acquiring the interest level of the individual and the group with respect to the content, and
A storage step of associating the acquired degree of interest in the content with the attribute information of the individual and the group and storing it in the storage unit.
Only including,
When providing content to a new individual or group, the content providing step refers to the storage unit and selects and provides content having a high degree of interest in the attributes of the individual or the group. /> A content provision method characterized by this. The control unit of the content providing device
A person recognition step that recognizes multiple people existing in the vicinity, acquires the position information, recognition time or attribute information of each recognized person, and stores it in association with the identification information that uniquely identifies each person.
An individual / group recognition step in which each recognized person is recognized separately as an individual or a group based on the position information, the recognition time, or the attribute information.
A content provision step that provides content to the recognized individual and the group, and
An interest level acquisition step for acquiring the interest level of the individual and the group with respect to the content, and
A storage step of storing the acquired degree of interest in the content in association with the attribute information of the individual and the group.
Only including,
In the interest level acquisition step, a representative of the group is selected from each person included in the group, and the interest level of the selected representative is acquired as the interest level of the group. Characteristic content provision method. A program characterized by functioning as the content providing device according to any one of claims 1 to 8.

Images