JP6752532B2 - Behavior analyzers, programs and methods for estimating improvement factors of service scenarios from user behavior targets - Google Patents

Description

The present invention relates to a technique for estimating user behavior from images taken by a camera.

In recent years, there has been a demand for commercial transactions that provide products and services that are optimal for users of different ages and genders. Therefore, the service provider conducts a monitor test in which general consumers actually use the product service and identify points for improvement before the product service is released or at the time of marketing research. In the monitor test, the user who uses the product service usually answers the impressions and problems in the form of a questionnaire.

In the questionnaire method, the user often does not fully describe the problem. Further, in the monitor test, even if the service provider recommends the handling procedure of the product service, it is not possible to grasp how the user handles it. Therefore, in the monitor test, it was necessary to actually gather the users in the same place in order to grasp the behavior of the users who handle the goods and services. In order to save such trouble, there is also a demand for the service provider to automatically grasp the user's behavior in the monitor test.

Conventionally, there is a technique of attaching a device for measuring pulse information to a user and analyzing the behavioral state (mental state, external environment, living place) of the user (see, for example, Patent Document 1). According to this technology, when a service provider detects a stress state of a user, it is necessary for the service provider to notify the user of the behavioral state at that time.

In addition, there is also a technique for executing operation support suitable for proficiency and the like based on the operation status of a player who operates an information processing device (see, for example, Patent Document 2). According to this technology, the operation status of the operation guide application by the player is acquired from the touch panel, and the operation by the player is dynamically supported. By determining whether or not the operation status matches the expected reaction, the player's proficiency level and learning motivation can be estimated.

Further, there is also a behavior factor analysis technique for calculating the emotional response of the user based on the position information and direction of the user and the position information of the object (see, for example, Patent Document 3). According to this technique, it is possible to identify an object that the user is paying attention to, determine an area showing interest in the object from the viewpoint position of the user, and then calculate the emotional reaction.

Further, there is also a technique of linking a user's action and the next action to be taken and presenting how the influence of the user's action is propagated to other users (see, for example, Patent Document 4). For example, it is possible to calculate a trend in which a user's behavior of "discarding an empty can in a trash can" affects another user.

Further, there is also a technique for appropriately displaying a preview help function that supports the user's operation depending on the difference in the user's operation proficiency (see, for example, Patent Document 5). According to this technology, it is possible to memorize whether or not a certain operation in the past has been performed smoothly, and display the preview help function only when it is determined that the user is unfamiliar with the operation to be performed. ..

JP-A-2016-202347 Japanese Unexamined Patent Publication No. 2015-132951 Japanese Unexamined Patent Publication No. 2010-061452 WO2012090697A1 JP-A-2009-020554

Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem and Juan Carlos Niebles, “ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding,” CVPR2015., [Online], [Search September 18, 2017], Internet <URL : http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Heilbron_ActivityNet_A_Large-Scale_2015_CVPR_paper.pdf>

The service provider wants the user to handle the service according to the service scenario in the product service. A service scenario defines a user's action target in chronological order as a handling procedure (instruction manual or operation procedure).

However, even if the service provider recommends the service scenario of the product service, it cannot grasp how the general consumer user actually handles it. Many users often take actions that are different from the service scenario, or the time is long.
Further, according to the above-mentioned conventional technology, it is not determined that the action target of the user proceeds according to the service scenario in the first place. In addition, these conventional techniques also require the user to attach the device or perform a specific operation.

On the other hand, the inventors of the present application wondered if it would be possible to determine whether or not the operation was performed according to the service scenario without requiring each user to attach the device or perform a specific operation.
In addition, when the user does not operate according to the service scenario, it is further determined whether the service scenario itself has an improvement factor, or whether it is caused by the user's personality or the surrounding environment other than the service scenario. I also wondered if I could do it.
If the service scenario itself can be improved, the inconvenience of various users who try to use the service can also be improved.

Therefore, an object of the present invention is to provide a behavior analysis device, a program, and a method for estimating improvement factors of a service scenario from a user's behavior target without requiring the user to wear a device or perform a specific operation.

According to the present invention, it is a behavior analysis device that analyzes a user's behavior from an image taken by a camera.
A service scenario registration method for registering service scenarios (procedures) in which each action target associated with action time is arranged in chronological order, and
An action target estimation means that sequentially estimates the user's action target from the video,
Whether the estimated action target is the first improper behavior that has shifted to an action target different from the time series, the second improper behavior that exceeds the action time, or other appropriate behavior for the service scenario. Judgment scenario Behavior judgment means and
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second failure, using the behavior results of a plurality of users. When the number or ratio of users of appropriate behavior is equal to or greater than the second predetermined number or the second predetermined ratio, the service scenario itself is characterized by having a behavior factor estimating means for presuming that there is an improvement factor.

According to another embodiment of the behavior analyzer of the present invention.
The action target estimation means further detects peripheral objects reflected in the image.
The service scenario registration method registers peripheral objects that are reflected in the video for each service scenario.
It is also preferable that the scenario action determination means does not determine the action result when the peripheral object registered in the service scenario is not reflected in the image.

According to another embodiment of the behavior analyzer of the present invention.
The service scenario registration method has registered multiple service scenarios.
It is also preferable to further have a service selection means for selecting one of the service scenarios based on the user's action target estimated from the video.

According to another embodiment of the behavior analyzer of the present invention.
The action target estimation means detects peripheral objects reflected in the image, and
The service scenario registration method registers peripheral objects that are reflected in the video for each service.
As the service selection means, it is also preferable to further narrow down and select one of the service scenarios from the peripheral objects reflected in the image.

According to another embodiment of the behavior analyzer of the present invention.
It further has a user group storage means for grouping and storing a plurality of users to which user attribute information is given based on predetermined conditions.
It is also preferable that the behavior factor estimation means estimates whether or not there is an improvement factor in the service scenario itself for each group.

According to another embodiment of the behavior analyzer of the present invention.
It is also preferable that the scenario action determining means does not determine the action result when the action target different from the time series of the service scenario is the action target not included in the service scenario.

According to another embodiment of the behavior analyzer of the present invention.
Furthermore, the user's utterance voice is collected from the terminal that recorded the user's utterance voice with the microphone.
It is also preferable that the action target estimation means sequentially estimates the user's action target by further using the spoken voice.

According to the present invention, the behavior analyzer described above and
A system in which a terminal having a speaker and / or a display capable of instructing an action to a user communicates via a network.
The behavior analysis device provides a behavior instruction transmitting means for transmitting behavior instruction media data based on the service scenario to the terminal when the scenario behavior determination means determines the first improper behavior or the second improper behavior. Have more
The terminal is characterized in that the action instruction media data received from the behavior analyzer is reproduced by the speaker and / or the display.

According to the present invention, it is a behavior analysis program that functions a computer mounted on a device that analyzes user behavior from images taken by a camera.
A service scenario registration method for registering service scenarios (procedures) in which each action target associated with action time is arranged in chronological order, and
An action target estimation means that sequentially estimates the user's action target from the video,
Whether the estimated action target is the first improper behavior that has shifted to an action target different from the time series, the second improper behavior that exceeds the action time, or other appropriate behavior for the service scenario. Judgment scenario Behavior judgment means and
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second failure, using the behavior results of a plurality of users. When the number or ratio of users with appropriate behavior is equal to or greater than the second predetermined number or the second predetermined ratio, the computer is characterized by functioning as a behavior factor estimation means for presuming that the service scenario itself has an improvement factor. ..

According to the present invention, it is a behavior analysis method of a device that analyzes a user's behavior from an image taken by a camera.
The device has a service scenario registration unit that registers service scenarios (procedures) in which each action target associated with action time is arranged in chronological order.
The device is
The first step of sequentially estimating the user's action target from the video,
Whether the estimated action target is the first improper behavior that shifts to an action target different from the time series, the second improper behavior that exceeds the action time, or other appropriate behavior for the service scenario. The second step to judge and
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second failure, using the behavior results of a plurality of users. When the number or ratio of users with proper behavior is equal to or greater than the second predetermined number or the second predetermined ratio, the service scenario itself is characterized by executing the third step of presuming that there is an improvement factor.

According to the behavior analyzer, the program and the method of the present invention, it is possible to estimate the improvement factor of the service scenario from the behavior target of the user without requiring the user to wear the device or perform a specific operation.

It is a system configuration diagram in the present invention. It is a functional block diagram of the behavior analysis apparatus in this invention. It is explanatory drawing which shows the service scenario registered in the service scenario registration part. It is explanatory drawing which shows the selection of the service scenario in the service selection part. It is a flowchart of a scenario action judgment part. It is explanatory drawing which shows the processing flow of the specific example in a scenario action determination part. It is a table stored by the user behavior storage unit.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram in the present invention.

According to the system of FIG. 1, the behavior analysis device 1 is connected to the Internet and analyzes the user's behavior from the video captured by the camera.
Each of the terminals 2 is equipped with a camera, and transmits a video of the user's behavior state to the behavior analysis device 1 via a network. The terminal 2 is a smartphone or mobile terminal owned by each user and connects to an access network such as a mobile phone network or a wireless LAN.
Of course, the terminal 2 is not limited to a smartphone or the like, and may be, for example, a Web camera installed in the house. Further, the video data captured by the Web camera may be recorded on the SD card, and the recorded video data may be input to the behavior analysis device 1.

FIG. 2 is a functional configuration diagram of the behavior analyzer according to the present invention.

According to FIG. 2, the behavior analysis device 1 has a service scenario registration unit 101, a user group storage unit 102, and a user behavior storage unit 103. Further, the behavior analysis device 1 has an action target estimation unit 11, a scenario action determination unit 12, an action factor estimation unit 13, a service selection unit 14, and an action instruction transmission unit 15. These functional components are realized by executing a program that makes the computer mounted on the device function. In addition, the processing flow of these functional components can be understood as a behavior analysis method of the device.

[Service Scenario Registration Department 101]
The service scenario registration unit 101 registers each action procedure (service scenario) of the user recommended by the service provider in chronological order with the action time. The service scenario itself may be one, or may be plural so that it can be selected by the service selection unit 14.
Further, as another embodiment, it is also preferable that the service scenario registration unit 101 registers peripheral objects to be reflected in the image for each service scenario.

FIG. 3 is an explanatory diagram showing a service scenario registered in the service scenario registration unit.

According to FIG. 3, for example, three service scenarios are registered.
First service scenario ("TVde photo" service)
Service setting procedure for displaying photos received on a mobile phone on a TV (action target) (action time)
Grasp the dongle 5 seconds Insert the dongle into the TV 10 seconds Launch the mobile phone app 15 seconds Press the button on the TV 20 seconds (Peripheral objects-> People, dongle, TV, mobile phone)
-------------------------------------------------- -------------------
Second service scenario ("streaming" service)
Setting procedure of service to stream smartphone videos to recorder (action target) (action time)
Launching the smartphone app 10 seconds Pressing the button on the recorder 5 seconds Holding the cable 5 seconds Inserting the cable into the recorder 10 seconds (Peripheral objects-> People, cables, recorders, smartphones)
-------------------------------------------------- -------------------
Third service scenario ("PC settings" service)
Service setting procedure for automatically setting PC settings from a USB memory (action target) (action time)
PC operation 20 seconds Holding the USB memory 5 seconds Inserting the USB memory into the PC 10 seconds PC operation 15 seconds (Peripheral objects-> People, PC, USB memory)

The "action target" is a user's behavior estimated from the video by the action target estimation unit 11.
The "action time" is the estimated duration of the action target, and is the time of the conforming operation assumed by the service provider.
The "peripheral object" is an object that is supposed to be reflected in the image by the service provider.

[User group storage unit 102]
The user group storage unit 102 groups and stores a plurality of users to which the user attribute information is assigned based on predetermined conditions.
The user attribute information may be, for example, age, gender, terminal used, or the like. This makes it possible to group user groups. Some groups, such as young men, can act according to the service scenario, while others, such as older women, are difficult to act according to the service scenario.
These user attribute information can be acquired in advance from, for example, contract information of a telecommunications carrier.

The service scenario registration unit 101 described above preferably registers different service scenarios for each group of the user group storage unit 102. For example, depending on the user attribute, one action target may require a longer action time than the other groups. Some groups, such as young men, can operate each service scenario in a short time, while others, such as elderly women, take time to operate each service scenario.

[Action target estimation unit 11]
The action target estimation unit 11 recognizes the peripheral objects reflected in the video, and sequentially estimates the user's action target from the video.
Such estimation of behavioral objects is realized by existing technology based on deep learning (see, for example, Non-Patent Document 1). According to this technology, it is possible to analyze a user's "action target" such as "walking", "speaking", and "holding" from a video in which the user's behavior is reflected. The action target estimation unit 11 also measures the estimated action time (duration) of the action target.
Then, the action target estimation unit 11 sequentially outputs the estimated "action target" and the "action time" to the scenario action determination unit 12 and the service selection unit 14.

In actual operation, for example, a user who participates in a monitor test is asked to take a picture of his / her behavior with the camera of his / her smartphone (terminal 2). The smartphone transmits a video in which the user's behavior is reflected to the behavior analysis device 1. The behavior target estimation unit 11 of the behavior analysis device 1 can estimate the user's behavior target from the video.

It is also preferable that the action target estimation unit 11 estimates the user's action target, such as "tilting his head" or "grimacing", as an emotion target. Then, the "emotional object" and the "action time" are associated and output. For example, when the "action time" is long, the user can determine that he / she is in the state of the "emotional object". In addition, for example, when an elderly person is performing the action target "insert the USB memory into the personal computer" in the service scenario of the personal computer setting, the emotion target "grimacing" state exceeds the registered action time. If so, the user can also be presumed to be in a confused state.

As another embodiment, the action target estimation unit 11 may further collect the user's utterance voice from the terminal 2 which has recorded the user's utterance voice by the microphone. The action target estimation unit 11 can sequentially estimate the action target of the user by using not only the video but also the spoken voice.
For example, if there is an image in which the user brings the mobile phone closer to the ear after the ringtone of the mobile phone rings, it is presumed that the action target is "the incoming call of the mobile phone".
In addition, if there is a video of an action target different from the service scenario after the user utters "bad" or "wrong", it is considered that the probability of being the wrong action target is high.

[Service selection unit 14]
The service selection unit 14 selects one of the service scenarios from the user's action target estimated from the video by the action target estimation unit 11. It is also preferable that the service selection unit 14 further narrows down and selects one of the service scenarios from the peripheral objects reflected in the image by the action target estimation unit 11.
The selected service scenario is output to the scenario action determination unit 12.

FIG. 4 is an explanatory diagram showing selection of a service scenario in the service selection unit.

According to FIG. 4, the action target estimation unit 11 estimates from the video that the “dongle”, “television”, and “mobile phone” are reflected, and that the user's action target is “grasping the dongle”. Service scenario 1 is selected from the service scenario registration unit 101 of.

Further, for example, when the action target estimation unit 11 estimates that the “cable”, “recorder”, and “smartphone” are reflected from the video and the user's action target is “launching the smartphone application”, FIG. Service scenario 2 is selected from the service scenario registration unit 101.
Further, for example, when the action target estimation unit 11 estimates that the "personal computer" and "USB memory" are reflected from the video and the user's action target is "operation of the personal computer", the service scenario registration unit of FIG. Service scenario 3 is selected from 101.

[Scenario behavior judgment unit 12]
The scenario action determination unit 12 determines inappropriate behavior / appropriate behavior from the action target estimated by the action target estimation unit 11 for the service scenario of the service scenario registration unit 101. The service scenario may be, for example, one selected by the service selection unit 14.
The action target estimated by the scenario action determination unit 12 is output to the user action storage unit 103.

FIG. 5 is a flowchart of the scenario action determination unit.

The scenario action determination unit 12 sequentially determines the action targets arranged in the time series of the service scenario from the image reflected by the user.

(S120) As an optional operation, the scenario action determination unit 12 determines whether or not the peripheral object registered in the service scenario of the service scenario registration unit 101 is reflected in the video. If it is determined to be false, it is assumed that the user is taking another action, and the action result is not determined.

(S121) It is determined whether or not the estimated action target is different from the time-series action target of the service scenario. When it is determined to be true (different), it means the following two cases.
(S121a) "The handling procedure is not fully understood by the user" (misunderstanding)
(S121b) "I am taking different actions depending on the surrounding environment" (occurrence of behavioral inhibition factors)

According to the example (s121a), for example, the following cases.
<Handling procedure>
"Grip the dongle"->"Insert the dongle into the TV"->"Launch the mobile phone app"
<Estimated action target>
"Grip the dongle"->"Launch the mobile phone app" (★ different procedure)

According to the example (s121b), for example, the following cases.
<Handling procedure>
"Grip the dongle"->"Insert the dongle into the TV"->"Launch the mobile phone app"
<Estimated action target>
"Grip the dongle"->"Put your mobile phone in your ear (incoming call)"

(S122) It is determined whether or not the estimated action target and the action target of the service scenario match, and the action time of the estimated action target is equal to or longer than the action time of the service scenario.
If it is determined to be true (more than the action time), it is regarded as "second improper action".
On the other hand, if it is determined to be false (shorter than the action time), it is regarded as "appropriate action".

The second improper behavior means the following two cases.
(S122a) "The handling procedure is not fully understood by the user" (confused)
(S122b) I am not at a loss, but it takes time depending on the "personality of the user". Specifically, for example, there are the following cases.
<Handling procedure>
"Grip the dongle"->"Insert the dongle into the TV"->"Launch the mobile phone app"
<Estimated action target>
"Grip the dongle"->"Insert the dongle into the TV" (★ Continue for 10 seconds or more)

(S123) If an action target different from the time series of the service scenario based on S121 is an action target that is not included in the entire service scenario, the action result is not determined. That is, as in the example (s121b), it is assumed that the user is taking a different action depending on the surrounding environment, and the action result is not determined.
On the other hand, as in the example (s121a), when the action target included in the service scenario and the order of the handling procedures is misunderstood, it is regarded as the "first improper action".

FIG. 6 is an explanatory diagram showing a processing flow of a specific example in the scenario action determination unit.

(S1) First, it is confirmed that a predetermined peripheral object in the selected service scenario 1 is reflected in the image.
(S2) The first action target “grasping the dongle” in service scenario 1 was estimated.
(S3) The action target "grasping the dongle" was completed in less than 5 seconds.
(S4) Here, the action target “grasping the dongle” is determined to be “appropriate action”.
(S5) Next, the action target “incoming call” that is not in the service scenario 1 was estimated.
(S6) Here, the improper behavior is not determined.
(S7) Next, the action target “grasping the dongle” in the service scenario 1 was estimated again.
(S8) The action target "grasping the dongle" was completed in less than 5 seconds.
(S9) Here, the action target “grasping the dongle” is determined to be “appropriate action”.
(S10) Although it is in the service scenario 1, it is estimated that the action target “launching the mobile phone application” is different from the time series.
(S11) Here, it is determined that the action target "launching the mobile phone application" is "first improper action".
(S12) The action instruction media data is transmitted to the terminal equipped with the camera of the image, and the user is notified that the action is "inappropriate action".
(S13) Next, the action target “termination of the mobile phone application” that is not in the service scenario 1 was estimated.
(S14) Here, the improper behavior is not determined.
(S15) Next, the next action target “insertion of the dongle into the television” in the service scenario 1 was estimated.
(S16) The action target "insertion of the dongle into the television" was estimated for 10 seconds or more.
(S17) Here, the action target "insertion of the dongle into the television" is determined to be "second improper action".
(S18) Next, the next action target “launching the mobile phone application” in the service scenario 1 was estimated.
(S19) The action target "launching the mobile phone application" was completed in less than 15 seconds.
(S20) Here, it is determined that the action target "launching the mobile phone application" is "appropriate action".
(S21) Next, the next action target "pressing the button on the TV" in the service scenario 1 was estimated.
(S22) The action target "pressing the button on the TV" was completed in less than 20 seconds.
(S23) Here, the action target "pressing the button on the TV" is determined to be "appropriate action".

[User behavior storage unit 103]
The user action storage unit 103 stores the action result for each user.

FIG. 7 is a table stored by the user behavior storage unit. According to FIG. 7, the action result of each user is stored for each group.

[Behavior factor estimation unit 13]
The behavior factor estimation unit 13 estimates that the service scenario itself has an improvement factor in the following cases by using the behavior results of a plurality of users for the same service scenario.
(S13a) When the number or ratio of users of the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio (s13b) The number or ratio of users of the second improper behavior is the second. When the number is equal to or greater than the predetermined number or the second predetermined ratio Here, the behavior factor estimation unit 13 estimates that the improper behavior based on the "user's individuality" has no improvement factor in the service scenario itself. There is. Further, it is considered that the improper behavior based on the "individuality of the user" is such that the number or ratio of such users is extremely small among the plurality of users.

It is preferable that the behavior factor estimation unit 13 estimates whether or not there is an improvement factor in the service scenario itself for each group. It is not necessary to improve the service scenario in a group that can act according to the service scenario such as a young man, but it is also necessary to improve the service scenario in a group that is difficult to act according to the service scenario such as an elderly woman.

<Inappropriate behavior based on "user individuality / lack of skills">
Inappropriate behavior based on "user individuality / lack of skills" is counted as a minority in the total number of users. Such minority behavior will be ignored as it does not improve the service scenario itself.
On the other hand, according to the present invention, since the user group is classified into groups (gender, age group) according to the user attributes, it is possible to determine the suitability of the improvement factor of the service scenario for each group. .. For example, even "insufficient" users are likely to be in the majority in service scenarios for "old women" groups. If the majority behaves improperly, there may be improvement factors in the service scenario for the group.
After all, improper behavior based on "user individuality / lack of skills" of users who are classified into groups according to user attributes and are a minority within that group should not be improved in the service scenario itself, but rather ignored. It is considered that it should be done.

As a service provider, for example, it is possible to know the percentage of users who cannot perform such an operation for a product advertised as "the operation time can be set in 3 seconds!". In particular, in the monitor test, it is possible to grasp the behavior that the user is confused and the time that the user is frustrated (for example, about 10 seconds). This can be understood as an improvement factor for service scenarios.

[Action instruction transmission unit 15]
It is assumed that the terminal 2 has not only a camera but also a speaker and / or a display capable of instructing the user to act.
Here, the action instruction transmission unit 15 transmits the action instruction media data based on the service scenario to the terminal 2 when the scenario action determination unit 12 determines the first improper action or the second improper action. Send.
Action instruction media data is based on audio, video and text. The action instruction media data may be used to explain to the user the procedure for handling the service scenario again, for example, when the user takes an improper action.
Then, the terminal 2 reproduces the action instruction media data by the speaker and / or the display. The user can redo his / her handling procedure by the action instruction.

As described in detail above, according to the behavior analyzer, program and method of the present invention, it is possible to estimate the improvement factor of the service scenario from the behavior target of the user without requiring the user to wear the device or perform a specific operation. Can be done.
In particular, the service scenario can be improved according to the user attributes.
Further, the service provider can estimate the improvement factor of the service scenario without imposing any burden on the user even in the monitor test in the user's home, for example.

With respect to the various embodiments of the present invention described above, various changes, modifications and omissions in the technical idea and scope of the present invention can be easily made by those skilled in the art. The above description is merely an example and is not intended to be any limitation. The present invention is limited only to the scope of claims and their equivalents.

1 Behavior analyzer 101 Service scenario registration unit 102 User group storage unit 103 User behavior storage unit 11 Action target estimation unit 12 Scenario behavior judgment unit 13 Action factor estimation unit 14 Service selection unit 15 Action instruction transmission unit 2 Terminal

Claims (10)

A behavior analysis device that analyzes user behavior from images taken by a camera.
A service scenario registration method for registering service scenarios (procedures) in which each action target associated with action time is arranged in chronological order, and
An action target estimation means that sequentially estimates the user's action target from the video,
For the service scenario, the presumed action target is the first improper action in which the action target is different from the time series, the second improper action that exceeds the action time, or other appropriate actions. Scenario behavior judgment means to judge whether
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second non-compliance, using the behavior results of a plurality of users. Behavior analysis characterized by having a behavior factor estimation means for presuming that the service scenario itself has an improvement factor when the number or percentage of users of appropriate behavior is equal to or greater than the second predetermined number or the second predetermined percentage. apparatus. The action target estimation means further detects peripheral objects reflected in the image.
The service scenario registration means registers peripheral objects reflected in the video for each service scenario.
The behavior analysis device according to claim 1, wherein the scenario behavior determination means does not determine the behavior result when the peripheral object registered in the service scenario is not reflected in the video. The service scenario registration means has registered a plurality of service scenarios.
The behavior analysis device according to claim 1 or 2, further comprising a service selection means for selecting one of the service scenarios based on the user's behavior target estimated from the video. The action target estimation means has detected a peripheral object reflected in the image.
The service scenario registration means registers peripheral objects reflected in the video for each service.
The behavior analysis device according to claim 3, wherein the service selection means further narrows down and selects one of the service scenarios from the peripheral objects reflected in the video. It further has a user group storage means for grouping and storing a plurality of users to which user attribute information is given based on predetermined conditions.
The behavior analysis device according to any one of claims 1 to 4, wherein the behavior factor estimating means estimates whether or not there is an improvement factor in the service scenario itself for each group. Any of claims 1 to 5, wherein the scenario action determining means does not determine an action result when an action target different from the time series of the service scenario is an action target not included in the service scenario. The behavior analyzer according to item 1. Furthermore, the user's utterance voice is collected from the terminal that recorded the user's utterance voice with the microphone.
The behavior analysis device according to any one of claims 1 to 6, wherein the behavior target estimation means sequentially estimates a user's behavior target by further using the spoken voice. The behavior analyzer according to any one of claims 1 to 7.
A system in which a terminal having a speaker and / or a display capable of instructing an action to a user communicates via a network.
When the behavior analyzer determines the first improper behavior or the second improper behavior by the scenario behavior determining means, the behavior analysis device transmits the behavior instruction media data based on the service scenario to the terminal. Has more means of transmission
The terminal is a system characterized in that the behavior instruction media data received from the behavior analyzer is reproduced by a speaker and / or a display. A behavior analysis program that activates a computer mounted on a device that analyzes user behavior from images taken by a camera.
A service scenario registration method for registering service scenarios (procedures) in which each action target associated with action time is arranged in chronological order, and
An action target estimation means that sequentially estimates the user's action target from the video,
For the service scenario, the presumed action target is the first improper action in which the action target is different from the time series, the second improper action that exceeds the action time, or other appropriate actions. Scenario behavior judgment means to judge whether
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second non-compliance, using the behavior results of a plurality of users. When the number or ratio of appropriate behavior users is equal to or greater than the second predetermined number or the second predetermined ratio, the computer is characterized by functioning as a behavior factor estimation means for presuming that the service scenario itself has an improvement factor. Behavior analysis program. It is a behavior analysis method of a device that analyzes user behavior from images taken by a camera.
The device has a service scenario registration unit that registers service scenarios (procedures) in which action targets associated with action times are arranged in chronological order.
The device is
The first step of sequentially estimating the user's action target from the video,
For the service scenario, the estimated action target is the first improper action in which the action target is different from the time series, the second improper action that exceeds the action time, or other appropriate actions. The second step to determine whether
For the same service scenario, when the number or ratio of users with the first improper behavior is equal to or greater than the first predetermined number or the first predetermined ratio, or the second failure, using the behavior results of a plurality of users. A device characterized in that when the number or ratio of users with proper behavior is equal to or greater than the second predetermined number or the second predetermined ratio, a third step of presuming that the service scenario itself has an improvement factor is executed. Behavior analysis method.

Images