JP7607272B2 - Learning support device and learning support system - Google Patents

Description

The present invention relates to a learning support device and a learning support system.

A device has been devised that measures the level of concentration of a user while performing a task. Patent Document 1 discloses a video playback device that measures the level of concentration of a user for tasks that the user actively performs, such as composing an email or browsing the web, and tasks that the user passively performs, such as watching a video, and plays video according to the measured level of concentration.

International Publication No. 2007/132566

However, the video playback device disclosed in Patent Document 1 was unable to appropriately switch between tasks that the user actively performed and tasks that the user passively performed.

Therefore, the present invention provides a learning support device, etc., that can appropriately switch between tasks that the user actively performs and tasks that the user passively performs in accordance with the user's level of concentration.

A learning support device according to one embodiment of the present invention is a learning support device for a user to perform a learning task, and comprises a first concentration level estimation unit that analyzes information from an imaging means that images the user and estimates a first concentration level of the user, a second concentration level estimation unit that analyzes information actively input by the user when the user performs the learning task and estimates a second concentration level of the user, a presentation switching unit that switches the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level, and a first learning task presentation unit that presents to the user a first learning task that the user actively learns, and while the first learning task presentation unit is presenting the first learning task to the user, the presentation switching unit switches the content presented to the user to the first learning task of different difficulty depending on the relative magnitude relationship between the first concentration level and the second concentration level .

A learning support system according to one embodiment of the present invention is a learning support system for a user to perform a learning task, and comprises a display, an imaging means for imaging the user, a first concentration level estimation unit for estimating a first concentration level of the user by analyzing information from the imaging means, a second concentration level estimation unit for estimating a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task, a presentation switching unit for switching the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level, and a first learning task presentation unit for presenting to the user a first learning task that the user actively learns, and while the first learning task presentation unit is presenting the first learning task to the user, the presentation switching unit switches the content presented to the user to the first learning task of different difficulty depending on the relative magnitude relationship between the first concentration level and the second concentration level .

A learning support device etc. relating to one aspect of the present invention can appropriately switch between tasks that are actively performed by the user and tasks that are passively performed by the user in accordance with the user's level of concentration.

FIG. 1 is a block diagram of a learning support device according to an embodiment. FIG. 2 is a flowchart showing the process of the learning support device according to the embodiment. FIG. 3A is a diagram illustrating a user performing an active task. FIG. 3B is a diagram illustrating a user performing a passive task. FIG. 4A is a diagram showing how the concentration level of a user is being measured while performing an active task. FIG. 4B is a diagram showing how the concentration level of the user is being measured while the user is performing a passive task. FIG. 5 is a flowchart showing a process of determining a first concentration level performed by the learning support device according to the embodiment. FIG. 6 is a diagram showing an example of a subject's habits that the learning support device according to the embodiment uses to determine the first concentration level. FIG. 7 is a diagram showing time slots for comparing the first and second concentration degrees performed by the concentration degree determination unit according to the embodiment. FIG. 8 is a diagram showing an outline of measurement of the first concentration level in the learning support device according to the embodiment. FIG. 9 is a diagram showing an outline of measurement of the second concentration level in the learning support device according to the embodiment. FIG. 10 is a diagram showing switching between an active task and a passive task in the learning support device according to the embodiment. FIG. 11 is a table showing details of switching between active tasks and passive tasks in the learning support device according to the embodiment. FIG. 12 is a flowchart illustrating an example of a process performed by the learning support device according to the embodiment. FIG. 13 is a flowchart illustrating another example of the process of the learning support device according to the embodiment. FIG. 14 is a diagram illustrating an example of a state determination of a user by comparing a first concentration level with a second concentration level in the learning support device according to the embodiment. FIG. 15 is a diagram showing guidance of a user's state based on a comparison between a first concentration level and a second concentration level in the learning support device according to the embodiment.

The following describes the embodiments with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, etc. shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim are described as optional components.

Note that each figure is a schematic diagram and is not necessarily a precise illustration. In addition, in each figure, the same reference numerals are used for substantially the same configurations, and duplicate explanations may be omitted or simplified.

(Embodiment)
[Configuration of the learning support device]
First, the configuration of the learning support device 100 will be described. Fig. 1 is a block diagram of the learning support device 100 according to an embodiment. The learning support device 100 includes an image capture unit 10, a body movement/pose determination unit 12, a gaze/facial expression determination unit 14, a first concentration level estimation unit 16, a concentration level determination unit 18, an answer input unit 20, a first learning task presentation unit 22, an information processing unit 24, a second concentration level estimation unit 26, a second learning task presentation unit 28, and a presentation switching unit 30.

The imaging means 10 captures an image of the user's face or body. The imaging means 10 is realized by a web camera built into a personal computer, or a digital camera connectable to a personal computer. The imaging means 10 also has an eye tracking function. The imaging means 10 may also be realized by an infrared camera. The imaging means 10 transmits the acquired image data to the body movement/pose determination unit 12 and the gaze/facial expression determination unit 14.

The body movement/pose determination unit 12 recognizes the position of each of two or more parts of the user's body in the image acquired by the image capture means 10. The body movement/pose determination unit 12 is also a processing device that calculates a target positional relationship, which is the positional relationship between each of the recognized positions of the two or more parts of the user's body. The body movement/pose determination unit 12 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

The body movement/pose determination unit 12 identifies the user's body from other parts in the image received from the image capture means 10 by image recognition. The body movement/pose determination unit 12 also identifies each part of the identified user's body and recognizes the position of each part in the image. This calculates a target positional relationship, which is the positional relationship between two or more parts of the user's body in the image. Here, the positional relationship between two or more parts is indicated by the distance between the two or more parts. For example, if the two or more parts are "a part of the user's face" and "the user's hand", the body movement/pose determination unit 12 calculates the target positional relationship such as "a part of the face and the hand are within a specific distance". The body movement/pose determination unit 12 transmits the calculated target positional relationship to the first concentration degree estimation unit 16.

A plurality of images are acquired, and the object positional relationship is calculated for each of them. More specifically, the images acquired by the imaging means 10 are moving images in which the images are arranged continuously in a chronological order. Therefore, the body movement/pose determination unit 12 determines whether the user is in a state of concentration for each frame of the image included in the moving image. In other words, the body movement/pose determination unit 12 outputs, based on the determination, a sequence of numbers in which values indicating either a state of concentration or a non-concentrated state that is not a state of concentration, corresponding to the moving image captured by the user, are arranged continuously in a chronological order.

The gaze/facial expression determination unit 14 identifies the gaze or facial expression of the user on the image received from the image capture unit 10 by image recognition. The gaze/facial expression determination unit 14 acquires images from a near-infrared LED (Light Emitting Diode) and the image capture unit 10, and performs calculation processing including image detection, a 3D eye model, and a gaze calculation algorithm. The gaze/facial expression determination unit 14 detects the gaze of the user looking at a display or the like. Specifically, the near-infrared LED generates a light reflection pattern on the user's cornea, and the image capture unit 10 acquires the reflection pattern. Then, the gaze/facial expression determination unit 14 estimates the position of the eyeball in space and the viewpoint using an image processing algorithm and a physiological 3D model of the eyeball based on the reflection pattern. Note that the gaze/facial expression determination unit 14 can also be configured using natural light illumination and a visible light color camera, and the above configuration is just one example.

Furthermore, the gaze/facial expression determination unit 14 learns the user's face, etc., by deep learning or the like, extracts features of the captured facial image of the user, and determines the user's facial expression based on the learned data and the extracted features. The gaze/facial expression determination unit 14 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The gaze/facial expression determination unit 14 transmits information related to the estimated user's viewpoint or information related to the determined user's facial expression to the first concentration level estimation unit 16.

The first concentration level estimation unit 16 is a processing device that determines whether the user is in a state of concentration based on the target positional relationship and the user's facial expression. The first concentration level estimation unit 16 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

The first concentration level estimation unit 16 estimates the first concentration level of the user based on the target positional relationship acquired from the body movement/pose determination unit 12. The first concentration level estimation unit 16 knows the user's habits in advance and determines whether the target positional relationship calculated by the body movement/pose determination unit 12 is the user's habit. If the target positional relationship matches the user's habit, it can be determined that the user is performing an action that can be taken when in a state of concentration. In other words, the first concentration level estimation unit 16 can determine that the user's first concentration level is high because the user is performing the relevant action. Here, the first concentration level is the concentration level when the user performs a task that is passively performed by the user (hereinafter referred to as a passive task). A passive task is, for example, watching a video.

In this specification, a habit is a movement that a person may take when he or she is in a state of concentration, and is a movement that is estimated from the positional relationship (i.e., distance) of two or more parts of the person's body. Therefore, a habit can be defined as the positional relationship of two or more parts of the person's body, or a movement that is estimated from the positional relationship.

The first concentration level estimation unit 16 calculates the ratio of the time during which the user was in a concentrated state to the total measured time for a preset time range, as the concentration level of user 1, using the target positional relationship output by the body movement/pose determination unit 12. For example, if the total time during which user 1 was in a concentrated state was 4 minutes out of a 5-minute video, 4/5 = 0.8 is calculated as the concentration level. Alternatively, for example, the first concentration level estimation unit 16 may calculate the concentration level as 0.8 x 100 = 80% using a percentage.

The first concentration level estimation unit 16 estimates the first concentration level of the user based on information on the user's viewpoint estimated by the gaze/facial expression determination unit 14 or information on the determined facial expression of the user. For example, the first concentration level estimation unit 16 determines that the first concentration level of the user is high when the gaze/facial expression determination unit 14 estimates that the user's viewpoint moves little over time in space. For example, the first concentration level estimation unit 16 may predetermine a facial expression when the concentration level is high, and determine that the first concentration level of the user is high when the gaze/facial expression determination unit 14 determines that the facial expression is the determined facial expression. The first concentration level estimation unit 18 outputs the calculated first concentration level of the user to the concentration level determination unit 18.

The answer input unit 20 is an interface such as a terminal where the user inputs an answer or a screen for answer input that is presented to the user. The user inputs an answer to the question presented by the first learning task presentation unit 22 into the answer input unit 20. The answer input unit 20 transmits the acquired answer to the information processing unit 24. The answer input unit 20 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The answer input unit 20 may include a display such as a touch panel display or a liquid crystal display, and an input button or a keyboard.

The first learning task presentation unit 22 is an interface such as a terminal or a screen that presents to the user a first learning task that the user actively learns. The first learning task presentation unit 22 presents to the user a first learning task that the user actively learns, such as a problem for intellectual training, such as a calculation problem, a problem related to knowledge of kanji, a problem related to English vocabulary, etc., and a problem that requires the user to input an answer. The first learning task is also called an active task. The first learning task presentation unit 22 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The answer input unit 20 may be equipped with a display such as a touch panel display or a liquid crystal display. The first learning task presentation unit 22 transmits information to the answer input unit 20 about what kind of problem the first learning task presentation unit 22 is presenting. In addition, the first learning task presentation unit 22 presents a problem based on a signal from the presentation switching unit 30.

The information processing unit 24 acquires the answer entered by the user from the answer input unit 20, and calculates indicators related to the question presented to the user, such as whether the answer is correct or incorrect, the progress speed of the question, the amount of processing for the question, the answer score, and the correct answer rate. The information processing unit 24 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

The second concentration level estimation unit 26 acquires an index related to the problem presented to the user from the information processing unit 24, and estimates the second concentration level of the user from the index. Here, the second concentration level is the concentration level when the user performs a task that is actively performed (hereinafter referred to as an active task). An actively performed task is, for example, answering a question that is presented. For example, the second concentration level estimation unit 26 estimates that the second concentration level of the user is high when the user's correct answer rate for the problem is high. Alternatively, for example, the second concentration level of the user may be estimated to be high when the user's progress speed in the problem is fast. The second concentration level estimation unit 26 outputs the calculated second concentration level of the user to the concentration level determination unit 18. The second concentration level estimation unit 26 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

The concentration level determination unit 18 determines the user's concentration level using the first concentration level or the second concentration level acquired from the first concentration level estimation unit 16 or the second concentration level estimation unit 26. Specifically, when the learning support device 100 presents a question to the user, the concentration level determination unit 18 acquires the first concentration level and the second concentration level from the first concentration level estimation unit 16 and the second concentration level estimation unit 26, and determines the user's concentration level by normalizing and comparing the first concentration level and the second concentration level.

Furthermore, when the learning support device 100 is presenting a video to the user, the concentration level determination unit 18 determines the user's concentration level by comparing the first concentration level obtained from the first concentration level estimation unit 16 with the first value. The concentration level determination unit 18 outputs information related to the determined concentration level of the user to the presentation switching unit 30. The concentration level determination unit 18 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

The presentation switching unit 30 switches between presenting a video or a question on the display based on at least one of the first and second concentration levels. Based on information about the user's concentration level acquired from the concentration level determination unit 18, it determines how to switch the content presented to the user. For example, when the learning support device 100 is presenting a video to the user and the first concentration level is higher than the second concentration level, the presentation switching unit 30 determines to switch the content presented to the user to a video with a lower level of difficulty than the currently presented video. The presentation switching unit 30 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The presentation switching unit 30 acquires a signal from the concentration level determination unit 18 and transmits a signal regarding the switching of the content presented to the user to the first learning task presentation unit 22 and the second learning task presentation unit 28.

The second learning task presentation unit 28 is an interface such as a terminal or a screen that presents to the user a second learning task that the user passively learns. The second learning task is, for example, a video. The second learning task is also called a passive task. The second learning task presentation unit 28 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The second learning task presentation unit 28 may be equipped with a display such as a touch panel display or a liquid crystal display. The second learning task presentation unit 28 presents the video based on a signal from the presentation switching unit 30.

[Processing of learning support device]
Next, a description will be given of the processing performed by the learning support device 100. Fig. 2 is a flowchart showing the processing performed by the learning support device 100 according to the embodiment.

First, the learning support device 100 presents a video or a question to the user (step S100). The learning support device 100 presents a video using the second learning task presentation unit 28, or presents a question using the first learning task presentation unit 22.

Next, the learning support device 100 estimates the first concentration level or the second concentration level (step S101). The learning support device 100 estimates the first concentration level using the first concentration level estimation unit 16, or estimates the second concentration level using the second concentration level estimation unit 26.

Next, the learning support device 100 compares the first concentration level with the second concentration level, or determines the value of the first concentration level (step S102). The learning support device 100 uses the concentration level determination unit 18 to compare the magnitude of the first concentration level with the second concentration level, or to determine the magnitude of the first concentration level with the first value.

Then, the learning support device 100 switches the content presented to the user depending on the comparison result between the first concentration level and the second concentration level, or the determination result of the value of the first concentration level (step S103). The learning support device 100 determines how to switch the content presented to the user using the presentation switching unit 30, and transmits the determined switching method to the first learning task presentation unit 22 or the second learning task presentation unit 28. The contents described in Figures 1 and 2 will be described in detail below.

[Active and Passive Tasks]
Next, the active task and the passive task presented by the learning support device 100 will be described in detail. The active task and the passive task are videos or questions presented in step S100 shown in Fig. 2. Here, the video (passive task) or the question (active task) presented in step S100 shown in Fig. 2 will be described. Fig. 3A is a diagram showing a state in which the user 1 is performing an active task. Also, Fig. 3B is a diagram showing a state in which the user 1 is performing a passive task.

The active task shown in FIG. 3A refers to a task in which user 1 actively inputs answers, etc.Specifically, active tasks are calculation problems, problems related to kanji, problems related to English vocabulary, other problems that require knowledge answers, geometric problems, problems to understand sentences, etc.The passive task shown in FIG. 3B refers to a task in which user 1 passively watches videos, etc.Specifically, passive tasks refer to watching videos of lessons in arithmetic, Japanese, English, science, social studies, etc., watching musical performances, watching paintings or visual art works, watching plays, watching videos with educational content, etc.

Figure 4A is a diagram showing how the concentration level of user 1 is measured while performing an active task. When the learning support device 100 presents the active task to user 1, user 1 looks at the problem displayed on the display 2 or the like, and inputs the answer to the learning support device 100 via a keyboard, touch panel, or the like. The learning support device 100 estimates the second concentration level of user 1 from the work information. Here, the work information is the touch rate of the touch panel display when user 1 answers the problem, the correct answer rate for the problem, the response time until the answer is input, the progress speed for the problem, the amount of processing for the problem, the answer score, etc. User 1 may also input the answer to the active task by voice via a microphone or the like.

In addition, while presenting the active task to user 1, learning support device 100 acquires an image of user 1's face or body from imaging means 10. Learning support device 100 analyzes the acquired image and estimates user 1's first concentration level. Specifically, learning support device 100 determines user 1's facial expression, information related to user 1's line of sight or viewpoint, target positional relationships indicating user 1's pose, etc., and estimates the first concentration level.

Figure 4B is a diagram showing how the concentration level of user 1 is measured while performing a passive task. When the learning support device 100 presents the passive task to user 1, user 1 watches a moving image displayed on the display 2 or the like. The learning support device 100 analyzes the image acquired by the image capturing means 10 and estimates the first concentration level of user 1 from the face image of user 1, information regarding the line of sight or viewpoint, the target positional relationship indicating the pose of user 1, or physiological indicators such as body temperature. The learning support device 100 may perform a more accurate analysis during the passive task than during the active task. The learning support device 100 may acquire physiological indicators of user 1, such as pulse or body temperature, from a wearable device or a smartphone.

[First Estimation of Concentration Degree]
Next, a process of estimating the first concentration level performed by the learning support device 100 will be described. Fig. 5 is a flowchart showing a process of determining the first concentration level performed by the learning support device 100 according to the embodiment. Here, the estimation of the first concentration level performed in step S101 shown in Fig. 2 will be described.

In this embodiment, the image capturing means 10 receives an image captured by the user 1, thereby acquiring the image, and performs an acquisition step (S101). The image capturing means 10 also transmits the acquired image to the body movement/pose determination unit 12.

Next, the body movement/pose determination unit 12 uses image recognition to distinguish between the body of user 1 and other parts of the image received from the image capture means 10, and further identifies each part of user 1's body. The body movement/pose determination unit 12 recognizes the position on the image for each part of user 1's body. The body movement/pose determination unit 12 further performs a recognition step (S102) in which, for combinations of two or more parts of user 1's body on the image, the body movement/pose determination unit 12 calculates a target positional relationship, which is the positional relationship between each combination, from the recognized positions.

Next, the first concentration level estimation unit 16 performs a determination step of determining whether or not the user 1 is in a state of concentration based on the target positional relationship in the acquired image and the positional relationship of two or more parts of the body that define the habit of the user 1. The first concentration level estimation unit 16 acquires the positional relationship of two or more parts of the body that define the habit of the user 1, for example, by using habit information of the user 1 stored in a memory unit. The first concentration level estimation unit 16 further performs a determination of whether or not the user 1 is in a state of concentration by determining whether or not the positional relationship that corresponds to the habit of the user 1 is included in the target positional relationship calculated from the image (S103).

For example, if the positional relationship corresponding to the habit of user 1 corresponds (matches or can be considered equivalent to) the target positional relationship calculated from the image (Yes in S103), the first concentration level estimation unit 16 determines that user 1 is in a state of concentration (S104). Also, for example, if the positional relationship corresponding to the habit of user 1 does not correspond to the target positional relationship calculated from the image (No in S103), the first concentration level estimation unit 16 determines that user 1 is not in a state of concentration (S105).

Next, the first concentration level estimation unit 16 judges whether or not the user 1 is in a state of concentration for each frame of image captured by the image capture means 10. Here, the learning support device 100 in this embodiment calculates the concentration level of the user 1 in a pre-set time range. In other words, the first concentration level estimation unit 16 judges whether or not the user 1 is in a state of concentration for a predetermined number (number of frames) of images corresponding to a pre-set time range.

The first concentration level estimation unit 16 determines whether the number of images for which a judgment has been performed has reached a predetermined number (S106), and if the predetermined number has not been reached (No in S106), repeats the acquisition step (S101) to step (S106). As a result, the first concentration level estimation unit 16 acquires images and determines whether user 1 in the images is in a state of concentration until the number of images for which a judgment has been performed reaches the predetermined number.

When the number of images for which the judgment has been performed reaches a predetermined number (Yes in S106), the first concentration level estimation unit 16 performs a calculation step (S107) of calculating the concentration level of the user 1 using the judgment result of whether the user 1 is in a concentration state or not, which is judged for the predetermined number of images. This allows the learning support device 100 to quantify the degree to which the user 1 was concentrated within a preset time range. The first concentration level estimation unit 16 transmits information regarding the calculated concentration level of the user 1 to an output unit (not shown) . This allows the user 1 or an administrator or the like who manages the user 1 to check the concentration level measured by the learning support device 100.

The determination of whether user 1 is in a state of concentration in the first concentration level estimation unit 16 will be explained in more detail with reference to FIG. 6. FIG. 6 is a diagram showing an example of a subject's habit used by the learning support device 100 according to the embodiment to determine the first concentration level. (a) of FIG. 6 is a diagram showing an image of user 1 captured when in a state of concentration. (b) of FIG. 6 is a diagram showing an image of user 1 captured when not in a state of concentration. Note that the habit of user 1 when concentrating in this case is the action of touching the chin (i.e. part of the face) with the hand.

As shown in (a) of Fig. 6, the body part recognition unit recognizes the positions of the chin, which is one part of the body, and the hands, which are other parts, of the user 1 on the image as coordinates on the image. Furthermore, based on the action of touching the chin with the hand, which is a habit of the user 1 when concentrating, the concentration habit determination unit determines whether the shortest distance between the chin and the hand is 0 or within a distance that can be considered equivalent to 0. In (a) of Fig. 6, the shortest distance between the chin and the hand on the image coordinates of the user 1 is 0. Therefore, the user 1 in (a) of Fig. 6 is determined to be in a state of concentration, which indicates a habit when concentrating.

On the other hand, as shown in ( d ) of Fig. 6, the body movement/pose determination unit 12 recognizes the position of only the chin, which is one part of the body of the user 1 in the image, as the coordinates on the image. Because the hand is not recognized in the image, the distance between the chin and the hand is not calculated, and the action of touching the chin, which is a habit of the user 1 when concentrating, is not shown. Therefore, it is determined that the user 1 in ( d ) of Fig. 6 is not in a state of concentration, which is a habit when concentrating.

[Estimation of the second concentration level]
Next, the estimation of the second concentration level performed by the learning support device 100 will be described. Here, the estimation of the second concentration level performed in step S101 shown in FIG. 2 will be described. The second concentration level is represented as a ratio of the time during which the user 1 was concentrating to the time during which the task was performed. The time during which the user 1 was concentrating is calculated by multiplying the expected value of the response time by the total number of responses. The response time is expressed by a mixed normal distribution. Specifically, it is represented by the following equations (1) to (5).

f(t) represents the distribution of response time. f _l and f _h are log-normal distributions used for the mixed normal distribution. f _l is defined by μ _l and σ _l . f _h is defined by μ _h and σ _h . Parameter p is a mixing coefficient. CT is the concentration time (the time during which user 1 was concentrated), and N is the total number of responses. CTR is the concentration time ratio (the ratio of the time during which user 1 was concentrated to the time during which the task was performed), and T _total is the total task execution time (the total amount of time during which the target task was performed). Here, the second concentration level is defined for the entire time during which the task was performed, but may be defined for a shorter time period (time slot). In that case, the second concentration level is also a value indicating temporal fluctuations, similar to the first concentration level.

[Determining User 1's Status]
Next, the determination of the state of the user 1 by the concentration level determination unit 18 will be described. Here, the process performed in step S102 shown in Fig. 2 will be described. Fig. 7 is a diagram showing time slots for comparing the first concentration level and the second concentration level performed by the concentration level determination unit 18 according to the embodiment.

The concentration level determination unit 18 determines the state of user 1 by comparing the first and second concentration levels of user 1. The concentration level determination unit 18 does not determine the state of user 1 continuously, but intermittently. Specifically, for each period during which the learning support device 100 is performing an active task or a passive task, the concentration level determination unit 18 calculates an average value of the first or second concentration level of user 1 estimated from data acquired during that period, and uses the average value to determine the state of user 1 during that period.

Alternatively, the concentration level determination unit 18 may estimate a first concentration level or a second concentration level of user 1 from data acquired during a time period when the learning support device 100 is performing an active task or a passive task, determine the concentration level of user 1 from the estimated first concentration level or the second concentration level, and use an average value of the multiple concentration levels of user 1 determined during that time period as a representative value of the concentration level of user 1 during that time period. For example, the concentration level determination unit 18 may determine the state of user 1 by comparing the magnitude of the first concentration level with that of the second concentration level.

In addition, the concentration determination unit 18 may use the median instead of the average value when performing the above process. The time during which the learning support device 100 performs one active task or passive task is, specifically, for example, 30 minutes.

In addition, the data indicating the first concentration level and the data indicating the second concentration level are normalized so that the first concentration level and the second concentration level can be compared. Any normalization method may be used.

For example, as shown in FIG. 7, the learning support device 100 first performs an active task. The execution time is, for example, 30 minutes. The concentration level determination unit 18 then determines the state of the user 1 during the execution time from the representative value of the first concentration level and the representative value of the second concentration level estimated during the execution time. Next, the learning support device 100 performs a passive task. The execution time is, for example, 30 minutes. The concentration level determination unit 18 then determines the concentration level of the user 1 during the execution time from the representative value of the first concentration level estimated during the execution time. Next, the learning support device 100 performs an active task. The execution time is, for example, 30 minutes. The concentration level determination unit 18 then determines the state of the user 1 during the execution time from the representative value of the first concentration level and the representative value of the second concentration level estimated during the execution time. In this way, the concentration level determination unit 18 does not continuously and sequentially determine the concentration level of the user 1, but determines the state of the user 1 in a predetermined period from the average value of the first concentration level or the second concentration level in the predetermined period. Note that the predetermined period may be the entire implementation time of 30 minutes, or may be a short time period (time slot) of 1 minute, 3 minutes, etc.

Next, we will explain in detail how the learning support device 100 measures the concentration level. Figure 8 is a diagram showing an overview of the measurement of the first concentration level in the learning support device 100 according to the embodiment.

First, consider the case where user 1 is watching a video (passive task). The time that user 1 watches the video is, for example, 30 minutes. While user 1 is watching the video, the learning support device 100 acquires image data of user 1 from the imaging means 10. The images acquired by the imaging means 10 are analyzed by the body movement/pose determination unit 12 and the gaze/facial expression determination unit 14.

For example, in the example shown in Fig. 10, analysis by the body movement/pose determination unit 12 and the gaze/facial expression determination unit 14 reveals that immediately after user 1 starts watching a video, user 1 is seen taking notes with a serious expression. Next, user 1 is seen yawning. Then, user 1 is seen to have a cheerful expression, and finally, user 1 is seen to have a tired expression with his chin resting on his hand.

8 shows a graph of the concentration level of user 1 estimated by first concentration level estimation unit 16 based on these poses and facial expressions. For example, in a time period in which user 1 is seen taking notes with a serious expression, the first concentration level is estimated to be relatively high, and in a time period in which user 1 is seen yawning, the first concentration level is estimated to be relatively low. In addition, in a time period in which user 1 is seen to have a cheerful expression, the first concentration level is estimated to be higher than the immediately preceding time period, and in the last time period in which user 1 is seen to have a tired expression with his chin resting on his hand, the first concentration level is estimated to be relatively low.

The concentration level determination unit 18 determines the state of user 1 based on a representative value, such as the average value of the first concentration level for the 30 minutes shown above.

Figure 9 is a diagram showing an overview of the measurement of the second concentration level in the learning support device 100 according to the embodiment. Next, consider the case where the user 1 is answering the problem (active task). The time for the user 1 to answer the problem is, for example, 30 minutes. While the user 1 is answering the problem, the learning support device 100 acquires image data of the user 1 from the imaging means 10. The image acquired by the imaging means 10 is analyzed by the body movement/pose determination unit 12 and the gaze/facial expression determination unit 14. In addition, the information processing unit 24 acquires, as work information, the touch rate on the touch panel display when the user 1 answers the problem, the correct answer rate of the problem, the response time until the answer input, the progress speed of the problem, the processing amount of the problem, the answer score, etc. Then, the second concentration level estimation unit 26 estimates the second concentration level of the user 1 based on the work information.

For example, the second concentration level estimation unit 26 estimates that the second concentration level is high when the rate of correct answers to questions is high. The second concentration level estimation unit 26 may also estimate that the second concentration level is high when the response time until answer input is short. As shown in FIG. 9, the second concentration level is sequentially estimated during the 30 minutes during which the active task is presented by the learning support device 100.

The concentration level determination unit 18 determines the state of user 1 based on a representative value, such as the average value of the first concentration level for the 30 minutes shown above.

[Switch between active and passive tasks]
Next, the switching between active tasks and passive tasks performed by the learning support device 100 according to the state of the user 1 will be described. Here, the process performed in step S103 shown in FIG. 2 will be specifically described. FIG. 10 is a diagram showing the switching between active tasks and passive tasks in the learning support device 100 according to the embodiment. As shown in FIG. 10, the learning support device 100 switches between a passive task of watching a lesson video and an active task of performing a quiz or exercise related to the lesson video according to the state of the user 1. Note that the learning support device 100 may switch from one passive task to a passive task having a different level of difficulty from the passive task, or may switch from one active task to an active task having a different level of difficulty from the active task, according to the state of the user 1.

Figure 11 is a table showing details of switching between active tasks and passive tasks in the learning support device 100 according to the embodiment. When the learning support device 100 presents an active task, if the second concentration level is higher than the first concentration level, the difficulty level of the active task is determined to be too low for user 1. This is because, although user 1's task performance, such as the answer rate to the questions, is high, this is not reflected in his/her facial expression. It is considered that user 1 has fully achieved learning and understanding of this task. Therefore, the presentation switching unit 30 switches to a more difficult passive task. This means that the learning support device 100 introduces user 1 to a lesson video of the next learning stage that is one step more advanced.

Also, when the learning support device 100 presents an active task, if the first concentration level is higher than the second concentration level, it is determined that the difficulty level of the active task is too high for the user 1. This is because the actual work performance, such as the answer rate to the questions, is low, even though the user 1 appears to be fully concentrated from the appearance of his/her facial expression. Alternatively, the learning support device 100 determines that the user 1 is in a so-called mind wandering state. Therefore, the presentation switching unit 30 switches to a passive task with a low level of difficulty. This means, for example, returning the assignment to the previous lesson video and having the user 1 review it again. In this case, the presentation switching unit 30 may also switch to a break. When the learning support device 100 presents an active task, if the first concentration level is higher than the second concentration level, it is determined whether the presentation switching unit 30 switches to a passive task with a low level of difficulty or switches to a break based on whether the first concentration level or the second concentration level is higher than the third value.

Furthermore, when the learning support device 100 presents an active task, the presentation switching unit 30 may switch to a passive task of different difficulty depending on the level of the second concentration level. For example, when the learning support device 100 presents an active task, if the second concentration level is higher than a first predetermined value, the presentation switching unit 30 may switch to a passive task of higher difficulty. Conversely, for example, when the learning support device 100 presents an active task, if the second concentration level is lower than a second predetermined value, the presentation switching unit 30 may switch to a passive task of lower difficulty.

Also, when the learning support device 100 presents a passive task, if the first concentration level is higher than the first value, the presentation switching unit 30 determines that the user 1 is sufficiently concentrating on watching the lesson video or the like, and switches to a more difficult active task as the next stage. Conversely, when the learning support device 100 presents a passive task, if the first concentration level is lower than the second value, the presentation switching unit 30 determines that the user 1 is not concentrating on the lesson video, and may switch to a break or to an active task that encourages the user to answer a relatively easy question. Here, the second value is assumed to be smaller than the first value. When the learning support device 100 presents a passive task, if the first concentration level is lower than the first value, the presentation switching unit 30 determines whether to switch to a break or to a less difficult active task based on whether the first concentration level is higher than a third value.

Next, an overview of the process of switching between active and passive tasks performed by the learning support device 100 will be described. FIG. 12 is a flowchart showing an example of the process of the learning support device 100 according to an embodiment. Using FIG. 12, the process of switching between active and passive tasks based on the concentration level of user 1 when the learning support device 100 presents a passive task to user 1 will be described. The process shown in FIG. 12 is a specific example of the process shown in FIG. 2.

First, the second learning task presentation unit 28 presents a video to user 1 (step S300).

Next, the first concentration level estimation unit 16 estimates the first concentration level of user 1 (step S301).

Next, the concentration determination unit 18 determines whether the first concentration level is higher than the first value (step S302).

If the concentration level determination unit 18 determines that the first concentration level is higher than the first value (Yes in step S302), the presentation switching unit 30 switches to question presentation (step S303). Specifically, the presentation switching unit 30 causes the second learning task presentation unit 28 to stop outputting the video, and causes the first learning task presentation unit 22 to output a question. Here, the presentation switching unit 30 causes the first learning task presentation unit 22 to output a question with a high level of difficulty.

If the concentration level determination unit 18 determines that the first concentration level is lower than the first value (No in step S302), the presentation switching unit 30 switches to a break (step S304). Specifically, the presentation switching unit 30 causes the second learning task presentation unit 28 to stop outputting the video, and causes the first learning task presentation unit 22 to output content that prompts the user 1 to take a break. Also, if the concentration level determination unit 18 determines that the first concentration level is lower than the first value and higher than the second value, the presentation switching unit 30 causes the first learning task presentation unit 22 to output a low-difficulty question instead of the content that prompts the user 1 to take a break.

Figure 13 is a flowchart showing another example of processing of the learning support device 100 according to an embodiment. Using Figure 13, we will explain the processing of switching between an active task and a passive task based on the concentration level of user 1 when the learning support device 100 presents an active task to user 1. The processing shown in Figure 13 is a specific example of the processing shown in Figure 2.

First, the first learning task presenter 22 presents a question to the user 1 (step S500 ).

Next, the first concentration level estimating unit 16 estimates a first concentration level of the user 1 (step S 501 ).

Next, the second concentration level estimating unit 26 estimates a second concentration level of the user 1 (step S502 ). Note that the order of steps S501 and S502 may be reversed.

Then, the concentration level determination unit 18 determines whether the second concentration level is higher than the first concentration level (step S503 ).

When the concentration level determination unit 18 determines that the second concentration level is higher than the first concentration level (Yes in step S503 ), the presentation switching unit 30 switches to presenting a video with a higher level of difficulty (step S504 ). Specifically, the presentation switching unit 30 causes the first learning task presentation unit 22 to stop outputting questions, and causes the second learning task presentation unit 28 to output a video with a higher level of difficulty.

When the concentration level determination unit 18 determines that the second concentration level is lower than the first concentration level (No in step S503 ), the presentation switching unit 30 switches to presenting a video with a low level of difficulty (step S505 ). Specifically, the presentation switching unit 30 causes the first learning task presentation unit 22 to stop outputting questions, and causes the second learning task presentation unit 28 to output a video with a low level of difficulty. Also, when the concentration level determination unit 18 determines that the second concentration level is lower than the first concentration level (No in step S503 ), the presentation switching unit 30 may switch to a break. Depending on whether the first concentration level or the second concentration level is higher than the third value, the second learning task presentation unit 28 may switch between outputting a video with a low level of difficulty or outputting content that encourages the user 1 to take a break.

[Specific examples of concentration determination and task switching]
Next, a specific description will be given of the determination of the concentration level of the user 1 by the learning support device 100 and the switching of tasks presented to the user 1 by the learning support device 100. Fig. 14 is a diagram showing an example of the state determination of the user 1 by comparing the first concentration level with the second concentration level in the learning support device 100 according to the embodiment. Fig. 15 is a diagram showing the guidance of the state of the user 1 by comparing the first concentration level with the second concentration level in the learning support device 100 according to the embodiment.

Figure 14 shows a graph plotting the concentration level of user 1 when an active task is presented to user 1 by the learning support device 100, with the first concentration level on the vertical axis and the second concentration level on the horizontal axis. Specifically, task A is plotted at a point where the first concentration level is 0.567 and the second concentration level is 0.477. Task B is plotted at a point where the first concentration level is 0.748 and the second concentration level is 0.384. The concentration level determination unit 18 interprets that in task A, the first concentration level and the second concentration level are approximately equal, and that user 1's working attitude and performance are balanced. In the state of task A, the presentation switching unit 30 switches to a passive task with a lower level of difficulty.

Furthermore, the concentration level determination unit 18 interprets that in task B, the first concentration level is greater than the second concentration level, and that user 1's working attitude is good but performance is declining. In other words, the concentration level determination unit 18 determines that user 1 is in a absent-minded state in task B. When in the state of task B, the presentation switching unit 30 switches to a break. At this time, the learning support device 100 presents to user 1 an image or music that has the effect of relaxing user 1.

Alternatively, when in the task B state, the presentation switching unit 30 may switch to an active task that is less difficult than the currently presented active task. Alternatively, when in the task B state, the presentation switching unit 30 may switch to a passive task. The passive task presented at this time is, for example, a video for reviewing the active task that was previously performed.

By switching tasks as described above, the learning support device 100 can guide user 1 to a state in which the first and second concentration levels of user 1 are balanced by either decreasing the first concentration level of user 1 when task B is performed or increasing the second concentration level of user 1 when task B is performed, as shown in FIG. 15. Thus, by switching tasks as described above, the learning support device 100 can guide user 1 to a more concentrated state.

[Effects, etc.]
The learning support device 100 of this embodiment is a learning support device for user 1 to perform a learning task, and includes a first concentration level estimation unit 16 that analyzes information from an imaging means 10 that images user 1 and estimates a first concentration level of user 1, a second concentration level estimation unit 26 that analyzes information actively input by user 1 when user 1 performs the learning task and estimates a second concentration level of user 1, and a switching unit 30 that switches the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level.

As a result, the learning support device 100 can present an appropriate learning task between a first learning task for which the user 1 actively learns and a second learning task for which the user 1 passively learns, depending on the state of the user 1 estimated from the user 1's concentration level.

For example, the learning support device 100 further includes a first learning task presentation unit 22 that presents to user 1 a first learning task that user 1 actively learns, and while the first task presentation unit 22 is presenting the first learning task to user 1, a switching unit 30 switches the content presented to user 1 to a first learning task of a different difficulty level depending on the magnitude relationship between the first concentration level and the second concentration level.

As a result, when the learning support device 100 presents a first learning task to user 1, it can switch the presentation to a second learning task of an appropriate level of difficulty depending on the state of user 1 estimated from the level of concentration of user 1.

Also, for example, in the learning support device 100, while the first learning task presentation unit 22 presents the first learning task to the user 1, the switching unit 30 switches the content presented to the user 1 to a second learning task that the user 1 passively learns and that has a different level of difficulty depending on the level of the second concentration level.

As a result, when the learning support device 100 presents a first learning task to user 1, it can switch the presentation to a second learning task of an appropriate level of difficulty depending on the state of user 1 estimated from the level of concentration of user 1.

For example, the learning support device 100 further includes a second learning task presentation unit 28 that presents a second learning task to user 1, and while the second learning task presentation unit 28 is presenting the second learning task to user 1, when the first concentration level is higher than the first value, the switching unit 30 switches the content presented to user 1 to the first learning task.

As a result, when the learning support device 100 presents the second learning task to the user 1, it can switch the presentation to the first learning task of an appropriate level of difficulty depending on the state of the user 1 estimated from the concentration level of the user 1.

In addition, for example, the learning support device 100 further includes a concentration level determination unit 18 that determines that user 1 is in a absent-minded state and prompts user 1 to take a break when the first learning task presentation unit 22 is presenting the first learning task to user 1 and the first concentration level is higher than the second concentration level.

As a result, when the learning support device 100 presents the first learning task to the user 1, it can prompt the user 1 to take a break depending on the state of the user 1 estimated from the level of concentration of the user 1, thereby improving the work efficiency of the user 1.

Also, for example, in the learning support device 100, while the second learning task presentation unit 28 is presenting the second learning task to the user 1, when the first concentration level is lower than the second value, the concentration level determination unit 18 prompts the user 1 to take a break.

As a result, when the learning support device 100 presents the second learning task to the user 1, it can prompt the user 1 to take a break depending on the state of the user 1 estimated from the level of concentration of the user 1, thereby improving the work efficiency of the user 1.

In addition, the learning support system disclosed herein is a learning support system for user 1 to perform a learning task, and includes a display 2, an imaging means 10 for imaging user 1, a first concentration level estimation unit 16 for estimating a first concentration level of user 1 by analyzing information from the imaging means 10 for imaging user 1, a second concentration level estimation unit 26 for estimating a second concentration level of user 1 by analyzing information actively input by user 1 when user 1 performs the learning task, and a presentation switching unit 30 for switching the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level.

As a result, the learning support system disclosed herein can achieve the same effects as the learning support device 100 described above.

[others]
Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

For example, in the above embodiment, the processing performed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

Also, for example, in the above embodiment, a learning support method for user 1 performing a learning task may be executed, which includes a first concentration level estimation step of estimating a first concentration level of user 1 by analyzing information from an imaging means 10 that images user 1, a second concentration level estimation step of estimating a second concentration level of user 1 by analyzing information actively input by user 1 when user 1 performs the learning task, and a switching step of switching the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level.

In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Each component may also be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Each of these circuits may also be a general-purpose circuit, or a dedicated circuit.

In addition, the general or specific aspects of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. In addition, the general or specific aspects of the present disclosure may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

For example, the present disclosure may be realized as a program for causing a computer to execute the learning support method of the above-described embodiment. The present disclosure may be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

In addition, this disclosure also includes forms that can be obtained by applying various modifications to each embodiment that a person skilled in the art may conceive, or forms that can be realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of this disclosure.

The learning support device and learning support system disclosed herein can provide users with an effective learning experience.

Reference Signs List 1 User 2 Display 10 Photography means 12 Body movement/pose determination unit 14 Gaze/facial expression determination unit 16 First concentration level estimation unit 18 Concentration level determination unit 20 Answer input unit 22 First learning task presentation unit 24 Information processing unit 26 Second concentration level estimation unit 28 Second learning task presentation unit 30 Switching unit 100 Learning support device

Claims (6)

A learning assistance device for a user to perform a learning task, comprising:
a first concentration level estimation unit that estimates a first concentration level of the user by analyzing information from an imaging means that images the user;
a second concentration level estimation unit that estimates a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task;
a presentation switching unit that switches a content of a learning task and a presentation method of the learning task based on at least one of the first concentration level and the second concentration level;
a first learning task presentation unit that presents to the user a first learning task that the user actively learns,
While the first learning task presentation unit is presenting the first learning task to the user,
the presentation switching unit switches the content to be presented to the user to the first learning task having a different level of difficulty depending on a magnitude relationship between the first concentration level and the second concentration level;
Learning aids. While the first learning task presentation unit is presenting a first learning task to the user,
the presentation switching unit switches the content presented to the user to a second learning task of a different difficulty level depending on the second concentration level, the second learning task being passively learned by the user;
The learning support device according to claim 1 . Further, a second learning task presenting unit presents a second learning task to the user,
When the first concentration level is higher than a first value while the second learning task presenter is presenting the second learning task to the user,
the presentation switching unit switches the content presented to the user to the first learning task.
The learning support device according to claim 1 or 2 . a concentration level determination unit that determines that the user is in a absent-minded state and prompts the user to take a break when the first concentration level is higher than the second concentration level while the first learning task presentation unit is presenting the first learning task to the user,
The learning support device according to any one of claims 1 to 3 . a second learning task presenting unit presenting a second learning task to the user, and when the first concentration level is lower than a second value,
The concentration level determination unit prompts the user to take a break.
The learning support device according to claim 4 . A learning support system for a user to perform a learning task, comprising:
A display and
An imaging means for imaging a user;
a first concentration level estimation unit that analyzes information from the image capturing means to estimate a first concentration level of the user;
a second concentration level estimation unit that estimates a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task;
a presentation switching unit that switches a content of a learning task and a presentation method of the learning task based on at least one of the first concentration level and the second concentration level;
a first learning task presentation unit that presents to the user a first learning task that the user actively learns,
While the first learning task presentation unit is presenting the first learning task to the user,
the presentation switching unit switches the content to be presented to the user to the first learning task having a different level of difficulty depending on a magnitude relationship between the first concentration level and the second concentration level;
Learning support system.

Images