JP7526931B2 - Environmental control system and environmental control method - Google Patents

Description

The present invention relates to an environmental control system and an environmental control method that determine the control content of an environmental control device to wake up a user.

Various technologies have been proposed to wake up users. Patent Document 1 discloses an alarm device that notifies a sleeper to wake up based on the sleep state of the sleeper.

JP 2014-023571 A

However, it is desirable for an environmental control system that controls the environment to wake up a user to be able to wake up the user effectively.

Therefore, the present invention provides an environmental control system and an environmental control method that can effectively wake up the user.

An environmental control system according to one aspect of the present invention includes a first acquisition unit that acquires input information including behavioral indicator information of a user working on a PC; a determination unit that determines, from the input information in accordance with a control content determination rule, a control content for waking up the user sleeping in a sleeping room, a control content for an environmental control device including a light-outputting device provided in the sleeping room; a control unit that executes control of the environmental control device based on the determined control content; a second acquisition unit that acquires evaluation information indicating the user's evaluation of the executed control; and an update unit that updates the control content determination rule by machine learning using a value based on the evaluation information as a reward, and the determination unit determines, as the control content, how the brightness of the light output by the light-outputting device is changed over time.

An environmental control method according to one aspect of the present invention is an environmental control method executed by a computer, and includes a first acquisition step of acquiring input information including behavioral index information of a user working on a PC, a decision step of determining, from the input information in accordance with a control content decision rule, a control content for waking up the user sleeping in a sleeping room, the control content of an environmental control device including a light-outputting device provided in the sleeping room, and a control step of executing control of the environmental control device based on the determined control content, a second acquisition step of acquiring evaluation information indicating the user's evaluation of the executed control, and an update step of updating the control content decision rule by machine learning using a value based on the evaluation information as a reward, and in the decision step, determining, as the control content, how to change the brightness of the light output by the light-outputting device over time.

These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an integrated circuit, a computer program, and a recording medium.

The environmental control system and method according to one aspect of the present invention can effectively wake up a user.

FIG. 1 is a block diagram showing a functional configuration of an environmental control system according to the first embodiment. FIG. 2 is a diagram for explaining an overview of the awakening control. FIG. 3 is a diagram showing a plurality of types of lighting devices. FIG. 4 is a flowchart of the operation during awakening control of the control device according to the first embodiment. FIG. 5 is a diagram showing information that can be used as input information. FIG. 6 is a diagram for explaining the control parameters of the awakening control. FIG. 7 is a chromaticity diagram for explaining changes in emitted light color in wakefulness control. FIG. 8 is a flowchart of the reward calculation operation of the control device according to the first embodiment. FIG. 9 is a diagram showing items that can be used to calculate remuneration. FIG. 10 is a diagram showing the relationship between items for determining an individual remuneration and physiological index information that can be used to determine the individual remuneration. FIG. 11 is a first diagram for explaining the control parameters of the sleep control. FIG. 12 is a second diagram for explaining the control parameters of the sleep control. FIG. 13 is a block diagram showing a functional configuration of an environmental control system according to the second embodiment.

The following describes the embodiments with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement positions, connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim are described as optional components.

Note that each figure is a schematic diagram and is not necessarily a precise illustration. In addition, in each figure, the same reference numerals are used for substantially the same configurations, and duplicate explanations may be omitted or simplified.

In addition, in this specification, numerical values and numerical ranges are not expressions that express only a strict meaning, but are expressions that include a substantially equivalent range, for example, a difference of about a few percent.

(Embodiment 1)
[overview]
An environmental control system according to a first embodiment will be described below. FIG. 1 is a block diagram showing a functional configuration of the environmental control system according to the first embodiment. The environmental control system 10 according to the first embodiment is an awakening assistance system for awakening a user while the user is sleeping. The environmental control system 10 performs awakening control to awaken the user using a device that outputs light, such as a lighting device. FIG. 2 is a diagram for explaining an overview of the awakening control.

As shown in FIG. 2, the awakening control is, for example, a control that gradually increases the brightness of the light emitted by the lighting device from the start time of the target period to the end time of the target period. In this way, the environmental control system 10 can wake up the user comfortably.

Here, it is considered that the optimal values of the wakefulness control content (length of target period, maximum brightness, amount of brightness change, brightness change function (i.e., shape of brightness curve), etc.) will differ depending on behavioral indicator information indicating the user's behavior before or during sleep. Therefore, if the wakefulness control content is fixed regardless of the user's behavior, there is a possibility that the effect of waking the user up comfortably will be reduced.

The environmental control system 10 therefore has a learning device 100a that has been constructed in advance by machine learning. When behavioral index information is provided to the learning device 100a as input information, the learning device 100a outputs the control content of wakefulness control that is considered to be optimal. This allows the environmental control system 10 to wake up the user effectively (i.e., comfortably).

The environmental control system 10 shown in FIG. 2 can also perform sleep control during the sleep induction period or sleep period to allow the user to sleep comfortably. The control performed during the sleep induction period is also referred to as light fluctuation control. As described below, the control content of the sleep control is used as input information for causing the learning device 100a to output the control content of the wakefulness control.

Note that sleep here includes naps. A nap is a short period of sleep, for example, a period during which work is interrupted, and is lighter than the long period of sleep (hereinafter also referred to as "main sleep") that people who are active during the day take at night. As shown in FIG. 1, the environmental control system 10 specifically comprises a control device 100, a sensor 200, an input device 300, and an environmental control device 400. Each of these devices will be described in detail below.

[Sensor]
The sensor 200 detects the user's behavior and outputs behavior index information directly or indirectly indicating the detected user's behavior to the control device 100. Such a sensor 200 may be, for example, a user interface system that detects the user's typing speed or the number of mouse clicks, a pressure sensor provided on a chair on which the user sits to detect the user's absence from the seat (frequency and number of times the user leaves the seat), etc. The sensor 200 may be a voice recognition system that detects the number of conversations or state of the user, or a vital sensor (specifically, a sleep meter or an electroencephalograph) that detects the user's sleep time, sleep quality, or wake-up time/bedtime. The user's sleep time, sleep quality, or wake-up time/bedtime may be detected by executing an application on a mobile terminal such as a smartphone, or may be detected by a dedicated wearable terminal.

The sensors 200 may also include sensors that detect the user's mental and physical state and output physiological index information that directly or indirectly indicates the detected mental and physical state of the user to the control device 100. Such sensors 200 are, for example, a heart rate monitor, a camera, a thermometer, an electroencephalograph, a saliva sensor, a sweat sensor, a breathing sensor, a body movement sensor, a blood flow sensor, etc. Each of the sensors 200 that output such physiological index information to the control device 100 may be a contact type sensor that comes into contact with the user, or a non-contact type sensor.

The sensor 200 may also include a sensor that detects environmental information around the user and outputs the detected environmental information to the control device 100. Such sensors 200 are, for example, a solar radiation sensor, a light reception sensor, a temperature sensor, an odor sensor, a microphone, a _CO2 concentration sensor, etc. The number of sensors 200 included in the environmental control system 10 is not particularly limited.

[Input device]
The input device 300 is a user interface device that accepts operations by the user to input information to the control device 100. The input device 300 outputs, for example, behavioral index information (medication history, eating and drinking history), subjective index information, schedule information, user information, and the like to the control device 100 based on the user's operations.

In addition, the input device 300 outputs user evaluation information regarding the awakening control that is being performed or has been performed to the control device 100 based on the user's operation. The evaluation information is used to calculate rewards in machine learning.

The input device 300 is, for example, a mobile terminal such as a smartphone or a tablet terminal, but may also be a wearable device such as a smart watch. The input device 300 may also be a microphone, a mechanical push button, a keyboard, a mouse, or the like. The number of input devices 300 included in the environmental control system 10 is not particularly limited.

[Environmental control equipment]
The environmental control device 400 is a device for controlling the environment (light environment, air environment, temperature environment, etc.) around the user, and is a device controlled by the control device 100 in wakefulness control. Specifically, the environmental control device 400 is a device that outputs light, such as a lighting device (i.e., a device that stimulates the user by light).

The environmental control system 10 may include multiple types of lighting devices as the environmental control devices 400. FIG. 3 is a diagram showing multiple types of lighting devices. As shown in FIG. 3, the multiple types of lighting devices include, for example, lighting devices that provide direct lighting such as downlights, lighting devices that provide indirect lighting such as cove lighting and cornice lighting, and upper and lower awakening lighting. In the awakening control, these multiple types of lighting devices may be controlled individually, or may be grouped and controlled on a group basis.

The environmental control device 400 may include other devices other than lighting devices that control the environment to wake up the user. The environmental control device 400 may include devices that stimulate the user with images, sounds, scents, vibrations, temperature and humidity, airflow, and tactile sensations in addition to light. Specifically, the environmental control device 400 may further include air conditioners, air purifiers, ventilation fans, electric fans, or floor heating. The environmental control device 400 may also include opening and closing devices for shading equipment (e.g., blinds and curtains) that can open and close windows. The environmental control device 400 may also include video devices, audio devices, or video-audio devices. The environmental control device 400 may also include an aroma diffuser. The environmental control device 400 may also include a massager.

[Control device]
Next, the control device 100 will be described. The control device 100 acquires behavior index information as input information, and determines the control content of the environmental control device 400 in wakefulness control based on the acquired input information. The control device 100 also outputs a control signal to the environmental control device 400 for controlling the environmental control device 400 according to the determined control content. The control device 100 is realized by, for example, a microcomputer, but may also be realized by a processor or the like.

Specifically, the control device 100 includes a first acquisition unit 110, a determination unit 120, a control unit 130, a second acquisition unit 150, a reward calculation unit 160, a reward condition setting unit 170, an update unit 180, and a memory unit 190. Of these components, the components other than the control unit 130 constitute the learning device 100a. The reward calculation unit 160, the reward condition setting unit 170, the update unit 180, and the memory unit 190 constitute the learning unit 100b.

The first acquisition unit 110 acquires input information including the user's behavioral indicator information.

The determination unit 120 determines the control content for waking up the user from the input information according to the control content determination rules. Specifically, the determination unit 120 determines the control content of the environmental control device 400 that is controlled in the awakening control for waking up the user. The control content determination rules are stored in the memory unit 190.

The control unit 130 controls the environmental control device 400 based on the control content determined by the determination unit 120. Specifically, the control unit 130 outputs a control signal corresponding to the control content to the environmental control device 400.

The second acquisition unit 150 acquires evaluation information indicating the user's evaluation of the awakening control. The evaluation information includes information indicating an evaluation of the control executed by the control unit 130, and includes, for example, information input by the user via the input device 300 after the awakening control. The second acquisition unit 150 acquires information output by the input device 300 as the evaluation information, but may also acquire information output by the sensor 200 as the evaluation information.

The reward calculation unit 160 calculates the reward based on the evaluation information acquired by the second acquisition unit 150. Details of the process of calculating the reward will be described later.

The reward condition setting unit 170 sets the conditions for the calculation of the reward in the reward calculation unit 160. As described below, the conditions are, for example, weighting coefficients. The conditions may be stored in advance in the storage unit 190, or if the conditions are fixed, the reward condition setting unit 170 may not be provided.

The update unit 180 updates the control content determination rules by machine learning using a value based on the evaluation information acquired by the second acquisition unit 150 as a reward.

The storage unit 190 is a storage device that stores the control content determination rules, usage history information of the environmental control system 10, user schedule information output by the input device 300, and the like. The storage unit 190 is realized, for example, by a semiconductor memory.

[Awakening Control Operation]
Next, a description will be given of the operation of the controller 100 during awakening control. Fig. 4 is a flowchart showing the operation of the controller 100 during awakening control.

First, the first acquisition unit 110 acquires input information including the user's behavior index information (S110). FIG. 5 is a diagram showing behavior index information that can be used as input information. As shown in FIG. 5, the behavior index information includes the typing speed at which the user operates the keyboard, sleep time, sleep quality, wake-up time, bedtime, number of mouse clicks, frequency and number of times the user leaves their desk, number of conversations and conversation state, break time, drug intake history, and eating and drinking history. The behavior index information may also include information (time information) indicating changes in these items over time.

The first acquisition unit 110 acquires at least one of these pieces of behavioral index information as input information. The first acquisition unit 110 acquires the behavioral index information from the sensor 200, for example, but may also acquire the behavioral index information from the input device 300, or may acquire the behavioral index information stored in the storage unit 190.

Next, the determination unit 120 determines, from the input information, the control content of the awakening control to awaken the user, which is the control content of the environmental control device 400 that includes a device that outputs light, in accordance with the control content determination rules stored in the memory unit 190 (S120).

The control content decision rule is expressed, for example, by an action value function that determines the value of the control content. The action value function is an example of a value function. The decision unit 120, for example, uses the action value function to decide that the control content that is estimated to provide the highest reward from the input information (for example, the control content that provides the maximum reward for the awakening effect) is the control content for the input information.

The control content that provides the highest reward may differ for each user. Therefore, when the environmental control system 10 is shared by multiple users, the determination unit 120 may calculate the reward for each user for the control content according to control content determination rules that differ for each user.

The control content of the awakening control includes the control parameters as shown in FIG. 6. FIG. 6 is a diagram for explaining the control parameters of the awakening control. The vertical axis of FIG. 6 indicates the brightness of the light emitted by the lighting device, and the horizontal axis of FIG. 6 indicates time.

Awakening control is a control in which the brightness of the light emitted by the lighting device is gradually increased from the start time of the target period to the end time of the target period, and then kept constant. In this way, the environmental control system 10 can wake up the user comfortably. In this case, examples of the control parameters determined by the determination unit 120 include (a) the length of the target period, (b) the maximum brightness, (c) the time required to reach the maximum brightness, (d) the time the maximum brightness is maintained, and (e) the shape of the brightness curve until the maximum brightness is reached. The shape of the curve includes a straight line, an upward convex shape, a downward convex shape, etc.

In addition, in wakefulness control, the emitted color (chromaticity of the light emitted by the lighting device) may be changed instead of or in addition to the brightness. When the emitted color is changed, the vertical axis of FIG. 6 is read as the emitted color. FIG. 7 is a chromaticity diagram for explaining the change in the emitted color in wakefulness control.

For example, when changing the chromaticity from point b to point a on the chromaticity diagram shown in Fig. 7, examples of the control parameters determined by the determination unit 120 include (a) the length of the target period, (b) the chromaticity of point a, (c) the time required for the chromaticity to reach point a from point b, (d) the time for which the chromaticity of point a is maintained, and (e) the shape of the curve of the chromaticity until the chromaticity of point a is reached. The shape of the curve includes a linear shape, an upward convex shape, a downward convex shape, and the like.

In addition, in the awakening control, air conditioners may be controlled in addition to lighting equipment. Awakening control with air conditioners as the control target is a control in which the strength of the airflow emitted by the air conditioner is gradually increased from the start time of the target period to the end time of the target period and then kept constant. In other words, the vertical axis in Figure 6 can be interpreted as the strength of the airflow.

In this case, examples of control parameters determined by the determination unit 120 include (a) the length of the target period, (b) the maximum airflow, (c) the time required to reach the maximum airflow, (d) the time the maximum airflow is maintained, and (e) the shape of the curve of the strength of the airflow until the maximum airflow is reached. The shape of the curve includes a straight line, an upward convex shape, a downward convex shape, etc.

After step S120, the control unit 130 executes control of the environmental control device 400 (i.e., awakening control) based on the determined control content (S130). Specifically, the control unit 130 outputs a control signal corresponding to the control content to the environmental control device 400.

Next, the second acquisition unit 150 acquires evaluation information indicating the user's evaluation of the awakening control executed in step S130 (S140). The second acquisition unit 150 acquires information output by the input device 300 as the evaluation information, but may also acquire information output by the sensor 200 as the evaluation information.

Next, the reward calculation unit 160 calculates a reward based on the evaluation information acquired in step S140 (S150). Details of the reward calculation operation will be described later. Note that the acquisition of the evaluation information in step S150 is performed after awakening control, but may also be performed during awakening control.

Next, the update unit 180 updates the control content determination rules by machine learning using the reward calculated by the reward calculation unit 160 (S160). The update unit 180 learns to determine the control content adapted to the user (i.e., the control content that will obtain the most reward for the user) by reinforcement learning based on the reward calculated by the reward calculation unit 160. As described above, in the first embodiment, the update unit 180 updates the control content determination rules by updating the action value function.

Below, a method for updating the action value function will be explained. Q-learning and TD-learning are known as typical methods of reinforcement learning. Below, Q-learning will be explained as an example. Q-learning is a method for learning a value Q(s, a) for selecting a control content a under a user's state s indicated by input information, and in a certain state s, the control content a with the highest value Q(s, a) is selected as the optimal control content. The learning device 100a (update unit 180) selects various control contents a under a certain state s, and a reward is given for the control content a at that time. As a result, the learning device 100a learns to select better control contents, that is, the correct value Q(s, a). The update formula for such a value Q(s, a) can be expressed, for example, by Equation 1.

Here, s _t represents the state at time t, and a _t represents the control content at time t. The state changes to s _t+1 due to the control content a _t . _{r t+1} represents the reward obtained by the change in state. In addition, the term with max is the Q value multiplied by γ when the control content a _t+1 with the highest Q value known at that time is selected under the state s _t+1 . Here, γ is a parameter with a range of 0<γ≦1 and is called the discount rate. In addition, α is a learning coefficient and is in the range of 0<α≦1.

The above method is an example of a reinforcement learning method. Any existing method may be used for reinforcement learning, such as a method using a neural network or a method that combines reinforcement learning with deep learning.

[Reward calculation operation]
Next, the remuneration calculation operation in step S150 will be described in detail with reference to Fig. 8, which is a flowchart showing the remuneration calculation operation.

First, the reward calculation unit 160 determines an individual reward Fa for the sense of comfort (S210). The reward calculation unit 160 determines the individual reward Fa using, for example, subjective index information indicating the user's subjective evaluation result output by the input device 300 as evaluation information, but may also determine the individual reward Fa using the user's heart rate detected by a heart rate meter used as the sensor 200 as evaluation information. In this case, the higher the sense of comfort indicated by the subjective index information or the heart rate, the larger the value of the individual reward Fa.

Next, the reward calculation unit 160 determines an individual reward Fb for the feeling of refreshment (S220). The reward calculation unit 160 determines the individual reward Fb using, for example, the user's subjective index information output by the input device 300 as evaluation information. In this case, the higher the sense of comfort indicated by the subjective index information, the larger the value of the individual reward Fb.

Next, the reward calculation unit 160 determines an individual reward Fc for the user's motivation (S230). The reward calculation unit 160 determines the individual reward Fc using, for example, the user's subjective index information output by the input device 300 as evaluation information, but may also determine the individual reward Fc using the user's typing speed detected by the user interface system used as the sensor 200 as evaluation information. In this case, the higher the degree of improvement in motivation indicated by the subjective index information, the higher the value of the individual reward Fc, and the faster the typing speed, the higher the value of the individual reward Fc.

Next, the reward calculation unit 160 determines an individual reward Fd for the drowsiness level (S240). The reward calculation unit 160 determines the individual reward Fd using, for example, the user's subjective index information output by the input device 300 as evaluation information, but may also determine the individual reward Fd using the number of times the user blinks in a predetermined period detected by a camera used as the sensor 200 as evaluation information. In this case, the greater the degree of reduction in the drowsiness level indicated by the subjective index information, the greater the value of the individual reward Fd, and the greater the number of times the user blinks after waking up compared to before falling asleep, the greater the value of the individual reward Fd.

Next, the reward calculation unit 160 determines an individual reward Fe for the improvement in concentration level after awakening (S250). The reward calculation unit 160 determines the individual reward Fe using, for example, the user's subjective index information output by the input device 300 as evaluation information, but may also determine the individual reward Fe using the user's typing speed detected by the user interface system used as the sensor 200 as evaluation information. In this case, the higher the degree of improvement in motivation indicated by the subjective index information, the larger the value of the individual reward Fe, and the faster the typing speed, the larger the value of the individual reward Fe.

Then, the reward calculation unit 160 determines the reward F for the control content based on the individual rewards Fa to Fe determined in steps S210 to S250 (S260). At this time, the reward calculation unit 160 may calculate the reward F by weighting and adding the individual rewards Fa to Fe. For example, the reward calculation unit 160 may calculate the reward F based on Equation 2.

F=w1×Fa+w2×Fb+w3×Fc+w4×Fd+w5×Fe...(Formula 2)

w1 to w5 are weights for each item set by the reward condition setting unit 170, and are examples of reward conditions. In other words, the reward condition setting unit 170 sets the weights w1 to w5 that the reward calculation unit 160 uses when performing weighting addition.

The reward condition setting unit 170 may change the conditions (e.g., weights) depending on at least one of the weather, season, and time of day when the user wakes up. For example, if the weather, season, and time of day are included in the input information acquired by the first acquisition unit 110, the reward condition setting unit 170 may acquire information related to the weather, season, and time of day from the first acquisition unit 110. The reward condition setting unit 170 may also acquire information related to the season and time of day from a general-purpose timer IC (timer circuit) that measures the current time, or a real-time clock IC, which is provided in the control device 100.

The reward calculation operation has been described above, but this reward calculation operation is one example. For example, the items for determining the individual reward (feeling of comfort, feeling of refreshment, motivation, sleepiness level, and concentration level) are one example, and the reward may be calculated by determining an individual reward for at least one item. In addition, individual rewards may be determined for other items in the reward calculation. Figure 9 is a diagram showing items that can be used to calculate the reward.

In addition, in the above-mentioned reward calculation operation, information indicating the subjective evaluation result, behavioral index information (typing speed), or physiological index information (heart rate and eye blinking) was used as evaluation information to determine the individual reward. Here, in the reward calculation operation, the individual reward may be determined using other physiological index information. FIG. 10 is a diagram showing the relationship between the items for determining the individual reward and the physiological index information that can be used to determine the individual reward.

As shown in FIG. 10, the items for determining the individual reward include items indicating the user's state during awakening control and items indicating the user's state after awakening. The items for determining the individual reward also include items that are determined based on changes in physiological index information before falling asleep and after awakening. In this way, the individual reward may be determined absolutely based on the user's state during awakening control or after awakening, or may be determined relatively based on changes in physiological index information before falling asleep and after awakening.

[Other input information 1]
The input information is not limited to the behavior index information. Other input information will be described below with reference to FIG.

The first acquisition unit 110 may acquire physiological index information indicating the user's mental and physical state as input information. The physiological index information includes heart rate (pulse wave), blinking/gaze, eye movement, pupil fluctuation, skin temperature (peripheral, nose, forehead), facial expression (emotion), brain wave, saliva, head movement, sweating (normal sweating, mental sweating), breathing, body movement, blood flow (brain/peripheral), and the like. Note that the heart rate used as the physiological index information includes not only the heart rate, but also the frequency components LF, HF, HF/LF of heart rate fluctuation. The physiological index information may include information indicating changes over time in these items (time information). The physiological index information is acquired, for example, from the sensor 200, but may also be acquired from the input device 300.

The first acquisition unit 110 may also acquire subjective index information as input information. The subjective index information includes drowsiness, motivation, physical condition/physical fatigue, stress, concentration, performance, degree of tension/relaxation, degree of irritation/anger/sadness, etc. The subjective index information may also include information indicating changes over time in these items (time information). The subjective index information is acquired, for example, from the input device 300.

The first acquisition unit 110 may also acquire schedule information indicating the user's schedule as input information. The schedule information includes planned activities (e.g., meetings) for the day of use, the day before use, and the day after use, the work status (busy, away from desk, in a meeting, etc.), the season to which the day of use belongs, the time period of use, etc. The schedule information is acquired, for example, from the input device 300.

The first acquisition unit 110 may also acquire usage history information of the user's environmental control system 10 (wakefulness control) as input information. The usage history information includes usage time/time of day, day of the week, timing, input information, output information, rewards, etc. The usage history information is acquired, for example, from the memory unit 190. In other words, the usage history information is stored in the memory unit 190.

The first acquisition unit 110 may also acquire environmental information around the user as input information. The environmental information includes weather (amount of solar radiation), amount of received light (i.e., light environment information), season/time period, environmental temperature, humidity, odor, sound environment, _CO2 concentration, and the like.

The first acquisition unit 110 may also acquire user information as input information. The user information includes gender, age, race, place of origin, occupation, physical constitution (photosensitivity, etc.), medical history (including insomnia, etc.), etc. The user information is acquired, for example, from the input device 300.

[Other input information 2]
The first acquisition unit 110 may acquire control parameters in the sleep control as input information. Specifically, the first acquisition unit 110 may acquire control parameters in the sleep control performed in a sleep introduction period (or sleep period) prior to (for example, immediately prior to) the wakefulness control to be performed, as input information. The control parameters of the sleep control will be described below. Figs. 11 and 12 are diagrams for explaining the control parameters of the sleep control. The vertical axis of Figs. 11 and 12 indicates the brightness of the light emitted by the lighting device, and the horizontal axis of Figs. 11 and 12 indicates time. In the following, the control content of the sleep introduction period (in other words, the control content of the light fluctuation control) will be mainly described among the relaxation period, the sleep introduction period, and the sleep period.

As shown in FIG. 11, the control parameters for sleep control include the length of the entire sleep induction period, maximum brightness and minimum brightness, the rise time required to increase the brightness, the time for which maximum brightness is maintained, the fall time required to decrease the brightness, the time for which minimum brightness is maintained, the period, an ascending curve showing the manner of change (e.g., slope) when the brightness is increased, and a descending curve showing the manner of change (e.g., slope) when the brightness is decreased. Note that maximum brightness and minimum brightness refer to the maximum and minimum values of brightness when the brightness is changed periodically.

In addition, at least one of the maximum brightness, minimum brightness, various times, and various curves may change over time. Figure 12 shows an example in which the maximum brightness changes over time.

As shown in FIG. 12, the control parameters for sleep control may further include a change start time at which the change to maximum brightness begins, a change end period at which the change to maximum brightness ends, a change curve indicating how the maximum brightness changes, and a change target value indicating the maximum brightness after the change.

In addition, in sleep control, the emitted color (chromaticity of the light emitted by the lighting device) may be changed instead of or in addition to the brightness. When the emitted color is changed, the control parameters of sleep control may include a control parameter related to the emitted color in addition to the control parameter related to the brightness. For example, when changing the emitted color from point b to point a on the chromaticity diagram shown in FIG. 7, the control parameters of sleep control include the chromaticity of point a and the chromaticity of point b, the time until the chromaticity of point a is reached, the time for which the chromaticity of point a is maintained, the time until the chromaticity of point b is reached, the time for which the chromaticity of point b is maintained, the period, the shape of the curve until the chromaticity of point a is reached, and the shape of the curve until the chromaticity of point b is reached.

In this case, at least one of the chromaticity of point a, the chromaticity of point b, the various periods, and the various curves may change over time. For example, if the chromaticity of point a changes over time, the control parameters for sleep control may include a change start time at which the chromaticity of point a starts to change, a change end period at which the change in the chromaticity of point a ends, a change curve that indicates how the chromaticity of point a changes, and a change target value that indicates the chromaticity of point a after the change.

[Modification 1]
As described above, the second acquisition unit 150 can acquire the evaluation information during the execution of the awakening control. Then, the reward calculation unit 160 can calculate the reward during the execution of the awakening control based on the acquired evaluation information, and the update unit 180 can update the control content decision rule (action value function) based on the calculated reward during the execution of the awakening control. As a result, the decision unit 120 can change the control content (i.e., re-determine the control content) during the execution of the awakening control.

In this way, if the control content is re-determined while awakening control is being performed, the environmental control system 10 can determine the control content that is more appropriate for the user's state at that time, thereby effectively waking up the user.

[Modification 2]
The storage unit 190 may store control contents previously determined by the determination unit 120. In this way, when the environmental control system 10 is unable to acquire input information for some reason and is therefore unable to determine the control contents, the control contents stored in the storage unit 190 can be used instead. The control contents stored in the storage unit 190 may be stored as part of the above-mentioned usage history information, or may be stored independently. For example, the storage unit 190 may store the control contents in association with the input information used to determine the control contents, or may store the control contents without association with the input information used to determine the control contents.

[Effects, etc.]
As described above, the environmental control system 10 includes a first acquisition unit 110 that acquires input information including behavioral index information of the user, a determination unit 120 that determines, from the input information in accordance with a control content determination rule, control content for waking up the user, the control content of the environmental control device 400 including a device that outputs light, a control unit 130 that executes control of the environmental control device 400 based on the determined control content, a second acquisition unit 150 that acquires evaluation information indicating the user's evaluation of the executed control, and an update unit 180 that updates the control content determination rule by machine learning using a value based on the evaluation information as a reward.

Such an environmental control system 10 can determine the control content based on the control content decision rules learned in association with the behavioral index information and the evaluation information. Therefore, the environmental control system 10 can effectively wake up the user.

Also, for example, the control content determination rules include a value function that determines the value of the control content, and the update unit 180 updates the value function.

Such an environmental control system 10 can determine the control content based on a value function learned in association with behavioral index information and evaluation information.

For example, the environmental control system 10 further includes a reward calculation unit 160 that calculates a reward based on the evaluation information, and a reward condition setting unit 170 that sets the conditions for the calculation of the reward in the reward calculation unit 160.

Such an environmental control system 10 can calculate rewards according to conditions set by the reward condition setting unit 170. For example, if conditions are set according to the user's preferences, it becomes easier to determine the control content according to the user's preferences.

For example, the environmental control system 10 further includes a memory unit 190 that stores the control content determined by the determination unit 120.

Such an environmental control system 10 can read out the control content (e.g., the previous control content) stored in the memory unit 190, and perform control to wake up the user even when input information cannot be obtained and control content cannot be determined.

For example, the update unit 180 updates the control content decision rule based on the evaluation information acquired by the second acquisition unit 150 while controlling the environmental control device 400 with the above control content. The decision unit 120 further re-determines the control content for the input information during control in accordance with the control content decision rule updated while controlling the environmental control device 400.

Such an environmental control system 10 can update the control content decision rules and change the control content depending on the state of the user during control. In other words, by learning during control, the environmental control system 10 can decide on control content that is more appropriate for the state of the user at that time, thereby waking up the user more effectively.

For example, the input information further includes at least one of the user's physiological index information, the user's schedule information, the user's usage history information of the environmental control system 10, and environmental information.

Such an environmental control system 10 can determine the control content based on control content determination rules learned by associating at least one of physiological index information, schedule information, usage history information, and environmental information with evaluation information.

Furthermore, for example, the control unit 130 can further execute sleep control to put the user to sleep using the environmental control device 400, and the input information further includes control parameters for the sleep control.

Such an environmental control system 10 can determine the control content based on control content determination rules learned by associating control parameters in sleep control with evaluation information.

For example, the environmental control device 400 further includes other devices in addition to the device that outputs light. The decision unit 120 decides the control content of the environmental control device 400 including the other devices.

Such an environmental control system 10 can control the use of light-emitting devices and other devices to wake up the user.

Also, for example, the other equipment is an air conditioning device.

Such an environmental control system 10 can control the light-emitting devices and air-conditioning devices to wake up the user.

In addition, for example, the control content for waking up the user includes at least one of (a) the length of the target period, (b) the maximum brightness, (c) the time required to reach the maximum brightness, (d) the time the maximum brightness is maintained, and (e) the shape of the brightness curve until the maximum brightness is reached, in the wakefulness control that increases the brightness output by the light-outputting device during the target period.

Such an environmental control system 10 can determine the control parameters in wakefulness control as the control content.

The environmental control method executed by a computer such as the environmental control system 10 includes a first acquisition step (S110) of acquiring input information including behavioral index information of the user, a determination step (S120) of determining, from the input information in accordance with a control content determination rule, a control content for waking up the user, the control content of the environmental control device 400 including a device that outputs light, a control step (S130) of executing control of the environmental control device 400 based on the determined control content, a second acquisition step (S140) of acquiring evaluation information indicating the user's evaluation of the executed control, and an update step (S160) of updating the control content determination rule by machine learning using a value based on the evaluation information as a reward.

Such an environmental control system 10 can determine the control content based on the control content decision rules learned in association with the behavioral index information and the evaluation information. Therefore, the environmental control system 10 can wake up the user effectively.

(Embodiment 2)
[composition]
In the second embodiment, another configuration of the environmental control system will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the functional configuration of the environmental control system according to the second embodiment.

As shown in FIG. 13, the environmental control system 10b according to the second embodiment includes a plurality of individual environmental control systems 10c and a server device 500 communicably connected to each of the plurality of individual environmental control systems 10c.

Each of the multiple individual environmental control systems 10c has a configuration similar to that of the environmental control system 10 of embodiment 1, for example, but in FIG. 13, configuration other than the first acquisition unit 110 and the memory unit 190 is omitted. The number of multiple individual environmental control systems 10c included in the environmental control system 10b is not particularly limited.

The server device 500 acquires at least one of the input information acquired by the first acquisition unit 110 and the learning results (e.g., control content, reward, and updated action value function) stored in the memory unit 190 from each of the multiple individual environmental control systems 10c, and centrally manages them. The server device 500 stores the acquired information in the memory unit 510. This makes it possible to share the learning results, etc., of each of the multiple individual environmental control systems 10c.

At least one of the multiple control devices 100 included in each of the multiple individual environmental control systems 10c may function as a server device. In other words, the environmental control system 10b does not need to include a server device 500 separate from the individual environmental control systems 10c. In this case, each of the multiple individual environmental control systems 10c is communicatively connected to each other and communicates at least one of the input information, evaluation information, and learning results with each other. The update unit 180 then updates the action value function based on at least one of the input information and learning results acquired from the other individual environmental control systems 10c.

[Effects, etc.]
As described above, the environmental control system 10b includes a plurality of individual environmental control systems 10c each having a first acquisition unit 110, a determination unit 120, a control unit 130, a second acquisition unit 150, and an update unit 180. Each of the plurality of individual environmental control systems 10c is communicatively connected to each other and communicates at least one of input information and learning results with each other. The update unit 180 updates the control content determination rule based on at least one of the input information and learning results acquired from the other individual environmental control systems 10c.

In such an environmental control system 10b, the update unit 180 can update the action value function of the own device based on input information acquired by the other individual environmental control system 10c. This improves the accuracy of learning in the own device, and a more appropriate action value function can be obtained.

The environmental control system 10b also includes a plurality of individual environmental control systems 10c and a server device 500 communicably connected to each of the plurality of individual environmental control systems 10c.

Such an environmental control system 10b can centrally manage at least one of the input information, the evaluation information, and the learning results. Furthermore, if the server device 500 is equipped with a learning unit, it is possible to update the action value function by machine learning based on the input information and the evaluation information, etc., acquired from each of the multiple individual environmental control systems 10c. This improves the accuracy of learning, and a more appropriate action value function can be obtained. In this case, it is preferable that the server device 500 is equipped with a processor or the like that can perform processing faster than the control device 100.

(Other embodiments)
Although the embodiment has been described above, the present invention is not limited to the above embodiment.

For example, in the above embodiment, the environmental control system may determine multiple patterns of control content as a recommended pattern for the user, and the user may select one of the multiple patterns of control content using an input device. In this case, the environmental control system executes control of the selected control content.

In addition, in the above embodiment, an example has been described in which the control parameters of sleep control (i.e., the control content) are used as input information to determine the control content of the environmental control device in wakefulness control. However, conversely, the control parameters of wakefulness control (i.e., the control content) may be used as input information to determine the control content of the environmental control device in sleep control.

In addition, in the above embodiment, the location where the wakefulness control is performed is not particularly limited. The wakefulness control is performed, for example, in an office nap room, but may also be performed in a home, a medical facility, or a care facility.

In addition, in the above embodiment, the environmental control system is realized by multiple devices, but it may be realized as a single device. When the environmental control system is realized by multiple devices, the components described in the above embodiment may be distributed in any manner among the multiple devices. Furthermore, the environmental control system may be realized as a client-server system.

In addition, the above embodiment does not impose any particular limitation on the method of communication between the devices. The communication between the devices is, for example, wireless communication using a communication standard such as specific low-power radio, ZigBee (registered trademark), Bluetooth (registered trademark), or Wi-Fi (registered trademark), but may also be wired communication. In addition, a relay device (not shown) may be involved in the communication between the devices.

In addition, in the above embodiment, the processing performed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, and multiple processes may be executed in parallel.

In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or an integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

The general or specific aspects of the present invention may be realized as a system, device, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM. The present invention may also be realized as any combination of a system, device, method, integrated circuit, computer program, and recording medium. For example, the present invention may be realized as a control device according to the above embodiment. The present invention may also be realized as a program for causing a computer to execute the environmental control method according to the above embodiment, or as a computer-readable non-transitory recording medium on which such a program is recorded.

In addition, the present invention also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art may conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the spirit of the present invention.

Reference Signs List 10, 10b Environmental control system 10c Individual environmental control system 110 First acquisition unit 120 Determination unit 130 Control unit 150 Second acquisition unit 160 Reward calculation unit 170 Reward condition setting unit 180 Update unit 190 Storage unit 400 Environmental control device 500 Server device

Claims (14)

A first acquisition unit that acquires input information including behavior index information of a user;
a determination unit that determines, from the input information in accordance with a control content determination rule, a control content for waking up the user sleeping in the sleeping room, the control content of an environmental control device that is provided in the sleeping room and includes a device that outputs light;
a control unit that controls the environmental control device based on the determined control content;
a second acquisition unit that acquires evaluation information indicating an evaluation of the user with respect to the executed control;
an update unit that updates the control content determination rule by machine learning using a value based on the evaluation information as a reward;
the determination unit determines, as the control content, how brightness of the light output by the device that outputs the light is changed over time;
The behavioral index information includes at least one of the following: typing speed, sleep time, sleep quality, wake-up time, bedtime, number of mouse clicks, frequency of leaving the desk, number of times of leaving the desk, number of conversations, conversation state, break time, drug intake history, and eating and drinking history of the user.
Environmental control system. the control content decision rule includes a value function that determines a value of the control content,
The environmental control system according to claim 1 , wherein the update unit updates the value function. moreover,
A remuneration calculation unit that calculates the remuneration based on the evaluation information;
The environmental control system according to claim 1 or 2, further comprising a reward condition setting unit that sets conditions for the calculation of the reward in the reward calculation unit. The environmental control system according to claim 1 , further comprising a storage unit that stores the control content determined by the determination unit. The update unit updates the control content determination rule based on the evaluation information acquired by the second acquisition unit while the environmental control device is being controlled with the control content,
The environmental control system according to any one of claims 1 to 4, wherein the determination unit further re-determines the control content for the input information during the control in accordance with the control content determination rule updated during the control of the environmental control device. The environmental control system according to any one of claims 1 to 5, wherein the input information further includes at least one of physiological index information of the user, schedule information of the user, usage history information of the user of the environmental control system, and environmental information. The control unit is further capable of executing sleep control to put the user to sleep using the environmental control device,
7. The environmental control system according to claim 1, wherein the input information further includes a control parameter in the sleep control. the environmental control device further includes other devices than the device that outputs the light,
The environmental control system according to claim 1 , wherein the determination unit determines the control content of the environmental control device including the other device. The environmental control system according to claim 8 , wherein the other equipment is an air conditioning equipment. The environmental control system according to any one of claims 1 to 9, wherein the control content for waking up the user includes at least one of (a) the length of a target period, (b) maximum brightness, (c) the time required to reach the maximum brightness, (d) the time for which the maximum brightness is maintained, and (e) the shape of the brightness curve until the maximum brightness is reached, in control for increasing the brightness of the light output by the light-outputting device during a target period. a plurality of individual environment control systems each including the first acquisition unit, the determination unit, the control unit, the second acquisition unit, and the update unit;
Each of the plurality of individual environmental control systems is communicatively connected to each other and communicates at least one of the input information and the learning result with each other;
The environmental control system according to any one of claims 1 to 10, wherein the update unit updates the control content determination rule based on at least one of the input information and the learning result acquired from another individual environmental control system. the plurality of individual environmental control systems;
The environmental control system according to claim 11 , further comprising: a server device communicatively connected to each of the plurality of individual environmental control systems. The environmental control system according to any one of claims 1 to 12, wherein the behavioral indicator information includes at least one of a typing speed at which the user operates a keyboard, a number of mouse clicks by the user, and a frequency at which the user leaves his/her desk. 1. A computer-implemented method for controlling an environment, comprising:
A first acquisition step of acquiring input information including behavior index information of a user;
a determination step of determining, from the input information in accordance with a control content determination rule, a control content for waking up the user sleeping in the sleeping room, the control content of an environmental control device provided in the sleeping room, the environmental control device including a device that outputs light;
a control step of executing control of the environmental control device based on the determined control content;
a second acquisition step of acquiring evaluation information indicating an evaluation of the user with respect to the executed control;
updating the control content determination rule by machine learning using a value based on the evaluation information as a reward;
In the determining step, a method for changing the brightness of the light output by the light-outputting device over time is determined as the control content ;
The behavioral index information includes at least one of the following: typing speed, sleep time, sleep quality, wake-up time, bedtime, number of mouse clicks, frequency of leaving the desk, number of times of leaving the desk, number of conversations, conversation state, break time, drug intake history, and eating and drinking history of the user.
Environmental control methods.

Images