JP7827599B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device, an information processing method, etc.

Conventionally, methods for selecting bedding suitable for a user have been known. For example, Patent Document 1 discloses a method for classifying body types based on height and weight and selecting bedding based on the parameters obtained. Furthermore, Patent Document 2 discloses a method for selecting bedding based on sleeping posture estimation using body pressure values.

Japanese Patent Application Laid-Open No. 2006-320647 Japanese Patent Application Laid-Open No. 2007-151817

We provide information processing devices and information processing methods that make suggestions suited to users.

One aspect of the present disclosure relates to an information processing device that includes an image acquisition unit that acquires a sleeping posture image of a user in a sleeping posture, a model estimation unit that estimates a skeletal model of the user based on the sleeping posture image, and a bedding selection unit that performs processing to select bedding recommended for the user based on the estimated skeletal model.

Another aspect of the present disclosure relates to an information processing method that acquires a sleeping posture image of a user in a sleeping posture, estimates a skeletal model of the user based on the sleeping posture image, and performs a process of selecting bedding recommended for the user based on the estimated skeletal model.

FIG. 1 illustrates an example of the configuration of an information processing system. FIG. 2 is a diagram illustrating an example of the flow of data in the processing of this embodiment. FIG. 1 illustrates an example of the configuration of a server system. FIG. 2 is a diagram illustrating an example of the configuration of a terminal device. FIG. 1 illustrates an example of the configuration of an information processing device. FIG. 2 is a diagram illustrating a detailed configuration example of an information processing device. FIG. 10 is a diagram illustrating an example of the flow of data when selecting bedding. 10 is a flowchart illustrating processing by the information processing device. 10A and 10B are diagrams illustrating an example of the positional relationship between a user and an imaging device when a standing position image is acquired. 10A and 10B are diagrams illustrating an example of the positional relationship between a user and an imaging device when acquiring a lying posture image. 10A and 10B are diagrams illustrating an example of the positional relationship between a user and an imaging device when acquiring a lying posture image. 10A and 10B are diagrams illustrating an example of the positional relationship between a user and an imaging device when acquiring a lying posture image. FIG. 1 is a diagram illustrating a skeletal model. FIG. 10 is a diagram illustrating the relationship between the neck tilt angle and the pillow height in the supine position. FIG. 10 is a diagram illustrating the relationship between the neck tilt angle and the pillow height in the supine position. FIG. 10 is a diagram illustrating the relationship between the neck tilt angle and the pillow height in the supine position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the supine position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the supine position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the supine position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the lateral position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the lateral position. FIG. 10 is a diagram illustrating the relationship between the waist inclination angle and mattress hardness in the lateral position. FIG. 10 is a diagram illustrating another example of the data flow in selecting bedding. FIG. 10 is a diagram illustrating an example of a position change technique. FIG. 10 is a diagram showing examples of recommended mattress characteristics. FIG. 10 is a diagram showing examples of recommended pillow characteristics. FIG. 10 is a diagram illustrating an example of a position change technique. FIG. 10 is a diagram showing examples of recommended mattress characteristics. FIG. 10 is a diagram showing examples of recommended pillow characteristics. FIG. 1 is a diagram illustrating a detection device that is an example of a sleep sensor. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of recommended bedding. FIG. 10 is a diagram illustrating an example of a data flow in advice output. 10 is an example of a user screen displayed on a terminal device. 10 is an example of a user screen displayed on a terminal device. 10 is an example of a user screen displayed on a terminal device. 10 is an example of a user screen displayed on a terminal device.

The present embodiment will be described below with reference to the drawings. In the drawings, identical or equivalent elements are designated by the same reference numerals, and duplicate explanations will be omitted. Note that the present embodiment described below does not unduly limit the content of the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

1. System Configuration Example Fig. 1 shows a configuration example of an information processing system 10 according to this embodiment. The information processing system 10 according to this embodiment acquires information from a user who is suffering from problems such as stiff shoulders, lower back pain, headaches, etc., caused by body misalignment or lifestyle habits, and presents measures to alleviate the problem. For example, the information processing system 10 may acquire a skeletal model of the user (see, for example, Fig. 10 described below), and based on the skeletal model, evaluate the bedding being used or select recommended bedding (see, for example, Figs. 11A to 13C described below).

Figure 2 is a diagram explaining the flow of data in this embodiment, showing specific examples of information (Input) about users, etc., and information (Output) output by the information processing system 10 based on that information. The information processing system 10 may acquire or calculate the information indicated in Input, and output the information indicated in Output based on that information.

For example, the information processing system 10 of this embodiment may acquire information such as personal data, personal attributes, facility data, and facility attributes. Personal data includes information such as the user's standing posture, sleeping posture (sleeping posture), skeletal length, and sleep score. Personal attributes include information such as age, gender, height, weight, illness, and residential area. Personal attributes may also include the user's method of changing position (turning over method). Facility attributes include information on the type and location of the facility in question. A facility here refers to a facility that addresses problems such as stiff shoulders and lower back pain, and includes chiropractic clinics, osteopathic clinics, and clinics. Facility data includes a visit score that indicates the effect of visiting the facility in question. Details of each piece of information will be provided later.

Based on this information, the information processing system 10 proposes measures to resolve the user's concerns. These measures include evaluating and selecting bedding, encouraging the user to visit the facility, and providing advice on sleep and lifestyle habits. The information processing system 10 may also calculate scores to objectively evaluate posture, sleep, pain/concerns, and the quality of the facility. Details of this information will be provided later.

As shown in FIG. 1, the information processing system 10 may include a server system 100, a terminal device 200, an imaging device 300, and a sensing device 400. Note that the configuration of the information processing system 10 is not limited to that shown in FIG. 1, and modifications such as omitting some components or adding other components are possible. For example, while the terminal device 200 and the imaging device 300 are shown as separate devices in FIG. 1, they may also be implemented as a single device. For example, the terminal device 200 with imaging capabilities may also function as the imaging device 300.

The server system 100 is connected to the terminal device 200, the imaging device 300, and the sensing device 400, for example, via a network. The network here is, for example, a public communications network such as the Internet. However, the network is not limited to a public communications network and may be a LAN (Local Area Network) or the like. For example, the server system 100 may communicate in accordance with the IEEE 802.11 standard. However, various variations in the communication method between the devices are possible.

The server system 100 may be a single server, or may include multiple servers. For example, the server system 100 may include a database server and an application server. The database server may store, for example, the information shown in Figure 2. The application server performs various processes. For example, the application server may calculate a skeletal model or select bedding recommended for the user, as will be described later using Figure 8. The multiple servers here may be physical servers or virtual servers. Furthermore, if a virtual server is used, the virtual server may be located on a single physical server, or may be distributed across multiple physical servers. As described above, the specific configuration of the server system 100 in this embodiment can be modified in various ways.

The terminal device 200 is, for example, a device used by a user of the information processing system 10. For example, the terminal device 200 is used to input personal attribute data such as age, gender, height, and weight, and to output suggestions obtained by the server system 100. The suggestions here may include, for example, information indicating recommended bedding selected by the server system 100. The terminal device 200 in this embodiment is, for example, a mobile terminal device such as a smartphone or tablet terminal. However, the terminal device 200 may also be other devices, such as a PC (Personal Computer), a headset, or a wearable device such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses.

The imaging device 300 is a device that captures images used to calculate a skeletal model of the user. The imaging device 300 includes an image sensor that captures an image of a predetermined imaging range and outputs image information. The image information here may be a still image or a moving image. The image information may also be color or monochrome. The imaging device 300 may also include a depth sensor that detects the distance to the subject, or a sensor that detects the heat of the subject (e.g., an infrared sensor). The imaging device 300 may be a camera, a smartphone with imaging capabilities, or the like.

The sensing device 400 is a device used to sense information related to the user's sleep and daily life. For example, the sensing device 400 has various sensors and acquires sensing data based on the sensors. The sensing data here may be the sensor output itself, or information obtained by arithmetic processing based on the sensor output. For example, as shown in FIG. 1, the sensing device 400 may be a detection device 430 (sleep scan) that is placed between the bed 610 and mattress 620 and senses the user's sleep state. Details of the detection device 430 will be described later using FIG. 17. The sensing device 400 may also be a device that senses the state of the user's work, and may include, for example, a wristwatch-type device or a wearable device that is attached to the user's clothing or skin.

Figure 3 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a memory unit 120, and a communication unit 130.

The processing unit 110 of this embodiment is composed of the following hardware. The hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware may be composed of one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices may be, for example, an IC (Integrated Circuit), an FPGA (Field-Programmable Gate Array), etc. The one or more circuit elements may be, for example, a resistor, a capacitor, etc.

The processing unit 110 may also be implemented by the following processor. The server system 100 of this embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information may be, for example, a program and various data. The memory may be the storage unit 120 or another memory. The processor includes hardware. Various processors can be used as the processor, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The memory may be semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, or may be a register, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores computer-readable instructions, and the processor executes these instructions to realize the functions of the processing unit 110 as processing. The instructions here may be instructions from an instruction set that constitutes a program, or instructions that instruct the processor's hardware circuitry to operate.

The memory unit 120 is a work area for the processing unit 110 and stores various information. The memory unit 120 can be implemented using various types of memory, and the memory may be semiconductor memory such as SRAM, DRAM, ROM (Read Only Memory), or flash memory, or it may be a register, a magnetic storage device, or an optical storage device.

The communication unit 130 is an interface for communicating over a network, and when the server system 100 performs wireless communication, it includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. However, the server system 100 may also perform wired communication, in which case the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 may operate under control of the processing unit 110, or may include a processor for communication control that is different from the processing unit 110. The communication unit 130 may perform communication according to a method specified in, for example, IEEE 802.11 or IEEE 802.3. However, the specific communication method can be varied in various ways.

Figure 4 is a block diagram showing a detailed example configuration of the terminal device 200. The terminal device 200 includes, for example, a processing unit 210, a memory unit 220, a communication unit 230, a display unit 240, and an operation unit 250. However, the configuration of the terminal device 200 is not limited to that shown in Figure 4, and modifications such as omitting some components or adding other components are possible. For example, the terminal device 200 may have various sensors such as an image sensor (imaging unit), motion sensors such as an acceleration sensor or gyro sensor, a pressure sensor, a GPS (Global Positioning System) sensor, etc.

The processing unit 210 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 210 may also be realized by a processor. Various types of processors can be used, such as a CPU, GPU, or DSP. The processor executes instructions stored in the memory of the terminal device 200, thereby realizing the functions of the processing unit 210 as processing.

The storage unit 220 is a work area for the processing unit 210 and is realized by various types of memory, such as SRAM, DRAM, and ROM. The storage unit 220 stores, for example, attribute information of the target user and captured images acquired from the imaging device 300. The storage unit 220 may also store information representing suggestions to be presented to the user.

The communication unit 230 is an interface for communicating over a network and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100, for example, over a network. The communication unit 230 may perform wireless communication with the server system 100, for example, in accordance with the IEEE 802.11 standard.

The display unit 240 is an interface that displays various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface that accepts user operations. The operation unit 250 may be buttons or the like provided on the terminal device 200. The display unit 240 and operation unit 250 may also be a touch panel that is integrally configured.

The terminal device 200 may also include components not shown in FIG. 4, such as a light-emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light-emitting unit is, for example, an LED (light emitting diode) and provides notifications by emitting light. The vibration unit is, for example, a motor and provides notifications by vibrating. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and provides notifications by sound.

Figure 5 is a diagram showing an example configuration of an information processing device 20 according to this embodiment. The information processing device 20 may correspond to the server system 100. For example, the units shown in Figure 5 may be included in the processing unit 110 of the server system 100. However, the information processing device 20 according to this embodiment is not limited to the server system 100, and may correspond to the terminal device 200. For example, the units shown in Figure 5 may be included in the processing unit 210 of the terminal device 200. The information processing device 20 according to this embodiment may also correspond to multiple devices. For example, the processing of the units in Figure 5 may be realized by distributed processing between the processing unit 110 and the processing unit 210.

As shown in FIG. 5, the information processing device 20 includes an image acquisition unit 21, a model estimation unit 22, and a bedding selection unit 23. The image acquisition unit 21 acquires a sleeping posture image of a user in a sleeping posture. The model estimation unit 22 estimates a skeletal model of the user based on the sleeping posture image. The bedding selection unit 23 performs processing to select bedding recommended for the user based on the estimated skeletal model.

The method of this embodiment allows a user's skeletal model to be estimated based on an image of the user's sleeping position. When sleeping, the body distorts due to environmental factors such as the pillow or mattress, so the skeletal model obtained based on the sleeping position image reflects the skeletal distortion caused by these environmental factors. Therefore, by using a skeletal model based on the sleeping position image, it is possible to provide appropriate advice regarding bedding, which is a factor that causes distortion of the body.

The image acquisition unit 21 may also acquire standing images of the user in a standing position. In this case, the model estimation unit 22 estimates a skeletal model of the user based on both the standing image and the lying image. Since a standing position is not affected by bedding such as pillows and mattresses, the skeletal model acquired based on the standing image reflects the basic body distortion of the target user. Therefore, by using both a skeletal model based on a standing image and a skeletal model based on a lying image, it is possible to distinguish whether a body distortion is a basic distortion or a distortion caused by environmental factors such as bedding, making it possible to accurately determine bedding. Details of the process will be described later.

Figure 6 is a diagram showing a detailed example configuration of the information processing device 20 according to this embodiment. The information processing device 20 may include an image acquisition unit 21, a model estimation unit 22, a bedding selection unit 23, a sleeping posture determination unit 24, a position change estimation unit 25, a sleep information acquisition unit 26, an evaluation processing unit 27, a notification processing unit 28, and a sharing processing unit 29. However, the configuration of the information processing device 20 is not limited to that shown in Figure 6, and various modifications are possible, such as omitting some components or adding other components. Note that the image acquisition unit 21, model estimation unit 22, and bedding selection unit 23 are the same as those in Figure 5, and therefore their description will be omitted.

The sleeping posture determination unit 24 determines whether the user's sleeping posture is one of a number of postures, including supine, right lateral, left lateral, and prone. The sleeping posture determination unit 24 may also determine the time ratio of each posture. Details of the time ratios will be described later. The sleeping posture determination unit 24 may be connected to the image acquisition unit 21 as shown in FIG. 6, and may determine the sleeping posture and time ratio based on images captured by the imaging device 300. Alternatively, although not shown in FIG. 6, the sleeping posture determination unit 24 may be connected to the sleep information acquisition unit 26, and may determine the sleeping posture and time ratio based on sensing data from the detection device 430, which will be described later.

The position change estimation unit 25 estimates the method used when the user changes position by turning over (hereinafter referred to as the position change method). The position change estimation unit 25 may estimate the position change method based on images captured by the imaging device 300, or may estimate the position change method based on sensing data from the detection device 430, which will be described later. The position change method will be described later using Figures 15 and 16.

The sleep information acquisition unit 26 acquires information related to the user's sleep. The sleep information acquisition unit 26 may perform processing to acquire sleep data based on sensing data from the detection device 430, which will be described later. The sleep data here may be the output of the detection device 430 itself, or may be information calculated by the sleep information acquisition unit 26 based on the output of the detection device 430.

The evaluation processing unit 27 calculates an index value that evaluates the effectiveness of the proposal made by the method of this embodiment. For example, when the bedding selection unit 23 suggests changing the bedding, the evaluation processing unit 27 evaluates the effectiveness of changing the bedding. Furthermore, when the evaluation processing unit 27 recommends a visit to a facility, it may evaluate the effectiveness of the visit.

The notification processing unit 28 performs processing to notify the user of the processing results of each unit of the information processing device 20. For example, the notification processing unit 28 may notify the user of information related to the bedding selected by the bedding selection unit 23. The notification processing unit 28 may also notify the user to visit a facility for medical treatment, or notify the user of advice related to sleep or daily life, based on the evaluation results of the evaluation processing unit 27. For example, the notification processing unit 28 may send a control signal to the terminal device 200. The control signal here is a signal that instructs the display unit 240, light-emitting unit, vibration unit, sound output unit, etc. of the terminal device 200 to output information according to the proposal content.

The sharing processing unit 29 performs processing to share the concerns of the user being processed and measures to resolve those concerns with other users. For example, when the concern of the target user is resolved, the sharing processing unit 29 may send information about the measures taken to other users. Furthermore, when the condition of other users who have the same concern as the target user is improved by some measure, the sharing processing unit 29 may notify the target user of the measure.

Furthermore, the output of the processing results of the information processing device 20 is not limited to be used for bedding notifications and information sharing, but may also be used for other feedback. For example, a user according to this embodiment may use an electric bed that can automatically change the height and posture of the bottom, an air mattress that can automatically adjust the hardness (thickness) of each part, a posture control pillow that encourages the user to turn over in bed by adjusting the thickness of each part, etc. The information processing device 20 may then control these electric bedding items in real time based on the processing results. For example, when the preferred pillow height, softness, mattress hardness, etc. are determined, the electric bed, air mattress, posture control pillow, etc. may be controlled based on this information.

Details of the processing performed by each part of the information processing device 20 will be described later along with specific examples.

Furthermore, some or all of the processing performed by the information processing device 20 (information processing system 10) of this embodiment may be realized by a program. The processing performed by the information processing device 20 may be processing executed by the processing unit 110 of the server system 100, processing executed by the processing unit 210 of the terminal device 200, or processing executed by a processor included in the sensing device 400. Furthermore, the processing performed by the information processing device 20 may be processing executed by two or more of the server system 100, the terminal device 200, and the sensing device 400.

The program according to this embodiment can be stored in, for example, a non-transitory information storage medium (information storage device), which is a medium readable by a computer. The information storage medium can be realized, for example, by an optical disc, memory card, HDD, or semiconductor memory. The semiconductor memory is, for example, ROM. The processing unit 110 and the like perform various processes of this embodiment based on the program stored in the information storage medium. In other words, the information storage medium stores programs that cause a computer to function as the processing unit 110 and the like. A computer is a device that includes an input device, a processing unit, a memory unit, and an output unit. Specifically, the program according to this embodiment is a program that causes a computer to execute each step described below using Figure 8 and the like.

The technique of this embodiment can also be applied to an information processing method including the following steps. The information processing method includes the steps of acquiring a standing image of a user in a standing position and a lying image of the user in a lying position, estimating a skeletal model of the user based on the standing image and the lying image, and performing a process of selecting bedding recommended for the user based on the estimated skeletal model.

2. Details of the Processing Next, the processing of this embodiment will be described in detail. First, the processing of selecting bedding based on a skeletal model will be described, followed by an example of using the time ratio of sleeping postures and a position change method. Furthermore, feedback using sleep information will also be described. Note that the following describes an example in which the information processing device 20 is a server system 100.

2.1 Skeleton Model Fig. 7 is a diagram summarizing the data flow in the processing of this embodiment. As shown in Fig. 7, in this embodiment, bedding evaluation values are analyzed based on the skeletal model, personal attribute information, and body position (sleeping posture) data, and measures related to the characteristics of the bedding are proposed based on the analysis results.

The skeletal model is calculated by the model estimation unit 22 based on the lying position image. The skeletal model may also be calculated based on the standing position image. The skeletal model may include information on the length of skeletal parts, as described below. Furthermore, the neck tilt angle and waist tilt angle may be calculated from the skeletal model, as described below using Figures 11A to 13C.

The individual's attribute information includes information such as age, gender, height, weight, and medical history. For example, the individual's attribute information may be displayed in a text box for input on the terminal device 200, and the target user may enter their own information in the text box. The terminal device 200 transmits the attribute information to the server system 100 via the communication unit 230.

The body position data is information indicating which of a number of positions the user is sleeping in, including supine, right lateral, left lateral, and prone, and is acquired by the sleep position determination unit 24. The information used by the sleep position determination unit 24 for determination may be an image, pressure information, acceleration, or a combination of these pieces of information. Details will be provided below.

The information processing device 20 analyzes the bedding evaluation value based on this information. For example, the bedding evaluation value is calculated by the bedding selection unit 23 based on the neck inclination angle and waist inclination angle.

The information processing device 20 then proposes measures based on the obtained bedding evaluation values. Specifically, the measures here are information about recommended bedding. The proposed information may be the height of the pillow 630, the hardness of the mattress 620, or other information. The processing flow is explained in detail below using flowcharts, etc.

Figure 8 is a flowchart illustrating an example of processing by the information processing device 20. Below, an example using both lying-down position images and standing-up position images will be described, but this is not required. For example, the information processing device 20 may perform processing based on lying-down position images, and the standing-up position images in the following description can be omitted. First, before the processing in Figure 8 begins, the imaging device 300 outputs a standing-up position image by capturing an image of the user in a standing position, and outputs a lying-down position image by capturing an image of the user in a lying-down position. Figures 9A to 9C are diagrams illustrating the relationship between the user and the imaging device 300 when capturing standing-up position images and lying-down position images.

For example, as shown in FIG. 9A, the imaging device 300 may be placed at a predetermined height and angle using a stand ST. A standing image is captured when the user stands in front of the imaging device 300. The imaging device 300 may be any device owned by the user according to this embodiment; for example, a terminal device 200 with imaging capabilities may be used as the imaging device 300. In this way, there is no need to prepare a dedicated device when capturing a standing image, etc., thereby reducing the burden on the user.

Similarly, as shown in Figures 9B and 9C, the imaging device 300 may be placed at a given height and angle near the side of a bed 610 or the like using a stand ST. A lying position image is acquired when the user lies down on the bed 610. The lying position image may be an image of the user in a supine position as shown in Figure 9B, or an image of the user in a lateral position as shown in Figure 9C, or both. The lateral position may be a right lateral position, a left lateral position, or both. Furthermore, an image of a prone position may also be captured as a lying position image.

Note that in this embodiment, it is sufficient to be able to capture standing position images and lying position images. The position, angle, and fixing method of the imaging device 300 are not limited to the examples shown in Figures 9A to 9C. For example, as shown in Figure 9D, multiple lying position images may be captured while changing the position of the imaging device 300. In the example of Figure 9D, the imaging device 300 rotates around an axis extending from the head to the feet, centered at the user's waist. For example, the imaging device 300 may stop at multiple points along its movement path to capture still images, or it may capture moving images while rotating. In this way, fewer parts of the body are not captured compared to using a fixed camera such as Figure 9B, thereby improving the accuracy of skeletal model estimation. Furthermore, Figure 9D shows an example in which the captured image 300 is moved to capture lying position images corresponding to the supine position. However, the captured image 300 may be moved to capture lying position images corresponding to other sleeping positions, such as the lateral position, or to capture standing position images.

In step S101 of FIG. 8, the image acquisition unit 21 acquires the standing position images and lying position images captured by the imaging device 300. For example, the image acquisition unit 21 is included in the processing unit 110 of the server system 100, and the processing unit 110 performs processing to receive the standing position images and lying position images from the imaging device 300 via the communication unit 130. Note that the server system 100 is not limited to being directly connected to the imaging device 300, and may be connected via another device such as a terminal device 200.

In step S102, the model estimation unit 22 estimates a skeletal model of the user based on the standing image and the lying position image. For example, the model estimation unit 22 may estimate the skeletal model of the user based on a trained model generated by machine learning. The trained model here is created based on training data in which a correct skeletal model is assigned as correct answer data for an input image. The correct answer data is assigned by a skeletal expert, such as a doctor. The trained model used in machine learning is, for example, a CNN (Convolutional Neural Network), and the trained model may be a CNN in which parameters (weights) are set by a learning process. The model estimation unit 22 may obtain a standing skeletal model by inputting a standing image into the trained model, and obtain a lying position skeletal model by inputting a lying position image into the trained model. Note that various methods for calculating skeletal models based on images are known, such as "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" (https://arxiv.org/pdf/1611.08050.pdf) and OpenPose disclosed by Zhe Cao et al., and these methods can be widely applied in this embodiment.

Figure 10 is an example of an estimated skeletal model. For example, the skeletal model may be information representing the respective positions of multiple pre-set body parts. In the example of Figure 10, the positions of body parts such as the head, neck, right shoulder, left shoulder, right elbow, left elbow, right hand, left hand, back, waist, right knee, left knee, right foot, and left foot are estimated. Note that the number and positions of the set body parts are not limited to the example of Figure 10, and various modifications are possible. Furthermore, as indicated by the arrows in Figure 10, the skeletal model may include information specifying the rotation state of rotatable body parts. For example, the skeletal model may include information on the rotation angle of the head. Having the skeletal model include rotation angle information makes it possible to specify, for example, the direction of the face when in a prone position.

For example, the model estimation unit 22 may perform a process to correct the lying-down skeletal model based on the standing skeletal model. For example, as shown in FIG. 9A, the standing skeletal model is a position in which the user's body is not affected by bedding such as pillows and mattresses. Therefore, the standing skeletal model calculated based on the standing image reflects the user's basic body distortion that is not caused by bedding. Therefore, by performing a correction process to remove the body distortion in the standing skeletal model from the body distortion in the lying-down skeletal model, it is possible to accurately calculate the body distortion caused by bedding, etc. Furthermore, as shown in FIG. 9A, in the standing skeletal model, the degree to which some parts of the body are occluded by other parts is lower than in the lying-down position, which increases the estimation accuracy of the skeletal model. Therefore, a parameter that is thought to change little between the standing and lying positions, such as the skeletal segment length, may be calculated based on the standing skeletal model, and the lying-down skeletal model may be corrected using this skeletal segment length. The skeletal segment length represents the distance between parts, and may represent, for example, the distance (edge length) between a given point and its adjacent point in FIG. 10.

Alternatively, the model estimation unit 22 may use two images, a standing image and a lying image, as input data and perform processing using a learned model that estimates one skeletal model based on the input data. For example, the training data used for learning may be data in which one skeletal model (more specifically, a skeletal model for lying position) is assigned as correct answer data for a pair of learning standing image and learning lying position image. The model estimation unit 22 then inputs the pair of standing image and lying position image into the learned model acquired from the training data to obtain a skeletal model. In this case, correct answer data that takes into account the relationship between standing position and lying position is assigned at the training data acquisition stage, and therefore the output of the learned model will also be a skeletal model that takes into account the relationship between standing position and lying position.

As shown in Figure 2, the method of this embodiment may also obtain a posture score that indicates whether the user's posture is good or bad. For example, the information processing device 20 (or, more narrowly, the model estimation unit 22) determines the degree of skeletal distortion based on the estimated skeletal model, and calculates a posture score that indicates the degree of distortion. The posture score here is numerical data that, for example, increases the smaller the distortion and decreases the larger the distortion. In this way, it becomes possible to objectively evaluate the user's posture.

In step S103, the sleeping posture determination unit 24 determines the sleeping posture of the target user. For example, the sleeping posture determination unit 24 may determine whether the user's sleeping posture is supine, right lateral, left lateral, or prone. For example, when capturing a sleeping posture image, the user may capture an image in their preferred sleeping posture. In this case, the sleeping posture determination unit 24 determines the user's sleeping posture using the sleeping posture image for determining a skeletal model.

Alternatively, an image of the sleeping user may be acquired separately from the sleeping posture image used to determine the skeletal model. The sleeping posture determination unit 24 may determine the user's sleeping posture based on this image. Alternatively, as described below with reference to FIG. 17 , a detection device 430 that detects information related to sleep may be used. For example, the detection device 430 may have a pressure sensor, and the sleeping posture determination unit 24 may determine the sleeping posture based on which area of the detection device 430 is under pressure and to what extent. In this embodiment, a motion sensor (not shown in FIG. 1 , etc.) may be used as the sensing device 400. The motion sensor is, for example, an acceleration sensor, but it may also be a gyro sensor. For example, the acceleration sensor may be attached to the user's skin or clothing. In this case, since the attachment position and direction of the motion sensor are known, the sleeping posture determination unit 24 can estimate the user's sleeping posture based on the detection result of the direction of gravity. In addition, various modifications are possible with regard to the information used for the determination by the sleeping posture determination unit 24.

In step S104, the bedding selection unit 23 evaluates the bedding currently used by the user based on the skeletal model and the results of the sleeping posture determination. For example, the bedding selection unit 23 may calculate a bedding evaluation value based on the angle of a given part of the estimated skeletal model. The bedding evaluation value may be a pillow evaluation value that is an evaluation value of the pillow 630, or a mattress evaluation value that is an evaluation value of the mattress 620, or both. Note that the bedding to be evaluated is not limited to the pillow 630 and mattress 620, and may also include the bed 610 (or, in a narrower sense, an electric bed).

Figures 11A to 11C are diagrams explaining a method for calculating pillow evaluation values in the supine position. Figure 11A shows a state in which the pillow 630 is at a desirable height. As shown in Figure 11A, when the pillow 630 is at an appropriate height, for example, the angle formed by the line connecting the neck and head of the skeletal model and the horizontal plane (hereinafter referred to as the neck inclination angle) is θ1 in Figure 11A. Note that instead of the horizontal plane here, the bottom surface of the bed 610 or the surface of the mattress 620 may be used.

Figure 11B shows a state in which the pillow 630 is too high, and Figure 11C shows a state in which the pillow 630 is too low. The neck tilt angles are θ2 and θ3, respectively. As is clear from the figure, θ3 < θ1 < θ2. Therefore, the bedding selection unit 23 may determine the neck tilt angle from the detection results of the skeletal model in the sleeping position, and determine as the pillow evaluation value an evaluation value that increases the closer the neck tilt angle is to θ2 and decreases the further it is from θ2. Note that the pillow evaluation value is not limited to numerical data, and may be two-level information indicating either "good" or "bad," or three-level information indicating "too low/appropriate/too high," or other information.

Figures 12A to 12C are diagrams illustrating a method for calculating mattress evaluation values in the supine position. Figure 12A shows a state in which the firmness of the mattress 620 is appropriate. As shown in Figure 12A, when the firmness of the mattress 620 is appropriate, for example, when considering three points on the skeletal model, namely the shoulders, waist, and hip joints, these three points are aligned in an approximately horizontal direction. For example, in this embodiment, the waist inclination angle may be a set of a first angle formed by the horizontal line connecting the hip joints and waist, and a second angle formed by the horizontal line connecting the waist and shoulders. In this case, in the example of Figure 12A, the first angle and the second angle are approximately 0 degrees. Note that here, as with Figures 11A to 11C, angles in the direction of increasing height in the direction toward the head are considered positive angles.

Figure 12B shows a state in which the mattress 620 is too soft. In the case of Figure 12B, the area from the buttocks to the back sinks in excessively compared to Figure 12A. As a result, the position of the lower back and shoulders becomes higher relative to the hip joints. For example, as in the above example, if the first angle and second angle are considered as the lower back inclination angle, at least the first angle will be larger than in the example of Figure 12A.

Figure 12C shows a state in which the mattress 620 is too hard. In the case of Figure 12C, the user sinks less deeply into the mattress 620 than in Figure 12A, and the spine is curved from the waist to the back. As a result, the first angle of the waist inclination angle is larger than in Figure 12A, and the second angle is smaller than in Figure 12A.

As described above, the lumbar inclination angle varies depending on the hardness of the mattress 620. Therefore, the bedding selection unit 23 calculates the mattress evaluation value based on the lumbar inclination angle. Specifically, the bedding selection unit 23 calculates the mattress evaluation value so that the value is large when the lumbar inclination angle is close to the angle shown in FIG. 12A and the value is small when the lumbar inclination angle approaches the angles shown in FIG. 12B or 12C. However, the method for calculating the mattress evaluation value based on the lumbar inclination angle is not limited to this. For example, in chiropractic treatment, a swayback (e.g., the posture shown in FIG. 12C) may be intentionally adopted, and the posture shown in FIG. 12C may be effectively used. Therefore, the bedding selection unit 23 may calculate the mattress evaluation value so that the value is large when the lumbar inclination angle approaches the angle shown in FIG. 12C. The same applies when the posture with the lower back lowered as shown in FIG. 12B is effective. The bedding selection unit 23 may calculate the mattress evaluation value so that the value is large when the lumbar inclination angle approaches the angle shown in FIG. 12C. For example, the bedding selection unit 23 may acquire information identifying a posture recommended for the target user. Based on this information, the bedding selection unit 23 may then switch between which of the waist inclination angles shown in Figures 12A to 12C the bedding evaluation value should be increased for. In this way, it becomes possible to calculate the bedding evaluation value from the perspective of whether it is easy for the target user to adopt the recommended posture. Note that information specifying the recommended posture may be included, for example, in the user's attribute information, which will be described later.

Furthermore, the above description focuses on the hardness of the mattress 620, with an example being described in which Figure 12A shows a desirable hardness, Figure 12B shows a soft state, and Figure 12C shows a state that is too hard. However, the characteristics of the mattress 620 are not limited to hardness, and the height may vary. Therefore, in this embodiment, the height of the mattress 620 (for example, the relationship between the heights of each position when the mattress 620 is divided into multiple parts) may be evaluated based on the lumbar inclination angle. For example, Figure 12A may correspond to a state in which the height of each part is appropriate, Figure 12B may correspond to a state in which the height of areas such as the waist and buttocks is too low, and Figure 12C may correspond to a state in which areas such as the waist and buttocks are too high. The fact that the height of the mattress 620 may also be evaluated applies to the following description.

Figures 13A to 13C are diagrams explaining a method for calculating mattress evaluation values in the lateral position. Figure 13A shows a state in which the firmness of the mattress 620 is appropriate. As shown in Figure 13A, when the firmness of the mattress 620 is appropriate, for example, the line connecting the waist, back, and neck of the skeletal model is approximately horizontal. For example, if the first angle and second angle are set as waist inclination angles, as in the example of Figures 12A to 12C, both angles will be close to 0 degrees.

Figure 13B shows a state in which the mattress 620 is too soft. In the case of Figure 13B, the area from the lower back to the back sinks excessively compared to Figure 13A. For example, the area from the lower back to the back sinks initially, and the height changes suddenly from the back to the lower back, so the first angle is smaller than in Figure 13A and the second angle is larger than in Figure 13A.

Figure 13C shows a state in which the mattress 620 is too hard. In the case of Figure 13C, the lower back and buttocks sink in too little, so the area from the lower back to the back sinks in relatively more. As a result, the first angle is smaller than in Figure 13A. When comparing Figure 13B and Figure 13C, the lower back and buttocks sink in Figure 13B, so if the angle between the line connecting the lower back and neck and the horizontal direction is taken as the third angle, the value of the third angle is relatively large. In contrast, in Figure 13C, the lower back and buttocks are lifted, so the third angle is relatively small. Therefore, for example, by taking the third angle into consideration, the bedding selection unit 23 can distinguish whether the mattress 620 is too soft or too hard.

As shown in Figures 13A to 13C, even in the lateral position, the lumbar inclination angle calculated from the skeletal model changes depending on the hardness of the mattress 620, so the bedding selection unit 23 calculates the mattress evaluation value based on this lumbar inclination angle. Specifically, the bedding selection unit 23 calculates the mattress evaluation value so that the value is large when the lumbar inclination angle is close to the angle shown in Figure 13A, and the value is small when the lumbar inclination angle is close to the angles shown in Figures 13B and 13C.

Although not explained here, when in the lateral position, the pillow evaluation value may be calculated based on the neck tilt angle.

Furthermore, the desirable neck tilt angle and waist tilt angle vary depending on the attributes of the target user, such as age, gender, height, weight, etc. Therefore, the bedding selection unit 23 may acquire the user's personal attribute information and calculate the bedding evaluation value based on the skeletal model, sleeping posture, and attribute information. For example, the bedding evaluation value may be calculated using a function whose variables are the neck tilt angle or waist tilt angle calculated based on the skeletal model and the user's attribute information. For example, the bedding selection unit 23 may calculate the bedding evaluation value by performing a correction process based on the individual's attribute information on the value calculated from the neck tilt angle or waist tilt angle.

The attribute information here may also include the user's medical history. For example, the attribute information may include the target user's illness, such as aspiration pneumonia or sleep apnea syndrome. For example, the information processing device 20 acquires data correlating illnesses with parameters used to calculate a bedding evaluation value. The bedding selection unit 23 may then determine the parameters to use to calculate a bedding evaluation value for the user based on the data and the user's illness. This makes it possible to calculate a bedding evaluation value that corresponds to the user's illness. For example, depending on the illness, a doctor may specify a sleeping position. Specifically, there may be cases where a patient with an illness such as sleep apnea syndrome is recommended to sleep in a lateral position to prevent snoring. In this case, the bedding selection unit 23 may prioritize information related to the lateral position over information related to other positions in the bedding evaluation value calculation process. The method of this embodiment makes it possible to calculate a bedding evaluation value that takes into account the impact of illness on the sleep state (sleeping position, etc.).

Returning to Figure 8, the explanation continues. In step S105, the bedding selection unit 23 determines whether the bedding currently being used by the user is appropriate based on the bedding evaluation value. For example, the bedding selection unit 23 determines that the bedding is appropriate if the bedding evaluation value is above a predetermined threshold, and determines that the bedding is inappropriate if the bedding evaluation value is below the threshold.

If the bedding selection unit 23 determines that the bedding is appropriate (step S105: Yes), this is output in step S107. For example, the server system 100 performs processing on the terminal device 200 used by the user to display text, an icon, or the like indicating that the bedding is appropriate.

If the bedding selection unit 23 determines that the bedding is inappropriate (step S105: No), in step S106 the bedding selection unit 23 performs a process of selecting bedding recommended for the user. In this embodiment, selecting bedding may be a process of selecting the characteristics of the recommended bedding, or a process of selecting a specific product that has those characteristics. The characteristics here include the height, hardness, shape, etc. of the bedding.

For example, because the bedding selection unit 23 determines the neck tilt angle when calculating the pillow evaluation value, if the pillow evaluation value is low, it can determine whether the pillow 630 is too high (closer to FIG. 11B) or too low (closer to FIG. 11C). Therefore, if the height is too high, the bedding selection unit 23 selects a pillow 630 that is lower than the current one, and if the height is too low, it selects a pillow 630 that is higher than the current one. The bedding selection unit 23 may perform a process to determine the appropriate pillow 630 height as part of the bedding selection process, or may obtain information identifying a product with a height close to the determined height, such as the product's manufacturer name or model number. For example, the memory unit 120 of the server system 100 may store bedding characteristic information that associates products with the characteristics of the products. The bedding selection unit 23 may perform a process to select bedding by comparing the characteristics determined based on the bedding evaluation value with the bedding characteristic information to determine a specific product recommended to the user. The same applies when the mattress 620 is the target; in the bedding selection process, the bedding selection unit 23 may determine the desired firmness of the mattress 620, or may obtain information on specific products with a firmness close to that desired firmness. The bedding selection process in this embodiment is not limited to the above, and various modifications are possible. Other examples of the bedding selection process will be described later using Figures 18A to 18G, etc.

After processing in step S106, a corresponding message is output in step S107. For example, the server system 100 outputs information indicating the bedding recommended by the bedding selection unit 23 to the terminal device 200 used by the user. For example, the display unit 240 of the terminal device 200 may display the desired pillow height, etc. Alternatively, if the bedding selection unit 23 recommends a specific product, link information to the product's manufacturer's web page, sales site web page, etc. may be output.

Note that although the above describes an example of proposing the height of the pillow 630 and the hardness of the mattress 620, as described above using Figure 7, bedding suggestions in this embodiment are not limited to this. For example, the bedding selection unit 23 may evaluate the hardness and shape of the pillow 630 and determine the hardness and shape recommended to the user based on the evaluation results. Furthermore, as will be described later using Figures 15 and 16, the mattress 620 may have different hardness in different parts. Therefore, the bedding selection unit 23 may evaluate the hardness distribution of the mattress 620 and determine the hardness distribution recommended to the user based on the evaluation results.

The recommended bedding may also include a comforter, as shown in FIG. 7. For example, the bedding selection unit 23 may evaluate the comforter and suggest to the user to change to the recommended comforter based on the evaluation results.

For example, the bedding selection unit 23 may evaluate, based on captured images, whether the user's limbs are protruding from the comforter, or whether the comforter is too large and causing the user to slip off the bed 610. Alternatively, a temperature sensor or humidity sensor may be provided around the bed 610 (or, more narrowly, in a position between the mattress 620 and the comforter when the user is sleeping), and the bedding selection unit 23 may evaluate the heat retention and moisture retention properties of the comforter based on the output of the temperature sensor or humidity sensor. Based on these evaluations, the bedding selection unit 23 may perform processing to suggest a comforter with a recommended size, heat retention, and moisture retention for the user. Note that, while the above describes an example of a user using the bed 610, users who use a mattress without using the bed 610 or users who use a mattress 620 directly on the floor may also be processed. The fact that users who do not use the bed 610 may also be processed applies to processes other than the process of calculating the comforter evaluation value.

2.2 Time Ratio, Posture Change Method, and Sleep Score Although the processing based on the skeletal model, sleeping posture, and attribute information has been described above, other information may also be used.

Figure 14 is a diagram summarizing the data flow in the processing of this embodiment. The example in Figure 14 differs from Figure 7 in that the position change method (turning over method) has been added to the personal attributes, and the sleep score and the proportion of time spent in each sleeping position have been added as sleep data. Each of these is explained below.

<Time ratio>
For example, the sleeping posture determination unit 24 of this embodiment may determine whether the user's sleeping posture is one of a plurality of postures, including supine, right lateral, left lateral, and prone, and may also calculate the time ratios of each of the plurality of postures. The time ratios here refer to, for example, the ratios of the time t1 spent in the supine position, the time t2 spent in the right lateral position, the time t3 spent in the left lateral position, and the time t4 spent in the prone position during one sleep session. For example, the time ratio for the supine position is t1/(t1+t2+t3+t4), the time ratio for the right lateral position is t2/(t1+t2+t3+t4), the time ratio for the left lateral position is t3/(t1+t2+t3+t4), and the time ratio for the prone position is t4/(t1+t2+t3+t4). However, various modifications are possible to the specific calculation method of the time ratios.

As mentioned above, the information used by the sleeping posture determination unit 24 to determine the sleeping posture can be modified in various ways. For example, the sleeping posture determination unit 24 may use pressure values from the detection device 430 or acceleration values from an acceleration sensor to determine the time series changes in sleeping posture during one sleep session, and calculate the time ratio from the length of time each posture is held. The sleeping posture determination unit 24 may also calculate the time ratio for a period that includes multiple sleep sessions, such as one week or one month.

The bedding selection unit 23 may perform a process of selecting bedding based on the time ratio. As can be seen from a comparison of Figures 12A to 12C (supine position) and Figures 13A to 13C (lateral position), the way the body sinks and the way the skeleton bends differs depending on the sleeping position, which may change the desirable characteristics of the bedding (height, hardness, etc.). In this regard, by taking the time ratio into consideration, it becomes possible to suggest bedding that is more suitable for the target user. For example, if a user spends a higher proportion of time in the supine position than in other positions, bedding suitable for the supine position is more likely to be recommended.

For example, the bedding selector 23 may calculate the pillow evaluation value using a function f1 shown in the following equation (1).
f1 = (neck tilt angle in supine position × time ratio in supine position × supine position coefficient + neck tilt angle in right lateral position × time ratio in right lateral position × right lateral position coefficient + neck tilt angle in left lateral position × time ratio in left lateral position × left lateral position coefficient + neck tilt angle in prone position × time ratio in prone position × prone position coefficient)
× personal attribute coefficient ... (1)

Here, the supine position coefficient, right lateral position coefficient, and left lateral position coefficient are given coefficients preset for each posture. The personal attribute coefficient is a coefficient set based on personal attribute information. When calculating the mattress evaluation value, the lumbar inclination angle may be used instead of the neck inclination angle. This makes it possible to take the time ratio of sleeping postures into account in the calculation process of the bedding evaluation value. However, the method using function f1 is one example of a method for calculating a bedding evaluation value taking time ratios into account, and the specific process is not limited to using function f1. For example, while the above formula (1) shows an example in which the value calculated from each posture is uniformly multiplied by the personal attribute coefficient, this is not limited to this. For example, the personal attribute coefficient may include multiple parameters that differ depending on the posture. In a broader sense, the above function f1 is a function whose variables are the neck inclination angle for each posture, the time ratio for each posture, the coefficient for each posture, and the personal attribute coefficient, and the specific formula is not limited to the above formula (1).

<Position change method>
6, the information processing device 20 may also include a position change estimation unit 25 that estimates a position change technique used by the user when changing position by turning over while sleeping. The bedding selection unit 23 may perform processing to select bedding based on the estimated position change technique. In this way, it becomes possible to select bedding that takes into account the way the user turns over in their sleep.

15A to 16C are diagrams explaining the relationship between position change techniques and recommended bedding. For example, if a user lying supine in the center of mattress 620 rolls over to change their position to a left lateral position, they may move to the left side of mattress 620 as seen from the user's perspective, as shown in FIG. 15A. Similarly, if a user lying supine in the center of mattress 620 rolls over to a right lateral position, they may move to the right side of mattress 620 as seen from the user's perspective, as shown in FIG. 15A. In the case of the position change shown in FIG. 15A, bedding such as mattress 620 and pillow 630 is used such that the center is used in the supine position and the left and right ends are used in the lateral position. Therefore, the bedding selection unit 23 may perform processing to suggest bedding with different characteristics in the center and ends for a user who uses a position change technique that changes position when rolling over. Note that the center and end portions here refer to positions along the shorter side of the mattress 620 (the left-right direction for a user lying on their back).

For example, the position change estimation unit 25 may detect a change in the position where the user's body pressure is detected based on the pressure value from the detection device 430. Then, if the amount of change in the position where the body pressure is detected in the short direction of the mattress 620 is equal to or greater than a threshold, the position change estimation unit 25 determines that the user's position change method is the method shown in FIG. 15A. Alternatively, the position change estimation unit 25 may determine the sleeping position based on the output of an acceleration sensor, and may obtain the displacement in the short direction of the mattress 620 when the sleeping position changes (when turning over) by integrating the acceleration value. If the displacement when turning over is equal to or greater than a predetermined threshold, the position change estimation unit 25 determines that the user's position change method is the method shown in FIG. 15A.

Figure 15B is a diagram showing the hardness distribution of a mattress 620 recommended for a user who uses the position change method shown in Figure 15A, and is a diagram of the mattress 620 observed, for example, from the same direction as Figure 15A. For example, as shown in Figure 15B, the recommended mattress 620 has characteristics suitable for the supine position, with the central part corresponding to the head and back having standard hardness, the lower back being slightly harder, and the feet being slightly softer. Furthermore, at the left and right ends, the mattress 620 has characteristics suitable for the lateral position, with the head part being slightly harder and the remaining parts being softer.

Figure 15C is a diagram showing the characteristics of a pillow 630 recommended for users who use the positioning technique shown in Figure 15A, and is a cross-sectional view showing the height distribution, for example, along the longitudinal direction of the pillow 630 (the short direction of the mattress 620 when placed on the mattress 620). As shown in Figure 15C, the recommended pillow 630 has a low center height that is suitable for the supine position, and high left and right end heights that are suitable for the lateral position.

In this way, by using bedding with different characteristics depending on the position in the short direction of the mattress 620, it is possible to propose bedding suitable for users who change positions using the method shown in Figure 15A.

Also, as shown in Figure 16A, some users do not change their position in the short direction of the mattress 620 when turning over, and instead change their position to, for example, a right-sided or left-sided position while remaining in the center of the mattress 620. In such cases, it is possible to adopt both supine and lateral sleeping positions in the same position.

For example, if the amount of change in the position where body pressure is detected in the short direction of the mattress 620 is less than a threshold, the position change estimation unit 25 determines that the user's position change method is the method shown in FIG. 16A. Alternatively, if the displacement when turning over is less than a predetermined threshold, the position change estimation unit 25 determines that the user's position change method is the method shown in FIG. 16A.

Figure 16B shows the hardness distribution of a mattress 620 recommended for a user using the position change method shown in Figure 16A, and is a diagram of the mattress 620 observed, for example, from the same direction as Figure 16A. For example, as shown in Figure 16B, the recommended mattress 620 has the following characteristics, regardless of the position in the short direction: the head section is relatively hard, the neck to back section is of standard hardness, the lower back section is relatively hard, and the foot section is relatively soft. As mentioned above, in this case, the position is unlikely to change between the supine and lateral positions, so a mattress 620 with characteristics that are not dependent on the position in the short direction is effective. However, in the position change method shown in Figure 16A, the characteristics of the bedding can be dynamically changed depending on the position.

Figure 16C is a diagram showing the characteristics of a pillow 630 recommended for users who use the position change method shown in Figure 16A, and is a cross-sectional view showing the height distribution along the longitudinal direction of the pillow 630, for example. As shown in Figure 16C, the recommended pillow 630 is set to a low height regardless of position.

In this way, by using bedding with consistent characteristics regardless of the position in the short direction of the mattress 620, it is possible to propose bedding suitable for users who change positions using the method shown in Figure 16A, and it is also possible to reduce the cost of the bedding.

When the position change method is taken into consideration, the bedding selection unit 23 may calculate the pillow evaluation value using a function f2 shown in the following equation (2).
f2 = (neck tilt angle in supine position × time ratio in supine position × supine position coefficient + neck tilt angle in right lateral position × time ratio in right lateral position × right lateral position coefficient + neck tilt angle in left lateral position × time ratio in left lateral position × left lateral position coefficient + neck tilt angle in prone position × time ratio in prone position × prone position coefficient)
× personal attribute coefficient × position conversion coefficient … (2)

Here, the position change coefficient is a coefficient whose value is determined based on the position change method. For example, for a user performing the position change shown in FIG. 15A, a first value is set as the position change coefficient, and for a user performing the position change shown in FIG. 16A, a second value different from the first value is set as the position change coefficient. Note that everything except the position change coefficient is the same as in equation (1) above. Similarly, when calculating the mattress evaluation value, the waist inclination angle is used instead of the neck inclination angle, which is also the same as equation (1) above. For example, as shown in equation (2) above, the time ratio of each position may be used to calculate the bedding evaluation value. For example, when a position change is performed between the supine position and the lateral position, in addition to the position change method, processing may be performed based on which position has a higher time ratio. For example, a determination may be made that prioritizes the position with a higher time ratio between the supine position and the lateral position. Furthermore, in this embodiment, the degree of deviation from the desired position may be determined for each of the supine position and the lateral position. The degree of imbalance may be determined from the neck tilt angle or waist tilt angle, as described above with reference to Figures 12A to 13C, or may be determined using other parts of the skeletal model. The bedding selection unit 23 may then prioritize the supine or lateral position, whichever is more imbalance in posture. Furthermore, the postural change coefficient is not limited to a uniform multiplication by a value determined from each posture, as in equation (2) above, but may include different parameters for each posture. In a broader sense, the above function f2 is a function whose variables are the neck tilt angle for each posture, the time ratio for each posture, the coefficient for each posture, the personal attribute coefficient, and the postural change coefficient, and the specific formula is not limited to equation (2) above. Various other variations on the method for calculating the bedding evaluation value are possible.

For example, the bedding selection unit 23 may calculate a bedding evaluation value using function f2 in equation (2) above, and if the bedding evaluation value is less than a threshold, perform processing to suggest bedding shown in Figures 15B, 15C, 16B, 16C, etc. depending on the position change method. In this case, as described above using Figures 11A to 13C, processing may be performed to adjust the hardness of each part based on the neck tilt angle and waist tilt angle of the current bedding. Furthermore, if the user is using bedding that allows the hardness and height of each part to be adjusted, such as an air mattress or a firmness adjustment sheet, the bedding selection unit 23 may control the air mattress or the like according to the determined bedding characteristics. In this way, if the characteristics of the current bedding differ from the characteristics of the desired bedding, feedback can be provided in real time to reduce the difference.

<Sleep data>
As shown in Fig. 14, sleep data, which is information related to the user's sleep, may be used to select bedding. Fig. 17 is a diagram illustrating an example of a detection device 430 that outputs sleep data and is placed on the bottom of a bed 610. As shown in Fig. 17, the detection device 430 is, for example, a sheet-like or plate-like device that is placed between the bottom of the bed 610 and a mattress 620. The detection device 430 is a device that senses information related to the user's sleep. The detection device 430 includes a pressure sensor that outputs a pressure value.

When the user gets into bed, the detection device 430 detects the user's body vibrations (body movements, vibrations) through the mattress 620. Based on the body vibrations detected by the detection device 430, information regarding the breathing rate, heart rate, activity level, posture, wakefulness/asleep, and whether the user is out of bed or in bed can be obtained. For example, the periodicity of body movements may be analyzed, and the breathing rate and heart rate may be calculated from the peak frequency. The periodicity may be analyzed using, for example, a Fourier transform. The breathing rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time may be, for example, one minute. Alternatively, body vibrations may be detected per sampling unit time, and the number of detected body vibrations may be calculated as the activity level. When the user gets out of bed, the detected pressure value decreases compared to when the user is in bed. Therefore, whether the user is out of bed or in bed can be determined based on the pressure value and changes over time. Furthermore, when the user is in bed and the activity level is equal to or greater than a predetermined level, the user may be determined to be in an awake state, and when the user is in bed and the activity level is less than a predetermined level, the user may be determined to be in a sleeping state. The detection device 430 may also subdivide sleep states to determine whether the sleep state is non-REM or REM sleep, and the depth of sleep.

The sleep data used in the processing of the bedding selection unit 23 broadly includes information related to sleep/wake-up, and may be pressure values from a pressure sensor, or information related to respiratory rate, heart rate, activity level, posture, awake/asleep, and getting out/staying in bed.

For example, the bedding selection unit 23 may obtain a sleep score that indicates the quality of sleep by determining the sleep time and depth of sleep from the output of the detection device 430. The sleep score is, for example, numerical data that has a high value when the user is getting good quality sleep. By using the sleep score in processing, it is possible to take into account the quality of sleep with the target bedding, making it possible to suggest bedding that will provide the user with good quality sleep.

For example, when taking the sleep score into consideration, the bedding selector 23 may calculate the pillow evaluation value using a function f3 shown in the following equation (3).
f3 = (neck tilt angle in supine position × time ratio in supine position × supine position coefficient + neck tilt angle in right lateral position × time ratio in right lateral position × right lateral position coefficient + neck tilt angle in left lateral position × time ratio in left lateral position × left lateral position coefficient + neck tilt angle in prone position × time ratio in prone position × prone position coefficient)
× personal attribute coefficient × position change coefficient × sleep coefficient … (3)

Here, the sleep coefficient is a coefficient whose value is determined according to the sleep score. Furthermore, all other factors are the same as in equation (2) above. Furthermore, when calculating the mattress evaluation value, the lumbar inclination angle is used instead of the neck inclination angle, which is also the same as equations (1) and (2) above. Note that while the sleep coefficient has been described here as information that is different from the other coefficients, the sleep coefficient may also be included in the personal attribute coefficient, and various modifications can be made to the specific process for calculating the bedding evaluation value.

<Others>
As shown in FIG. 2 , the information processing device 20 of this embodiment may calculate a pain/distress score that indicates the level of pain felt due to factors such as stiff shoulders or lower back pain, and how much the factors are bothering the user. For example, the information processing device 20 may accept a user input indicating which of the pain-causing events that may be experienced daily resembles the pain caused by stiff shoulders or lower back pain, and calculate a pain score that quantifies the pain based on the user input. Alternatively, when using a dedicated device, a state in which the pain caused by passing an electric current is comparable to the pain caused by stiff shoulders or lower back pain may be searched for, and the pain score may be calculated based on the current value in that state. Furthermore, the information processing device 20 may accept a user input indicating the extent to which various activities in daily life are hindered by stiff shoulders or lower back pain, and calculate a distress score that quantifies the distress based on the user input.

In the method of this embodiment, for example, the pain score or distress score may be used to suggest measures. For example, the information processing device 20 may suggest changing the bedding more actively the higher the pain score or distress score. For example, the pain score or distress score may be used in the calculation process of the bedding evaluation value, or a threshold value to be compared with the bedding evaluation value may be set based on the pain score or distress score. Furthermore, as described below, when recommending a visit to a specialist facility, the information processing device 20 may recommend a visit to a specialist facility more actively the higher the pain score or distress score.

2.3 Specific Example of Bedding Selection Process A specific example of the bedding selection process in this embodiment will be described. A possible reason for a low bedding evaluation value being calculated in the method of this embodiment is that body distortion occurs due to sleeping position. Furthermore, if distortion occurs, it may cause a bias in sleeping position, further reducing the bedding evaluation value. For example, the bedding selection unit 23 may perform a process of selecting (1) bedding that makes it difficult to adopt a sleeping position that promotes distortion, (2) bedding that induces a change in position, or (3) bedding that corrects distortion. Specific examples will be described below.

First, we will explain the process of selecting bedding that makes it difficult to assume a sleeping position that promotes distortion. For example, consider a user who tends to assume a position with their head turned to the right when in the prone position. This sleeping position involves a significant twisting of the neck, and continuing this sleeping position is likely to promote distortion of the body. Therefore, the bedding selection unit 23 may suggest the use of specific bedding for a user who has a low bedding evaluation value, spends more than a predetermined amount of time in the prone position, and is determined to have a high proportion of their face facing a predetermined direction based on a skeletal model while in the prone position. The bedding in this case may be, for example, a pillow 630 with a central ventilation hole. By using such a pillow 630, breathing is not obstructed by the pillow 630, making it easier to assume a position with the head facing forward when in the prone position. As a result, it is possible to prevent users from assuming positions that are prone to distortion.

Furthermore, if it is determined that there is a large bias in sleeping positions, the bedding selection unit 23 may perform processing to select bedding that makes it difficult to assume a sleeping position with a high proportion of time spent in that position. For example, suppose a user only assumes a right lateral position due to body distortion. In this case, continuing to sleep in that position may worsen the body distortion. Therefore, the bedding selection unit 23 may suggest the use of specific bedding for a user who is determined to have a low bedding evaluation value and spends more than a predetermined amount of time in the right lateral position. Note that this determination may be made based on the difference or ratio between the proportion of time spent in the right lateral position and the proportion of time spent in another sleeping position (for example, the left lateral position), and the specific processing content can be modified in various ways.

18A shows an example of bedding that makes it difficult to assume a right lateral position. The bedding in this case may be, for example, a mattress 620 with different heights in the short direction (left-right direction). In the example of FIG. 18A, the right side of the mattress 620, as seen from the user's perspective when sleeping, is set higher than other areas. Using such a mattress 620 makes it difficult for the user to maintain a position facing right, thereby encouraging a sleeping position other than the right lateral position. Note that while FIG. 18A illustrates a mattress 620 with different heights in the short direction, bedding that makes it difficult to assume a specific position is not limited to this. For example, a mattress 620 may be used that intentionally makes the side that the user does not want to face uncomfortable by adjusting the shape, material, texture, etc. of the mattress 620. For example, using a mattress 620 with a protrusion on the right side as seen from the user's perspective when sleeping, or a mattress 620 made of a rough-textured material, can also encourage a sleeping position other than the right lateral position.

Next, we will explain bedding that induces positional changes. Note that the bedding proposed in this embodiment is not limited to static bedding with fixed characteristics. For example, bedding with dynamically changeable characteristics may be used. Dynamic bedding includes a bed 610 with a changeable bottom angle, a mattress 620 with a changeable shape, a pillow 630, etc. Dynamic bedding may also include bedding that provides physical stimulation to the user using vibration, sound, light, scent, etc.

The bedding selection unit 23 may select the characteristics of bedding that are desirable for the user based on the above-described processing, and may control the dynamic bedding to match those characteristics. For example, if it is known that a particular sleeping position is undesirable, the bedding selection unit 23 may control the dynamic bedding to induce a change in position if the undesirable position continues for a predetermined period of time.

18B and 18C are diagrams illustrating an example of dynamic bedding control. Here, an example is described in which the user is encouraged to change their position to the right lateral position in order to eliminate an undesirable posture. For example, as shown in FIG. 18B, the bedding here is a shape-adjustable mattress 620, and the bedding selection unit 23 may control the left side of the mattress 620 to be higher than the right side. In this way, the difference in height and inclination make it easier for the user to change their position to the right lateral position. Alternatively, as shown in FIG. 18C, the bedding here may be a bed 610 with a changeable bottom angle and a shape-adjustable mattress 620. The bedding selection unit 23 controls the bottom of the bed 610 to tilt so that the right side is lower, and controls the right side of the mattress 620 to raise to create a flat section. In this case, too, the inclination of the bed 610 relatively lowers the height of the user's right side, making it easier for the user to change their position to the right lateral position. Furthermore, by creating a flat section on the mattress 620, it is possible to prevent the user from falling off the bed 610. In this case, from the perspective of further encouraging the user to change position to a specific sleeping position, control may be performed to apply a stimulus to the side to which movement is not desired. For example, when encouraging the user to change position to the right lateral position, the bedding selection unit 23 may perform control to generate vibrations, sounds, lights, etc. on the left side of the mattress 620.

Next, we will explain bedding that corrects distortions. For example, in fields such as osteopathy, the body may be forced to assume a specific posture in order to correct distortions. Figures 18D and 18E show examples of bedding used in this case. The bedding in this case may be a mattress 620 with different heights depending on the area. The mattress 620 in this case may be a static bedding with a fixed height, or a dynamic bedding.

Figure 18D shows the mattress 620 as viewed from the side, and Figure 18E shows the cross-sectional shape of the mattress 620 in the dashed line area (buttocks region) of Figure 18D as viewed from the user's head. For example, as shown in Figure 18D, the height of the mattress 620 in the area corresponding to the user's buttocks is relatively higher than other areas, and as shown in Figure 18E, the right side of this area is relatively higher than the left side. In this case, the user lifts their buttocks and assumes a posture in which their body is slightly twisted to the left at the buttocks. This posture may cause distortion in the body, but by shifting the direction of the distortion in the opposite direction to the existing distortion (a direction that suppresses distortion), it is possible to correct the distortion. As can be seen from the above explanation, the characteristics of bedding that correct distortion may be determined based on the distortion actually occurring in the user, and the specific shape, etc., can be modified in various ways. Additionally, a traction device or rubber may be installed on the bed board of the bed 630, and the traction device or rubber may be connected to a specific part of the user's body, allowing the user to adopt a posture that corrects distortion. Therefore, the bedding selection process of this embodiment may include a process that suggests the use of such a traction device or the like.

Furthermore, while the above describes the process of selecting bedding to address distortions and sleeping posture imbalances, the method of this embodiment is not limited to this. For example, in this embodiment, a process may be performed to suggest a chair or desk to correct distortions. In this way, it becomes possible to adjust furniture that affects posture during the day based on the results of the bedtime assessment. For example, the bedding selector 23 may perform a process to select the angle of inclination of a chair or the height of a desk. Furthermore, recommendations for exercise or stretching, or recommendations for chiropractic clinics, osteopathic clinics, clinics, etc. may be made to address distortions. These recommendations will be described later.

In addition, in this embodiment, not only can the device address body distortion, which is a relatively long-term effect of sleeping posture, but also stiff necks, which are a relatively short-term effect. For example, if the user adopts an unnatural sleeping posture, the user may experience pain the next day, and the bedding selector 23 may execute processing to suppress this. This processing is, in a narrow sense, dynamic bedding control. For example, the bedding selector 23 may encourage the user to change their sleeping posture if the user's sleeping posture is determined to be poor. For example, the bedding selector 23 may determine that the sleeping posture is poor if the sleeping posture is biased toward the edge of the bed 610 (mattress 620) and vital signs (e.g., sleep data) are below a predetermined threshold. The sleeping posture change may be achieved by the control described above using Figures 18B and 18C, and may involve control of any of the bed 610, mattress 620, and pillow 630, or may include output of vibration, sound, light, etc.

Figures 18F and 18G are diagrams showing examples of bedding control to prevent stiff necks. An unnatural sleeping position here may be, for example, a prone position with the head turned to the side. This position not only causes distortion of the body but can also be a cause of stiff necks. For example, as shown in Figure 18F, the bedding selection unit 23 may increase the height of one side of the mattress 620 in the left-right direction compared to the other side. This makes it possible to encourage the user to change their sleeping position to a position other than prone. Even if the user does not change their position, tilting the body angle reduces the rotation angle of the head relative to the body, thereby preventing stiff necks. Alternatively, the bedding selection unit 23 may control at least one of the bed 610 and the mattress 620 to lower the height of the user's head compared to other parts. Figure 18F illustrates an example of control to lower the height of the mattress 620. This makes it easier for the user to face their head forward when in a prone position, as breathing is less likely to be obstructed by the bedding.

2.4 Feedback In the method of this embodiment, the user may be given feedback on whether the proposed measures are effective or not.

For example, as shown in FIG. 6, the information processing device 20 of this embodiment may include an evaluation processing unit 27 that calculates a bedding effect index that indicates the effect of the bedding selected by the bedding selection unit 23 based on the sleep data. The sleep data represents the sensing results regarding the user's sleep obtained by a sleep sensor, and may be the output of the sleep sensor itself, or information calculated by the information processing device 20 based on the output of the sleep sensor.

The sleep sensor here refers to, for example, the detection device 430 described above with reference to Figure 17, but other devices capable of sensing information related to the user's sleep may also be used. For example, a wristwatch-type device including an acceleration sensor or a photoelectric sensor that detects pulse may be used as the sleep sensor. Furthermore, the sleep data may be, for example, the sleep score that indicates the quality of sleep described above, but may also include other information related to sleep. For example, the sleep data may include information on respiratory rate, heart rate, activity level, posture, awake/asleep, and getting out/being in bed, as described above.

For example, the evaluation processing unit 27 may perform a process to compare the user's sleep score before introducing the bedding with the user's sleep score after introducing the bedding proposed by the information processing device 20. The evaluation processing unit 27 then determines that the introduced bedding is effective if the quality of sleep represented by the sleep score has improved, and determines that the introduced bedding is ineffective if the quality has not changed or has deteriorated. For example, the evaluation processing unit 27 calculates the degree of improvement in the sleep score (difference, ratio, etc.) as a bedding effectiveness index. In this way, it is possible to determine whether the bedding proposed by the information processing device 20 was effective for the user, thereby enabling appropriate feedback to be provided to the user. In particular, the use of the sleep score allows the effectiveness of the bedding to be objectively evaluated. Furthermore, the use of the sleep score allows the calculation of the bedding effectiveness index to be automated, thereby reducing the burden on both the service provider and the user of the service.

Specific feedback methods can be considered in various ways. For example, if the evaluation processing unit 27 determines that the effectiveness of the bedding is below a predetermined level, the bedding selection unit 23 may perform at least one of the following processes: adding input information used to select bedding, changing the bedding selection algorithm that selects bedding based on the input information, and adding types of bedding to be selected.

As described above, the bedding selection unit 23 of this embodiment can use various information, such as a skeletal model, sleeping position, personal attributes, time ratio, position change method, and sleep score, in processing related to bedding (or, more narrowly, processing to calculate bedding evaluation values). Therefore, the bedding selection unit 23 may first perform processing to select bedding using a relatively small number of types of information, and then perform processing to add new information if the effectiveness of the bedding selected by this processing is determined to be low. For example, the bedding selection unit 23 may perform processing based on a skeletal model, sleeping position, and personal attributes by default, and add at least one of the time ratio, position change method, and sleep score if the effectiveness of the proposed bedding is low. This reduces the processing load by default, while improving processing accuracy by adding information if the proposal is not sufficiently effective. As a result, it is possible to increase the effectiveness of changing bedding.

Furthermore, the bedding evaluation value calculation process can use, for example, the above-mentioned functions f1 to f3. In this case, various settings are possible for the supine position coefficient, right lateral position coefficient, left lateral position coefficient, and prone position coefficient. By adjusting these values, it is possible to adjust which sleeping position is emphasized from a perspective other than the time ratio. The magnitudes of the personal attribute coefficient, position change coefficient, sleep coefficient, etc. can also be changed. For example, if the calculation algorithm is changed so that the personal attribute coefficient is more likely to increase, it is possible to perform processing that emphasizes personal attribute information over the position change method and sleep score. The same applies to the other two coefficients. Therefore, for example, the bedding selection unit 23 may change the values of these coefficients as a way to change the bedding selection algorithm. Furthermore, the bedding selection unit 23 may use functions other than the above-mentioned f1 to f3 as functions for calculating the bedding evaluation value. As described above, by adjusting the algorithm used in the bedding evaluation value calculation process, etc. for each user, it is possible to improve processing accuracy. As a result, it is possible to increase the effectiveness of bedding changes.

The bedding selection unit 23 may also change the type of bedding to be selected. For example, by default, the bedding selection unit 23 may only calculate the evaluation value and suggest changes for the pillow 630, which is easy to change, and omit processing for the mattress 620 and comforter. If the proposed pillow 630 is not sufficiently effective, the bedding selection unit 23 may start processing for the mattress 620 in addition to the pillow 630. In this way, the number of types of bedding suggested increases, making it possible to improve the effectiveness of changing bedding. Note that the order in which the target bedding is added is, for example, pillow 630, mattress 620, and comforter, but this is not limited to this and various modifications are possible.

As another feedback method, the bedding selection unit 23 may accept input of an evaluation value for the bedding by the user when the evaluation processing unit 27 determines that the effectiveness of the bedding is below a predetermined level, or when a bedding effectiveness index cannot be calculated. For example, the bedding selection unit 23 may accept input of an evaluation value for each of multiple evaluation items related to the bedding by the user.

For example, the bedding selection unit 23 may output information to the terminal device 200 instructing it to display a questionnaire screen regarding bedding. The terminal device 200 then accepts user input on the questionnaire screen and transmits the user input to the bedding selection unit 23. The evaluation items here are, for example, items provided for each piece of bedding, such as the pillow 630, mattress 620, and comforter, and the terminal device 200 may accept a selection input for each item, indicating whether the comfort is "good/average/bad." The evaluation items are not limited to these and may include, for example, the height, hardness, and shape of the pillow 630, the hardness and distribution of the mattress 620, and the heat retention, moisture retention, and size of the comforter, as shown in the countermeasure proposal in Figure 7. In this case, for example, the questionnaire screen may display options such as "too low, appropriate, or too high" for each evaluation item and accept any of the selection inputs. The evaluation items are not limited to these and various variations are possible.

In this way, it becomes possible to obtain the user's subjective opinion on bedding. As a result, it becomes possible to propose bedding that better matches the user's preferences. Furthermore, as described above, the input of evaluation values may be accepted when a bedding evaluation index cannot be calculated. For example, it is difficult to calculate a bedding effectiveness index for a user who has not installed a sleep sensor such as the detection device 430, or for a user who has installed a sleep sensor but is unable to obtain sensing data due to some error. In this way, even when it is difficult to automatically determine the effectiveness of bedding, by prompting the user for input, it becomes possible to appropriately obtain feedback on the proposed bedding.

Furthermore, in the method of this embodiment, if a bedding recommendation alone is insufficient, a consultation at a specialized facility may be suggested. For example, as shown in FIG. 6, the information processing device 20 may include a notification processing unit 28 that outputs information to the user recommending a consultation at a specialized facility including at least one of a chiropractic clinic, an osteopathic clinic, and a clinic when the evaluation processing unit 27 determines that the effectiveness of the bedding is below a predetermined level. Here, a chiropractic clinic is a facility that provides private therapies such as massage. An osteopathic clinic is a facility where medical treatment is performed by a specialist such as a judo therapist. A clinic is a facility where medical treatment is performed by a specialist such as an orthopedic surgeon. However, the specialized facilities in this embodiment are not limited to these and may include other facilities used to relieve user concerns such as stiff shoulders, back pain, and headaches.

In this manner, the method of this embodiment first attempts to change bedding at home, and if this does not produce sufficient results, a visit to a specialized facility is recommended. As described above with reference to FIG. 2, the method of this embodiment may also use information such as the type of facility, its location, and the individual's residential area. In this case, the information processing device 20 may perform processing to select a specific specialized facility to recommend a visit to based on the distance from the target user's residential area and the facility's type. This method of this embodiment enables smooth collaboration with specialized facilities. Furthermore, since bedding change takes priority over a visit to a facility in this embodiment, in cases where a bedding change can be sufficient, less-necessary visits to a facility can be avoided. Furthermore, when a visit to a specialized facility is made, previously attempted measures, such as bedding change, are known, making it possible to efficiently determine the treatment details at the facility. For example, the bedding selection unit 23 may perform processing to output a report including a bedding change history and the resulting changes in sleep scores. Using this report facilitates smooth communication between the user and the facility. For example, the report may contain information equivalent to a pre-medication interview with the user. Furthermore, medical examinations at facilities are not limited to outpatient visits, and remote medical examinations may also be conducted.

The method of this embodiment may also determine the effect of visiting the specialized facility. For example, the evaluation processing unit 27 may calculate a facility effect index that represents the effect of the specialized facility based on sleep data before and after the examination at the specialized facility.

For example, the evaluation processing unit 27 calculates the degree of improvement in the sleep score (difference, ratio, etc.) as a facility effectiveness index. In this way, it is possible to determine whether the facility visited was effective for the user, thereby enabling appropriate feedback to be provided to the user. For example, if the information processing device 20 determines that visiting a specialized facility was not effective, it may suggest changing to another specialized facility.

For example, the information processing device 20 may calculate a facility score and a visit score, as shown in Figure 2. The facility score is a numerical representation of the effect of visiting a specialized facility for each facility. For example, the facility score may be the average or median of the facility effect index calculated for that facility. The visit score is a numerical representation of the effect of an individual visit, and may be, for example, the facility effect index itself. Note that, as mentioned above, if outpatient and remote medical consultations are selectable, a visit score may be calculated for each.

In this way, the information processing device 20 of this embodiment can suggest appropriate specialized facilities and specific consultation details to the user by quantifying and evaluating specialized facilities and consultation details. For example, when recommending a consultation at a specialized facility, the information processing device 20 may perform a process of selecting a recommended specialized facility based on the facility score and consultation score.

In addition, in this embodiment, suggestions other than bedding selection and visiting a specialist facility may be made. For example, if the evaluation processing unit 27 determines that the effectiveness of the bedding is below a predetermined level, the notification processing unit 28 of the information processing device 20 may output information to the user recommending that they start monitoring using at least one of an activity meter and a vital sensor. Alternatively, the information processing device 20 may acquire the user's snoring level and sleep index (for example, the sleep score described above), and determine whether to start monitoring based on this information.

The activity meter here refers to a sensor device that calculates the amount of activity, which is information indicating the level of a user's activity. The activity meter may be the detection device 430 described above using FIG. 17, or a device including a motion sensor such as an acceleration sensor. The information acquired based on the activity meter may be the number of steps measured as the user walks, exercise intensity, or other information indicating the level of the user's activity. Exercise intensity is, for example, numerical data in units of METs, but other information may also be used. The vital sensor is a device that detects the user's vital information, and may be the detection device 430, a device including a motion sensor such as an acceleration sensor, or a device including a photoelectric sensor that detects pulse rate, blood oxygen saturation, etc. Vital information is information that indicates the state of the user's biological activity, such as pulse (heart rate), respiratory rate, blood pressure, and body temperature.

This makes it possible to monitor the user's condition in their daily life. This means that it will be possible to try to alleviate pain and worries by improving lifestyle habits, rather than relying on bedding or specialized facilities.

Furthermore, if the evaluation processing unit 27 determines that the effectiveness of the bedding is below a predetermined level, the notification processing unit 28 may output advice information to the user including at least one of advice regarding the timing of caffeine intake, bathing, and urination. For example, the notification processing unit 28 may push this advice information to the terminal device 200 at the corresponding timing. In this way, it becomes possible to present specific lifestyle habits to the user.

Figure 19 is a diagram illustrating the flow of data related to advice. For example, the information processing device 20 may acquire sleep data using the sleep information acquisition unit 26. The sleep data is information acquired from, for example, the detection device 430, and includes bedtime, wake-up time, sleep duration, time to fall asleep, and time to wake up during sleep. The time to go to bed may be the time to stay in bed, the time to transition to a sleeping state, or both. The time to wake up may be the time to get out of bed, the time to transition to a waking state, or both. Sleep duration is the time during a day when a person is determined to be asleep. The time to fall asleep is the time from staying in bed to transitioning to a sleeping state. The time to wake up during sleep is the total time spent awake between going to bed at night and waking up the next morning. Note that when targeting workers who work night shifts, the calculation period for time to wake up during sleep may be set to daytime hours. Furthermore, the sleep data is not limited to the above examples and may include other information, such as the number of times a person wakes up during sleep per day. Furthermore, this sleep data is not limited to data from a single sleep session, but may be acquired continuously over a period corresponding to multiple sleep sessions.

The information processing device 20 may obtain analytical data by performing an analysis based on the sleep data. The analytical data here may include daytime behavior, behavior before going to bed, and sleeping habits. For example, when the detection device 430 is used, daytime behavior may be whether or not the person takes a nap (goes to bed during the day). Pre-bed behavior may be data such as whether the person spends a long time in bed 610 before going to sleep or falls asleep immediately after getting into bed 610, based on data from when the person becomes in bed until when the person goes to sleep. Sleeping habits may also be statistical data such as the above-mentioned bedtime and wake-up time. Sleeping habits may also include information such as the proportion of time spent in each sleeping position and the number of times the person turns over in sleep.

Furthermore, the analysis data here is not limited to data obtained based on information from the detection device 430, but may be obtained using other sensors, or may be obtained from the results of a questionnaire given to the user, etc. For example, information such as whether or not the user exercised may be obtained as information on how the user spent their day. Furthermore, information on the timing of urination, the timing of caffeine intake, the timing of bathing, etc. may be obtained as information on how the user spent their time before going to bed.

The information processing device 20 may then output lifestyle advice based on the sleep data and analysis data. Lifestyle advice may include advice to encourage urination before bedtime, advice to encourage going to bed only after feeling sufficiently sleepy, advice to encourage bathing at a predetermined time before sleep, advice to encourage limiting caffeine intake after dinner, etc. For example, if the sleep state determined from the sleep data (which may be the level of the sleep score described above, for example) is determined to be below a predetermined level, and undesirable behavior regarding urination, going to bed, bathing, or caffeine is observed, the information processing device 20 may send a push notification of the corresponding advice.

For example, the information processing device 20 may estimate a standard bedtime based on sleeping habits, and send a push notification encouraging urination a predetermined time before that bedtime. The information processing device 20 may also estimate a first time to get into bed 610 and a second time to transition to a sleeping state based on sleeping habits, and send a push notification around the first time to inform the user that they should not get into bed 610 yet, as well as a push notification encouraging them to go to bed a predetermined time before the second time. The information processing device 20 may also send a push notification encouraging them to take a bath a predetermined time before their standard bedtime. The information processing device 20 may also estimate the timing of dinner based on how the user spent their time during the day and before going to bed, and send a push notification after dinner encouraging them to curb their future caffeine intake.

The order and manner of the push notification are arbitrary, but for example, the following chronological notification may be performed using the display unit 240, a sound output unit, or the like.
10:00 PM "This is the last time I'll have caffeine."
10:30 PM "It's time to take a bath and get ready for bed."
11:00 PM "Are you feeling sleepy? Go to bed when you're sleepy enough."
11:30 PM "It's almost time to go to bed. Let's go to bed without overdoing it."

As shown in FIG. 19 , the information processing device 20 may acquire, as activity data, information on activity levels and vital signs during work. The activity data includes information on work hours, vital signs during meetings (hereinafter referred to as MTGs), MTG participants, and MTG times. Work hours may refer to, for example, the time spent working in a day, or the time spent working in units such as a week or a month. MTG-time vital signs refer to vital signs information during a MTG. Alternatively, MTG-time vital signs may refer to the degree to which vital signs during a MTG deviate from normal. MTG participants refer to information identifying the people who participated in the MTG. MTG time refers to the time the MTG took place. Whether a MTG is being held may be detected using a sensor installed in a conference room or the like, or may be determined based on schedule data entered by the user.

The information processing device 20 may obtain analytical data by performing an analysis based on the activity data. The analytical data here includes work habits, compatibility with work, and compatibility with business partners. Work habits may be, for example, the frequency of meetings and average working hours. Work habits may also include information such as arrival and departure times, and whether or not work is performed on holidays. Compatibility with work represents the degree to which work can be performed without stress. For example, the information processing device 20 determines that the closer the meeting vitals or vitals during working hours are to a normal state (for example, vitals outside of working hours or vitals while sleeping), the better the compatibility with work. Compatibility with business partners represents compatibility with people related to the work, such as colleagues or business partners. For example, the information processing device 20 determines that the closer the meeting vitals are to a normal state, the better the compatibility between the target user and the meeting participants.

The information processing device 20 may then output business reform advice based on the activity data and analysis data. Business reform advice includes advice on changing work hours, team composition, etc. For example, if working hours are longer than a predetermined amount or if vital signs during meetings deviate significantly from normal, the information processing device 20 may consider that work is a burden on the user and suggest reducing work hours. Alternatively, if vital signs during early morning work or late-night overtime deviate from normal, advice on changing arrival and departure times may be provided. Furthermore, if the information processing device 20 determines that compatibility with a specific person is below a predetermined level, it may provide advice such as assigning the person and the target user to different teams. Furthermore, if a decline in physical strength is observed based on activity data, or if sleep data indicates poor sleep quality, the information processing device 20 may output advice to encourage a nap. Such advice may be output to a user (e.g., a supervisor, human resources officer, etc.) or the target company who has the authority to act in accordance with the advice. For example, the information processing device 20 may encourage the user to take a nap, and may also output advice to the user's company, such as encouraging employees to take naps and providing nap spaces.

Furthermore, since the target user's daytime activity level (activity time, activity intensity, stress level), etc. can be estimated based on activity data, etc., the information processing device 20 may output advice regarding diet and exercise based on this information. For example, if activity intensity is low, such as when the amount of time spent sitting in a seat is above a predetermined threshold, the information processing device 20 may advise reducing the amount of food eaten. The information processing device 20 may also advise recommended meal times or specific meal contents based on the user's daily routine during work. If the target company has a staff cafeteria, etc., advice may be provided suggesting specific recommended menus by acquiring menu information. The information processing device 20 may also output advice recommending exercise if the activity level is below a predetermined threshold. In this case, the information processing device 20 may advise specific exercise contents, such as climbing stairs or commuting on foot, by using structural information about the target company's building or map information from the station to the company, etc.

Note that while the above describes lifestyle advice based on sleep data and work reform advice based on activity data, this is not limited to this. For example, activity data may be used for lifestyle advice, and sleep data may be used for work reform advice. Furthermore, advice other than lifestyle advice and work reform advice may also be provided.

2.5 User Interface Figures 20A to 20D are examples of user screens presented to a user using the information processing system 10 of this embodiment. The screens shown in Figures 20A to 20D are displayed, for example, on the display unit 240 of the terminal device 200. For example, the terminal device 200 of this embodiment may operate according to an application program that performs processes such as sending images of sleeping positions and standing positions, and receiving bedding selection results and advice. The screens shown in Figures 20A to 20D are displayed, for example, by the application program.

As shown in FIG. 20A, the user screen may display information related to the target user's sleeping posture, subjective symptoms, bedding evaluation results, posture evaluation results, and countermeasures. The information related to sleeping posture may include, for example, the time ratio determined for each of the supine, right lateral, left lateral, and prone positions. Also, as shown in FIG. 20A, the information related to sleeping posture may include information on distortion for each posture. For example, the information processing device 20 determines the degree of distortion of the neck and lower back based on a skeletal model for each posture. Furthermore, the information related to sleeping posture is not limited to time ratio and distortion, and other information, such as information indicating the degree of snoring in each posture, may be added. For example, the information processing device 20 may determine the degree of snoring by receiving input regarding whether the user is aware of snoring, or may determine the degree of snoring by acquiring audio data during sleep using a microphone or the like.

In addition, Figure 20A may display, for example, information on the time ratio and distortion for one sleep session, or information on the time ratio and distortion for a period including multiple sleep sessions (for example, one month). Although omitted from Figure 20A, a graph showing the time-series changes in the time ratio and distortion may also be displayed. In this case, multiple graphs corresponding to multiple postures may be displayed side by side. Furthermore, along with the graphs of the time ratio and distortion, graphs showing changes in other information such as the sleep score may also be displayed.

In addition, the information regarding subjective symptoms may be numerical data that represents the user's thoughts about the degree of pain, distress, and comfort for each body part, such as the neck, shoulders, or lower back. In the example of FIG. 20A, each degree is displayed on a scale of 1 to 10, and the subjective symptoms are presented in an easy-to-understand manner by displaying objects that are illustrations of faces with different expressions. In addition, the information processing device 20 may display the screen shown in FIG. 20B on the display unit 240 of the terminal device 200, for example, when a selection operation is performed on any of the objects. In FIG. 20B, a slider (seek bar) extending horizontally is displayed, allowing the user to input operations. Using the screen shown in FIG. 20B, the user can easily input values for each item.

The bedding evaluation results also include values based on, for example, pillow evaluation values and mattress evaluation values. In the example of Figure 20A, a pillow deviation value, which is the deviation value of the pillow evaluation values of the target user, is calculated from the pillow evaluation values of multiple users, and the pillow deviation value is displayed. Similarly, a mattress deviation value calculated based on the mattress evaluation values may also be displayed.

As shown in Figure 20A, the information processing device 20 may also perform a process to estimate the evaluation value of the recommended bedding selected by the bedding selection unit 23. For example, since the target user's skeletal model and the characteristics (height, hardness, etc.) of the recommended bedding are known, the information processing device 20 may estimate the bedding evaluation value that will be obtained if the target user uses the recommended bedding based on this information. In the example of Figure 20A, two bedding items have been selected (Recommended 1 and Recommended 2), and the pillow standard deviation and mattress standard deviation are displayed for each. In this way, the user can confirm the degree of improvement in the evaluation value if they change their bedding, making it possible to appropriately encourage them to change their bedding.

The posture evaluation result may also be information about a skeletal model calculated based on a lying posture image, for example. The skeletal model information may be the skeletal model itself, the magnitude of body distortion calculated based on the skeletal model, or an evaluation value representing the degree of distortion calculated based on the magnitude. The posture evaluation result may also include information about a skeletal model calculated based on a standing posture image. For example, as shown in FIG. 20A, the posture evaluation item displays two objects: "standing posture" and "lying posture." When either selection operation is performed, the information processing device 20 performs processing to display a screen including information about the skeletal model corresponding to the selected posture.

In addition, as information regarding countermeasures, two objects, "Change bedding" and "Consult an expert," may be displayed as shown in FIG. 20A. Each object can be selected by the user, and when "Change bedding" is selected, the information processing device 20 performs processing to display the bedding change screen shown in FIG. 20C on the display unit 240 of the terminal device 200, and when "Consult an expert" is selected, the information processing device 20 performs processing to display the expert consultation screen shown in FIG. 20D on the display unit 240 of the terminal device 200.

As shown in FIG. 20C, the bedding change screen displays detailed information about the bedding selected by the bedding selection unit 23. For example, here, a first option (recommended 1) is presented, which involves changing the pillow 630 to "XX," and a second option (recommended 2) is presented, which involves changing the pillow 630 to "XX" and changing the mattress 620 to "△△." Recommendation 1 and Recommendation 2 here are the same bedding items used in the bedding evaluation section of FIG. 20A. This allows the user to confirm the specific recommended bedding. The bedding change screen may also display information about purchasing or subscribing to the recommended bedding. For example, as shown in FIG. 20C, the bedding change screen may display the purchase cost of the bedding and the monthly subscription fee. The bedding change screen may also allow transition to an e-commerce (EC) site where purchases or subscriptions can be made. The bedding selection screen is not limited to recommended bedding for purchase or subscription, and may display objects for the user to individually customize each of the pillow 630, mattress 620, bed 610, and other items (such as comforters).

As shown in FIG. 20D , the expert consultation screen displays information about consultations with experts or specialized facilities, such as sleep improvement instructors, chiropractors, osteopathic clinics, and clinics. For example, the expert consultation screen may display two objects for each item: "in-person" and "online." When "in-person" is selected, the information processing device 20 displays a screen for booking an in-person consultation with the corresponding expert or specialized facility. For example, the information processing device 20 may provide a reservation service for each expert or specialized facility within the same application software, or may link with an external service for making reservations. Similarly, when "online" is selected, the information processing device 20 displays a screen for booking a remote consultation via a network with the corresponding expert or specialized facility. In this way, if the user determines that a consultation with an expert is necessary based on the presented analysis data, the consultation can be quickly realized, thereby improving convenience.

3. Modified version <Special seat>
The above describes a method for selecting bedding based on a skeletal model. However, the method of this embodiment is not limited to this, and processing may also be performed using a dedicated sheet in addition to the skeletal model. Here, the dedicated sheet is, for example, a sheet printed with moiré fringes. For example, a user places the dedicated sheet on their bedding and places a weight in a specific position (e.g., the position where their lower back rests when sleeping). The user then captures an image of the dedicated sheet in this state using the imaging device 300. In this case, the dedicated sheet deforms according to the shape and hardness of the bedding, distorting to a shape corresponding to the bedding. The moiré fringes printed on the dedicated sheet change their pattern depending on the degree of distortion of the dedicated sheet. Therefore, by detecting changes in the moiré fringe pattern, the information processing device 20 can determine the characteristics of the user's bedding, for example, the hardness and degree of wear of the mattress 620. Furthermore, the characteristics of a pillow 630 may be determined by placing the dedicated sheet on the pillow 630.

For example, the bedding selection unit 23 may perform a process to select bedding recommended for the target user based on posture distortion detected from the skeletal model and bedding characteristics detected from the dedicated sheet.

<Share>
In this embodiment, information may be shared on a community site, etc. For example, as shown in Fig. 6, the information processing device 20 may include a sharing processing unit 29 that performs a process of outputting input information used to select bedding for a user in a bedding selection unit 23 and output information including a bedding selection result, and a process of acquiring input information and output information used in processing related to other users.

In this way, each user can share information about their own situation, proposed measures, and the effects of implementing those measures. Therefore, for example, by referencing the data of other users who have the same concerns, it becomes possible to resolve their own concerns. The information shared here may be information corresponding to Input (input information) shown in Figure 2, information related to Output (output information), or both. Furthermore, it is not limited to sharing all of the information shown in Figure 2; only items permitted by the target user may be shared.

In addition to the information shown in Figure 2, users may also share their own experiences of difficulties, ideas they have used, etc. as additional information. Each user may also rate the shared information as useful or not. For example, points may be awarded to users who find the information useful. When displaying information about other users, the information processing device 20 may perform processing such as sorting the information in order of points. Furthermore, the targets of evaluation are not limited to users; each user may also rate specialized facilities such as chiropractic clinics, osteopathic clinics, and clinics, or specific treatment contents.

<Sleeping position when taking a nap>
The sleeping posture in the above description refers, in a narrow sense, to a sleeping posture during serious sleep that lasts for several hours or more (nighttime sleep for a typical user). However, the method of the present embodiment is not limited to this, and a sleeping posture during a nap (daytime sleep for a typical user) may also be used in each of the above-mentioned processes.

Although the present embodiment has been described in detail above, those skilled in the art will readily understand that many modifications are possible that do not substantially deviate from the novel features and effects of the present embodiment. Therefore, all such modifications are intended to be within the scope of the present disclosure. For example, a term that appears at least once in the specification or drawings together with a different term with a broader or equivalent meaning may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of the present embodiment and modifications are also within the scope of the present disclosure. Furthermore, the configurations and operations of information processing systems, information processing devices, server systems, terminal devices, etc. are not limited to those described in the present embodiment, and various modifications are possible.

10...information processing system, 20...information processing device, 21...image acquisition unit, 22...model estimation unit, 23...bedding selection unit, 24...sleeping posture determination unit, 25...posture change estimation unit, 26...sleep information acquisition unit, 27...evaluation processing unit, 28...notification processing unit, 29...shared processing unit, 100...server system, 110...processing unit, 120...storage unit, 130...communication unit, 200...terminal device, 210...processing unit, 220...storage unit, 230...communication unit, 240...display unit, 250...operation unit, 300...imaging device, 400...sensing device, 430...detection device, 610...bed, 620...mattress, 630...pillow, θ1 to θ3...neck tilt angles, ST...stand

Claims (11)

an image acquisition unit that acquires a sleeping posture image of a user in a sleeping posture;
a model estimation unit that estimates a skeletal model of the user based on the sleeping posture image;
a sleeping posture determination unit that determines whether the sleeping posture of the user is one of a plurality of postures including a supine position, a right lateral position, a left lateral position, and a prone position, and determines a time ratio of each of the plurality of postures;
a bedding selection unit that performs processing to select bedding recommended for the user based on the estimated skeletal model and the time ratio ;
An information processing device comprising: In claim 1,
The image acquisition unit
acquiring a standing image of the user in a standing position;
The model estimation unit
An information processing device that estimates the skeletal model of the user based on the standing position image and the lying position image. In claim 2,
The model estimation unit
An information processing device calculates body distortion caused by the bedding from a sleeping posture skeletal model estimated based on the sleeping posture image and a standing posture skeletal model estimated based on the standing posture image, and corrects the sleeping posture skeletal model. In claim 1,
The device further includes a position change estimation unit that estimates a position change technique that is a technique used when the user changes their position by turning over while sleeping,
The bedding selection unit
An information processing device that performs processing to select the bedding based on the skeletal model and the position change method. In any one of claims 1 to 4,
The information processing device further includes an evaluation processing unit that calculates a sleep score that indicates the quality of sleep based on sleep data, which is the sensing result regarding the user's sleep obtained by a sleep sensor, and calculates a bedding effect index that indicates the effect of the bedding by performing a process of comparing the sleep score before and after introducing the bedding selected by the bedding selection unit. In claim 5,
The bedding selection unit
When the evaluation processing unit determines that the effectiveness of the bedding is below a predetermined level, the information processing device performs at least one of the following processes: adding input information to be used to select the bedding, changing the bedding selection algorithm that selects the bedding based on the input information, and adding types of the bedding to be selected. In claim 5,
The bedding selection unit
An information processing device that accepts input of an evaluation value of the bedding by the user when the evaluation processing unit determines that the effectiveness of the bedding is below a predetermined level or when the bedding effectiveness index cannot be calculated. In claim 5,
The device further includes a notification processing unit that, when the evaluation processing unit determines that the effectiveness of the bedding is equal to or less than a predetermined level, outputs to the user information recommending that the user be examined at a specialized facility including at least one of a chiropractic clinic, an osteopathic clinic, and a clinic ;
The bedding selection unit is an information processing device that performs processing to output a report including at least one of the bedding change history and the sleep score during the examination at the specialized facility . In claim 8,
The evaluation processing unit
An information processing device that calculates a facility effect index that represents an effect of the specialized facility based on the sleep data before the examination at the specialized facility and the sleep data after the examination at the specialized facility. In claim 5,
The information processing device further includes a notification processing unit that, when the evaluation processing unit determines that the effectiveness of the bedding is below a predetermined level, outputs information to the user recommending that the user start monitoring using at least one of an activity meter and a vital sensor. Acquire a sleeping posture image capturing a user in a sleeping posture;
Estimating a skeletal model of the user based on the sleeping posture image;
determining whether the user's sleeping position is one of a plurality of positions including a supine position, a right lateral position, a left lateral position, and a prone position, and also determining a time ratio of each of the plurality of positions;
performing a process of selecting bedding recommended for the user based on the estimated skeletal model and the time ratio ;
Information processing methods.