JP7621091B2 - Information processing device and biological condition determining method - Google Patents

Description

The present invention relates to an information processing device and a method for determining a biological condition.

Patent Document 1 describes a drowsiness estimation rule update device as comprising: "RRI acquisition means for acquiring RRI data; index value calculation means for calculating index values for multiple types of indexes related to autonomic nerve activity based on the RRI data; storage means for storing drowsiness estimation rules; drowsiness data acquisition means for acquiring information output from a drowsiness information output device that outputs information on the presence or absence of drowsiness, which is information obtained from the subject or information obtained by analyzing the subject's biological state; reference data generation means for generating reference data using the drowsiness data for evaluating the suitability of the drowsiness estimation rules; and update means for updating the drowsiness estimation rules stored in the storage means based on the index values and the reference data."

JP 2016-120062 A

In recent years, attention has been focused on technology that measures a driver's biological information, such as pulse rate and heart rate, to determine their biological state, such as their level of drowsiness, fatigue or concentration, and then performs automatic control, such as issuing an alert to prevent accidents or stopping the vehicle, depending on the results of the assessment.

When determining the biological condition, the measurement data is corrected to take into account the characteristic differences that vary from one sensor model to another, the individual differences of the subject being measured, or differences in measurement methods. However, this is problematic because it is necessary to set the processing and conditions for correction in advance for each sensor, which is cumbersome. For this reason, there is a demand for a method to determine the biological condition more accurately without the need for correction to absorb differences in sensor characteristics, etc.

Patent Document 1 discloses a device that uses RRI (R-R Interval: heartbeat interval) to determine drowsiness. In the technology of this document, RRI data is corrected to absorb characteristic differences due to differences in sensor models, and the sensor characteristic information used for the correction is set in advance. In other words, the technology of this document does not take into consideration the determination of biological conditions that do not require correction to absorb differences in sensor characteristics, etc.

The present invention aims to determine the biological condition more accurately without the need for corrections that take into account differences in sensor characteristics, etc.

The present application includes a plurality of means for solving at least a part of the above-mentioned problems, examples of which are as follows: An information processing device according to one aspect of the present invention for solving the above-mentioned problems includes an image information generating unit that generates image information showing a Lorenz plot graph representing fluctuations in heartbeat intervals using RRI (R-R Interval) data, which is human biological information, a model constructing unit that constructs a learning model by machine learning using the image information, a biological condition determining unit that inputs the image information generated based on the biological information acquired from the user of the vehicle into the learning model and obtains output values that are biological condition determination results regarding the presence or absence of drowsiness, fatigue, and concentration, and a communication unit that outputs the biological condition determination result to a predetermined vehicle device implemented in the vehicle in order to control in-vehicle equipment based on the determination result.

According to the present invention, it is possible to determine the biological condition more accurately without the need for corrections that take into account differences in sensor characteristics, etc.

Problems, configurations, effects, etc. other than those mentioned above will become clear from the description of the embodiments below.

1 is a diagram showing an example of a schematic configuration of an information processing device according to a first embodiment. FIG. 2 is a functional block diagram showing an example of a functional configuration of a calculation unit in the information processing device. 3A is a diagram showing an example of image information of a Lorenz plot representing fluctuations in heartbeat intervals when the user is not feeling sleepy, and FIG. 3B is a diagram showing an example of image information of a Lorenz plot representing fluctuations in heartbeat intervals when the user is feeling sleepy. FIG. 11 is a flow diagram illustrating an example of a learning model construction process. FIG. 11 is a flow chart showing an example of a biological condition determining process. FIG. 11 is a functional block diagram showing an example of a functional configuration of an information processing device and a server device according to a second embodiment.

The following describes an embodiment of the present invention.

First Embodiment
1 is a diagram showing an example of a schematic configuration of an information processing device 100. The information processing device 100 is a device that judges a biological state using biological information of a user (e.g., a driver) riding on a moving body such as a vehicle. Specifically, the information processing device 100 uses biological information (e.g., RRI data: R-R Interval) generated from measurement results of heart rate, pulse rate, etc. to generate image information of a graph (Lorenz plot) showing fluctuations in heart rate intervals, and inputs this to a learning model to judge a biological state such as sleepy or not sleepy.

As shown in the figure, the information processing device 100 has a calculation device 110, a sensor 120, a communication device 130, and a storage device 140. Note that the information processing device 100 may have the calculation device 110, the communication device 130, the storage device 140, and the sensor 120 in the same housing, or the calculation device 110, the communication device 130, the storage device 140, and the sensor 120 may each be housed in a separate housing. The calculation device 110 and the sensor 120 are connected to each other so as to be able to communicate with each other via a communication cable or a specified wireless communication standard (e.g., Bluetooth: registered trademark).

The arithmetic device 110 is a device that performs various arithmetic processes of the information processing device 100. As shown in the figure, the arithmetic device 110 has a CPU (Central Processing Unit) 111 that executes various arithmetic processes performed by the information processing device 100, a ROM (Read Only Memory) 112 that stores programs executed by the CPU 111, etc., a RAM (Random Access Memory) 113 that temporarily stores various information read from the ROM 112, an I/F (interface) 114 for electrically connecting the arithmetic device 110 and the sensor 120, and a bus 115 that connects them to each other.

The arithmetic device 110 acquires biological information (e.g., RRI data indicating the heartbeat interval) from the sensor 120, and uses this to generate image information showing a Lorentz plot graph that represents the fluctuation of the heartbeat interval (hereinafter, sometimes referred to as "Lorentz plot image information"). In addition, the arithmetic device 110 constructs (generates) a learning model by performing machine learning using the generated image information.

The calculation device 110 also inputs image information generated using the biometric information acquired from the sensor 120 into the learning model to obtain an output value that is the result of calculation processing by the learning model. The output value is information that indicates the result of determining the biometric state using the image information, and indicates, for example, the biometric state, such as sleepy, not sleepy, tired, not tired, focused, or not focused.

The calculation device 110 also converts the output values output from the learning model into a predetermined data format that can be used by the Main Unit 200 (for example, a navigation device, an in-vehicle device such as a D/A (Digital Audio), or an ECU (Engine Control Unit)), and outputs the converted data to these devices. The in-vehicle device, etc., uses the biological condition determination result obtained from the information processing device 100 to execute processing according to the user's biological condition (for example, playing music in the car, or controlling the air conditioner to change the temperature inside the car).

Sensor 120 is a measuring device that measures vital data such as the user's heart rate and pulse rate. Specifically, sensor 120 is a contact sensor capable of detecting the user's heart rate (or pulse rate), such as a piezoelectric meter embedded in the seat of a vehicle or a wearable device such as a watch that is worn in contact with the body. Sensor 120 may also be a non-contact sensor such as a Doppler sensor that irradiates the human body with microwaves or millimeter waves and detects the user's heart rate based on the reflected waves. Alternatively, sensor 120 may be a camera that can detect the heart rate based on the flow rate of hemoglobin contained in the blood.

The sensor 120 measures the user's heart rate for a predetermined period (e.g., 5 minutes, 10 minutes, or 30 minutes from the start of measurement), generates RRI data, which is bioinformation, as the measurement result, and outputs it to the information processing device 100. The RRI data is information that indicates the heartbeat interval (interval between R waves), which is the time interval between heartbeats.

The type of sensor 120 is not particularly limited, and may be any measuring device capable of measuring vital data such as the user's heart rate or pulse rate. For example, sensor 120 may be a measuring device capable of calculating the user's heart rate interval over a predetermined period of time in a backward calculation based on the detected pulse rate, and generating RRI data.

The communication device 130 is a communication module for communicating information with an external device (e.g., the Main Unit 200). The communication device 130 is connected to the Main Unit 200 via a communication cable or a predetermined wireless communication standard (e.g., Bluetooth) so that they can communicate with each other, and transmits the biological condition determination result to the Main Unit 200.

The storage device 140 is a so-called hard disk drive, a solid state drive (SSD), a flash memory, or other non-volatile storage device capable of storing digital information. The storage device 140 stores, for example, various information used to construct a learning model and the constructed learning model.

Figure 2 is a functional block diagram showing an example of the functional configuration of the calculation device 110 in the information processing device 100. As shown in the figure, the calculation device 110 has a calculation unit 300, a control unit 310, a memory unit 320, and a communication unit 330.

The calculation unit 300 is a functional unit that performs various processes executed by the calculation device 110. Specifically, the calculation unit 300 has an image information generation unit 301, a labeling unit 302, a model construction unit 303, a user authentication unit 304, a biological condition determination unit 305, and a data format conversion unit 306. In the following, the determination of a biological condition related to drowsiness will be described as an example.

The image information generating unit 301 is a functional unit that generates image information using bioinformation acquired from the sensor 120. Specifically, the image information generating unit 301 uses the RRI data acquired from the sensor 120 to generate image information of a Lorenz plot that represents fluctuations in heartbeat intervals over a predetermined period of time.

More specifically, the image information generating unit 301 uses the RRI data to plot the time interval from the start of measurement to the first (Nth (N=1)) heartbeat on the horizontal axis (X-axis), the time interval from the first (Nth (N=1)) heartbeat to the next (N+1th (N=1)) heartbeat on the vertical axis (Y-axis), and plots the intersections between them. Next, the image information generating unit 301 plots the time interval from the first (Nth (N=1)) heartbeat to the next (N+1th (N=1)) heartbeat on the horizontal axis (X-axis), the time interval from the next (N+1th (N=1)) heartbeat to the next (N+1 (N=2)) heartbeat on the vertical axis (Y-axis), and plots the intersections between them. In this way, the image information generating unit 301 generates a Lorentz plot graph by repeatedly performing the same process for all heartbeat intervals measured during a specified period.

In addition, the image information generating unit 301 generates image information by converting the generated Lorenz plot graph into a specified format.

3A shows an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is not feeling sleepy. FIG. 3B shows an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is feeling sleepy. As shown in these figures, when the user is not feeling sleepy, the fluctuations in the heartbeat intervals are small, so the positions of the plots are not spread out and the graph shape is within a certain range. On the other hand, when the user is feeling sleepy, the fluctuations in the heartbeat intervals are large, so the positions of the plots are scattered over a wide range. In this way, image information of a Lorenz plot based on RRI data shows a unique shape that represents the user's biological state (e.g., sleepy, not sleepy, etc.), so that the user's biological state can be determined using such image information.

The labeling unit 302 is a functional unit that performs labeling processing. Specifically, the labeling unit 302 performs labeling processing to associate a correct answer label indicating the user's biological state with image information of a Lorenz plot. More specifically, the labeling unit 302 obtains a correct answer label indicating whether or not the user felt drowsy during the measurement period, and performs labeling processing to associate the correct answer label with image information of a Lorenz plot generated using the user's biological information during the corresponding period.

The labeling unit 302 acquires the correct label by a predetermined method. For example, after the end of the measurement period, the labeling unit 302 displays screen information on a display device provided in the information processing device 100 or the Main Unit 200 to receive a response from the user as to whether or not the user felt sleepy during the measurement period, and acquires the correct label by receiving the response directly from the user.

Alternatively, the labeling unit 302 captures the facial expression of the user during the measurement period via an in-vehicle camera capable of capturing images inside the vehicle, and performs facial recognition using the image information to obtain a correct label indicating whether the user felt drowsy during the measurement period. Note that such facial recognition technology may be a publicly known technology such as the evaluation method of NEDO (New Energy Industrial Technology Development Organization).

The method for obtaining the correct label is not particularly limited, and any method may be used as long as it can obtain a correct label that indicates whether or not the user felt drowsy during the measurement period.

The model construction unit 303 is a functional unit that constructs the learning model 322. Specifically, the model construction unit 303 constructs the learning model 322 by performing machine learning using the image information of the Lorenz plot generated by the image information generation unit 301 and the correct answer label associated with the image information. The construction process of the learning model 322 will be described later.

The user authentication unit 304 is a functional unit that performs user authentication based on user information. The user information includes a user ID for identifying the user and a category to which the user belongs among predetermined categories classified based on physical characteristics. The user authentication unit 304 searches the memory unit 320 for a learning model 322 associated with the user ID or a learning model 322 associated with the category to which the user belongs.

The biometric condition determination unit 305 is a functional unit that inputs image information of a Lorenz plot generated using biometric information (RRI data) acquired from the sensor 120 into the learning model 322 and acquires an output value indicating the result of the biometric condition determination.

The data format conversion unit 306 is a functional unit that acquires the output value (information indicating the biological state determination result) output as the calculation result of the learning model 322, and converts the output value into a data format that can be used by the Main Unit 200. The data format conversion unit 306 also outputs the converted output value, which is the biological state determination result, to the Main Unit 200 via the communication unit 330. The data format to be converted is not particularly limited as long as it is a data format that can be used by the Main Unit 200.

The control unit 310 is a functional unit that controls the operation of the sensor 120. Specifically, the control unit 310 outputs an instruction to the sensor 120 to measure biometric information. The control unit 310 also outputs instructions to the sensor 120 regarding specific control objects, such as recognition of the user's position, the measurement time for the user, and the frequency range of the radio waves to be irradiated.

The memory unit 320 is a functional unit that stores various data. Specifically, the memory unit 320 stores various base models 321 that serve as a foundation for constructing the learning model 322. The base models 321 include, for example, a convolutional neural network (CNN), which exhibits excellent performance in image recognition. The memory unit 320 also stores the constructed learning model 322.

The learning models 322 include a learning model 322 constructed by measurements of a user of a vehicle equipped with the information processing device 100, and a plurality of learning models 322 constructed by measurements of people (people other than the user) belonging to each category classified based on physical characteristics. These learning models 322 are stored in the storage unit 320 in association with a user ID and the category.

The communication unit 330 is a functional unit that communicates information with the Main Unit 200. Specifically, the communication unit 330 transmits the biological condition determination result to the Main Unit 200. The communication unit 330 may also obtain the correct answer label from the Main Unit 200.

The above describes the functional configuration of the information processing device 100.

The calculation unit 300 and the control unit 310 are realized by a program that causes the CPU 111 of the calculation device 110 to perform processing. These programs are stored in, for example, ROM 112, and are loaded onto RAM 113 for execution and executed by the CPU 111. The control unit 310 may be realized by a predetermined control circuit. The storage unit 320 is realized by the RAM 113, ROM 112, storage device 140, or a combination of these. The communication unit 330 is realized by the communication device 130.

Furthermore, each functional block of the information processing device 100 is classified according to the main processing content in order to facilitate understanding of each function realized in this embodiment. Therefore, the present invention is not limited by the manner in which each function is classified or its name. Furthermore, each configuration of the information processing device 100 can be further classified into more components according to the processing content. Furthermore, it is also possible to classify each component so that it performs even more processes.

Furthermore, all or part of each functional unit may be constructed by hardware (such as an integrated circuit such as an ASIC) implemented in a computer. Furthermore, the processing of each functional unit may be executed by a single piece of hardware, or may be executed by multiple pieces of hardware.

[Operation Description]
Next, a learning model construction process executed by the information processing device 100 will be described.

Figure 4 is a flow diagram showing an example of the learning model construction process. This process is started, for example, when a user (or a person other than the user) receives an instruction to execute the learning model construction process while driving. The following describes an example in which a learning model 322 is constructed for a user.

When the process starts, image information to be used for machine learning is generated by the processes in steps S001 to S005 below. Specifically, the control unit 310 outputs an instruction to the sensor 120 to perform (start) measurement of biometric information of a user (e.g., a driver) (step S001). Based on the instruction, the sensor 120 performs measurement of the user for a predetermined period (e.g., 5 minutes), generates RRI data based on the measurement data, and outputs it to the image information generation unit 301.

Next, the model construction unit 303 acquires the RRI data from the sensor 120 (step S002). The model construction unit 303 temporarily stores the acquired RRI data in the storage unit 320.

Next, the model construction unit 303 performs data cleaning (step S003). Specifically, the model construction unit 303 removes outliers from the acquired RRI data and performs a process of complementing information that cannot be read as a numerical value (NaN). The complementation process is performed using a predetermined method, such as by using the midpoint between the values before and after a data point that is NaN.

Next, the image information generating unit 301 generates a Lorenz plot graph based on the RRI data and generates image information of the graph (step S004).

Next, the labeling unit 302 performs a labeling process on the generated image information (step S005). Specifically, the labeling unit 302 acquires from the user a correct answer label that indicates the biological condition during the measurement period of the RRI data that corresponds to the generated image information, and associates this with the image information.

By performing the processes in steps S001 to S005, machine learning information is generated that associates image information of a Lorenz plot generated using RRI data obtained by measuring the user for a specified period of time with a correct answer label indicating the biological condition during that period.

Next, the model construction unit 303 determines whether a predetermined number (e.g., 100) of pieces of machine learning information have been generated (step S006). Note that the predetermined number of pieces of machine learning information includes a predetermined ratio (e.g., approximately equal numbers) of Lorenz plot image information showing fluctuations in heartbeat intervals in each of the cases where the user feels sleepy and where the user does not feel sleepy.

If it is determined that the predetermined number of pieces of information for machine learning have not been generated (No in step S006), the model construction unit 303 returns the process to step S001. On the other hand, if it is determined that the predetermined number of pieces of information for machine learning have been generated (Yes in step S006), the model construction unit 303 transitions the process to step S007.

In step S007, the model construction unit 303 divides the machine learning information at an arbitrary ratio. Specifically, in order to use the hold-out method for accuracy verification described below, the model construction unit 303 divides the machine learning information, i.e., image information of the Lorenz plot to which the correct label indicating the biological state is associated, into training data and validation data. The division ratio of the training data and validation data is arbitrary. For example, the training data and validation data are divided at an arbitrary ratio such as 6:4 or 7:3.

Next, the model construction unit 303 constructs the learning model 322 by the processing of steps S008 to S011. Specifically, the model construction unit 303 determines the base model 321 (step S008). More specifically, the model construction unit 303 obtains a predetermined base model 321 (e.g., a convolutional neural network) from the storage unit 320, and determines this as the base model 321 to be used for model construction.

Next, the model construction unit 303 performs machine learning using the training data (step S009). Here, machine learning will be described. The model construction unit 303 inputs image information of the training data into a transfer function indicated by a determined base model 321 such as a convolutional neural network, and compares the output value output from the base model 321 as a prediction result for the correct label with the correct label associated with the input image information. Furthermore, if the output value differs from the correct label, the model construction unit 303 feeds it back and changes the objective variable of the transfer function. Furthermore, the model construction unit 303 performs machine learning by executing such prediction and feedback processing for all training data, and constructs a learning model 322.

Next, the model construction unit 303 performs accuracy verification of the learning model 322 using the verification data (step S010). Specifically, the model construction unit 303 inputs the verification data to the constructed learning model 322. The model construction unit 303 also verifies the accuracy of the constructed learning model 322 based on a comparison between the output value from the learning model 322 and the correct answer label associated with the image information of the verification data.

Next, the model construction unit 303 determines whether the accuracy of the learning model 322 is sufficient (step S011). Specifically, the model construction unit 303 makes this determination based on whether the accuracy of the learning model 322 verified in step S010 meets a predetermined standard value (e.g., an accuracy rate of 80% or more) that is deemed sufficient. If the model construction unit 303 determines that the accuracy is sufficient (Yes in step S011), the model construction unit 303 proceeds to step S012. On the other hand, if the model construction unit 303 determines that the accuracy is not sufficient (No in step S011), the model construction unit 303 returns the process to step S001.

In addition, in the processing of steps S009 and S010, which are executed via step S011, the model construction unit 303 executes these processes using new training data and validation data. Specifically, in step S001, the model construction unit 303 outputs an instruction to the sensor 120 via the control unit 310 to execute a new measurement, and performs the same processes as the above-mentioned steps S002 to S010 using the acquired new bioinformation.

In addition, in step S012, the model construction unit 303 stores (saves) in the memory unit 320 a predetermined category classified based on the user ID or physical characteristics in association with the constructed learning model 322.

In addition, once the model construction unit 303 saves the learning model 322, it ends the processing of this flow.

When constructing a learning model 322 for a person other than the user, the model construction unit 303 constructs a learning model 322 corresponding to each category by measuring people who belong to each category classified based on physical characteristics. The categories are classified according to the type and range of physical characteristics. For example, examples of categories include "gender = male, age = 30s, height = 170 cm to 180 cm, weight = 60 kg to 70 kg, chronic illness = none" and "gender = female, age = 40s, height = 150 cm to 160 cm, weight = 50 kg to 60 kg, chronic illness = none".

The model construction unit 303 performs measurements on individuals from the population who belong to the categories classified in this way, and uses image information generated based on the biometric information acquired thereby to construct a learning model 322 corresponding to each category.

As a result, in the biometric condition determination process described below, the user can determine the biometric condition based on their own user authentication by applying a learning model 322 constructed by measuring themselves or a learning model 322 constructed by measuring a person belonging to the same category as the user.

Next, the biological condition determination process executed by the information processing device 100 will be described.

Figure 5 is a flow diagram showing an example of a biological condition determination process. This process is started, for example, when the vehicle engine is started (when the ignition key is turned ON) or when an instruction to start the process is received from the user. The start of the engine may be detected by the user authentication unit 304 acquiring information indicating that the engine has started from the ECU via the communication unit 330. The user may issue an instruction to start the process by, for example, pressing a process start instruction button provided in the information processing device 100, which may be detected by the user authentication unit 304.

When the process starts, the user authentication unit 304 performs personal authentication of the user (step S021). Specifically, the user authentication unit 304 searches the storage unit 320 for a learning model 322 associated with the user ID input via a specified input device. If such a learning model 322 does not exist, the user authentication unit 304 searches the storage unit 320 for a learning model 322 associated with the category to which the user belongs.

The specified input device may be, for example, a Main Unit 200 such as a navigation device. In this case, the user authentication unit 304 may obtain user input information from the Main Unit 200 via the communication unit 330. The input device may also be provided in the information processing device 100.

The authentication method is not limited to inputting user information. For example, if the vehicle is equipped with an in-vehicle camera that captures images inside the vehicle, the user authentication unit 304 may acquire image information captured by the in-vehicle camera and use this information to perform face authentication to identify the physical characteristics of the user. When this method is adopted, the information processing device 100 only needs to be connected to the in-vehicle camera via the communication device 130 so that information can be communicated.

The authentication method may also be fingerprint authentication. When this method is adopted, the information processing device 100 may be provided with a fingerprint authentication sensor. Alternatively, when a fingerprint authentication sensor is installed at a specific location of the vehicle (e.g., the steering wheel, etc.), the information processing device 100 may acquire fingerprint information from an ECU connected via the communication device 130, for example, and identify the user's physical characteristics by comparing the fingerprint information with fingerprint information previously stored in the memory unit 320.

Next, the user authentication unit 304 determines whether or not there is a corresponding learning model 322 (step S022). Specifically, the user authentication unit 304 makes this determination based on whether or not a learning model 322 associated with the user ID or the category to which the user belongs is stored in the storage unit 320. If it is determined that there is a corresponding learning model 322 (Yes in step S022), the user authentication unit 304 transitions the process to step S023. On the other hand, if it is determined that there is no corresponding learning model 322 (No in step S022), the user authentication unit 304 transitions the process to step S030.

In step S030, the biometric condition determination unit 305 notifies the user that there is no corresponding learning model 322. For example, the biometric condition determination unit 305 generates a predetermined message for notifying the user that there is no corresponding learning model 322, and displays the message on the display device of the vehicle-mounted device, which is the Main Unit 200, via the communication unit 330. In addition, when the biometric condition determination unit 305 displays the message on the Main Unit 200, it ends the processing of this flow.

In addition, in the process of step S023, which is performed when it is determined that a corresponding learning model 322 exists, the user authentication unit 304 retrieves the corresponding learning model 322 from the memory unit 320.

Next, the biological condition determination unit 305 outputs an instruction to the sensor 120 via the control unit 310 to measure vital data such as heart rate (step S024). Next, the biological condition determination unit 305 acquires the RRI data output from the sensor 120 (step S025) and inputs the RRI data into the acquired learning model 322 (step S026).

Next, the biological condition determination unit 305 acquires the output value output as the calculation result of the learning model 322, i.e., the determination result regarding the presence or absence of drowsiness of the user during the measurement period (sleepy/not sleepy) (step S027).

The data format conversion unit 306 also converts the output value into a predetermined data format (step S028) and outputs it to the Main Unit 200 via the communication unit 330 (step S029). When the data format conversion unit 306 outputs the output value, which is the result of the biological condition determination, to the Main Unit 200, it ends the processing of this flow.

The above describes the information processing device 100 of this embodiment.

According to such an information processing device, the biological condition can be determined more accurately without the need for correction that takes into account differences in sensor characteristics, etc. In particular, the information processing device uses RRI data, which is biological information, to generate image information of a Lorenz plot that visually represents fluctuations in heartbeat intervals, and determines the biological condition using this image information. For example, when RRI data is input into a learning model to determine the biological condition, differences in sensor characteristics are reflected in the numerical data, so correction processing and its conditions must be set in advance before correction is performed. However, according to the information processing device, the biological condition is determined using image information that allows visual discrimination of tendencies indicating the biological condition, so correction and settings for correction are not required, and the biological condition can be determined with high accuracy.

Second Embodiment
6 is a functional block diagram showing an example of the functional configuration of the information processing device 100 and the server device 400 according to this embodiment. As shown in the figure, the information processing device 100 is connected to the server device 400 so as to be able to communicate with each other via a predetermined network N such as a LAN (Local Area Network).

The image information generating unit 411, labeling unit 412, and model constructing unit 413, the memory unit 420, the base model 421, and the communication unit 430 in the server device 400 correspond to the image information generating unit 301, the labeling unit 302, and model constructing unit 303, the memory unit 320, the base model 321, and the communication unit 330 in the information processing device 100 of the first embodiment, respectively, and are functional units and information that perform similar processing, so detailed explanations will be omitted in this embodiment.

In the first embodiment described above, the learning model 322 was constructed by the information processing device 100. In this embodiment, however, the biometric information acquired from the sensor 120 is transmitted to the server device 400, and the learning model 322 is constructed by the image information generating unit 411, the labeling unit 412, and the model constructing unit 413 of the calculation unit 410 of the server device.

Specifically, when the calculation unit 300 of the information processing device 100 acquires bioinformation (RRI data) from the sensor 120 in the learning model construction process, it transmits it to the server device 400 via the communication unit 330. When the image information generation unit 411, labeling unit 412, and model construction unit 413 of the server device 400 acquire the RRI data via the communication unit 430, they construct the learning model 322 in a manner similar to the above-mentioned steps S003 to S011.

The learning model 322 constructed by the model construction unit 413 of the server device 400 is transmitted to the information processing device 100 via the communication unit 430 of the server device 400 and stored in the memory unit 320. The information processing device 100 also uses the learning model 322 stored in the memory unit 320 to execute the same biological condition determination process as described above.

According to the information processing device 100 and server device 400 of the second embodiment, the construction of the learning model 322 can be executed by a server device with high processing performance, so that the specifications of the information processing device 100 can be suppressed, and as a result, manufacturing costs can be saved.

The present invention is not limited to the information processing device 100 of the first and second embodiments, and various modifications are possible. For example, the calculation device 110 (calculation unit 300 and control unit 310), communication device 130, and storage device 140 in the first embodiment may be built into an on-board device such as a navigation device, which is the Main Unit 200, or an ECU.

In addition, in the above-described embodiment, the output value is the judgment result of sleepy/not sleepy, but the present invention is not limited to this, and the information processing device 100 may output a judgment result including an intensity level of the biological state, such as "slightly sleepy," "very sleepy," "low sleepiness," "medium sleepiness," and "high sleepiness."

This type of determination can be achieved, for example, by subdividing the correct answer labels into multiple levels of biological states, associating them with image information, and using these to construct a learning model 322 through machine learning.

The information processing device 100 according to this modified example can determine the user's biological condition at a more detailed level.

In addition, in the above-described embodiment, image information of a Lorenz plot is generated, but the present invention is not limited to this. For example, image information showing the user's electrocardiogram waveform may be generated and used to determine the biological condition.

Even with this type of modified information processing device, the biological condition is determined using image information that allows visual identification of trends indicating the biological condition, so correction and settings for correction are not required, and the biological condition can be determined with high accuracy.

The present invention is not limited to the above-mentioned embodiments and modifications, but includes various other embodiments and modifications. For example, the above-mentioned embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment or modification, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

100: Information processing device, 110: Calculation device, 111: CPU, 112: ROM, 113: RAM, 114: I/F (interface), 115: Bus, 120: Sensor, 130: Communication device, 140: Storage device, 200: Main unit, 300: Calculation unit, 301: Image information generation unit, 302: Labeling unit, 303: Model construction unit, 304: User authentication unit, 305: Biometric condition determination unit, 306: Data format conversion unit, 310: Control unit, 320: Storage unit, 321: Base model, 322: Learning model, 330: Communication unit

Claims (4)

an image information generating unit that generates image information showing a Lorentz plot graph representing fluctuations in heartbeat intervals using RRI (R-R Interval) data, which is human biological information;
A model construction unit that constructs a learning model by machine learning using the image information;
a biological condition determination unit that inputs the image information generated based on the biological information acquired from the user of the vehicle into the learning model and obtains output values that are biological condition determination results regarding the presence or absence of drowsiness, fatigue, and concentration ;
and a communication unit that outputs the result of the biological condition determination to a predetermined vehicle device implemented in the vehicle in order to control in-vehicle equipment based on the result of the determination . 2. The information processing device according to claim 1 ,
An information processing device further comprising a memory unit that stores the learning model constructed using the biometric information obtained by measuring people belonging to specified categories classified according to combinations of types of physical features and their ranges. 3. The information processing device according to claim 2 ,
the learning model is stored in the storage unit in association with user identification information or the user identification information and the predetermined category;
An information processing device further comprising a user authentication unit that searches the memory unit for the learning model corresponding to the user's identification information or the category to which the user belongs. A biological condition determining method performed by an information processing device, comprising:
The information processing device includes:
A step of generating image information showing a Lorentz plot graph representing fluctuations in heartbeat intervals using RRI (R-R Interval) data, which is human biological information;
constructing a learning model by machine learning using the image information;
inputting the image information generated based on the biological information acquired from the user of the vehicle into the learning model, and acquiring output values which are the results of the biological state determination regarding the presence or absence of drowsiness, the presence or absence of fatigue, and the presence or absence of concentration ;
and a communication step of outputting the result of the biological condition determination to a predetermined vehicle device implemented in the vehicle in order to control in-vehicle equipment based on the result of the determination .

Images