JP7688601B2 - Prediction device, program, and method for predicting user behavior sequences using sequence prediction model - Google Patents

Description

The present invention relates to sequence prediction model technology. In particular, it relates to applications for predicting user behavior sequences.

Conventionally, there is a GPT2 (Generative Pre-trained Transformer - 2) model as a sequence prediction model (see, for example, Non-Patent Document 1). This is a self-supervised language generation model that predicts the next word (next word prediction) of a word string (sentence) extracted from a large amount of corpus. When an arbitrary word string is input, the GPT2 model can sequentially predict the next word, and can generate sentences that look like they were written by a human. For example, when the word strings "today" and "good" are input, the word "weather" with the highest probability as the next word is output. Next, when "today", "good", and "weather" are input, the word "is" with the highest probability as the next word is output.
Sentence: "Today is a good day"

Figure 1 is an explanatory diagram showing the data structure during training of the GPT2 model in the prior art.

Assume that there are multiple texts (sentences) as training data. The GPT2 model is executed as follows.
(S1) The text is broken down into word strings, and words of a predetermined length are packed into sequence data.
(S2) At that time, if the number of words is less than the predetermined length, the words of the immediately next text are padded into the sequence data.
(S3) When a predetermined length is reached while packing the words of the text, the subsequent words are packed into the next sequence data.

According to FIG. 1, the process is carried out as follows.
Text 1 consists of 10 words (0 to 9) and is packed into sequence data 1 of a given length = 14.
Here, since the number of words in Text 1 is less than the specified length, the words (0-2) from the next Text 2 are stuffed into Sequence Data 1. However, since there is still space in Sequence Data 1, the word (0) from the next Text 3 is stuffed into it. The remaining tokens (1-5) in Text 3 that were cut off midway are stuffed into the next Sequence Data 2.
By repeating this process, for example, five texts 1 to 5 are packed into three sequence data 1 to 3.

Figure 2 is an explanatory diagram showing the process of training the GPT2 model in the prior art.

GPT2 is based on Transformer and has an attention mechanism (scoring that indicates the relevance between tokens). The attention of each token is always focused on the token preceding it. This allows each token to learn which part of the previous token to focus on in order to infer the next token.

The GPT2 model is trained using the training data constructed according to FIG.
According to FIG. 2, the predetermined length that can be input to the GPT2 model is 14, and training is performed by associating input data with correct answer data.
Input data sequence: 0,1,2,3,4,5,6,7,8,9,0,1,2,0
Correct data sequence : 1,2,3,4,5,6,7,8,9,0,1,2,0,1
In this way, in the GPT2 model, the correct answer data is obtained by deleting the first token from the input data and adding the last token.
In response to this, a result data string is output.
Then, the difference between the resulting data sequence and the correct data sequence is expressed as a loss function (generally called cross entropy loss) and training is carried out.

According to the example of input data and correct answer data mentioned above, to generate token 1, it is sufficient to focus only on token 0 (to begin with, token 0 is the only token to focus on before token 1). Also, to generate token 3, it is sufficient to focus on token 1 using the attention mechanism (tokens 0 and 2 can be ignored).

In this way, the GPT2 model in Figure 2 can be trained by deleting one token from the beginning and adding one token to the end of the sequence data 1 to 3 in Figure 1 described above.

Another conventional technique is to control sentence length, which was difficult with the GPT2 model (see, for example, non-patent document 2). With this technique, a "value indicating sentence length" is added to the beginning of a sentence during training. By adding a "value indicating sentence length" during estimation as well, sentences of any length can be generated.

Furthermore, as an application other than sentence prediction, there is also a technology for predicting a user's behavioral sequence. According to this technology, data is constantly acquired from sensors placed in the home and the user's next action is predicted.
User actions in the home can generally be recognized as "conceptual actions (complex actions)" that will take a certain amount of time, such as "cooking" or "washing dishes."
In addition, a conceptual behavior is composed of a number of simple "concrete actions" in a time series. For example, the conceptual behavior "cooking" is composed of a series of actions such as the following:
"Approach the kitchen"->"Open the refrigerator"->"Close the refrigerator"->"Turn on the stove"

If it were possible to predict the user's next sequence of actions at an early stage (time t), it would be possible to optimally control in-home appliances prior to the user's actions. For example, it would be possible to turn on the lights prior to the user's actions. It would also be possible to complement and control (turn off the stove) the actions that the user did not take at the end (not turning off the stove).

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language Models are Unsupervised Multitask Learners, 2019, pp.1-24., [online], [Retrieved April 8, 2022], Internet <URL: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf> Koichi Nagatsuka, Masayasu Atsumi, Conditional Language Model for Sentence Length Control, 2020 Annual Conference of the Japanese Society for Artificial Intelligence (JSAI2020), 2020, pp. 1-4

The aforementioned GPT2 can also be applied to predict a user's behavioral sequence. In this case, the user's conceptual behavior corresponds to the text in GPT2, and the user's specific behavior (action) corresponds to the token in GPT2. Even data observed by a sensor can be applied by considering it equivalent to a word string.

However, in the case of a user's action sequence, it is necessary to determine the end of each conceptual action, and if GPT2 is applied as is, the problem arises that sequence data will continue to be generated endlessly. Of course, it is possible to simultaneously learn the sequence length as in Non-Patent Document 2, but in the case of a user's action sequence, the sequence length cannot be specified in advance. Of course, the sequence length can be fixed as a known value, but it is not realistic to apply it to diverse and real-time user actions.

That is, if the data structure of GPT2 is applied as is to predict a user's behavioral sequence, the following problems arise.
(1) There may be no correlation between successive conceptual actions.
According to the conventional technology, multiple sequential data are packed in succession as shown in FIG. 1. However, when there is no correlation between the preceding and succeeding sequential data, the token of the succeeding sequential data is predicted from the token of the preceding sequential data. In particular, there are many cases where there is no correlation between different conceptual actions. Moreover, GPT2 only predicts the time series of concrete actions (actions) as tokens, and does not predict each conceptual action.

(2) It is not possible to recognize the user's conceptual behavior.
When only one concrete action is specified, multiple conceptual actions can be candidates . Even if the initial concrete action is the same, "approaching the kitchen," the sequence of subsequent concrete actions will be different depending on the conceptual actions, "cooking" and "washing dishes."

(3) It is not possible to predict the end of a conceptual action in a user's action sequence.
When the GPT2 model is applied, the end of sequential data is not recognized, and sequential data continues to be generated indefinitely.

The present invention aims to provide a prediction device, program, and method that can recognize conceptual behavior in a user's actions and predict the specific behavior that will become the next token using a sequence prediction model.

According to the present invention, a data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a prediction device for inputting a token sequence of a predetermined length and predicting the next token comprises:
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a control token assigning means for assigning a control token related to each piece of sequential data;
a padding means for padding a portion of each of the sequence data that does not reach a maximum length with padding;
a sequence prediction model for inputting a token sequence of a predetermined length for each of the sequence data filled with padding and predicting a next shifted token sequence ;
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
It is characterized by:

According to another embodiment of the prediction device of the present invention,
The control code classification model outputs a control code representing a user surrounding state,
It is also preferable that the control token providing means includes the control code output from the control code classification model as the control token.

According to another embodiment of the prediction device of the present invention,
It is also preferable that the control code classification model inputs measured values from one or more environmental sensors arranged around the user, and outputs, as a control code, a feature quantity based on the measured values.

According to another embodiment of the prediction device of the present invention,
The control code classification model inputs teacher data in which measurement values of environmental sensors arranged around the user are used as explanatory variables and the conceptual behavior of the user is associated with a target variable as a training phase, and inputs measurement values of environmental sensors arranged around the user as explanatory variables as an estimation phase, and outputs the conceptual behavior of the user and a first likelihood;
It is also preferable that the control code classification model outputs, as a control code, only a conceptual action of the user for which the first likelihood is equal to or greater than a predetermined threshold value.

According to another embodiment of the prediction device of the present invention,
The sequence prediction model outputs a second likelihood for the estimation stage;
It is also preferable that, when the sequence prediction model receives an input of a token sequence including a control token and outputs a next token sequence in which the second likelihood is equal to or greater than a predetermined threshold, the sequence prediction model further stores the input token sequence including the control token as an explanatory variable and the output next token sequence as a target variable as training data.

According to another embodiment of the prediction device of the present invention,
The sequence prediction model is trained to receive a token sequence of a predetermined length from 1 to n and output a token sequence of a predetermined length from 2 to n+1 that is the correct answer, and it is also preferable to recognize the end of one piece of sequence data by padding.

According to another embodiment of the prediction device of the present invention,
In the case where the nth token at the end of a token string of a predetermined length containing successive actions 1 to m (m<n) is padding, it is also preferable to input a token string of a predetermined length containing successive actions 1 to m-1, and train the system to output a token string of a predetermined length containing successive actions 2 to m, which is the correct answer.

According to another embodiment of the prediction device of the present invention,
It is also preferable that the sequence prediction model sets a loss function that reduces the error between the token sequence of the output data and the token sequence of the correct answer data.

According to another embodiment of the prediction device of the present invention,
It is also preferable that the tokens contained in the sequence data are arranged in chronological order.

According to another embodiment of the prediction device of the present invention,
It is also preferable that the sequence prediction model is a Generative Pre-trained Transformer (GPT).

According to the present invention, a data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a program for inputting a token string of a predetermined length and causing a computer to function to predict the next token comprises:
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a control token assigning means for assigning a control token related to each piece of sequential data;
a padding means for padding a portion of each of the sequence data that does not reach a maximum length with padding;
For each padded sequence data, a sequence prediction model is used to input a token sequence of a given length and predict the next shifted token sequence.
and make it work.
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
The present invention is characterized in that the computer is made to function in such a manner .

According to the present invention, a data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a prediction method for a device for inputting a token sequence of a predetermined length and predicting the next token comprises the steps of:
The device is
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a first step of assigning a control token associated with each piece of sequential data to the piece of sequential data;
a second step of filling the trailing portion of each sequence data with padding if the trailing portion does not reach the maximum length;
a third step of inputting a token sequence of a predetermined length for each of the padded sequence data using a sequence prediction model, and predicting a next shifted token sequence ;
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
It is characterized by:

The prediction device, program, and method of the present invention can recognize conceptual behavior in a user's actions and then use a sequence prediction model to predict the specific behavior that will become the next token.

FIG. 1 is an explanatory diagram showing a data structure during training of the GPT2 model in the prior art. FIG. 1 is an explanatory diagram showing the process of training the GPT2 model in the prior art. FIG. 2 is a functional configuration diagram of a prediction device according to the present invention. FIG. 2 is an explanatory diagram of sequence data in the present invention. FIG. 2 is an explanatory diagram of a control code classification model in the present invention. FIG. 1 is a first explanatory diagram illustrating a process during training of a prediction device according to the present invention. FIG. 2 is a second explanatory diagram showing the processing during training of the prediction device in the present invention.

The following describes in detail the embodiments of the present invention with reference to the drawings.

Figure 3 is a functional configuration diagram of the prediction device of the present invention.

As shown in FIG. 3, the prediction device 1 has a control token assignment unit 11, a padding unit 12, a sequence prediction model 13, and a control code classification model 14. In order to apply an environmental sensor, the prediction device 1 further has a measurement value collection unit 101 and a sequence data generation unit 102. These functional components are realized by executing a program that causes a computer installed in the prediction device to function. The processing flow of these functional components can also be understood as a prediction method.

A wide variety of environmental sensors are placed around the user as IoT (Internet of Things) devices. Examples of environmental sensors include a position sensor, a temperature sensor, an illuminance sensor, a motion sensor, a refrigerator sensor, a vibration sensor, a power sensor, and a water sensor. The measured values may also include the date and time. Furthermore, the measured values may also include weather information received from a weather server.
An embodiment of the present invention aims to predict the next behavior of a user with as high accuracy as possible from the most recent past measurement values of an environmental sensor based on the user's behavior.

[Measurement value collection unit 101]
The measurement value collector 101 collects measurement values from a plurality of environmental sensors. The collected measurement values are output to the sequence data generator 102. The measurement values may also be output to the control code classification model 14. These measurement values can be recognized as part of the user's behavior.

[Series Data Generation Unit 102]
The sequence data generator 102 generates "sequence data" as a data set of conceptual behaviors based on the measured values of the environmental sensors. The sequence data in the present invention is made up of "tokens" including multiple concrete behaviors (actions) classified for each conceptual behavior.

According to FIG. 3, for example, one conceptual behavior "cooking" is made up of specific behaviors (actions) in a time series as follows:
t1: "Approach the kitchen"
t2: "Open the refrigerator"
t3: "Close the refrigerator"
t4: "Turn on the stove"
...
The information for each specific action is based on the measurement values obtained from the sensors.

Each specific action is then included in a "token." A token is an information unit that includes a specific action. Multiple tokens are arranged in chronological order to form a single "sequence data." Sequential data is an information unit that represents a "conceptual action" consisting of multiple specific actions in chronological order.

Figure 4 is an explanatory diagram of sequence data in the present invention.

The sequence data generator 102 sets the maximum number of tokens among the multiple sequence data plus the control token as the "maximum length" of the sequence data.
Maximum length = (maximum number of tokens among multiple sequence data) + (number of control tokens)
The number of control tokens is basically one, but it may of course be two or more.

4, among sequential data sets 1 to 5, sequential data set 4 has the longest data length of 16. If the number of control tokens is 1, the maximum length of sequential data set is set to 17 as follows.
Maximum length 17 = Maximum number of tokens in multiple sequence data 16 + Number of control tokens 1

In other words, each conceptual behavior is composed as a single piece of sequence data. This unifies the number of tokens in all sequence data, and prevents multiple tokens in one sequence data from being split into different sequence data.

[Control token granting unit 11]
The control token assigner 11 assigns a "control token" related to each piece of sequential data to the corresponding piece of sequential data.
Control tokens represent the "conceptual actions" of a user.
The control code included in the control token may be specified by the user or may be output from the control code classification model 14. The control code is based on information representing the user surroundings in response to the user's conceptual behavior.

Clarifying the conceptual behavior first increases the accuracy of predicting the time series of subsequent concrete behaviors. For example, if the conceptual behavior is "cooking," it will be easier to predict that "approaching the kitchen" will be followed by "opening the refrigerator." Similarly, if the conceptual behavior is "washing dishes," it will be easier to predict that "approaching the kitchen" will be followed by "turning on the tap." In this way, by including conceptual behaviors in tokens, it becomes easier to predict concrete behaviors over time.

[Padding section 12]
The padding unit 12 fills the remaining part of each sequence data with padding to indicate the end of the sequence of conceptual actions.
According to FIG. 4, for example, padding is embedded as follows:
For sequence data 1, one control token and 10 tokens (0 to 9) are embedded. Then, padding is added to the last 6 tokens that do not reach the maximum length of 17.
For sequence data 2, one control token and three tokens (0 to 2) are embedded. Then, padding is added to the last 13 tokens that do not reach the maximum length of 17.
For sequence data 3, one control token and six tokens (0 to 5) are embedded. Then, padding is added to the last 10 tokens that do not reach the maximum length of 17.
For sequence data 4, one control token and 16 tokens (0 to 15) are embedded.
For sequence data 5, one control token and five tokens (0 to 4) are embedded. Then, padding is added to the last 11 tokens that do not reach the maximum length of 17.

Figure 5 is an explanatory diagram of the control code classification model in the present invention.

[Control Code Classification Model 14]
The control code classification model 14 outputs a "control code" representing the user surroundings state. The user surroundings information may be measurements received from environmental sensors.
In addition, the control code classification model 14 continues to estimate the control code according to one or more measurement values collected from a wide variety of environmental sensors (motion sensors, opening/closing sensors, etc.) at predetermined time intervals (for example, every 30 seconds).

As an embodiment of the control code classification model 14, for example, the following control codes are output.
First embodiment: Output of control code serving as feature quantity
Second embodiment: Output of control code representing conceptual behavior

First embodiment: Output of control code serving as feature quantity
The control code classification model 14 receives measurement values from one or more environmental sensors arranged around the user, and outputs a feature based on the measurement values as a control code. For example, it may output a time-series feature using a neural network.
According to FIG. 5, for example, if a refrigerator sensor detects whether the door is open or closed, and a power sensor detects the power of a stove, the feature extracted from the control code classification model 14 may represent, for example, the conceptual action “cooking.”
Also, for example, if a vibration sensor detects vibrations in the kitchen and a water sensor detects water use, the feature extracted from the control code classification model 14 may represent, for example, the conceptual action "washing dishes."
That is, a token represents a concrete ongoing action as a conceptual action of sequence data, and is considered to have some sort of relationship with the measured values of the environmental sensors.

Second embodiment: Output of control code representing conceptual behavior
The control code classification model 14 is a supervised learning engine and functions, for example, as follows.
(Training stage) Training is performed by inputting teacher data that associates the "measurement values of one or more environmental sensors" placed around the user as explanatory variables with the user's "conceptual behavior" as a response variable.
(Estimation stage) "Measurement values of one or more environmental sensors" placed around the user are input as explanatory variables, and the user's "conceptual behavior" and "first likelihood" are output.
Then, the control code classification model 14 outputs, as a control code, only the conceptual actions of the user for which the first likelihood is equal to or greater than a predetermined threshold value. By determining the first likelihood, it is possible to output only control codes with a high degree of certainty of estimation.

To output the first likelihood from the control code classification model 14, for example, a LightGBM (Light Gradient Boosting Machine) or a multilayer perceptron can be used. LightGBM is a gradient boosting machine learning engine based on a decision tree algorithm. A multilayer perceptron is a forward propagation neural network machine learning engine consisting of at least three layers.

[Series Prediction Model 13]
The sequence prediction model 13 inputs a token sequence of a predetermined length n for each padded sequence data, and predicts the next shifted token sequence.
The predetermined length n is the data length that can be input to the sequence prediction model 13 and is equal to or less than the maximum length mentioned above (n≦max_length). The predetermined length n depends on the sequence prediction model.

The sequence prediction model 13 itself may be a GPT2 model. According to the present invention, an existing language generation model is applied to predicting user behavior.

The sequence prediction model 13 is trained to input a token sequence of a predetermined length from 1 to n and output a token sequence of a predetermined length from 2 to n+1 which is the correct answer, and also recognizes the end of one piece of sequence data by padding.
According to the present invention, it is important to recognize the end of each conceptual action (sequence data) of the user, but the number of tokens until the end is different. The present invention does not make endless predictions from the preceding tokens like a sentence, but predicts the user's next concrete action in concrete action units, and predicts the end timing of the action, which differs for each conceptual action.

Figure 6 is a first explanatory diagram showing the processing during training of the prediction device of the present invention.

According to FIG. 6, for example, the sequence prediction model 13 is trained as follows, with the predetermined length=14.
Input data token sequence: F,0,1,2,3,4,5,6,7,8,9,10,11,12
Correct answer token sequence: 0,1,2,3,4,5,6,7,8,9,10,11,12,13
Output data token sequence: *,*,*,*,*,*,*,*,*,*,*,*,*, *, *, *, *
In this case, the machine learns to predict token 13 at the end of the correct data.

The sequence prediction model 13 sets a loss function that reduces the error between the token sequence of the output data as prediction data and the token sequence of the correct answer data. The loss function may be, for example, a cross entropy error.
This allows training to be performed so that the token sequence of the output data is trained to reduce the error between the token sequence of the correct answer data and the output data.

According to Figure 6, only token 13 is predicted from tokens 0 to 12, which include control token F. It is possible to learn by associating the control tokens representing conceptual behavior with tokens 0 to 12. In addition, the predicted token 13 can also be predicted based on the control tokens representing conceptual behavior and tokens 0 to 12.

Note that, according to Figure 6, it appears that the model is trained to predict only token 13, but inside sequence prediction model 13, it learns to be able to predict the next token, 1, when only token 0 is input. Also, when tokens 0 and 1 are input, it learns to be able to predict the next token, 2. By repeating this process, eventually, when tokens F, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12 are input, it will be able to predict token 13.

Figure 7 is a second explanatory diagram showing the processing during training of the prediction device of the present invention.

According to FIG. 7, the end of the token string of the input data is padding.
<Token sequence of sequence data: F,0,1,2,3,4,5,6,7,8,9,P,P,P> Predetermined length n = 14
Input data token sequence: F,0,1,2,3,4,5,6,7,8,P,P,P,P,
Token sequence of the correct data: 0,1,2,3,4,5,6,7,8,9,P,P,P,P
Output data token sequence: *,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*

Here, if the nth token at the end of a token string of a given length containing successive actions 1 to m (m < n) is padding, the system is trained to input a token string of a given length containing successive actions 1 to m-1, and output a token string of a given length containing successive actions 2 to m, which is the correct answer.

In the above example, the action token sequence in sequence data 1 is 10 tokens (m=10) from 0 to 9, which is shorter than the predetermined length n (=14), so the end is padded. In this case, a token sequence of the predetermined length n, in which actions 1 to m-1 (tokens 0 to 8) are consecutive, is input, and training is performed so that a token sequence of the predetermined length n, in which actions 2 to m (tokens 1 to 9) are consecutive, which is the correct answer, is output.
This means training the sequence prediction model 13 to predict the last token 9 before the padding. Finally, when the sequence prediction model 13 predicts only the padding, it is considered that the prediction of one conceptual behavior is completed.

<Further accumulation of training data>
It is assumed that the sequence prediction model 13 outputs a second likelihood for the estimation stage.
Here, when the sequence prediction model 13 receives a token sequence including a control token and outputs a next token sequence in which the second likelihood is equal to or greater than a predetermined threshold, the sequence prediction model 13 further stores the input token sequence including the control token (control token of conceptual behavior + token sequence of time-series specific behavior) as an explanatory variable and the output next token sequence (token sequence of time-series specific behavior) as a target variable as teacher data. The predetermined threshold is used as a reliability index of the likelihood.
Since the control tokens are based on the conceptual behavior of the user, the sequence prediction model 13 can be retrained with updated teacher data for the training phase.
This allows the amount of training data to be increased, and the accuracy of the sequence prediction model 13 to be improved.

As described above in detail, the prediction device, program, and method of the present invention can recognize conceptual behavior in a user's actions and then use a sequence prediction model to predict the specific behavior that will become the next token.

Furthermore, this will "promote the use of big data based on user behavior collected from home devices," which will contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation."

With respect to the various embodiments of the present invention described above, various changes, modifications, and omissions within the scope of the technical ideas and viewpoints of the present invention can be easily made by a person skilled in the art. The above description is merely an example and is not intended to be restrictive in any way. The present invention is limited only by the scope of the claims and their equivalents.

REFERENCE SIGNS LIST 1 Prediction device 101 Measurement value collection unit 102 Sequence data generation unit 11 Control token assignment unit 12 Padding unit 13 Sequence prediction model 14 Control code classification model

Claims (12)

A data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a prediction device for inputting a token sequence of a predetermined length and predicting the next token,
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a control token assigning means for assigning a control token related to each piece of sequential data;
a padding means for padding a portion of each of the sequence data that does not reach a maximum length with padding;
a sequence prediction model for inputting a token sequence of a predetermined length for each of the sequence data filled with padding and predicting a next shifted token sequence ;
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
A prediction device comprising: The control code classification model outputs a control code representing a user surrounding state,
2. The prediction device according to claim 1 , wherein the control token providing means includes a control code output from the control code classification model as a control token. 3. The prediction device according to claim 2 , wherein the control code classification model inputs measured values of one or more environmental sensors arranged around the user, and outputs a feature quantity based on the measured values as a control code. The control code classification model inputs teacher data in which measurement values of environmental sensors arranged around the user are used as explanatory variables and the conceptual behavior of the user is associated with a target variable as a training phase, and inputs measurement values of environmental sensors arranged around the user as explanatory variables as an estimation phase, and outputs the conceptual behavior of the user and a first likelihood;
4. The prediction device according to claim 3 , wherein the control code classification model outputs, as a control code, only a conceptual action of the user for which the first likelihood is equal to or greater than a predetermined threshold value. The sequence prediction model outputs a second likelihood for the estimation stage;
5. The prediction device according to claim 4, wherein the sequence prediction model inputs a token sequence including a control token, and when a next token sequence in which the second likelihood is equal to or greater than a predetermined threshold is output, further stores the input token sequence including the control token as an explanatory variable and the output next token sequence as a target variable as training data. The prediction device according to any one of claims 1 to 5, characterized in that the sequence prediction model is trained to input a token sequence of 1 to n of a predetermined length and output a token sequence of 2 to n+1 of a predetermined length as a correct answer, and recognizes the end of one piece of sequence data by padding. The prediction device according to claim 6, characterized in that, when the nth token at the end of a token string of a predetermined length in which actions 1 to m (m<n) are consecutive is padding, the prediction device is trained to input a token string of a predetermined length in which actions 1 to m-1 are consecutive, and to output a token string of a predetermined length in which actions 2 to m are consecutive, which is the correct answer. 6. The prediction device according to claim 1, wherein the sequence prediction model sets a loss function that reduces an error between a token sequence of the output data and a token sequence of the correct answer data. 6. The prediction device according to claim 1, wherein tokens included in the sequence data are arranged in a time series. 6. The prediction device according to claim 1 , wherein the sequence prediction model is a Generative Pre-trained Transformer (GPT). A data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a program for inputting a token sequence of a predetermined length and causing a computer to function to predict the next token,
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a control token assigning means for assigning a control token related to each piece of sequential data;
a padding means for padding a portion of each of the sequence data that does not reach a maximum length with padding;
For each padded sequence data, a sequence prediction model is used to input a token sequence of a given length and predict the next shifted token sequence.
and make it work.
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
A program that causes a computer to function in such a manner . A data set is made up of a plurality of sequence data, each sequence data being made up of a plurality of tokens, and a prediction method for a device for inputting a token sequence of a predetermined length and predicting the next token, comprising:
The device is
The maximum number of tokens among the multiple sequential data is added to the control token, and the sum is set as the maximum length of the sequential data.
a first step of assigning a control token associated with each piece of sequential data to the piece of sequential data;
a second step of filling the trailing portion of each sequence data with padding if the trailing portion does not reach the maximum length;
a third step of inputting a token sequence of a predetermined length for each of the padded sequence data using a sequence prediction model, and predicting a next shifted token sequence ;
Control tokens represent conceptual actions of a user;
Tokens in sequence data represent multiple concrete actions that divide conceptual actions.
A prediction method comprising:

Images