JP7769207B2 - Information processing program and information processing method - Google Patents

Description

The present invention relates to an information processing program and an information processing method.

There is currently a technology for identifying the time at which a target action is performed. The target action is, for example, performed by a person. For example, one method recognizes the basic movements of a person in each frame of a video, and identifies the time at which a pattern matching a predetermined rule appears in the time series of the recognized basic movements as the time at which the target action was performed.

Prior art, for example, involves identifying one or more second users who provide exercise support to a first user, extracting videos from a plurality of videos based on information indicating the support ratings of the identified second users, and generating a video list of the extracted videos. Another example is technology that generates a first image using a three-dimensional virtual human body model that mimics the human body, generates a training image in which each body part of the human body model in the first image is marked, generates a second image that includes noise, and trains the first model using the second image and the training image.

Japanese Patent Application Laid-Open No. 2019-180539 Japanese Patent Application Laid-Open No. 2021-043839

However, with conventional technology, it is difficult to accurately identify the time when a target action was performed. For example, if it is not possible to accurately define the pattern of basic actions that appear at the time when the target action was performed, it is not possible to accurately identify the time when the target action was performed.

In one aspect, the present invention aims to accurately identify the time when a target action was performed.

In one embodiment, an information processing program and information processing method are proposed that acquire first information identifying a period during which a target action was performed multiple times, acquire second information identifying multiple times within that period during which partial actions related to the target action were detected, calculate the length of time during which the target action was performed once based on the acquired second information, and identify each time during that period during which the target action was performed based on the acquired first information and the calculated length.

According to one aspect, it is possible to accurately identify the time when the target action was performed.

FIG. 1 is an explanatory diagram illustrating an example of an information processing method according to an embodiment. FIG. 2 is an explanatory diagram illustrating an example of an information processing system 200 . FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. FIG. 4 is a block diagram showing an example of the functional configuration of the information processing device 100. As shown in FIG. FIG. 5 is a block diagram showing a specific example of the functional configuration of the information processing device 100. FIG. 6 is an explanatory diagram (part 1) showing an example of the operation of the information processing device 100. FIG. 7 is an explanatory diagram (part 2) showing an example of the operation of the information processing device 100. FIG. 8 is an explanatory diagram (part 3) showing an example of the operation of the information processing device 100. FIG. 9 is an explanatory diagram (part 4) showing an example of the operation of the information processing device 100. FIG. 10 is an explanatory diagram (part 5) showing an example of the operation of the information processing device 100. FIG. 11 is an explanatory diagram (part 1) showing another example of calculating the length of each exercise time period. FIG. 12 is an explanatory diagram (part 2) showing another example of calculating the length of each exercise time period. FIG. 13 is an explanatory diagram (part 1) showing another example of identifying individual exercise time periods. FIG. 14 is an explanatory diagram (part 2) showing another example of identifying individual exercise time periods. FIG. 15 is an explanatory diagram (part 3) showing another example of identifying individual exercise time periods. FIG. 16 is an explanatory diagram (part 4) showing another example of identifying individual exercise time periods. FIG. 17 is an explanatory diagram (part 5) showing another example of identifying individual exercise time periods. FIG. 18 is an explanatory diagram (part 1) showing an example in which the end point of the target time period is used as the reference. FIG. 19 is an explanatory diagram (part 2) showing an example in which the end point of the target time period is used as the reference point. FIG. 20 is an explanatory diagram showing an example of evaluating athletic ability. FIG. 21 is an explanatory diagram showing an example of the effect of the information processing device 100. FIG. 22 is a flowchart illustrating an example of an overall processing procedure.

Below, embodiments of the information processing program and information processing method according to the present invention will be described in detail with reference to the drawings.

(Example of information processing method according to an embodiment)
FIG. 1 is an explanatory diagram illustrating an example of an information processing method according to an embodiment. An information processing device 100 is a computer for identifying a time when a target action is performed. The target action is, for example, performed by a person. The target action may be, for example, performed repeatedly. The target action is, for example, an exercise action.

In the past, it was sometimes desirable to analyze the status of a person's movements. For example, for purposes such as nursing care, rehabilitation, and fitness, it was sometimes desirable to analyze the status of a person's movements over a specified period of time. Specifically, it was sometimes desirable to identify the time during which a person made a movement or the timing at which a person's movement changed.

One possible approach is to attach pressure sensors to a person's hands and feet and recognize the person's movements based on the pressure sensor measurements. However, this approach can make it difficult to recognize the specific type of movement the person made. Therefore, it can be difficult to identify the time when the target movement was made. For more information on this approach, see, for example, Reference 1 below.

Reference 1: JP 2013-135874 A

Another possible method is to recognize the basic movements of a person for each frame contained in a video, and identify the time at which a pattern matching a predetermined rule appears in the time series of the recognized basic movements as the time at which the target movement was performed. However, with this method, it is difficult to accurately identify the time at which the target movement was performed. For example, if it is not possible to accurately determine the pattern of basic movements that appear at the time the target movement was performed, it is not possible to accurately identify the time at which the target movement was performed. Specifically, only a portion of the time at which the target movement was actually performed can be identified as the time at which the target movement was performed. Therefore, for example, if the target movement is performed repeatedly, it is not possible to identify the timing at which the target movement changed. For more information on this method, see, for example, Reference 2 below.

Reference 2: International Publication No. 2018/069981

As such, conventionally, it has not been possible to accurately identify the time when a target action was performed. Therefore, in this embodiment, we will describe an information processing method that can accurately identify the time when a target action was performed.

In FIG. 1, (1-1) the information processing device 100 acquires first information that identifies a period 110 during which a target action was performed multiple times. The first information is, for example, information that indicates at least one of the start and end points of the period 110. In the example of FIG. 1, the first information is specifically information that indicates the start point of the period 110.

(1-2) The information processing device 100 acquires second information that identifies multiple times 111 during which partial movements related to the target movement were detected within the period 110. A partial movement is, for example, any of multiple movements that form the target movement. The second information is, for example, information indicating the start or end of each time 111. In the example of FIG. 1, the second information is specifically information indicating the start of each time 111. The period 110 includes, for example, times 111 during which partial movements were detected and times during which partial movements were not detected.

The information processing device 100 obtains the second information by, for example, analyzing the video relating to the period 110, identifying multiple times 111 at which each partial action was detected, and generating the second information. Specifically, the information processing device 100 analyzes the video relating to the period 110, recognizes basic actions for each frame included in the video, and identifies the time at which a pattern representing a partial action appears in the time series of the basic actions as the time 111 at which the partial action was detected.

(1-3) The information processing device 100 calculates the length of the time 112 during which the target action was performed once, based on the acquired second information. For example, the information processing device 100 calculates the length of the time 112 during which the target action was performed once as the difference between the start of the time during which the first target action was performed and the start of the time during which the second target action was performed, among the multiple times.

(1-4) Based on the acquired first information and the calculated length, the information processing device 100 identifies each of the times 113 during which the target action was performed within the period 110. For example, based on the second information, the information processing device 100 identifies, as the times 113 during which the target action was performed, each of the times with a specified length corresponding to the number of times the target action was performed from the start of the period 110.

This allows the information processing device 100 to accurately identify each time 113 when the target action was performed. Based on each time 113 when the target action was performed, the information processing device 100 can easily and accurately identify the timing when the target action was switched.

The information processing device 100 outputs the times 113 at which the identified target actions were performed so that the user can refer to them. This allows the information processing device 100 to make the times 113 at which the target actions were performed available for purposes such as nursing care, rehabilitation, and fitness.

Here, we have described a case in which the information processing device 100 acquires the second information by analyzing video relating to the period 110 and generating it, but this is not limiting. For example, the information processing device 100 may acquire the second information by receiving it from another computer. The other computer has the function of, for example, analyzing video relating to the period 110 and identifying multiple times 111 at which each partial action was detected.

Here, we have described a case in which the information processing device 100 calculates the length of the time 112 during which one target action is performed based on the difference between the start of the time during which the first target action is performed and the start of the time during which the second target action is performed, but this is not limited to this. For example, the information processing device 100 may calculate the length of the time 112 during which one target action is performed based on the difference between the start of the time during which the i-th target action is performed and the start of the time during which the j-th target action is performed. i = positive integer ≦ N. j = positive integer ≦ N. i ≠ j. N = the number of times the target action is performed. An example of this case will be described in detail later using Figures 11 and 12.

Here, we have described a case in which the information processing device 100 identifies each of the times 113 at which the target action was performed using the number of times the target action was performed, but this is not limited to this. For example, the information processing device 100 may identify each of the times 113 at which the target action was performed without using the number of times the target action was performed. Specifically, the information processing device 100 may identify the amount of deviation between the start of the time 111 at which the partial action was detected and the start of the time 113 at which the target action was performed, and use the identified amount of deviation to identify each of the times 113 at which the target action was performed. An example of this case will be described later using Figures 13 to 17.

Here, we have described a case in which the information processing device 100 outputs each of the times 113 at which the target action was performed so that the user can refer to it, but this is not limited to this. For example, the information processing device 100 may identify the timing at which the target action was switched based on each of the times 113 at which the target action was performed, and output each of the identified timings so that the user can refer to it. An example of this case will be described in detail later using Figures 6 to 10.

Here, we have described a case in which the information processing device 100 determines the time 113 at which the target action was performed, starting from the start of the period 110 and working towards the end, but this is not limited to this. For example, the information processing device 100 may determine the time 113 at which the target action was performed, starting from the end of the period 110 and working towards the start. An example of this case will be described in detail later using Figures 18 and 19.

(An example of the information processing system 200)
Next, an example of an information processing system 200 to which the information processing device 100 shown in FIG. 1 is applied will be described with reference to FIG.

Figure 2 is an explanatory diagram showing an example of an information processing system 200. In Figure 2, the information processing system 200 includes an information processing device 100, a motion detection device 201, a client device 202, and a sensing device 203.

In the information processing system 200, the information processing device 100 and the motion detection device 201 are connected via a wired or wireless network 210. The network 210 may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. In the information processing system 200, the information processing device 100 and the client device 202 are connected via a wired or wireless network 210. In the information processing system 200, the motion detection device 201 and the sensing device 203 are connected via a wired or wireless network 210.

The information processing device 100 is a computer for identifying the time when a target action was performed by a subject. The information processing device 100 receives first information from the action detection device 201 that identifies a period during which the target action was performed multiple times. The information processing device 100 receives second information from the action detection device 201 that identifies multiple times during the period during which partial actions related to the target action were each detected.

The information processing device 100 calculates the length of time during which the target action was performed once based on the acquired second information. The information processing device 100 identifies each time during the period during which the target action was performed based on the acquired first information and the calculated length. The information processing device 100 identifies the timing at which the target action was switched based on each time during which the target action was performed.

The information processing device 100 performs a predetermined analysis process based on each time the target action is performed or the timing when the target action is switched. The predetermined analysis process is, for example, analyzing the implementation status of a therapy. The predetermined analysis process is, for example, analyzing a health status. The predetermined analysis process is, for example, analyzing cognitive ability. The information processing device 100 transmits the results of the predetermined analysis process to the client device 202. The information processing device 100 is, for example, a server or a PC (Personal Computer).

The motion detection device 201 is a computer for detecting that a partial motion has been performed. The motion detection device 201 receives information relating to a period during which a target motion was performed multiple times from the sensing device 203. The motion detection device 201 receives, for example, video relating to the period from the sensing device 203. The motion detection device 201 detects that a partial motion was performed within the period based on the received information. The motion detection device 201 transmits, to the information processing device 100, first information specifying the period during which the target motion was performed multiple times. The information processing device 100 transmits, to the information processing device 100, second information specifying multiple times during the period during which the partial motions were detected. The motion detection device 201 is, for example, a server or a PC.

The client device 202 is a computer used by a user. The client device 202 receives the results of a predetermined analysis process from the information processing device 100. The client device 202 outputs the results of the predetermined analysis process so that the user can refer to them. The client device 202 is, for example, a PC, a tablet terminal, or a smartphone.

The sensing device 203 is a computer for acquiring information regarding a period during which a target action was performed multiple times. The acquired information is information for detecting partial actions related to the target action. The sensing device 203, for example, acquires video images of a target person over a period of time and transmits them to the action detection device 201. The sensing device 203 may also acquire sensor values of the target person over a period of time and transmit them to the action detection device 201. The sensing device 203 is, for example, a smartphone, a wearable device, or a video capture device.

Here, we have described a case where the information processing device 100 is a device different from the motion detection device 201, but this is not limited to this. For example, the information processing device 100 may have the functions of the motion detection device 201 and also operate as the motion detection device 201.

Here, we have described a case where the information processing device 100 is a device different from the client device 202, but this is not limited to this. For example, the information processing device 100 may have the functionality of the client device 202 and also operate as the client device 202.

Here, we have described a case where the motion detection device 201 is a device different from the sensing device 203, but this is not limited to this. For example, the motion detection device 201 may have the functions of the sensing device 203 and also operate as the sensing device 203.

(Example of use of information processing system 200)
The information processing system 200 can be used, for example, to analyze whether a subject is exercising in sync with others and evaluate the subject's athletic ability or cognitive ability. The information processing system 200 can be used, for example, to analyze whether a subject is exercising as instructed according to an exercise menu, such as radio calisthenics, and evaluate the subject's athletic ability or cognitive ability. Therefore, the information processing system 200 can easily evaluate whether a subject is appropriately undergoing therapy, such as rehabilitation, to maintain physical function. The information processing system 200 can reduce the workload of a user who cares for or instructs the subject. The user can be, for example, a therapist.

(Example of hardware configuration of information processing device 100)
Next, an example of the hardware configuration of the information processing device 100 will be described with reference to FIG.

Figure 3 is a block diagram showing an example of the hardware configuration of information processing device 100. In Figure 3, information processing device 100 has a CPU (Central Processing Unit) 301, memory 302, network I/F (Interface) 303, recording medium I/F 304, and recording medium 305. Furthermore, each component is connected to each other via bus 300.

Here, CPU 301 is responsible for overall control of information processing device 100. Memory 302 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as a work area for CPU 301. Programs stored in memory 302 are loaded into CPU 301, causing CPU 301 to execute the coded processes.

Network I/F 303 is connected to network 210 via a communication line, and is connected to other computers via network 210. Network I/F 303 manages the internal interface with network 210 and controls the input and output of data from other computers. Network I/F 303 is, for example, a modem or LAN adapter.

The recording medium I/F 304 controls the reading and writing of data from and to the recording medium 305 under the control of the CPU 301. The recording medium I/F 304 is, for example, a disk drive, a solid state drive (SSD), or a universal serial bus (USB) port. The recording medium 305 is non-volatile memory that stores data written under the control of the recording medium I/F 304. The recording medium 305 is, for example, a disk, a semiconductor memory, or a USB memory. The recording medium 305 may be detachable from the information processing device 100.

In addition to the components described above, the information processing device 100 may also include, for example, a keyboard, mouse, display, printer, scanner, microphone, and speaker. The information processing device 100 may also include multiple recording medium I/Fs 304 and recording media 305. The information processing device 100 may also not include recording medium I/Fs 304 and recording media 305.

(Example of hardware configuration of the motion detection device 201)
A specific example of the hardware configuration of the motion detection device 201 is similar to the example of the hardware configuration of the information processing device 100 shown in FIG. 3, and therefore a description thereof will be omitted.

(Example of hardware configuration of client device 202)
The hardware configuration of the client device 202 is similar to that of the information processing device 100 shown in FIG. 3, and therefore will not be described further.

(Example of hardware configuration of sensing device 203)
Specifically, an example of the hardware configuration of the sensing device 203 is similar to the example of the hardware configuration of the information processing device 100 shown in Fig. 3, but may further include a sensor. The sensor measures information about the subject.

The sensor may include, for example, an imaging element and capture video images of the subject. The sensor may detect, for example, the state of the sensing device 203 as the state of the subject. The state of the sensing device 203 may be, for example, the position, orientation, or movement of the sensing device 203. The sensor may specifically include an acceleration sensor, a geomagnetic sensor, an optical sensor, or a vibration sensor and detect the state of the sensing device 203. The sensor may specifically include a GPS (Global Positioning System) receiver and detect the position of the sensing device 203.

(Example of functional configuration of information processing device 100)
Next, an example of the functional configuration of the information processing device 100 will be described with reference to FIG.

Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a memory unit 400, an acquisition unit 401, a detection unit 402, a calculation unit 403, an identification unit 404, and an output unit 405.

The storage unit 400 is realized, for example, by a storage area such as the memory 302 or recording medium 305 shown in FIG. 3. Below, a case where the storage unit 400 is included in the information processing device 100 will be described, but this is not limited to this. For example, the storage unit 400 may be included in a device different from the information processing device 100, and the contents stored in the storage unit 400 may be accessible from the information processing device 100.

The acquisition unit 401 to the output unit 405 function as an example of a control unit. Specifically, the acquisition unit 401 to the output unit 405 realize their functions by, for example, having the CPU 301 execute a program stored in a storage area such as the memory 302 or recording medium 305 shown in FIG. 3, or by using the network I/F 303. The processing results of each functional unit are stored in a storage area such as the memory 302 or recording medium 305 shown in FIG. 3.

The storage unit 400 stores various information that is referenced or updated during the processing of each functional unit. The storage unit 400 stores, for example, first information. The first information is, for example, information that makes it possible to identify a predetermined period during which a target action was performed multiple times. The first information may be, for example, information that makes it possible to identify a predetermined period during which different types of target actions were each performed multiple times. Specifically, the first information is information that indicates at least either the start or end of the predetermined period. The first information is acquired, for example, by the acquisition unit 401.

The specified period includes, for example, a time during which a partial action related to the target action is detected and a time during which a partial action related to the target action is not detected. A partial action is, for example, an action that forms the target action. Specifically, a partial action is an action that is part of the target action. A partial action may, for example, be an action incidental to the target action. Specifically, a partial action may be an action performed as a pre-stage of the target action. Specifically, a partial action may be an action performed as post-processing of the target action. A time during which a partial action is not detected can be considered to be, for example, a time during which the target action is performed even if a partial action is not detected.

The storage unit 400 stores, for example, second information. The second information is, for example, information that enables identification of multiple times during a predetermined period when partial actions related to the target action were each detected. The second information may be, for example, information that enables identification of multiple times during a predetermined period when multiple partial actions related to different types of target action were each detected. Specifically, the second information is information that indicates at least either the start or end of the time when the partial actions were detected. The second information is, for example, acquired by the acquisition unit 401. The second information may also be generated by the detection unit 402.

The acquisition unit 401 acquires various information used for processing by each functional unit. The acquisition unit 401 stores the acquired various information in the storage unit 400 or outputs it to each functional unit. The acquisition unit 401 may also output the various information stored in the storage unit 400 to each functional unit. The acquisition unit 401 acquires various information based on, for example, a user's operational input. The acquisition unit 401 may also receive various information from, for example, a device other than the information processing device 100.

The acquisition unit 401 acquires the first information. For example, the acquisition unit 401 acquires the first information by receiving it from another computer. The other computer may be, for example, the motion detection device 201. The acquisition unit 401 may acquire the first information by accepting input of the first information based on operation input by the user, for example.

The acquisition unit 401 acquires the second information. For example, the acquisition unit 401 acquires the second information by receiving it from another computer. The other computer may be, for example, the motion detection device 201. The acquisition unit 401 may acquire the second information by accepting input of the second information based on operation input by the user, for example.

The acquisition unit 401 may acquire moving images for a predetermined period. For example, if the acquisition unit 401 does not acquire second information, it acquires moving images for a predetermined period. For example, if the acquisition unit 401 acquires moving images for a predetermined period, it does not need to acquire first information. For example, the acquisition unit 401 acquires moving images by receiving them from another computer. The other computer is, for example, the motion detection device 201. For example, the acquisition unit 401 may acquire moving images by accepting input of moving images based on operation input by the user. This allows the acquisition unit 401 to enable the detection unit 402 to generate second information.

The acquisition unit 401 may receive a start trigger to start processing by one of the functional units. The start trigger may be, for example, a predetermined operational input by the user. The start trigger may be, for example, the receipt of predetermined information from another computer. The start trigger may be, for example, the output of predetermined information by one of the functional units. The acquisition unit 401 may, for example, receive the first information and the second information as a start trigger to start processing by the calculation unit 403. The acquisition unit 401 may, for example, receive a moving image as a start trigger to start processing by the detection unit 402.

The detection unit 402 generates second information. For example, the detection unit 402 recognizes basic movements for each frame included in the acquired video, and detects patterns representing partial movements from the time series of the recognized basic movements. The detection unit 402 generates second information that identifies, for example, multiple times during a predetermined period at which patterns representing partial movements were detected. This allows the detection unit 402 to acquire the second information even if the second information is not generated externally.

The detection unit 402 may generate first information. For example, the detection unit 402 identifies a predetermined period based on the acquired video and generates first information that identifies the predetermined period. This allows the detection unit 402 to acquire the first information even if the first information is not generated externally.

The calculation unit 403 calculates the length of time during which the target action was performed once based on the acquired second information. For example, the calculation unit 403 calculates the length of time during which the target action was performed once based on the acquired second information, using the difference between the start or end points of any two of the time periods included in the multiple time periods.

Specifically, the calculation unit 403 calculates the difference between the start or end points of two adjacent periods included in the multiple periods as the length of time during which the target action was performed once. More specifically, the calculation unit 403 calculates the difference between the start point of the period during which the nth partial action was performed and the start point of the period during which the n+1th partial action was performed as the length of time during which the target action was performed once. n = positive integer≦N. N = the number of times the target action was performed. This allows the calculation unit 403 to obtain a guideline for identifying the time during which the target action was performed.

Specifically, the calculation unit 403 may calculate the length of time during which the target action was performed once by dividing the difference between the start or end points of two non-adjacent time periods included in the multiple time periods by the number of time periods during which the partial action included in the difference was detected. More specifically, the calculation unit 403 calculates the difference between the start point of the time during which the i-th target action was performed and the start point of the time during which the j-th target action was performed. i = positive integer ≦ N. j = positive integer ≦ N. i ≠ j. More specifically, the calculation unit 403 calculates the value obtained by dividing the calculated difference by |j - i| as the length of time during which the target action was performed once. This allows the calculation unit 403 to obtain a guideline for identifying the time during which the target action was performed.

For example, the calculation unit 403 may calculate, based on the acquired second information, the length of time for which each of a plurality of target actions of different types was performed once. This allows the calculation unit 403 to obtain a guideline for identifying the length of time for which each of a plurality of target actions of different types was performed.

The calculation unit 403 may, for example, perform statistical processing on the differences between the start or end points of two time periods included in the multiple time periods based on the acquired second information, and calculate the length of time during which the target action was performed once. Specifically, the calculation unit 403 calculates the difference between the start points of each of two adjacent time periods included in the multiple time periods for each pair of the two time periods. Specifically, the calculation unit 403 calculates the statistical value of the calculated difference as the length of time during which the target action was performed once. The statistical value may be, for example, a maximum value, minimum value, average value, mode, or median. This allows the calculation unit 403 to obtain a guideline for identifying the time during which the target action was performed. For example, the calculation unit 403 can accurately calculate the length of time during which the target action was performed once, even if there is a blank period during which no action was performed between the start or end points of any two of the multiple time periods.

The identification unit 404 identifies each time during a predetermined period when the target action was performed based on the acquired first information and the calculated length. The identification unit 404 identifies, for example, the number of times the partial action was detected based on the acquired second information. The identification unit 404 identifies, for example, each time during which the target action was performed, having a calculated length, using the number of times the partial action was detected, based on the start or end of the predetermined period, based on the acquired first information and the calculated length. Specifically, the identification unit 404 identifies, as the time during which the target action was performed, each time having a calculated length equal to the number of times the partial action was detected, based on the start of the predetermined period. This allows the identification unit 404 to make the time during which the target action was performed available.

The identification unit 404 may, for example, identify a predetermined deviation amount using the difference between the start or end of a predetermined period and the start or end of any of the times when a partial action was detected, based on the acquired first information and the acquired second information. The predetermined deviation amount is the deviation amount between the start or end of each of the multiple times and the start or end of the time when the target action, including that time, was performed. The identification unit 404, for example, uses the identified deviation amount to identify each of the times during the predetermined period when the target action was performed, having the calculated length.

Specifically, the identification unit 404 identifies the difference between the start of a predetermined period and the start of one of the times when a partial action was detected as a predetermined deviation amount based on the acquired first information and the acquired second information. The predetermined deviation amount is the deviation between the start of each of the multiple times and the start of the time when a target action was performed, including that time. Specifically, the identification unit 404 identifies a time that is the identified deviation amount back from the start of each of the multiple times as the start of the time when the target action was performed, including that time. Specifically, the identification unit 404 identifies a period of time having a calculated length, starting from the identified start point, as the time when the target action was performed. This allows the identification unit 404 to make the time when the target action was performed available. For example, the identification unit 404 can accurately identify the time when the target action was performed even if there is a blank period in which no action was performed between the start or end of any two of the multiple times.

The identification unit 404 may identify a predetermined amount of deviation, for example, by using the difference between the start or end of any known time period during which the target action was performed and the start or end of any time period during which a partial action was detected. The identification unit 404 may, for example, use the identified amount of deviation to identify each time period during which the target action was performed, having a calculated length, within a predetermined period. This allows the identification unit 404 to make the time during which the target action was performed available. The identification unit 404 can accurately identify the time during which the target action was performed, even if, for example, there is a blank period during which no action was performed between the start or end of any two of the multiple times.

The identification unit 404 may, for example, identify the order in which multiple target actions of different types were performed, starting from the beginning of a predetermined period. The identification unit 404, for example, identifies the times during which each of the multiple target actions was performed within a predetermined period, starting with the target action that was performed first, based on the acquired first information, the identified order, and the calculated length. This allows the identification unit 404 to make available the times during which each of the multiple target actions of different types was performed.

The identification unit 404 may identify the timing at which the time at which the target action is performed changes, based on each of the times at which the identified target action is performed. For example, the identification unit 404 identifies the timing after the end of a first time period in which the identified target action is performed and before the start of a second time period in which the identified target action, adjacent to the first time period, is performed, as the timing at which the time at which the target action is performed changes. This allows the identification unit 404 to make available the timing at which the time at which the target action is performed changes.

The identification unit 404 may evaluate the motor ability of the subject of the target action. The identification unit 404 evaluates the motor ability of the subject of the target action, for example, based on each of the times at which the identified target action was performed. Specifically, the identification unit 404 identifies the amount of deviation between each of the times at which the identified target action was performed and a predetermined time. The predetermined time is, for example, a predetermined time at which the subject of the target action is instructed to perform the target action.

Specifically, if any of the identified deviation amounts is equal to or greater than a threshold, the identification unit 404 evaluates that the motor ability of the subject of the target movement is relatively low. Specifically, if any of the identified deviation amounts is less than a threshold, the identification unit 404 evaluates that the motor ability of the subject of the target movement is relatively high. This allows the identification unit 404 to evaluate the motor ability of the subject of the target movement, making it easier to manage the health condition, etc., of the subject of the target movement.

The identification unit 404 may evaluate the motor ability of the subject of the target action, for example, based on the timing at which the time at which the identified target action is performed changes. Specifically, the identification unit 404 determines whether each identified timing is delayed by a certain amount of time or more from a predetermined timing. The predetermined timing is, for example, a preset timing at which the subject of the target action is instructed to perform the target action.

Specifically, if any of the identified timings is delayed from the predetermined timing by more than a certain time, the identification unit 404 evaluates that the motor ability of the subject of the target movement is relatively low. Specifically, if none of the identified timings is delayed from the predetermined timing by more than a certain time, the identification unit 404 evaluates that the motor ability of the subject of the target movement is relatively high. This allows the identification unit 404 to evaluate the motor ability of the subject of the target movement, making it easier to manage the health condition, etc., of the subject of the target movement.

The output unit 405 outputs the processing results of at least one of the functional units. The output format may be, for example, display on a display, printout on a printer, transmission to an external device via the network I/F 303, or storage in a storage area such as the memory 302 or recording medium 305. This allows the output unit 405 to notify the user of the processing results of at least one of the functional units, thereby improving the convenience of the information processing device 100.

The output unit 405 outputs each of the times at which the target actions were performed, identified by the identification unit 404. The output unit 405, for example, outputs each of the times at which the target actions were performed so that the user can refer to it. The output unit 405 may, for example, transmit each of the times at which the target actions were performed to another computer. This allows the output unit 405 to make each of the times at which the target actions were performed available. The output unit 405, for example, allows the motor skills of the subject of the target actions to be evaluated externally.

The output unit 405 outputs each of the timings at which the time at which the target action was performed changes, identified by the identification unit 404. For example, the output unit 405 outputs each of the timings at which the time at which the target action was performed changes so that the user can refer to it. For example, the output unit 405 may transmit each of the timings at which the time at which the target action was performed changes to another computer. In this way, the output unit 405 can make each of the timings at which the time at which the target action was performed changes available. For example, the output unit 405 can make it possible to externally evaluate the motor skills of the subject of the target action.

The output unit 405 outputs information indicating the athletic ability of the subject of the target action, evaluated by the identification unit 404. This allows the output unit 405 to make available information indicating the athletic ability of the subject of the target action. The output unit 405 can, for example, make it easier to manage the health status of the subject of the target action.

Here, we have described a case in which the information processing device 100 includes an acquisition unit 401, a detection unit 402, a calculation unit 403, an identification unit 404, and an output unit 405, but this is not limited to this. For example, the information processing device 100 may not include any of the functional units. Specifically, the information processing device 100 may not include the detection unit 402. In this case, the information processing device 100 may be able to communicate with another computer that includes the detection unit 402.

(Specific example of functional configuration of information processing device 100)
Next, a specific example of the functional configuration of the information processing device 100 will be described with reference to FIG.

Figure 5 is a block diagram showing a specific example of the functional configuration of the information processing device 100. The information processing device 100 includes a target action time calculation unit 501, an action start lag calculation unit 502, an action start time calculation unit 503, and an action end time calculation unit 504.

The information processing device 100 acquires the start and end points of a target time period during which a target action is performed multiple times. The target action is, for example, exercise. The information processing device 100 acquires the start and end points of a time period during which a partial action related to the target action is detected within the target time period. In the following description, the time period during which a partial action is detected may be referred to as the "detection time period." The information processing device 100 acquires the number of times the target action is performed within the target time period. In the following description, the number of times the target action is performed may be referred to as the "number of actions." Multiple target actions of different types may each be performed multiple times within the target time period. In the following description, the type of target action may be referred to as the "type of action."

For each action type, the target action time calculation unit 501 calculates the length of the time period during which the target action of that action type was performed. In the following explanation, the time period during which the target action was performed may be referred to as the "action time period." For example, the target action time calculation unit 501 acquires the start or end point of the detection time period corresponding to that action type for each action type. For example, the target action time calculation unit 501 calculates the length of the action time period corresponding to that action type based on the start or end point of the acquired detection time period for each action type. Specifically, the target action time calculation unit 501 calculates the difference between the start points of the first detection time period and the second detection time period among the acquired detection time periods for each action type as the length of the action time period.

The action start delay calculation unit 502 calculates, for each action type, the magnitude of the delay between the start of the detection time period and the start of the action time period that includes that detection time period. In the following description, this delay may be referred to as the "action start delay." The action start delay calculation unit 502, for example, acquires the start of the target time period. For example, the action start delay calculation unit 502 acquires the start of the first detection time period for the first action type performed out of multiple action types. The action start delay calculation unit 502 calculates, for example, the difference between the start of the acquired target time period and the start of the acquired first detection time period as the action start delay corresponding to the first action type performed out of multiple action types.

The action start delay calculation unit 502 obtains, for example, the latest end time calculated by the action end time calculation unit 504 for the action type performed for the i-th time out of multiple action types. i = positive integer ≧ 2. The action start delay calculation unit 502 obtains, for example, the start time of the first detection time period for the action type performed for the i-th time. The action start delay calculation unit 502 calculates, for example, the difference between the latest end time obtained and the start time of the first detection time period obtained as the action start delay corresponding to the action type performed for the i-th time out of multiple action types.

The operation start time calculation unit 503 calculates the start time of an operation time period for each operation type. For example, the operation start time calculation unit 503 acquires the operation start delay calculated by the operation start delay calculation unit 502 for the operation type that was performed nth out of multiple operation types and for which the operation start delay has been calculated. The operation start time calculation unit 503 acquires the start time of each detection time period for the operation type that was performed nth and for which the operation start delay has been calculated. For the operation type that was performed nth and for which the operation start delay has been calculated, the operation start time calculation unit 503 calculates the start time of the operation time period including the detection time period as the time that is backdated by the acquired operation start delay from the start time of each detection time period.

The operation end time calculation unit 504 calculates the end time of the operation time period for each operation type. For example, for the nth operation type, the operation end time calculation unit 504 obtains the length of the operation time period calculated by the target operation time calculation unit 501. For example, for the nth operation type, the operation end time calculation unit 504 obtains the start time of each operation time period calculated by the operation start time calculation unit 503. For example, for the nth operation type, the operation end time calculation unit 504 calculates the end time of the operation time period to be the time that is the length of the obtained operation time period after the start time of each obtained operation time period.

For each type of movement, the information processing device 100 outputs the start and end times of the movement time period corresponding to that movement type. This allows the information processing device 100 to make available each movement time period for each type of movement. The information processing device 100 can also make available the timing at which the time when the target movement is performed changes for each type of movement. This allows the information processing device 100 to evaluate the motor skills of the subject of the target movement.

(Example of operation of information processing device 100)
Next, an example of the operation of the information processing device 100 will be described with reference to Figures 6 to 10. In the following description, the target operation is assumed to be "exercise" by the subject. The type of operation is assumed to be "type of exercise." The type of exercise is assumed to be, for example, "squats" and "lunges."

Figures 6 to 10 are explanatory diagrams showing an example of the operation of the information processing device 100. In Figure 6, the information processing device 100 acquires a video image showing a target person during a target time period 600. In the example of Figure 6, the target time period 600 is represented by a row of rectangles arranged in chronological order, each representing a unit of time.

Specifically, during target time period 600, squats are performed twice and lunges are performed twice. There is a time when the type of exercise changes between the second squat and the first lunge.

The information processing device 100 determines the start time of the target time period 600 and the end time of the target time period 600. The information processing device 100 determines the number of times that squats were performed during the target time period 600 = 2. The information processing device 100 determines the number of times that lunges were performed during the target time period 600 = 2.

For each type of exercise, the information processing device 100 detects frames from the video in which characteristics of partial movements related to that type of exercise appear. A partial movement is, for example, a part of an exercise. For each type of exercise, the information processing device 100 identifies the time period corresponding to the detected frame as the time period in which part of the exercise of that type of exercise was detected. In the following description, the time period in which part of the exercise was detected may be referred to as the "detection time period."

In the example of Figure 6, the information processing device 100 identifies a detection time period 601 in which part of the squat exercise was detected. The information processing device 100 identifies a detection time period 602 in which part of the lunge exercise was detected. For convenience, among the row of rectangles representing the target time period 600, rectangles corresponding to the identified detection time period are represented with diagonal hatching. Next, we move on to the explanation of Figure 7.

In Figure 7, the information processing device 100 sorts multiple exercise types in the order in which they were performed and identifies the first exercise type performed, "squats." The information processing device 100 calculates the length of a time period during which one session of the identified squat exercise was performed. In the following explanation, the time period during which one session of exercise was performed may be referred to as an "individual exercise time period" or simply as an "exercise time period." For example, for the identified squat exercise, the information processing device 100 calculates the difference between the start of the first detection time period 601 and the start of the second detection time period 601 as the length 701 of the individual exercise time period. Next, we move on to the explanation of Figure 8.

In Figure 8, the information processing device 100 sets the start of the target time period 600 as the start of the overall exercise time period during which squats were performed. The overall exercise time period includes individual exercise time periods. The information processing device 100 identifies the time period from the set start time that corresponds to the length 701 of the identified individual exercise time period x the number of times squats were performed = 2 as the overall exercise time period during which squats were performed. The information processing device 100 divides the identified overall time period equally by the number of times squats were performed = 2, and identifies each time period as an individual exercise time period 801 during which squats were performed. Next, we will move on to the explanation of Figure 9.

In Figure 9, the information processing device 100 identifies the type of exercise performed after squats, "lunges." The information processing device 100 calculates the length of a time period during which one session of the identified lunge exercise was performed. For example, for the identified lunge exercise, the information processing device 100 calculates the difference between the start of the first detection time period 602 and the start of the second detection time period 602 as the length 901 of the individual exercise time period. Next, we move on to the explanation of Figure 10.

In FIG. 10, the information processing device 100 sets the end point of the overall exercise time period during which squats were performed as the start point of the overall exercise time period during which lunges were performed. The overall exercise time period includes individual exercise time periods. The information processing device 100 identifies the time period from the set start point, which is the length 901 of the identified individual exercise time period x the number of lunges performed = 2, as the overall exercise time period during which lunges were performed. The information processing device 100 divides the identified overall time period equally by the number of lunges performed = 2, and identifies each time period as an individual exercise time period 1001 during which lunges were performed.

The information processing device 100 may identify the timing when the squat exercise was switched based on the identified individual exercise time period 801. The information processing device 100 may identify the timing when the lunge exercise was switched based on the identified individual exercise time period 1001. The information processing device 100 may identify the timing when the exercise type was switched based on the identified overall exercise time period during which the squat was performed and the identified overall exercise time period during which the lunge was performed.

This allows the information processing device 100 to accurately identify individual exercise time periods 801 so that time periods in which squats were performed that exist outside of the detection time period 601 are included in the individual exercise time period 801 in which the identified squats were performed. Similarly, the information processing device 100 can accurately identify individual exercise time periods 1001 so that time periods in which lunges were performed that exist outside of the detection time period 601 are included in the individual exercise time period 1001 in which the identified lunges were performed.

The information processing device 100 can then accurately identify the timing when the squat exercise is switched, the timing when the lunge exercise is switched, and the timing when the type of exercise is switched. Therefore, the information processing device 100 can accurately analyze the motor ability or cognitive ability of the subject. The information processing device 100 can accurately analyze the motor ability or cognitive ability of the subject, for example, for purposes such as nursing care, rehabilitation, or fitness. An example of the effect of the information processing device 100 will be described in detail later using FIG. 21.

Here, we have described a case in which the information processing device 100 calculates, for each type of exercise, the difference between the start of the first detection time period and the start of the second detection time period for that type of exercise as the length of an individual exercise time period, but this is not limited to this. For example, the information processing device 100 may calculate, for each type of exercise, the difference between the start of the nth detection time period and the start of the (n+1)th detection time period for that type of exercise as the length of an individual exercise time period.

(Another example of calculating the length of an individual exercise window)
Next, another example in which the information processing device 100 calculates the length of each exercise time period will be described with reference to FIGS. 11 and 12. FIG.

Figures 11 and 12 are explanatory diagrams showing another example of calculating the length of an individual exercise period. For example, the examples in Figures 6 to 10 describe a case in which the information processing device 100 calculates, for each type of exercise, the difference between the start point of the first detection period and the start point of the second detection period as the length of the individual exercise period.

In contrast, the example in Figure 11 describes a case in which the information processing device 100 calculates the length of an individual exercise time period for each exercise type based on the difference between the start of the nth detection time period and the start of the (n+m)th detection time period for that exercise type, where m is greater than or equal to 2.

In FIG. 11, the information processing device 100 acquires video images of a subject during a target time period 1100. In the example of FIG. 11, the target time period 1100 is represented by a series of rectangles representing unit times arranged in chronological order. Specifically, during the target time period 1100, squats were performed four times and lunges were performed a number of times, where a≧1.

The information processing device 100 identifies the start point of the target time period 1100 and the end point of the target time period 1100. The information processing device 100 identifies the number of times that squats were performed during the target time period 1100 as 4. The information processing device 100 identifies the number of times that lunges were performed during the target time period 1100 as a.

For each type of exercise, the information processing device 100 detects frames from the video in which characteristics of partial movements related to that type of exercise appear. For each type of exercise, the information processing device 100 identifies the time period corresponding to the detected frames as the detection time period in which part of the movement of that type of exercise was detected.

In the example of FIG. 11, the information processing device 100 identifies a detection time period 1101 in which part of the squat exercise was detected. The information processing device 100 identifies a detection time period 1102 in which part of the lunge exercise was detected. For convenience, among the row of rectangles representing the target time period 1100, rectangles corresponding to the identified detection time period are represented with diagonal hatching.

The information processing device 100 sorts the multiple exercise types in the order in which they were performed and identifies the exercise type "squats" that was performed first. For the identified squat exercise, the information processing device 100 calculates the length of each individual exercise time period in which one session of the exercise was performed.

For example, for the identified squat exercise, the information processing device 100 identifies the difference between the start of the first detection time period 1101 and the start of the fourth and final detection time period 1101 = 12 unit times. For example, the information processing device 100 divides the identified 12 unit times by the number of times the squat exercise has been performed minus 1 = 3, to calculate the length of the individual exercise time period = 4 unit times. Similarly, the information processing device 100 calculates the length of the individual exercise time period for, for example, lunges.

As a result, the information processing device 100 can accurately calculate the length of each individual exercise time period for each exercise type, even if there is variation in the time intervals between the detection time periods among the multiple detection time periods corresponding to that exercise type. For example, for squat exercise, the information processing device 100 can accurately calculate the length of each individual exercise time period even if there is a blank period in which no movement is performed between the nth detection time period 1201 and the n+1th detection time period 1201. Therefore, the information processing device 100 can accurately identify each individual exercise time period.

The example in Figure 12 illustrates a case in which the information processing device 100 calculates the length of an individual exercise time period for each type of exercise based on the statistical value of the difference between the start times of two adjacent detection time periods.

In FIG. 12, the information processing device 100 acquires video images of a subject during a target time period 1200. In the example of FIG. 12, the target time period 1200 is represented by a series of rectangles representing unit times arranged in chronological order. Specifically, during the target time period 1200, squats were performed four times and lunges were performed a number of times, where a≧1.

The information processing device 100 identifies the start point of the target time period 1200 and the end point of the target time period 1200. The information processing device 100 identifies the number of times that squats were performed during the target time period 1200 as 4. The information processing device 100 identifies the number of times that lunges were performed during the target time period 1200 as a.

For each type of exercise, the information processing device 100 detects frames from the video in which characteristics of partial movements related to that type of exercise appear. For each type of exercise, the information processing device 100 identifies the time period corresponding to the detected frames as the detection time period in which part of the movement of that type of exercise was detected.

In the example of FIG. 12, the information processing device 100 identifies a detection time period 1201 in which part of the squat exercise was detected. The information processing device 100 identifies a detection time period 1202 in which part of the lunge exercise was detected. For convenience, among the row of rectangles representing the target time period 1200, rectangles corresponding to the identified detection time period are represented with diagonal hatching.

The information processing device 100 sorts the multiple exercise types in the order in which they were performed and identifies the exercise type "squats" that was performed first. For the identified squat exercise, the information processing device 100 calculates the length of each individual exercise time period in which one session of the exercise was performed.

For example, for the identified squat exercise, the information processing device 100 determines that the difference between the start of the first detection time period 1201 and the start of the second detection time period 1201 is 5 unit times. Similarly, for example, for the identified squat exercise, the information processing device 100 determines that the difference between the start of the second detection time period 1201 and the start of the third detection time period 1201 is 4 unit times. Similarly, for example, for the identified squat exercise, the information processing device 100 determines that the difference between the start of the third detection time period 1201 and the start of the fourth detection time period 1201 is 4 unit times.

The information processing device 100, for example, calculates the statistical value of the identified difference as the length of each individual exercise time period. The statistical value may be, for example, a maximum value, minimum value, average value, mode, or median. Specifically, the information processing device 100 calculates the minimum value of the identified difference = 4 unit times as the length of each individual exercise time period.

As a result, the information processing device 100 can accurately calculate the length of each individual exercise time period for each exercise type, even if there is variation in the time intervals between the detection time periods among the multiple detection time periods corresponding to that exercise type. For example, for squat exercise, the information processing device 100 can accurately calculate the length of each individual exercise time period even if there is a blank period in which no movement is performed between the nth detection time period 1201 and the n+1th detection time period 1201. Therefore, the information processing device 100 can accurately identify each individual exercise time period.

(Another example of identifying individual exercise times)
Next, another example in which the information processing device 100 identifies individual exercise time periods will be described with reference to FIGS.

Figures 13 to 17 are explanatory diagrams showing other examples of identifying individual exercise time periods. For example, the examples in Figures 6 to 10 describe a case where the information processing device 100 identifies individual exercise time periods for each type of exercise using the number of times exercise is performed. In contrast, the examples in Figures 13 to 17 describe a case where the information processing device 100 identifies individual exercise time periods for each type of exercise using the amount of deviation between the start of the detection time period for that type of exercise and the start of the exercise time period for that type of exercise.

In FIG. 13, the information processing device 100 acquires video images of a subject during a target time period 1300. In the example of FIG. 13, the target time period 1300 is represented by a series of rectangles representing unit times arranged in chronological order. Specifically, during the target time period 1300, squats were performed four times and lunges were performed a number of times, where a≧1.

The information processing device 100 identifies the start and end points of the target time period 1300. For each type of exercise, the information processing device 100 detects frames from the video in which characteristics of partial movements related to that type of exercise appear. For each type of exercise, the information processing device 100 identifies the time period corresponding to the detected frames as the detected time period in which part of the movement of that type of exercise was detected.

In the example of FIG. 13, the information processing device 100 identifies a detection time period 1301 in which part of the squat exercise was detected. The information processing device 100 identifies a detection time period 1302 in which part of the lunge exercise was detected. For convenience, among the row of rectangles representing the target time period 1300, rectangles corresponding to the identified detection time period are represented with diagonal hatching.

The information processing device 100 sorts the multiple exercise types in the order in which they were performed and identifies the exercise type "squats" that was performed first. For the identified squat exercise, the information processing device 100 calculates the start offset 1303 between the start of the detection time period 1301 and the start of each individual exercise time period that includes the detection time period 1301.

For example, for the identified squat exercise, the information processing device 100 sets the start point of the target time period 1300 as the start point of the first individual exercise time period. The information processing device 100 calculates the difference between the set start point and the start point of the first detection time period 1301 (= 1 unit time) as the start deviation amount 1303. Next, we will move on to the explanation of Figure 14 and explain another example in which the information processing device 100 calculates the start deviation amount.

As shown in FIG. 14, it is possible that the start time 1400 of the second individual exercise time period for squat exercise is known. In this case, the information processing device 100 may calculate the start deviation amount 1401 based on the start time 1400 of the second individual exercise time period. The start deviation amount 1401 is the deviation amount between the start time of the detection time period 1301 and the start time of the individual exercise time period that includes the detection time period 1301.

For example, the information processing device 100 calculates the difference between the start time 1400 of the second individual exercise time period and the start time of the second detection time period 1301 (i.e., one unit of time) as the start delay 1401. This allows the information processing device 100 to accurately calculate the start delay 1401 for the first exercise type performed, even if there is a blank period during which no activity is performed between the start time of the target time period and the start time of the first exercise time period. Therefore, the information processing device 100 can accurately identify individual exercise time periods. Next, we move on to the explanation of Figure 15.

In FIG. 15, it is assumed that the information processing device 100 has already determined the start delay 1303 as shown in FIG. 13. It is assumed that the information processing device 100 has calculated the length 1501 of the individual exercise time period for the squat exercise as shown in FIGS. 6 to 12. The information processing device 100 calculates the end delay 1502 for the squat exercise based on the start delay 1303 and the length 1501 of the individual exercise time period. The end delay 1502 is the amount of delay between the end of the detection time period and the end of the individual exercise time period that includes that detection time period.

For example, for squat exercise, the information processing device 100 identifies the start point of the first individual exercise time period as the point that is located back by the start deviation amount 1303 minutes from the start point of the first detection time period 1301. For example, the information processing device 100 identifies the end point of the first individual exercise time period as the point that is the calculated length of the individual exercise time period 1501 minutes from the start point of the identified first individual exercise time period. For example, the information processing device 100 calculates the difference between the end point of the identified first individual exercise time period and the end point of the first detection time period as the end deviation amount 1502. Next, we move on to the explanation of Figure 16.

In FIG. 16, the information processing device 100 determines the start and end points of the individual exercise time periods for the second and subsequent squat exercises based on the determined start delay 1303 and end delay 1502.

For example, for squat exercise, the information processing device 100 identifies the start point of the second individual exercise time period as the point that is earlier than the start point of the second detection time period 1301 by the start delay amount 1303 minutes. For example, for squat exercise, the information processing device 100 identifies the end point of the second individual exercise time period as the point that is later than the end point of the second detection time period 1301 by the end delay amount 1502 minutes.

As a result, the information processing device 100 can accurately calculate the start and end points of individual exercise time periods for squat exercise, even if there is a blank time during which no movement is performed between two different detection time periods 1301. For example, for squat exercise, even if there is a blank time period 1610, the information processing device 100 can accurately identify the start and end points of the second individual exercise time period. Next, we will move on to the explanation of Figure 17.

In FIG. 17, the information processing device 100 identifies the end point 1700 of the fourth and final individual exercise period for the squat exercise based on the end deviation amount 1502. The information processing device 100 sets the end point 1700 of the identified fourth individual exercise period as the start point of the first individual exercise period for the lunge exercise performed after the squat.

For lunge exercises, the information processing device 100 determines the start and end points of each individual exercise time period, similar to FIGS. 13 to 16, based on the start point of the first individual exercise time period that has been set. Based on the start and end points of the determined individual exercise time periods, the information processing device 100 may also determine the timing when the squat exercise was switched to, the lunge exercise was switched to, or the type of exercise was switched.

This allows the information processing device 100 to accurately identify individual exercise time periods. The information processing device 100 can also accurately identify the timing when the squat exercise is switched to the lunge exercise, and the timing when the type of exercise is switched. Therefore, the information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject. The information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject, for purposes such as nursing care, rehabilitation, or fitness.

(An example based on the end of the target time period)
Next, an example in which the information processing device 100 identifies an individual exercise time period based on the end time of the target time period will be described with reference to FIGS. 18 and 19 .

Figures 18 and 19 are explanatory diagrams showing an example in which the end point of a target time period is used as the reference. In Figure 18, the information processing device 100 acquires video images of a target person during a target time period 1800. In the example of Figure 18, the target time period 1800 is represented by a row of rectangles arranged in chronological order, each representing a unit of time.

Specifically, during target time period 1800, squats are performed twice and lunges are performed twice. There is a time when the type of exercise changes between the second squat and the first lunge.

The information processing device 100 determines the start time of the target time period 1800 and the end time of the target time period 1800. The information processing device 100 determines the number of times that squats were performed during the target time period 1800 = 2. The information processing device 100 determines the number of times that lunges were performed during the target time period 1800 = 2.

For each type of exercise, the information processing device 100 detects frames from the video in which characteristics of partial movements related to that type of exercise appear. A partial movement is, for example, a part of an exercise. For each type of exercise, the information processing device 100 identifies the time period corresponding to the detected frame as the time period in which part of the movement of that type of exercise was detected.

In the example of FIG. 18 , the information processing device 100 identifies a detection time period 1801 in which part of the squat exercise was detected. The information processing device 100 identifies a detection time period 1802 in which part of the lunge exercise was detected. For convenience, among the row of rectangles representing the target time period 1800, rectangles corresponding to the identified detection time period are represented with diagonal hatching.

The information processing device 100 sorts the multiple exercise types in the order in which they were performed and identifies the last exercise type performed, "lunges." The information processing device 100 calculates the length of a time period during which one lunge exercise was performed for the identified lunge exercise. For example, for the identified lunge exercise, the information processing device 100 calculates the difference between the end point of the second-last detection time period 1802 and the end point of the first detection time period 1802 immediately before as the length 1803 of the individual exercise time period. Next, we move on to the explanation of Figure 19.

In FIG. 19, the information processing device 100 sets the end point of the target time period 1800 as the end point of the overall exercise time period during which lunges were performed. The overall exercise time period includes individual exercise time periods. The information processing device 100 counts back from the set end point and identifies the time period (length 1803 of the identified individual exercise time period x number of lunges performed = 2) as the overall exercise time period during which lunges were performed. The information processing device 100 divides the identified overall time period equally by the number of lunges performed = 2, and identifies each time period as an individual exercise time period 1901 during which lunges were performed.

Similarly, the information processing device 100 identifies the individual exercise time periods in which the squat exercise was performed. The information processing device 100 may further identify the timing when the squat exercise was switched to another exercise, the timing when the lunge exercise was switched to another exercise, or the timing when the type of exercise was switched.

This allows the information processing device 100 to accurately identify individual exercise time periods. The information processing device 100 can also accurately identify the timing when the squat exercise is switched to the lunge exercise, and the timing when the type of exercise is switched. Therefore, the information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject. The information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject, for purposes such as nursing care, rehabilitation, or fitness.

(An example of assessing athletic ability)
Next, an example in which the information processing device 100 evaluates athletic ability will be described with reference to FIG.

Figure 20 is an explanatory diagram showing an example of assessing athletic ability. In Figure 20, the subject performs squats and lunges according to a predetermined exercise menu that instructs the timing of exercise.

The exercise menu, for example, specifies the time point at which the subject is instructed to start exercising. The exercise menu, for example, specifies the time point at which the subject is instructed to switch to squats, the time point at which the subject is instructed to switch from squats to lunges, and the time point at which the subject is instructed to switch to lunges. The exercise menu may, for example, specify a time period 2001 during which the subject is instructed to perform squats, and a time period 2002 during which the subject is instructed to perform lunges.

The point at which an instruction to switch to squats is given is specifically the point at which an instruction to end the nth squat and start the n+1th squat. The point at which an instruction to switch from squats to lunges is given is specifically the point at which an instruction to end the squat and start the lunge. The point at which an instruction to switch to lunges is given is specifically the point at which an instruction to end the nth lunge and start the n+1th lunge.

As shown in Figures 6 to 10, the information processing device 100 is assumed to have identified individual exercise time periods 801 in which squats were performed and individual exercise time periods 1001 in which lunges were performed, based on video images of the subject. As shown in Figures 6 to 10, the information processing device 100 is assumed to have identified the timing when the squat exercise was switched to a lunge exercise, the timing when the exercise type was switched, and the like.

The information processing device 100 evaluates the subject's athletic ability based on the exercise menu. For example, the information processing device 100 calculates the amount of deviation between the time period during which the subject is instructed to perform squats as specified in the exercise menu and the individual exercise time period 801 during which the squats were performed. For example, if the calculated amount of deviation is equal to or greater than a threshold, the information processing device 100 determines that the subject's athletic ability is relatively low, and if the calculated amount of deviation is less than the threshold, the information processing device 100 determines that the subject's athletic ability is relatively high.

The information processing device 100 calculates, for example, the amount of deviation between the time period specified in the exercise menu when instructing to perform lunge exercises and the individual exercise time period 1001 when the lunges were performed. If the calculated amount of deviation is equal to or greater than a threshold, the information processing device 100 determines that the subject's athletic ability is relatively low, and if the calculated amount of deviation is less than the threshold, the information processing device 100 determines that the subject's athletic ability is relatively high.

The information processing device 100, for example, calculates the amount of deviation between the time when an instruction to switch to a squat exercise specified in the exercise menu is given and the timing when the squat exercise is switched. If the calculated amount of deviation is equal to or greater than a threshold, the information processing device 100 determines that the subject's athletic ability is relatively low, and if the calculated amount of deviation is less than the threshold, the information processing device 100 determines that the subject's athletic ability is relatively high.

The information processing device 100 calculates the amount of deviation between, for example, the point in time when an instruction to switch from squats to lunges as specified in the exercise menu is given and the timing at which the exercise type is switched. If the calculated amount of deviation is equal to or greater than a threshold, the information processing device 100 determines that the subject's athletic ability is relatively low, and if the calculated amount of deviation is less than the threshold, the information processing device 100 determines that the subject's athletic ability is relatively high.

The information processing device 100, for example, calculates the amount of deviation between the time when an instruction to switch lunge exercises specified in an exercise menu is given and the timing at which the lunge exercise is switched. If the calculated amount of deviation is equal to or greater than a threshold, the information processing device 100 determines that the subject's athletic ability is relatively low, and if the calculated amount of deviation is less than the threshold, determines that the subject's athletic ability is relatively high. In this way, the information processing device 100 can accurately analyze the subject's athletic ability or cognitive ability, for example. The information processing device 100 can accurately analyze the subject's athletic ability or cognitive ability, for example, for purposes such as nursing care, rehabilitation, or fitness.

(Example of effect of information processing device 100)
Next, an example of the effect of the information processing device 100 will be described with reference to FIG.

Figure 21 is an explanatory diagram showing an example of the effect of the information processing device 100. In Figure 21, a conventional method is considered in which the basic movements of a subject are recognized for each frame included in a moving image, and the time at which a pattern defined by a predetermined rule appears in the time series of the recognized basic movements is identified as the time at which the target movement was performed.

Here, as shown in FIG. 21, a predetermined rule may define a pattern corresponding to, for example, a portion of a squat exercise, but not a pattern corresponding to the remaining portion of the squat exercise, represented by cross hatching. Similarly, as shown in FIG. 21, a predetermined rule may define a pattern corresponding to, for example, a portion of a lunge exercise, but not a pattern corresponding to the remaining portion of the lunge exercise, represented by cross hatching.

Therefore, conventional methods may only be able to identify a portion 2101 of the exercise period during which squats are actually performed as the exercise period during which squats are actually performed. Similarly, conventional methods may only be able to identify a portion 2102 of the exercise period during which lunges are actually performed as the exercise period during which lunges are actually performed.

Furthermore, conventional methods cannot identify the timing when the squat exercise is switched to the lunge exercise, or when the exercise type is switched. As a result, conventional methods have difficulty accurately analyzing the subject's motor or cognitive abilities.

In response to this, the information processing device 100 is assumed to have identified, based on video images of the subject, individual exercise time periods 801 during which squats were performed and individual exercise time periods 1001 during which lunges were performed, as shown in Figures 6 to 10. The information processing device 100 is assumed to have identified, as shown in Figures 6 to 10, the timing at which the squat exercise was switched, the timing at which the lunge exercise was switched, and the timing at which the type of exercise was switched.

In this way, the information processing device 100 can more accurately identify the individual exercise time periods 801 when squats were performed and the individual exercise time periods 1001 when lunges were performed, compared to conventional methods. The information processing device 100 can also accurately identify the timing when the squat exercise was switched to the lunge exercise, and the timing when the exercise type was switched. Therefore, the information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject. The information processing device 100 can accurately analyze the motor ability or cognitive ability of a subject, for example, for purposes such as nursing care, rehabilitation, or fitness.

(Overall processing procedure)
Next, an example of an overall processing procedure executed by the information processing device 100 will be described with reference to Fig. 22. The overall processing is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I/F 303 shown in Fig. 3.

Figure 22 is a flowchart showing an example of the overall processing procedure. In Figure 22, the information processing device 100 acquires the start and end times of the target time period, the number of times exercise was performed for each type of exercise, and the start and end times of the detection time period for each type of exercise (step S2201).

Next, the information processing device 100 determines whether or not there is an exercise type for which the overall exercise time period is unknown among the multiple exercise types (step S2202). If there is no exercise type for which the overall exercise time period is unknown (step S2202: No), the information processing device 100 ends the overall processing. On the other hand, if there is an exercise type for which the overall exercise time period is unknown (step S2202: Yes), the information processing device 100 proceeds to processing in step S2203.

In step S2203, the information processing device 100 selects the exercise type that appears first among the exercise types whose overall exercise time period is unknown (step S2203).

Next, the information processing device 100 sets the known end time of the last exercise type among the exercise types for which the end time has been identified, or the start time of the target time period, as the start time of the entire exercise time period in which the selected exercise type was performed (step S2204).

In step S2204, for example, if the selected type of exercise is the first type of exercise, the information processing device 100 sets the start point of the target time period to the start point of the entire exercise time period in which the selected type of exercise was performed. Furthermore, for example, if the selected type of exercise is the second or subsequent type of exercise, the information processing device 100 sets the known end point of the last type of exercise among the exercise types whose end points have been identified to the start point of the entire exercise time period in which the selected type of exercise was performed.

In step S2204, for example, if the end time of the exercise of the exercise type performed immediately before the selected exercise type is unknown, the information processing device 100 may set the start time of the first exercise of the selected exercise type as the start time of the entire exercise time period during which the selected exercise type was performed.

Then, the information processing device 100 calculates the difference T between the start time of the first exercise session and the start time of the second exercise session of the selected exercise type after the set start time as the length of the individual exercise time period during which one session of the selected exercise type was performed (step S2205).

Next, the information processing device 100 identifies the time period from the set start time as the calculated difference T multiplied by the number of times the selected type of exercise was performed, as the total exercise time period during which the selected type of exercise was performed multiple times (step S2206).

The information processing device 100 then divides the identified time period equally into the number of times the selected type of exercise was performed, and identifies individual exercise time periods in which one session of the selected type of exercise was performed (step S2207). The information processing device 100 then returns to the processing of step S2202.

As described above, the information processing device 100 can acquire first information that identifies a period during which a target action was performed multiple times. The information processing device 100 can acquire second information that identifies multiple times during the period during which partial actions related to the target action were detected. The information processing device 100 can calculate the length of time during which the target action was performed once based on the acquired second information. The information processing device 100 can identify each time during the period during which the target action was performed based on the acquired first information and the calculated length. This allows the information processing device 100 to accurately identify the time during which the target action was performed.

The information processing device 100 can calculate the length of time that a target action was performed once, based on the acquired second information, using the difference between the start or end points of any two of the multiple time periods. This allows the information processing device 100 to accurately calculate the length of time that a target action was performed once, improving the accuracy of identifying the time that the target action was performed.

The information processing device 100 can calculate, based on the acquired second information, the difference between the start or end points of two adjacent time periods included in the multiple time periods as the length of time during which the target action was performed once. This allows the information processing device 100 to accurately calculate the length of time during which the target action was performed once, thereby improving the accuracy of identifying the time during which the target action was performed.

The information processing device 100 can calculate the difference between the start or end points of two non-adjacent periods included in a plurality of periods based on the acquired second information. The information processing device 100 can calculate the length of time during which the target action was performed once by dividing the calculated difference by the number of periods during which the partial action included in the difference was detected. This allows the information processing device 100 to accurately calculate the length of time during which the target action was performed once, thereby improving the accuracy of identifying the time when the target action was performed.

The information processing device 100 can identify each period of time during which a target action was performed, having a calculated length, based on the acquired first information, second information, and calculated length, using the number of times the partial action was detected, with the start or end of the period as the reference point. This allows the information processing device 100 to accurately identify the period during which the target action was performed.

The information processing device 100 can identify the amount of deviation between the start or end of each of multiple time periods and the start or end of the time period in which the target action was performed, including that time period. The information processing device 100 can use the identified amount of deviation to identify each time period in which the target action was performed, having the calculated length. This allows the information processing device 100 to accurately identify the time period in which the target action was performed.

The information processing device 100 can identify the timing at which the time at which the target movement is performed changes based on each of the times at which the identified target movement is performed. This makes it easier for the information processing device 100 to evaluate the motor ability of the subject of the target movement.

The information processing device 100 can identify the timing at which the time of the target movement changes as the timing after the end of the first time period in which the identified target movement was performed and before the start of the second time period in which the identified target movement, adjacent to the first time period, was performed. This makes it easier for the information processing device 100 to evaluate the motor ability of the subject of the target movement.

The information processing device 100 can acquire first information that identifies a period during which each of different types of target actions was performed multiple times. The information processing device 100 can acquire second information that identifies multiple times during which each of multiple partial actions related to different types of target actions was detected within the period. The information processing device 100 can identify the order in which each of the multiple target actions was performed from the beginning of the period based on the acquired second information. The information processing device 100 can calculate the length of time during which each of the multiple target actions was performed once based on the acquired second information. The information processing device 100 can identify the times during which each of the multiple target actions was performed within the period, starting with the target action that was performed first, based on the acquired first information, the identified order, and the calculated length. This allows the information processing device 100 to accurately identify the time during which each of the different types of target actions was performed.

The information processing device 100 can evaluate the motor skills of the subject of the target action based on the time at which each of the identified target actions was performed. This allows the information processing device 100 to accurately evaluate the motor skills of the subject of the target action and make the evaluation results available.

The information processing device 100 can acquire video of a period, recognize basic movements for each frame included in the acquired video, and detect patterns representing partial movements from the time series of the recognized basic movements. The information processing device 100 can generate second information that identifies multiple times within the period at which patterns representing partial movements were detected. This eliminates the need for the information processing device 100 to generate the second information externally.

The information processing device 100 can identify the time when the target action was performed within a period that includes both a time when the partial action was detected and a time when the partial action was not detected. This allows the information processing device 100 to accurately identify the time when the target action was performed, even if there is a time when the partial action was not detected.

The information processing method described in this embodiment can be implemented by executing a prepared program on a computer such as a PC or workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by the computer by reading it from the recording medium. Recording media include hard disks, flexible disks, CD (Compact Disc)-ROMs, MO (Magneto Optical Discs), and DVDs (Digital Versatile Discs). The information processing program described in this embodiment may also be distributed via a network such as the Internet.

The following additional notes are provided regarding the above-described embodiment.

(Supplementary Note 1) First information specifying a period in which the target action was performed multiple times is acquired;
acquiring second information that identifies a plurality of times during which partial movements related to the target movement were detected within the period;
Calculating the length of time during which the target action was performed once based on the acquired second information;
Identifying each time period during which the target action was performed based on the acquired first information and the calculated length.
An information processing program that causes a computer to execute a process.

(Note 2) The calculation process is
The information processing program described in Appendix 1 is characterized in that, based on the acquired second information, the length of time during which the target action was performed once is calculated using the difference between the start or end points of any two of the times included in the multiple times.

(Note 3) The calculation process is
An information processing program as described in Appendix 1 or 2, characterized in that, based on the acquired second information, the difference between the start or end points of each of two adjacent times included in the plurality of times is calculated as the length of time during which the target action was performed once.

(Note 4) The calculation process is
An information processing program described in any one of appendices 1 to 3, characterized in that, based on the acquired second information, the length of time during which the target action was performed once is calculated by dividing the difference between the start or end points of two non-adjacent times included in the plurality of times by the number of times during which the partial action included in the difference was detected.

(Note 5) The process of specifying
An information processing program described in any one of appendices 1 to 4, characterized in that, based on the acquired first information, the acquired second information, and the calculated length, each period of time during which the target action was performed having the calculated length is identified using the number of times the partial action was detected, with the start or end of the period as the reference point.

(Supplementary Note 6) The identifying process is
An information processing program as described in any one of appendices 1 to 5, characterized in that, based on the acquired first information, the acquired second information, and the calculated length, the difference between the start or end of the period and the start or end of any of the times when the partial action was detected is used to determine the amount of deviation between the start or end of each of the multiple times and the start or end of the time when the target action was performed, including that time, and the determined amount of deviation is used to determine each of the times within the period when the target action was performed, which has the calculated length.

(Supplementary Note 7) Based on each of the identified times at which the target motion is performed, a timing at which the time at which the target motion is performed changes is identified.
7. An information processing program according to any one of claims 1 to 6, which causes the computer to execute processing.

(Supplementary Note 8) The process of specifying the timing includes:
The information processing program described in Appendix 7 is characterized in that the timing at which the time at which the target action is performed changes is determined to be after the end of a first time period in which the identified target action was performed and before the start of a second time period in which the identified target action was performed, which is adjacent to the first time period.

(Supplementary Note 9) First information specifying a period in which each of different types of target actions was performed multiple times is acquired;
acquiring second information that identifies a plurality of times during the period when each of a plurality of partial movements associated with a different type of target movement was detected;
Identifying an order in which each of the different types of target actions was performed from the beginning of the period based on the acquired second information;
Calculating the length of time during which each of the different types of target actions was performed once based on the acquired second information;
Identifying, based on the acquired first information, the identified order, and the calculated length, times during which the target actions were performed in the period, in order from the first target action performed among the different types of target actions;
9. An information processing program according to any one of claims 1 to 8, which causes the computer to execute processing.

(Supplementary Note 10) Evaluating the motor ability of the subject of the target movement based on each of the times when the identified target movement was performed.
10. An information processing program according to any one of claims 1 to 9, which causes the computer to execute processing.

(Supplementary Note 11) The process of acquiring the second information includes:
An information processing program described in any one of appendices 1 to 10, characterized in that it acquires a moving image of the period, recognizes a basic movement for each frame included in the acquired moving image, detects a pattern representing the partial movement from the time series of the recognized basic movement, and generates second information that identifies multiple times within the period at which the patterns representing the partial movement were detected.

(Appendix 12) The information processing program described in any one of Appendices 1 to 11, wherein the period includes a time during which the partial movement is detected and a time during which the partial movement is not detected.

(Supplementary Note 13) Acquire first information specifying a period during which the target action was performed multiple times;
acquiring second information that identifies a plurality of times during which partial movements related to the target movement were detected within the period;
Calculating the length of time during which the target action was performed once based on the acquired second information;
Identifying each time period during which the target action was performed based on the acquired first information and the calculated length.
An information processing method characterized in that the processing is executed by a computer.

100 Information processing device 110 Period 111, 112, 113 Time 200 Information processing system 201 Motion detection device 202 Client device 203 Sensing device 210 Network 300 Bus 301 CPU
302 Memory 303 Network I/F
304 Recording medium I/F
305 Recording medium 400 Storage unit 401 Acquisition unit 402 Detection unit 403 Calculation unit 404 Identification unit 405 Output unit 501 Target movement time calculation unit 502 Movement start deviation calculation unit 503 Movement start time calculation unit 504 Movement end time calculation unit 600, 1100, 1200, 1300, 1800 Target time period 601, 602, 1101, 1102, 1201, 1202, 1301, 1302, 1801, 1802 Detection time period 701, 901, 1501, 1803 Length 801, 1001, 1901 Exercise time period 1303, 1401, 1502 Deviation amount 1400 Start time 1610 Blank time 2101, 2102 part

Claims (11)

acquire first information identifying a period during which the target action was performed multiple times;
acquiring second information that identifies a plurality of times during which partial movements related to the target movement were detected within the period;
Calculating the length of time during which the target action was performed once based on the acquired second information;
Identifying the times during which the target actions were performed within the period based on the acquired first information, the acquired second information, and the calculated length.
Have the computer execute the process,
The identifying process includes:
determining a first deviation amount between the start of each of the plurality of time periods and the start of a time period in which the target action was performed, the first deviation amount being a difference between the start of each of the plurality of time periods and the start of a time period in which the target action was performed, the difference being a difference between the start of each of the plurality of time periods and the start of any of the time periods in which the partial action was detected, based on the acquired first information and the acquired second information;
identifying a time point that is located back by the identified first deviation amount from the start time point of each of the plurality of time periods as the start time point of a time period in which the target action was performed, including the time period, and identifying a time period having the calculated length that starts from the identified start time point as the time period in which the target action was performed.
or
determining a second deviation amount between the end of each of the plurality of time periods and the end of a time period in which the target action was performed, the second deviation amount being a difference between the end of each of the plurality of time periods and the end of a time period in which the target action was performed, the difference being a difference between the end of each of the plurality of time periods and the end of any of the time periods in which the partial action was detected, based on the acquired first information and the acquired second information;
An information processing program characterized by identifying a point in time that is the identified second deviation amount from the start point of each of the multiple times as the start point of the time when the target action was performed, including that time, and identifying a time having the calculated length that starts from the identified start point as the time when the target action was performed . The calculation process includes:
The information processing program according to claim 1, characterized in that, based on the acquired second information, the length of time during which the target action was performed once is calculated using the difference between the start or end points of any two of the times included in the plurality of times. The calculation process includes:
The information processing program according to claim 1 or 2, characterized in that, based on the acquired second information, the difference between the start or end points of each of two adjacent times included in the plurality of times is calculated as the length of time during which the target action was performed once. The identifying process includes:
The information processing program according to any one of claims 1 to 3, characterized in that, based on the acquired first information, the acquired second information, and the calculated length, each period of time during which the target action was performed having the calculated length is identified using the number of times the partial action was detected, with the start or end of the period as the reference point. Identifying a timing at which the time at which the target action is performed changes based on each of the identified times at which the target action is performed.
5. The information processing program according to claim 1, which causes the computer to execute processing. The process of specifying the timing includes:
The information processing program of claim 5, characterized in that the timing at which the time at which the target action is performed changes is identified as the timing after the end of a first time period in which the identified target action is performed and before the start of a second time period in which the identified target action is performed, which is adjacent to the first time period. acquiring first information identifying a period in which each of different types of target actions was performed multiple times;
acquiring second information that identifies a plurality of times during the period when each of a plurality of partial movements associated with a different type of target movement was detected;
Identifying an order in which each of the different types of target actions was performed from the beginning of the period based on the acquired second information;
Calculating the length of time during which each of the different types of target actions was performed once based on the acquired second information;
Identifying, based on the acquired first information, the identified order, and the calculated length, times during which the target actions were performed in the period, in order from the first target action performed among the different types of target actions;
7. The information processing program according to claim 1, which causes the computer to execute processing. Evaluating the motor ability of the subject of the target movement based on each of the times when the identified target movement was performed.
8. The information processing program according to claim 1, which causes the computer to execute processing. The process of acquiring the second information includes:
An information processing program according to any one of claims 1 to 8, characterized in that it acquires a moving image of the period, recognizes a basic movement for each frame included in the acquired moving image, detects a pattern representing the partial movement from the time series of the recognized basic movement, and generates second information that identifies multiple times within the period at which the patterns representing the partial movement were detected. 10. The information processing program according to claim 1, wherein the period includes a time during which the partial movement is detected and a time during which the partial movement is not detected. acquire first information identifying a period during which the target action was performed multiple times;
acquiring second information that identifies a plurality of times during which partial movements related to the target movement were detected within the period;
Calculating the length of time during which the target action was performed once based on the acquired second information;
Identifying the times during which the target actions were performed within the period based on the acquired first information, the acquired second information, and the calculated length.
The computer executes the processing,
The identifying process includes:
determining a first deviation amount between the start of each of the plurality of time periods and the start of a time period in which the target action was performed, the first deviation amount being a difference between the start of each of the plurality of time periods and the start of a time period in which the target action was performed, the difference being a difference between the start of each of the plurality of time periods and the start of any of the time periods in which the partial action was detected, based on the acquired first information and the acquired second information;
identifying a time point that is located back by the identified first deviation amount from the start time point of each of the plurality of time periods as the start time point of a time period in which the target action was performed, including the time period, and identifying a time period having the calculated length that starts from the identified start time point as the time period in which the target action was performed.
or
determining a second deviation amount between the end of each of the plurality of time periods and the end of a time period in which the target action was performed, the second deviation amount being a difference between the end of each of the plurality of time periods and the end of a time period in which the target action was performed, the difference being a difference between the end of each of the plurality of time periods and the end of any of the time periods in which the partial action was detected, based on the acquired first information and the acquired second information;
An information processing method characterized by identifying a point in time that is the identified second deviation amount from the start point of each of the multiple times as the start point of the time when the target action was performed, including that time, and identifying a time having the calculated length that starts from the identified start point as the time when the target action was performed.