JP7659293B2 - Physical Exercise Support System - Google Patents

Description

The present invention relates to a physical exercise support system.

There is a wide range of physical exercises that use tools (including devices). For example, in ball games, which are one category of sports, the tool is the ball, and depending on the sport, the ball is handled with tools that are held or attached mainly to the hands, feet, or body. In addition, the use of tools used in sports, training tools, and rehabilitation tools in practice for these sports, as well as in strength training and rehabilitation, is widespread.

Here, we focus on training machines. An invention has been disclosed that detects the behavior of a training machine and measures the amount of exercise when a user uses the machine to train (Patent Document 1).

JP 2005-198867 A

However, the technology in Patent Document 1 requires that a measuring device be attached to the training machine itself, which poses problems in terms of simplicity and safety. Also, depending on the type of tool and the way the tool is used, it may be difficult to attach a measuring device to the tool.

The present invention was made in light of this background, and aims to provide technology that can support exercise by easily detecting and evaluating the movements of tools used in physical exercise and providing advice for further improving skills and improving the effects of training, rehabilitation, etc.

The present invention is a physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool, and is characterized by comprising a reference value storage unit that stores a reference value relating to the position of at least one of the tools, a part identification unit that analyzes an image of the tool to identify a part of the tool, and an evaluation unit that compares the position of the part in the image with the reference value to determine an evaluation value for the exercise.

Other problems and solutions disclosed in this application will be made clear in the description of the embodiments of the invention and the drawings.

The present invention makes it easy to evaluate the skills and effectiveness of physical exercise using tools, and provides support that contributes to improving skills and effectiveness.

1 is a diagram illustrating an example of the overall configuration of a physical exercise support system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal 10. FIG. 2 is a diagram illustrating an example of a software configuration of the user terminal 10. 4 is a diagram showing an example of the configuration of physical information stored in a physical information storage unit 130. FIG. 13 is a diagram showing an example of the configuration of a rating request transmitted by a rating request transmitting unit 112 to a server device 20. FIG. 13 is a diagram showing an example of the structure of evaluation information received by an evaluation information receiving unit 113 from a server device 20. FIG. 13 is a diagram showing an example of the configuration of an improvement measure request transmitted by an improvement measure request transmitting unit 116 to a server device 20. FIG. FIG. 13 is a diagram illustrating an example of a configuration of improvement measure information. FIG. 2 illustrates an example of a hardware configuration of a server device 20. FIG. 2 is a diagram illustrating an example of the software configuration of the server device 20. 4 is a diagram showing an example of the configuration of image information stored in an image data storage unit 231. FIG. 13 is a diagram showing an example of the configuration of reference information stored in a reference information storage unit 232. FIG. 13 is a diagram showing an example of the configuration of evaluation condition information stored in an evaluation condition information storage unit 233. FIG. 13 is a diagram showing an example of the configuration of remediation measure information stored in a remediation measure information storage unit 234. FIG. 13 is a diagram showing an example of the configuration of group analysis information stored in a group analysis information storage unit 235. FIG. FIG. 2 is a diagram showing a flow of processing executed in the physical exercise support server of the present embodiment. FIG. 13 is a diagram showing an example of a video evaluation screen in a weightlifting mode. FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. FIG. 13 is another diagram showing an example of a video evaluation screen in squat mode presented on the user terminal 10. FIG. 13 is a diagram showing an example of an evaluation screen for a video in group training, which is presented on a supporter terminal 50. 13 is another diagram showing an example of the evaluation screen for a video in group training, which is presented on the supporter terminal 50. FIG.

The details of the embodiment of the present invention will be listed and explained below. The physical exercise support server according to the embodiment of the present invention has the following configuration.

[Item 1]
A physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool,
A reference value storage function for storing a reference value relating to at least one position of the tool;
A part identification function that identifies a part of the tool by analyzing an image of the tool;
an evaluation function for determining an evaluation value of the movement based on the position of the portion in the image and the reference value;
A physical exercise support system comprising:
[Item 2]
The evaluation function determines the evaluation value according to a distance from the reference value to the position;
2. The physical exercise support system according to claim 1,
[Item 3]
The evaluation function determines an evaluation value of the exercise based on a relationship between a part of the tool and a part of the body;
3. The physical exercise support system according to item 1 or 2,
[Item 4]
a comment storage function for storing a comment on a position of the part in association with the part and a condition for the part;
a comment output function for outputting the comment corresponding to the condition satisfied by the position;
4. The physical exercise support system according to any one of claims 1 to 3,
[Item 5]
an improvement information storage function for storing advice on an improvement measure related to the physical exercise in association with a goal of improvement related to the exercise and a condition for the position;
an advice output function for outputting the advice corresponding to the condition satisfied by the position;
5. The physical exercise support system according to any one of claims 1 to 4,
[Item 6]
A physical exercise support server that supports a user in improving the effectiveness of physical exercise performed using a tool,
a reference value storage unit configured to store a reference value relating to at least one of the tool positions;
A part identification unit that analyzes an image of the tool and identifies a part of the tool;
an evaluation unit for determining an evaluation value of the movement by comparing a position of the part in the image with the reference value;
A physical exercise support server comprising:
[Item 7]
A physical exercise support method for supporting a user in improving the effectiveness of physical exercise performed using a tool, comprising:
a reference value storage step of storing a reference value relating to at least one of the positions of the tool;
A part identification step of analyzing an image of the tool to identify a part of the tool;
an evaluation step for determining an evaluation value of the movement by comparing the position of the portion in the image and the reference value;
A physical exercise support method comprising:

The physical exercise support server according to one embodiment of the present invention is designed to evaluate the movement of a tool when a user performs physical exercise using the tool.

Physical exercise using a user's equipment includes, but is not limited to, sports such as soccer, volleyball, and basketball, which use only a ball as the equipment; sports such as baseball, tennis, table tennis, and golf, which use a ball and other equipment; sports such as kendo and fencing, which use equipment other than a ball; physical exercise using equipment such as juggling; physical exercise using barbells or training equipment or machines; various types of rehabilitation using equipment; movements of the person being cared for using equipment such as a cane; and playing musical instruments.

The physical exercise support server of this embodiment identifies the tool and parts of the tool from an image (which may be a still image or a moving image, but in this embodiment it is a moving image) captured of a user performing physical exercise using a tool, and evaluates the movement of the tool based on the absolute position of the part and the relative positional relationship of multiple different parts. It is also possible to identify a part of the body and evaluate the movement of the tool based on the relative positional relationship between the part of the tool and the part of the body.

Figure 1 is a diagram showing an example of the overall configuration of a physical exercise support server according to this embodiment. As shown in the figure, the physical exercise support server according to this embodiment is configured to include a user terminal 10 and a server device 20. It may further include an imaging terminal 30. The user terminal 10, the server device 20, and the imaging terminal 30 are connected to each other so that they can communicate with each other via a communication network 40. The communication network 40 is, for example, the Internet or a LAN (Local Area Network), and is constructed by a public telephone line network, a dedicated telephone line network, a mobile phone line network, Ethernet (registered trademark), a wireless communication path, etc.

==User Terminal 10==
The user terminal 10 is a computer operated by a user performing physical exercise or by his/her supporter. The user terminal 10 is, for example, a smartphone, a tablet computer, a personal computer, etc. The user terminal 10 is equipped with an imaging device such as a camera, which can capture images of the user's body during exercise. In this embodiment, video images of the user's body during exercise are transmitted from the user terminal 10 to the server device 20. The user terminal 10 may also function as the imaging terminal 30. Also, although only one user terminal 10 is shown in FIG. 1, it goes without saying that there may be multiple user terminals 10.

Server Device 20
The server device 20 is a computer that evaluates the movement of the tool. The server device 20 is, for example, a workstation, a personal computer, or a virtual computer logically realized by cloud computing. The server device 20 receives the video captured by the user terminal 10, analyzes the received video, and evaluates the physical movement. The server device 20 also makes suggestions regarding improvements to the physical movement. Details of the evaluation of the physical movement and the suggestions of improvements will be described later.

==Imaging Terminal 30==
The imaging terminal 30 is a computer that captures an image of the user, or a camera with a communication function, which is installed or attached at a location where the user performs physical exercise. The imaging terminal 30 is, for example, a smartphone, a tablet computer, a personal computer, or the like. The imaging terminal 30 is equipped with an imaging device such as a camera, which can capture an image of the user's body during exercise. In this embodiment, a moving image of the user's body during exercise is transmitted from the imaging terminal 30 to the server device 20. Note that the data of the user captured and stored in the imaging terminal 30 may be input directly to the server device 20 by the user or his/her supporter, or a business operator who uses a physical exercise support server to perform a business. Alternatively, the data may be input via a network to the server device 20.

Supporter Terminal 50
The supporter terminal 50 is a computer operated by a person (supporter) who is responsible for providing guidance, instruction, explanation, support, etc. to a user performing physical exercise, such as a trainer, a physical therapist, a caregiver, etc. The supporter terminal 50 is, for example, a smartphone, a tablet computer, a personal computer, etc.

2 is a diagram showing an example of the hardware configuration of the user terminal 10. The user terminal 10 includes a CPU 101, a memory 102, a storage device 103, a communication interface 104, a touch panel display 105, and a camera 106. The storage device 103 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 104 is an interface for connecting to the communication network 40, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The touch panel display 105 is a device for inputting and outputting data. The user terminal 10 may also include input devices such as a keyboard, a mouse, buttons, and a microphone, and output devices such as a speaker and a printer.

Figure 3 is a diagram showing an example of the software configuration of the user terminal 10. The user terminal 10 includes functional units, namely, an imaging unit 111, an evaluation request sending unit 112, an evaluation information receiving unit 113, an evaluation display unit 114, a checkpoint display unit 115, an improvement request sending unit 116, an improvement information receiving unit 117, and an improvement information display unit 118, as well as storage units, namely, a physical information storage unit 130, an image storage unit 131, an evaluation information storage unit 132, and an improvement storage unit 133.

Each of the above functional units is realized by the CPU 101 of the user terminal 10 reading a program stored in the storage device 103 into the memory 102 and executing it, and each of the above storage units is realized as part of the storage area provided by the memory 102 and storage device 103 of the user terminal 10.

The imaging unit 111 captures images, including video, of the user performing physical exercise using the tool. The imaging unit 111 can obtain video images of the physical exercise using the tool by controlling the camera 106. The user or the user's supporter places the user terminal 10 on a flat surface or a wall, points the optical axis of the camera 106 toward the location where the user is performing exercise, and issues an instruction to start video recording. In response, the imaging unit 111 operates the camera 106 to obtain video images. The imaging unit 111 stores the obtained video images in the image storage unit 131.

The image storage unit 131 stores images captured by the imaging unit 111. In this embodiment, the images are moving images, but are not limited to this. The image storage unit 131 can store moving images, for example, as files.

The physical information storage unit 130 stores information (hereinafter referred to as physical information) relating to the user's body, physical abilities, factors that affect training effects, etc. FIG. 4 is a diagram showing an example of the configuration of physical information stored in the physical information storage unit 130. As shown in the figure, the physical information includes height, weight, sex, dominant hand, arm length, leg length, hand size, finger length, grip strength, muscle strength, flexibility, shoulder strength, genome, epigenome, gene polymorphism, intestinal flora, diet, etc.

The evaluation request sending unit 112 sends a request to the server device 20 to evaluate the physical exercise using the tool based on the image captured by the imaging unit 111 (hereinafter referred to as an evaluation request).

Figure 5 is a diagram showing an example of the configuration of an evaluation request sent by the evaluation request sending unit 112 to the server device 20. As shown in the figure, the evaluation request includes a user ID, a mode, physical information, and image data. The user ID is information that identifies the user. The mode is information that indicates the exercise performed by the user. The mode can be, for example, "bench press," "tennis serve," or "walking rehabilitation." Note that the mode is selected from a predetermined selection of options. The physical information is physical information stored in the physical information storage unit 130. The image data is data of a moving image acquired by the imaging unit 111.

The evaluation information receiving unit 113 receives information regarding the evaluation of the physical exercise using the tool (hereinafter referred to as evaluation information) that is returned from the server device 20 in response to the evaluation request. The evaluation information receiving unit 113 registers the received evaluation information in the evaluation information storage unit 132.

Figure 6 is a diagram showing an example of the configuration of evaluation information received by the evaluation information receiving unit 113 from the server device 20. As shown in the figure, the evaluation information includes a mode, a user ID, tool position information, body part position information, posture information, movement information, and checkpoint information.

The user ID and mode are the user ID and mode included in the rating request. The captured image shows the user's body performing the exercise indicated by the mode.

The tool position information indicates the position of each part of the tool (for example, the entire bat used in baseball, both ends, the grip, the place where the ball is met, the center of gravity, any other part, etc., as well as the entire barbell, the shaft, the plates (weight parts), the gripped part, the center of the shaft, the center of gravity, any other part, etc., including the entire tool) in the image. The tool position information includes each part of the tool and the position of the part, corresponding to a time on the time axis of the video. Based on the tool position information, the movement of the tool and the relationship with the body part can be displayed. That is, for example, a figure indicating a part (for example, a circle, etc.) can be displayed superimposed on the image at the position indicated by the tool position information. Note that the positions of multiple parts can be included for one time point. Furthermore, the tool position information does not need to be included for a part connecting two parts (for example, the midpoint between the parts where the barbell is gripped by the right hand and the left hand). In this case, the part connecting these two parts can be expressed by connecting a pair of marks (for example, a circle) indicating two specified parts with a line. The tool part position information may be included for each frame that constitutes the video, for each key frame (including frames related to checkpoints, which will be explained later), for every arbitrary number of frames, or for a random point in time. If My Frame does not include position information, the figure can be displayed based on the position information of the most recent past point in time.

The body position information indicates the position of each part of the body (e.g., head, shoulders, elbows, waist, knees, ankles, etc.) in the image. The body position information includes the part and the position of the part in association with a time on the time axis of the video. Based on the body position information, the state of the body skeleton (bones) can be displayed. That is, for example, a figure indicating the part (e.g., a circle, etc.) can be displayed by superimposing it on the image at the position indicated by the body position information. Note that the positions of multiple parts can be included for one time point. Note that position information does not need to be included for parts connecting two parts (e.g., the forearm connecting the wrist and elbow, or the thigh connecting the waist and knee, etc.). In this case, the part connecting these two parts can be expressed by connecting a pair of marks (e.g., a circle) indicating two specific parts with a line. The position information may be included for each frame constituting the video, may be included for each key frame (including frames related to checkpoints, which will be described later), may be included for every arbitrary number of frames, or may be included for a random time point. If My Frame does not include position information, the bones can be displayed based on the position information of the most recent past time point.

The tool orientation information is information related to the orientation of the tool used by the user and the direction in which the parts of the tool are facing. The tool orientation information includes the part of the tool to be evaluated, the tool movement value, the evaluation rank, the evaluation comment, etc., in association with a time point on the time axis of the video. The tool orientation value is a value that represents the orientation of the tool. The tool orientation value is, for example, the distance from the ground to a certain part of the tool, the distance between two parts of the tool, the angle of the part (the angle between a first part of the tool and, for example, a straight line to the part where the user is holding the tool, and a second part of the tool and, for example, a straight line to the part where the user is holding the tool), the movement of a certain part of the tool, etc. The evaluation rank is a value that represents the evaluation value by rank. The evaluation rank is expressed, for example, from 1 to 5 on a five-point scale, or ABC. The evaluation comment is a comment related to the evaluation of the posture. For example, when the mode is "upright row" and the distance from the ground to the right end and the left end of the barbell is different, an evaluation comment such as "Different forces are being applied to the left and right" may be included.

The tool movement information is information related to the movement of the tool used by the user. The tool movement information includes the part of the tool to be evaluated, a list of the tool orientation values, an evaluation rank, and an evaluation comment, in association with a period on the time axis of the video. The list of tool orientation values is a chronological series of tool orientation values within a period. The evaluation comment is a comment related to the evaluation of the tool movement. For example, if the mode is "upright row" and the up and down movement of the barbell is insufficient, an evaluation comment such as "the barbell is not lifted sufficiently" may be included.

Posture information is information related to the posture of the user's body. Posture information includes the part to be evaluated, posture value, evaluation rank, and evaluation comment, which are associated with a time point on the time axis of the video. Posture value is a value that represents posture. For example, posture value is the distance from the ground to the part, the distance between two parts, the angle of the joint part (the angle made by the straight line from the first end part to the joint part and the straight line from the second end part to the joint part), etc. Evaluation rank is a value that expresses the evaluation value by rank. The evaluation rank is expressed, for example, from 1 to 5 on a 5-point scale, or ABC. Evaluation comment is a comment related to the evaluation of posture. For example, if the mode is "lifting" and bending is insufficient, an evaluation comment such as "your hips are not lowered" may be included. (Flexibility such as the range of motion of the joints, numerical values (expressed as angles and distances) → also affect (confirmation) advice)

The body movement information is information related to the user's body movements. The body movement information includes the parts to be evaluated, a list of posture values, an evaluation rank, and an evaluation comment, all of which are associated with a period on the time axis of the video. The list of posture values is a time series of posture values within a period. The evaluation comment is a comment related to the evaluation of the movement. For example, if the mode is "lifting" and the knee extension is not smooth, an evaluation comment such as "the knee movement is not smooth" may be included.

The relationship information indicates one or more pieces of positional relationship information for the tool and the body. The relationship information includes information on the position, orientation, and movement of the tool, and information on the relationship between the body parts, posture, and movement, etc., in association with a time point on the time axis of the video. For example, based on the tool position information and the body position information, it is possible to display the positional relationship between the two. That is, for example, a figure indicating a part (such as a circle) can be superimposed on the image at the position indicated by the tool position information, and a figure indicating a part (such as a circle) can be displayed at the position indicated by the body position information. Note that the positions of multiple parts and parts can be included for one time point. Note that the position information does not need to be included for the part connecting the parts (for example, the tip of the bat and the center point of the part where the user is holding the bat). In this case, the part connecting the two parts and parts can be expressed by connecting the pair of figures (such as a circle) indicating two specified parts and parts with a line. The relationship information may be included for each frame that constitutes the video, for each key frame, for each checkpoint (details will be described later), for every arbitrary number of frames, or for a random point in time. If My Frame does not include location information, shapes showing parts or regions and lines connecting them can be displayed based on the location information of the most recent past point in time.

Checkpoint information is information that indicates points (hereinafter referred to as checkpoints) where the orientation of the tool or the posture of the body should be checked in the movement of the tool used by the user or in a series of movements of the user's body. For example, in the "weightlifting" mode, checkpoints include when the barbell reaches its highest position, when it reaches its lowest position, and the moment it is lifted. In the "pitching" mode, checkpoints include when the foot is raised, when the raised foot is lowered and the weight is transferred, and when the ball is released. The checkpoint information stores information indicating the checkpoint (hereinafter referred to as checkpoint ID) in association with a time point on the time axis of the video. In other words, it is possible to identify the frame (still image) in the video in which the checkpoint indicated by the checkpoint ID is displayed.

The evaluation display unit 114 displays the evaluation information. For example, the evaluation display unit 114 can display a figure representing a part of the tool or a part of the body (for example, a circle representing the end of the tool or the center of gravity of the body, and a line connecting them) superimposed on the video based on the tool position, tool orientation, tool movement information, body position, posture, body movement information, etc. included in the evaluation information, thereby superimposing the tool part or the movement of the body part on the video. Furthermore, the evaluation display unit 114 can display, for example, a graph of the position, orientation, and movement of the tool part, as well as the position, posture, and movement of the body part over time.

The evaluation display unit 114 can also display an evaluation rank and an evaluation comment in conjunction with the display of the video, based on the tool orientation information and tool movement information included in the evaluation information. For example, the evaluation display unit 114 can display the evaluation rank and evaluation comment included in the tool orientation information when the playback time of the video reaches a point around the time point included in the tool orientation information (for example, this can be any length, such as around 5 seconds). The evaluation display unit 114 can also display the evaluation rank and evaluation comment included in the tool movement information when the playback time of the video reaches a period included in the tool movement information. The evaluation display unit 114 can also display the orientation value included in the orientation information. The evaluation display unit 114 can also display a graph of the time-series conversion of the tool orientation value, based on a list of tool orientation values included in the tool movement information.

The evaluation display unit 114 can display the evaluation rank and evaluation comment in accordance with the display of the video based on the posture information and movement information included in the evaluation information. For example, the evaluation display unit 114 can display the evaluation rank and evaluation comment included in the posture information when the playback time of the video reaches a point around the time point included in the posture information (for example, it can be any length, such as around 5 seconds). The evaluation display unit 114 can display the evaluation rank and evaluation comment included in the movement information when the playback time of the video reaches a period included in the movement information. The evaluation display unit 114 can display the posture value included in the posture information. The evaluation display unit 114 can display a graph of the time-series conversion of the posture value based on a list of posture values included in the movement information. The evaluation rank and the evaluation comment may be displayed together with the evaluation rank and evaluation comment displayed in accordance with the display of the video based on the tool orientation information and tool movement information described in the previous paragraph.

The checkpoint display unit 115 can extract and display images of checkpoints from the video. The checkpoint display unit 115 can read frames corresponding to time points included in the checkpoint information from the image data of the video stored in the image storage unit 131 and display them as still images. Furthermore, the checkpoint display unit 115 may, for example, extract and display only body parts from the read frames.

The improvement measure request sending unit 116 sends a request for obtaining improvement measures related to tool handling and physical exercise (hereinafter referred to as an improvement measure request) to the server device 20. FIG. 7 is a diagram showing an example of the configuration of an improvement measure request. As shown in the figure, the improvement measure request includes a user ID, a mode, and a purpose. The purpose is the purpose for which the user makes an improvement. Examples of purposes can be "increasing ball speed," "increasing muscle strength," "stabilizing the lower body," and the like. The purpose is also selected from a predetermined list of options.

The improvement information receiving unit 117 receives information on the improvement measures (hereinafter referred to as improvement information) transmitted from the server device 20 in response to the improvement measure request. The improvement information receiving unit 117 stores the received improvement measure information in the improvement measure storage unit 133. An example of the configuration of the improvement measure information is shown in FIG. 8. As shown in the figure, the improvement measure information includes a purpose, advice, and reference information. In this embodiment, the advice is assumed to be a character string representing the improvement measure, but it may be content that presents the improvement measure using an image or video. The reference information is the preferred orientation and movement of the tool (the position, orientation, movement, speed, etc. of each part) and the body posture (the position and angle of each part, etc.). The improvement information receiving unit 117 may receive the improvement measure information transmitted by the improvement measure information transmitting unit 216 based on the evaluation result and the reference value even if there is no improvement measure request.

The improvement information display unit 118 displays the improvement measures. The improvement information display unit 118 displays advice included in the improvement information. In addition, if the improvement information includes a suitable position or angle for a part,

FIG. 9 is a diagram showing an example of the hardware configuration of the server device 20. The server device 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to the communication network 40, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data.

Fig. 10 is a diagram showing an example of the software configuration of the server device 20. As shown in the figure, the server device 20 includes functional units, namely, an evaluation request receiving unit 211, an image analysis unit 212, an evaluation unit 213, an evaluation information transmitting unit 214, an improvement request receiving unit 215, and an improvement information transmitting unit 216, as well as storage units, namely, an image data storage unit 231, a reference information storage unit 232, an evaluation condition information storage unit 233, and an improvement information storage unit 234.

The above-mentioned functional units are realized by the CPU 201 of the server device 20 reading out a program stored in the storage device 203 into the memory 202 and executing it, and the above-mentioned storage units are realized as part of the storage area provided by the memory 202 and storage device 203 of the server device 20.

The evaluation request receiving unit 211 receives an evaluation request sent from the user terminal 10. The evaluation request receiving unit 211 registers information including image data included in the received evaluation request (hereinafter referred to as image information) in the image data storage unit 231. FIG. 11 is a diagram showing an example of the configuration of image information stored in the image data storage unit 231. As shown in the figure, the image information includes image data associated with a user ID indicating the user who captured the image. The image data was included in the evaluation request.

The reference information storage unit 232 stores information (hereinafter referred to as reference information) including reference values related to the relationship derived from the relationship between the tool and the body, such as the tool position, tool movement (tool orientation, movement, etc.), posture (position and angle of a part, etc.) related to the physical exercise using the tool, and the relationship between the tool and the body. FIG. 12 is a diagram showing an example of the configuration of reference information stored in the reference information storage unit 232. As shown in the figure, the reference information includes, but is not limited to, the absolute position of the tool part when performing the physical exercise using the tool, information on how the tool part moved (movement speed, movement distance, movement direction, etc.), reference information on the absolute position of the part or its relative position with respect to other parts or other reference objects (hereinafter referred to as position reference information), and, for three parts including a joint part, reference information on the angle formed by the straight line connecting each of two parts and the joint part (hereinafter referred to as angle reference information), and further information on the relationship between the tool part and the body part.

The tool position reference information includes a part of the tool and a reference position for that part, in association with the mode and the checkpoint ID. There may be multiple parts. The vertical position of the part may be, for example, the height from the ground, or the distance from either foot. For example, if the mode is "weightlifting", the distance may be from a body part or a line connecting parts, such as the distance between the line connecting both shoulders and the shaft. The horizontal position of the part may be the distance from a specified reference object (for example, a mound plate or a mark on the floor) or the distance from a reference part such as the shoulders, chest, or foot. The position reference information is assumed to be registered in advance.

The tool movement reference information includes reference values for information such as the movement speed and distance of tool parts, the direction of movement at a given point in time, and the trajectory of movement over a given period of time, in association with the mode and checkpoint ID.

The position reference information includes a body part and a reference position for that body part, in association with a mode and a checkpoint ID. There may be multiple body parts. Regarding the position, the vertical position may be, for example, the height from the ground, or the distance from one of the toes. The horizontal position may be the distance from a specified reference object (for example, a mound plate or a mark on the floor) or the distance from a reference body part such as the shoulders, chest, or feet. The position reference information is assumed to be registered in advance.

The angle reference information corresponds to a mode and a checkpoint ID and includes two parts (part 1 and part 2), one joint part, and a reference value of the angle between a line connecting part 1 and the joint part, and a line connecting part 2 and the joint part.

The related criteria information includes information on criteria expressed in the relationship between parts of the tool and parts of the body, in association with the mode and checkpoint ID. The related criteria information includes information obtained from the movement speed, movement distance, angle, etc., for one or more parts and parts, in association with the mode and checkpoint ID. For example, when the mode is batting, the related criteria information includes the movement speed of the tip of the bat at the time the ball is met, and the angle formed by the bat and the dominant arm holding the bat, as reference information.

The evaluation condition information storage unit 233 stores information for evaluation (hereinafter, referred to as evaluation condition information). FIG. 13 is a diagram showing an example of the configuration of the evaluation condition information stored in the evaluation condition information storage unit 233. The evaluation condition information includes a category, a condition, an evaluation rank, and a comment. The category is an evaluation category. Examples of categories include "muscle strength," "ball speed," and "control." The conditions are conditions for the position, orientation, or movement (change in position in time series) of each part of the tool in the image, and also conditions for the position or movement (change in position in time series) of each part of the body. For example, when analyzing weightlifting movements, the evaluation condition information can set conditions for the elbow angle and the speed at which the arms are extended for the checkpoint at the moment the barbell is lifted, as well as conditions for the movement of the shaft and the speed at which the barbell is raised and lowered. When analyzing pitching form, the evaluation condition information can set conditions for the elbow angle and the line speed of the arms for the checkpoint at which the ball is released. The evaluation rank is the evaluation value when the above conditions are satisfied. The comments are an explanation of the body posture and movement when the above conditions are met.

The image analysis unit 212 (part/region identification unit) analyzes the image data. The image analysis unit 212 analyzes the image data to extract the features of each part of the tool and each part of the body, and identifies the position of each part and each region in the image. The image analysis unit 212 also analyzes the image data to extract the features of each part of the tool and identifies the direction in which each part is facing. Note that a general method of image analysis is adopted by the image analysis unit 212, and detailed explanations are omitted here. The image analysis unit 212 may analyze the image data for each frame or key frame, may analyze the image data for each checkpoint, or may analyze at random timing.

The image analysis unit 212 also compares the position of each part extracted from the image data for each checkpoint ID with the position reference information stored in the reference information storage unit 232, and identifies the closest time as the checkpoint time.

The evaluation unit 213 evaluates the movement of the tool used by the user based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including conditions satisfied by the positions of each part of the tool identified from the image data and the movements of the parts, and if there is evaluation condition information for which the conditions are satisfied, obtains the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the movement of the tool and count the number of physical movements.

The evaluation unit 213 evaluates the user's body movements based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including the conditions satisfied by the position of each body part and the movement of the part identified from the image data, and if there is evaluation condition information whose conditions are satisfied, obtains the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the body movements and count the number of body movements.

The evaluation unit 213 evaluates the tool used by the user and the body movements based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including the positions of each part of the tool and each part of the body identified from the image data, and the conditions satisfied by the movements or relationships of the parts and parts, and if there is evaluation condition information for which the conditions are satisfied, obtains the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the tool and the body movements and count the number of body movements.

The evaluation information transmission unit 214 transmits the evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates tool position information including the time on the time axis of the video identified by the image analysis unit 212 and the position of each part of the tool. For the evaluation rank and comment acquired by the evaluation unit 213, if the position of the part of the tool satisfies the condition, the evaluation unit 214 generates posture information including the time, part, and tool orientation value, and the evaluation rank and comment, and if the movement of the part (change in position in time series) satisfies the condition, the evaluation information transmission unit 214 generates tool movement information including a list of the time, part, and tool orientation value, and the evaluation rank and comment. In addition, the evaluation information transmission unit 214 generates checkpoint information including the time corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated tool position information, tool orientation information, tool movement information, and checkpoint information, and transmits it to the user terminal 10. Note that the evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

The evaluation information transmission unit 214 transmits the evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates position information including the time on the time axis of the video identified by the image analysis unit 212 and the position of each part. For the evaluation rank and comment acquired by the evaluation unit 213, if the position of the part satisfies the condition, posture information including the time, part, and posture value, and the evaluation rank and comment is generated, and if the movement of the part (change in position in time series) satisfies the condition, movement information including a list of the time, part, and posture value, and the evaluation rank and comment is generated. In addition, the evaluation information transmission unit 214 generates checkpoint information including the time corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated position information, posture information, movement information, and checkpoint information and transmits it to the user terminal 10. Note that the evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

The improvement measure information storage unit 234 stores information related to improvement measures (hereinafter referred to as improvement measure information). FIG. 14 is a diagram showing an example of the configuration of improvement measure information stored in the improvement measure information storage unit 234. As shown in the figure, the improvement measure information includes advice associated with the purpose, category, and conditions. The conditions may be conditions on the tool itself (such as the weight of a barbell), how to use the tool, physical conditions (such as flexibility), conditions on the position, orientation, and movement of a part of the tool, or conditions on the position and movement of a part of the body.

The improvement request receiving unit 215 receives an improvement request sent from the user terminal 10.

The improvement information transmission unit 216 searches for improvement information corresponding to the mode and purpose included in the improvement request that satisfies the conditions of the user's body information included in the evaluation request and the position, orientation, movement, etc. of each part or each part identified by the image analysis unit 212. The improvement information transmission unit 216 obtains advice for the searched improvement information, creates improvement information in which the purpose and advice are set, and responds to the created improvement information to the user terminal 10. The improvement information transmission unit 216 also transmits the position, orientation, speed, angle, etc. of each part or each part included in the reference information, including it in the improvement information. Note that the improvement information transmission unit 216 may search for improvement measures based on the evaluation information and the reference value even without an improvement request, and the improvement information transmission unit 216 may transmit the improvement measures to the user terminal 10.

Figure 15 shows an example of the flow of processing executed by the physical exercise support server of this embodiment.

In the user terminal 10, the imaging unit 111 accepts the input of the mode, images the user's body while exercising, and obtains video data (S321). The evaluation request sending unit 112 sends an evaluation request including the user ID indicating the user, the accepted mode, physical information, and video data to the server device 20 (S322).

When the evaluation request receiving unit 211 in the server device 20 receives an evaluation request, the image analysis unit 212 analyzes the video data to extract features (S323) and identifies the position of each part and each part (S324). Here, the image analysis unit 212 may identify the position on the image, or may identify the actual position (height from the ground, distance from a reference point such as the center of gravity of the body, etc.) using body information. The evaluation unit 213 obtains an evaluation rank and comments from evaluation condition information that satisfies the conditions of the position of each part and each part, or the movement of each part and part (change in position over time) (S325). The evaluation information sending unit 214 creates evaluation information and sends it to the user terminal 10 (S326).

The evaluation display unit 114 in the user terminal 10 displays the position, orientation, movement, etc. of the tool on the video data based on the received evaluation information (S327). In addition, the evaluation display unit 114 in the user terminal 10 may display the position (bones) of each part indicating the posture of the body, as well as an evaluation rank and comments (S327). Here, the evaluation display unit 114 may display a graph of the position, orientation, movement, etc. of parts, as well as chronological changes in the position and movement of parts. In addition, the checkpoint display unit 115 may extract and display images of checkpoints from the video. The improvement measure request sending unit 116 sends an improvement measure request to the server device 20 in response to an instruction from the user (S328).

In the server device 20, when the improvement request receiving unit 215 receives an improvement request sent from the user terminal 10, the improvement information sending unit 216 searches for improvement information that satisfies the conditions, acquires advice contained in the searched improvement information (S329), creates improvement information including the acquired advice, and sends it to the user terminal 10 (S330).

When the improvement information receiving unit 117 receives improvement information in the user terminal 10, the improvement information display unit 118 displays the advice contained in the received improvement information and can also display the appropriate way to use the tool by superimposing it on the video data (S331).

When the improvement information receiving unit 117 receives improvement information in the user terminal 10, the improvement information display unit 118 displays the advice contained in the received improvement information and can display an appropriate body posture in the form of bones superimposed on the video data (S331).

In this way, the physical exercise support server of this embodiment makes it easy to evaluate physical exercise. In particular, for physical exercise related to sports, the positional relationships and movements of each part of the equipment and each part of the body can be evaluated, which is likely to lead to concrete efforts to improve and improve performance. In addition, the physical exercise support server of this embodiment also provides comments and advice, allowing the user to easily understand the current situation and improvement measures.

Fig. 17 is a diagram showing an example of a screen that displays an evaluation of physical exercise using a tool. Fig. 17 illustrates a case where a video is captured in weightlifting mode. As shown in Fig. 17, a mark 411 indicating the position of the barbell shaft is displayed on screen 41. The movement of the barbell shaft is displayed by a line 412.

Fig. 18 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Fig. 18 illustrates a case where a video is captured in weightlifting mode. As an example of the results of evaluation performed by the evaluation unit 213 in weightlifting mode, the inclination of the barbell shaft, the distance traveled by the shaft, and the speed of travel (line 421) are displayed. In Fig. 18, a reference value (line 422) is displayed. The actual measurement result (line 423) may be displayed, and the difference from the reference value may be displayed numerically or as a graph. The user can use this as a reference to consider movements and postures that need to be corrected.

Fig. 19 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Fig. 19 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation by the evaluation unit 213 in weightlifting mode, for example, the evaluation result (line 432) is displayed together with a reference value (line 431) such as the angle of the body joints at the lowest point of the barbell. The difference from the reference value may also be displayed numerically or as a graph. Furthermore, the evaluation result based on the relationship between the body and the tool may also be displayed. The user can use this as a reference to consider movements and postures that should be corrected.

Fig. 20 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Fig. 20 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation by the evaluation unit 213 in weightlifting mode, lines 441 show bones that are lines connecting parts of the tool identified from the image and predetermined positions of body parts. The bones may be displayed superimposed on the captured image. Lines 442 show the acceleration of each body part. Furthermore, counting results such as the number of times the weight was lifted may be displayed. As shown by line 443, the evaluation results and the next training results according to the purpose, etc. may also be displayed.

Figure 21 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Figure 21 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation performed by the evaluation unit 213 in weightlifting mode, an evaluation (line 441) obtained by comparison with various reference values is displayed.

The group analysis unit 217 is intended to be used in situations where multiple users use the system at the same time and a small number of supporters provide instruction remotely (for example, training instruction using online meeting tools, online communication tools, etc., or remote rehabilitation support). In such situations, there may be multiple users for one supporter. Although image information of multiple users is displayed in real time on the supporter terminal, it is difficult to grasp the detailed status of the users, such as which users are training effectively and which users are tired, and as a result, the effect is reduced. In addition, it is difficult to communicate between the supporter and the user based on the content of the training that was performed, which is an issue of low continuity. The group analysis unit 217 solves such problems.

The group analysis unit 217 analyzes the evaluation information performed by the evaluation unit 213 for the image of each user, and stores the analyzed information in the group analysis information storage unit 235. Based on the evaluation information evaluated by the evaluation unit 213 for the physical exercise (including exercise using tools) performed by the user, the group analysis unit 217 creates a ranking of users whose physical exercise (which may be analyzed in real time) was effective (or ineffective), such as the number of repetitions of the training performed and the order close to (or far from) the reference value. In addition, each user may be categorized into a top group, a bottom group, etc. in the ranking. In addition, as the physical exercise is continued, the position, movement speed, movement distance, joint angle, etc. of the tool part or body part may be determined to be deviating from the reference value, and the degree of fatigue may be estimated based on the deviation from the reference value. Furthermore, the time to complete a predetermined number of repetitions may be measured for each set, and the degree of fatigue may be determined based on the increase in the completion time, or the degree of fatigue may be estimated based on the increase in the time or the completion time of the previous set.

Furthermore, the group analysis unit 217 may estimate the concentration level of the user by evaluating the time until the actual start of the physical exercise, such as the supporter's signal to start or end the exercise. Furthermore, the group analysis unit 217 may calculate a deviation value from the evaluation information between users who are participating in training at the same time, or may create a ranking using the deviation value. Furthermore, the group analysis unit 217 may calculate a deviation value from the evaluation information between users who performed the same physical exercise at different times, or may create a ranking using the deviation value. Furthermore, from the ranking, fatigue level, concentration level, etc., users who need guidance or users for whom direct comments are thought to be effective in terms of continuity, etc. may be identified. Furthermore, the group analysis unit 217 may compare evaluation information from when the user previously performed a similar physical exercise with evaluation information evaluated from an image of a newly performed physical exercise.

The group analysis information storage unit 235 stores the group analysis information performed by the group analysis unit 217. FIG. 15 is a diagram showing an example of the configuration of group analysis information stored in the group analysis information storage unit 235. The group analysis information broadly includes ranking information, status information, and warning information. The ranking information includes rankings and other category information based on the evaluation information, such as whether the ranking is in a higher or lower group. The status information includes fatigue levels and concentration levels. The warning information includes information such as users who need special guidance.

The group analysis information presentation unit 218 presents the group analysis information stored in the group analysis information storage unit 235 to the user terminal 10 and the supporter terminal 50. The group analysis information presentation unit 218 presents to the user terminal 10 the ranking, deviation value, and categories such as top and bottom, of the user among users who are performing physical exercise at the same time, and among users including users who performed similar physical exercise at different times. Furthermore, the group analysis information presentation unit 218 may also present information stored in the evaluation information storage unit 132. Furthermore, the group analysis information presentation unit 218 may present the ranking and deviation value information of each user to the supporter terminal 50, or may color the users who are ranked high (or low) so that they are easily visible on the screen (such as by displaying the screen showing the user large or at the top).

Figure 22 is a diagram showing an example of a screen created by the group analysis information presentation unit 218 and presented on the user terminal 10. Figure 22 illustrates a case where a video is captured in squat mode. As shown in Figure 22, the screen 46 displays the user's ranking within the group with line 461. Line 462 indicates the number of exercises performed. Line 463 displays a real-time video (or a recorded video) of a model trainer.

Figure 23 is a diagram showing an example of a screen created by the group analysis information presentation unit 218 and presented to the supporter terminal 50. The supporter terminal 50 displays each user's evaluation results and group analysis information, etc., along with a video of each user.

24 is a diagram showing an example of a screen created by the group analysis information presentation unit 218 presenting it to the supporter terminal 50. The screen 47 shows images of users lined up, and users in the group who need more guidance (including support, etc.) are displayed in a way that is easy for the supporter to understand based on the evaluation and group analysis. As an example, a mark is displayed in the upper right corner of the screen showing the user who needs guidance. The screen 48 shows details of each user, and displays the evaluation results (bones and posture values, tool orientation values, number of repetitions of physical exercise, etc.), group analysis results (ranking within the group and its progress, etc.), advice sent to the user by the improvement information sending unit 216, a function to view recorded images (or a part of a video, such as a few seconds before and after the point when the evaluation was lowered), etc. Also, the screen 47 and the screen 48 may be combined.

Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the interpretation of the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

For example, in this embodiment, the image analysis is performed in the server device 20, but this is not limited to the above. The image analysis may be performed in the user terminal 10 to identify the positional relationship of each part and each region.

In addition, in this embodiment, the positions of the parts and regions are assumed to be positions on a two-dimensional image, but this is not limited to this, and may be three-dimensional positions. For example, if the user terminal 10 is equipped with a depth camera in addition to the camera 106, the three-dimensional positions of the parts and regions can be identified based on the image from the camera 106 and the depth map from the depth camera. Also, for example, the three-dimensional position of the parts and regions may be identified by estimating the three-dimensionality from the two-dimensional image. It is also possible to provide a depth camera instead of the camera 106 and identify the three-dimensional position only from the depth map from the depth camera. In this case, the user terminal 10 can transmit the depth map to the server device 20 together with the image data or instead of the image data, and the image analysis unit 212 of the server device 20 can analyze the three-dimensional position.

In addition, in this embodiment, an image of the user's body while exercising using a tool is transmitted from the user terminal 10 to the server device 20, but this is not limited to the above. The user terminal 10 may extract features from the image and transmit the features to the server device 20. Alternatively, the user terminal 10 may estimate the parts of the tool or the body parts based on the features, obtain the absolute positions of the parts or parts (which may be positions on the XY coordinates of the image, or the actual distance from a reference position (for example, the ground, the toes, the head, the center of gravity of the body, etc.), or the positions in any other coordinate system) or the relative positional relationships between multiple parts, multiple parts, or multiple parts and parts, and transmit these absolute positions or relative positional relationships to the server device 20.

In addition, in this embodiment, the improvement information is provided as content prepared by the server device 20, but this is not limited thereto. For example, it is also possible to include reference values, and superimpose marks or bones that represent correct movements and postures (such as the position and orientation of each part, the position and angle of each body part, etc.) based on the reference values onto a video or a still image extracted from the video. This makes it easy to understand what movements and postures should be used.

In addition, in this embodiment, the part or orientation of the tool, the position or movement of a body part, etc. (position over time) are evaluated, but this is not limited to this, and the position of the tool worn by the user may be identified and evaluated.

In addition, in this embodiment, the content such as advice is provided as a remedy, but for example, a tool recommendation may be performed. In this case, the server device 20 stores the tool and a reference value of the size (length, etc.) of the tool in association with the user's physical information (height, weight, etc.), extracts the feature amount of the tool used by the user from the image data to identify the shape of the tool, estimates the size of the tool based on the shape and the size of the user (e.g. height) included in the physical information, and if the difference between the estimated tool size and the reference value is equal to or greater than a predetermined threshold, recommends a tool of the reference size. Furthermore, a tool according to the purpose may be recommended based on information such as the conditions of the tool itself (weight of a barbell, etc.), how to use the tool, physical conditions (flexibility, etc.), the position, orientation, and movement of the tool parts, etc.

In addition, in this embodiment, the content such as advice is provided as a remedy, but for example, the physical exercise being performed may be interrupted. In this case, the server device 20 stores a reference value for when the physical exercise should be interrupted in association with the user's physical information (purpose, height, weight, etc.), and interrupts the physical exercise if the number of times or speed of the physical exercise being performed by the user from the image data (for example, the speed at which the barbell is lifted drops drastically, or too many times are performed at once) deviates from the reference value. In this case, the server device 20 may issue a comment to the user terminal 10 to stop, or may inform the user by changing the display such as turning off the screen, or may issue a sound such as an alert, or may inform the user by vibration.

In addition, in this embodiment, content such as advice is provided regarding improvement measures, but for example, a diagnosis of illness and physical exercises aimed at improving the illness may be presented. In this case, the server device 20 extracts candidates for illnesses that the user is suspected to have from the symptoms and evaluation information entered by the user in the physical information, and presents a screening test to narrow down the candidates. Once the user has performed the screening test and narrowed down the illness, the server device 20 may recommend seeing a doctor, or recommend physical exercises aimed at improving the illness, or items such as tools and food for performing physical exercises.

In addition, by estimating the positions of the tool parts, the server device 20 can estimate the speed, acceleration, distance traveled, trajectory, etc. of the tool. In addition, by extracting a pattern of changes in the tool position over time, the server device 20 can estimate the number of times a pattern occurs as the number of actions using the tool.

In addition, in this embodiment, the exercise is evaluated, but the present invention is not limited to this. When a certain posture or movement is detected, a task for that movement may be suggested. In this case, the server device 20 may store a task in association with one or a series of postures or movements instead of an evaluation comment, and output the task.

In addition, in this embodiment, the exercise is evaluated, but the present invention is not limited to this. When the movement of a certain tool, the orientation of the tool, the posture, or the movement of a body part is detected, the content for improving the physical exercise according to the purpose, etc., such as training to be performed, rehabilitation, playing, or preparatory steps such as stretching, strength training, and posture, may be presented. In this case, the server device 20 may store the contents of the training, etc., in place of the evaluation comment, in association with one or a series of tool part movements, tool part orientations, body postures, or body part movements, and output the contents.

In addition, in this embodiment, the motion is evaluated, but the present invention is not limited to this, and it is also possible to automatically detect the motions performed by the user. In this case, the server device 20 stores the positions and postures of each part of the tool used to perform a specific motion, such as shooting or passing (the positions of each body part) as reference information, and can identify the motions performed by the user in the image by comparing the positions of the tool parts and body parts analyzed from the image with the reference information.

In addition, in this embodiment, images captured in the past are analyzed to evaluate the movement, but this is not limited to the above. The analysis process may be performed in real time, and when a specific movement is detected, the next tactic to be taken may be recommended. In this case, the tactic may be stored in place of the evaluation comment in association with the posture or movement, and the tactic may be output in real time.

In addition, in this embodiment, the execution of a specific function and the storage of information are performed by the user terminal 10 or the server device 20, but this is not limited thereto, and the execution of the function and the storage of information may be performed by either one of the devices. Alternatively, the functional unit and the storage unit may be provided in a distributed manner in a form different from this embodiment.

10 User terminal 20 Server device 30 Imaging terminal 40 Communication network 50 Supporter terminal 111 Imaging unit 112 Evaluation request transmission unit 113 Evaluation information reception unit 114 Evaluation display unit 115 Checkpoint display unit 116 Improvement measure request transmission unit 117 Improvement measure information reception unit 118 Improvement measure information display unit 130 Physical information storage unit 131 Image storage unit 132 Evaluation information storage unit 133 Improvement measure storage unit 211 Evaluation request reception unit 212 Image analysis unit 213 Evaluation unit 214 Evaluation information transmission unit 215 Improvement measure request reception unit 216 Improvement measure information transmission unit 217 Group analysis unit 218 Group analysis presentation unit 231 Image data storage unit 232 Criteria information storage unit 233 Evaluation condition information storage unit 234 Improvement measure information storage unit 235 Group analysis information storage unit

Claims (5)

A physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool,
A reference value storage function that stores a reference value related to a positional relationship between a part of the tool and a part of the user's body;
a part identification function that analyzes an image of the tool to identify a part of the tool and a part of the body and acquires a positional relationship between the part and the part;
an evaluation function that determines an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired by the part identification function and the reference value;
A physical exercise support system comprising: a comment storage function for storing a comment on the relative positional relationship in association with the part and a condition for the relative positional relationship between the part and the body part;
a comment output function that outputs the comment corresponding to the condition satisfied by the positional relationship acquired by the part identification function;
The physical exercise support system according to claim 1 . an improvement information storage function for storing advice on an improvement measure for the physical exercise in association with a goal of improvement for the exercise and a condition for a relative positional relationship between a part of the tool and a part of the body;
an advice output function that outputs the advice corresponding to the condition satisfied by the positional relationship acquired by the part identification function;
The physical exercise support system according to claim 1 or 2, A physical exercise support server that supports a user in improving the effectiveness of physical exercise performed using a tool,
a reference value storage unit that stores a reference value related to a positional relationship between a part of the tool and a part of the user's body;
a part identification unit that analyzes an image of the tool to identify a part of the tool and a part of the body and acquires a positional relationship between the part and the part;
an evaluation unit that determines an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired by the part identification unit and the reference value;
A physical exercise support server comprising: A physical exercise support method for supporting a user in improving the effectiveness of physical exercise performed using a tool, comprising:
The computer
A reference value storage step of storing a reference value related to a positional relationship between a part of the tool and a part of the user's body;
a part identification step of identifying a part of the tool and a part of the body by analyzing an image of the tool and acquiring a positional relationship between the part and the part;
an evaluation step of determining an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired in the part identification step and the reference value;
A physical exercise support method comprising the steps of: