JP7728655B2 - VIDEO MANAGEMENT SYSTEM, VIDEO MANAGEMENT METHOD, AND VIDEO MANAGEMENT PROGRAM - Google Patents

Description

One aspect of the present disclosure relates to a video management system, a video management method, and a video management program.

Mechanisms for assigning tags to video are known. For example, Patent Document 1 discloses technology for checking a subject's behavior tag associated with video showing the subject's behavior.

Japanese Patent Application Laid-Open No. 2017-28565

There is a need for a mechanism to analyze desired scenes in video.

A video management system according to one aspect of the present disclosure includes at least one processor. The at least one processor receives a request specifying a video set including at least one video and supplementary information for identifying scenes, extracts scenes corresponding to the supplementary information from the at least one video, generates a playlist for playing the scenes, and outputs the playlist.

A video management method according to one aspect of the present disclosure is executed by a video management system including at least one processor. The video management method includes the steps of: receiving a request specifying a video set including at least one video and accompanying information for identifying scenes; extracting scenes corresponding to the accompanying information from the at least one video; generating a playlist for playing the scenes; and outputting the playlist.

A video management program according to one aspect of the present disclosure causes a computer to execute the steps of: acquiring a request specifying a video set including at least one video and accompanying information for identifying a scene; extracting scenes corresponding to the accompanying information from the at least one video; generating a playlist for playing the scenes; and outputting the playlist.

In this aspect, a scene corresponding to the specified supplementary information is extracted from at least one video in the specified video set. A playlist is then generated from the extracted scenes and output. The playlist corresponds to an edited video that brings together the specified scenes in the specified video set. This provides an environment for analyzing desired scenes.

In another aspect of the video management system, at least one processor may receive a request specifying multiple video sets, extract a scene from at least one video for each of the multiple video sets, and generate a playlist for playing the multiple scenes corresponding to the multiple video sets. With this configuration, a scene corresponding to the specified supplementary information is extracted from at least one video for each of the specified multiple video sets. A playlist is then generated from the extracted multiple scenes. This provides an environment for analyzing desired scenes in multiple video sets.

In another aspect of the video management system, at least one processor may generate a playlist for consecutively playing multiple scenes. This allows desired scenes from multiple video sets to be combined into a single playlist.

In another aspect of the video management system, at least one processor may generate a playlist for each of the multiple video sets. This makes it possible to provide a playlist that collects desired scenes from the multiple video sets for each video set.

In a video management system according to another aspect, at least one processor may extract, from at least one video, a scene corresponding to the start time and end time indicated by the supplementary information. With this configuration, scenes are extracted using the start time and end time indicated by the supplementary information, making scene extraction easier.

In another aspect of the video management system, the additional information may be a tag assigned by a user to at least one video. At least one processor may extract, from at least one video, a scene corresponding to the start time and end time indicated by the tag. With this configuration, the tag assigned by the user is specified to identify the scene. The tag defines the start time and end time of the scene, making it easier to extract the scene.

In another aspect of the video management system, the additional information may be a measurement value category indicating a category of measurement data values obtained at the same time as the video set. At least one processor may extract scenes from at least one video that correspond to start and end times determined by the measurement value category. With this configuration, a measurement value category based on the measurement data is specified to identify a scene. Because the video set and the measurement data are synchronized, the start and end times of a scene are automatically determined when the measurement value category is specified. This makes it easier to extract scenes.

One aspect of the present disclosure provides an environment for analyzing a desired scene.

1 is a diagram illustrating an example of an overall configuration including a video management system according to an embodiment. FIG. 1 is a diagram illustrating an example of a general hardware configuration of a computer that constitutes a video management system according to an embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration related to a video management system according to an embodiment. FIG. 2 is a diagram illustrating an example of a video window. 5A and 5B are diagrams illustrating an example of an aggregation window, in which the additional information is a tag, and the additional information is a measurement value category. FIG. 10 is a sequence diagram showing an example of an operation of the video management system according to the embodiment. FIG. 10 is a diagram showing an example of a playlist. FIG. 10 is a diagram showing an example of a playlist. FIG. 10 is a diagram showing an example of a playlist.

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the description of the drawings, identical or equivalent elements will be designated by the same reference numerals, and duplicate explanations will be omitted.

[System Overview]
A video management system according to an embodiment is a computer system that provides a user with an environment for analyzing a video set including at least one video. A video set is a collection of at least one video captured at the same time. The individual videos in the video set may be of different subjects, or may be of the same subject captured from different sides. A user is a person who wishes to analyze a desired scene in a video. A scene includes a scene and a landscape. The video may be provided by the same user as the analyst, or by a different user than the analyst.

The video management system receives a request from a user terminal specifying information about a desired scene. Based on the received request, the video management system extracts the desired scenes and generates a playlist. A playlist is data representing edited video that brings together the desired scenes. By sending the playlist to the user terminal, the video management system provides the user with an environment for analyzing a set of videos. In one example, the video management system sends a playlist containing three videos to the user terminal. The user can analyze the desired scene by playing back the three videos simultaneously on the user terminal.

[System configuration]
1 is a diagram showing an example of the overall configuration including a video management system 1 according to an embodiment. The video management system 1 according to this embodiment includes a server 10. The server 10 is connected to a database 20 and a user terminal 30 via a communication network N. The configuration of the communication network N is not limited. For example, the communication network N may be configured to include the Internet or an intranet.

The server 10 is a computer that executes processing based on requests from the user terminal 30. The server 10 is composed of one or more computers. The server 10 receives a request from the user terminal 30 specifying information about a desired scene. The server 10 extracts scenes based on the received request and generates a playlist. The server 10 transmits the playlist to the user terminal 30.

Database 20 is a non-transitory storage medium or storage device that stores data used by video management system 1. Database 20 is composed of one or more databases. Database 20 may be a component of video management system 1 or a component of an external system. Database 20 stores video data provided by user terminal 30 and various data for extracting scenes from the video data.

The user terminal 30 is a computer used by a user. There are no limitations on the type or configuration of the user terminal 30. For example, the user terminal 30 may be a stationary terminal such as a desktop personal computer. Alternatively, the user terminal 30 may be a mobile terminal such as a high-function mobile phone (smartphone), tablet terminal, laptop personal computer, or mobile phone. While Figure 1 shows three user terminals 30, the number of user terminals 30 is not limited. The user terminal 30 sends various requests to the server 10. The user terminal 30 displays various screens received from the server 10. The user terminal 30 plays playlists.

Figure 2 is a diagram showing an example of a general hardware configuration of a computer 100 constituting the video management system 1 according to an embodiment. For example, the computer 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a communication control unit 104, an input device 105, and an output device 106. The processor 101 executes an operating system and application programs. The main memory unit 102 is composed of, for example, ROM and RAM. The auxiliary memory unit 103 is composed of, for example, a hard disk or flash memory, and generally stores larger amounts of data than the main memory unit 102. The communication control unit 104 is composed of, for example, a network card or wireless communication module. The input device 105 is composed of, for example, a keyboard, mouse, touch panel, etc. The output device 106 is composed of, for example, a monitor and speakers.

Each functional element of the video management system 1 is realized by a program 110 pre-stored in the auxiliary storage unit 103. Specifically, each functional element is realized by loading the program 110 onto the processor 101 or the main storage unit 102 and having the processor 101 execute the program 110. In accordance with the program 110, the processor 101 operates the communication control unit 104, input device 105, or output device 106 to read and write data from and to the main storage unit 102 or the auxiliary storage unit 103. Data or databases required for processing may be stored in the main storage unit 102 or the auxiliary storage unit 103.

Program 110 may correspond to a video management program. Program 110 may be provided by being permanently recorded on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, program 110 may be provided via a communications network as a data signal superimposed on a carrier wave.

The video management system 1 may be composed of one computer 100, or multiple computers 100. When multiple computers 100 are used, these computers 100 are connected via a communication network such as the Internet or an intranet, thereby logically constructing a single video management system 1.

The type of computer 100 is not limited. For example, the computer 100 may be a desktop or portable personal computer (PC), a workstation, or a mobile terminal such as a high-function mobile phone (smartphone), a mobile phone, or a personal digital assistant (PDA). The video management system 1 may be constructed by combining multiple types of computers.

Figure 3 is a diagram showing an example of the functional configuration related to the video management system 1 according to the embodiment. The server 10 includes, as functional elements, a screen management unit 11, a data management unit 12, a counting unit 13, and a playlist generation unit 14.

The screen management unit 11 is a functional element that manages the screens displayed on the user terminal 30. The screens are web pages written in, for example, HTML (HyperText Markup Language). The language used to write web pages is not limited to this. The screen management unit 11 generates screens based on requests from the user terminal 30 and transmits these screens to the user terminal 30. For example, the screen management unit 11 generates an input screen for entering conditions related to the compilation of video data, a compilation screen for displaying the compilation results of video data, and the like, and transmits these screens to the user terminal 30.

The data management unit 12 is a functional element that collectively manages the data in the database 20. The data management unit 12 acquires video data and various data for identifying scenes from the video data from the user terminal 30. The data management unit 12 associates these pieces of data and registers them in the database 20. The data management unit 12 may assign an identifier that uniquely identifies each piece of data acquired from the user terminal 30. The data management unit 12 synchronizes multiple pieces of video data (corrects time differences) as needed. The data management unit 12 also synchronizes video sets and measurement data as needed.

The aggregation unit 13 is a functional element that generates table data based on a request from the user terminal 30. The aggregation unit 13 receives an aggregation request from the user terminal 30, which requests aggregation of video data under specified conditions. The aggregation unit 13 aggregates the data in the database 20 based on the aggregation request. The aggregation unit 13 generates table data that is the aggregation result. The generated table data can be used on the aggregation screen by the screen management unit 11.

The playlist generation unit 14 is a functional element that generates a playlist based on a request from the user terminal 30. The playlist generation unit 14 acquires link information from the user terminal 30 that requests the generation of a playlist under specified conditions. The playlist generation unit 14 generates a playlist based on the link information. The playlist generation unit 14 transmits the generated playlist to the user terminal 30.

The database 20 stores video data, tag data, and measurement data in association with each other. Video data is data on the video itself provided by the user terminal 30. Tag data is data indicating tags assigned by a user to at least one video. Measurement data is data obtained at the same time as the video set.

Database 20 stores, as video data, a video set ID, at least one video ID, and at least one piece of video data in association with each other. A video set ID is an identifier that uniquely identifies a video set. A video ID is an identifier that uniquely identifies a video. When a video set ID is specified, at least one video ID and its video data included in the specified video set are identified.

Database 20 stores tag data, associating a video ID, at least one tag, and at least one set time frame for each tag. For example, a tag is text information that indicates attributes that characterize a scene. Examples of scene attributes include the content, location, and purpose of the scene. In one example, a tag may be quantitative information such as "30 seconds from the start of viewing." A tag may also be qualitative information such as "smile." A set time frame is the start and end times of a video at which a tag is assigned. The set time frame can also be considered the duration of a tagged scene. When a video ID is specified, at least one tag assigned to the video and the start and end times indicated by each tag are identified.

The database 20 stores measurement data in association with a video set ID, at least one measurement ID, and at least one piece of measurement data. The measurement ID is an identifier that uniquely identifies the measurement data. When a video set ID is specified, the measurement ID of the measurement data obtained at the same time as the specified video set and the value of that measurement data are identified. The measurement data may be measurement values related to the shooting scene, such as measurement values related to the subject or measurement values related to the environment of the location where the shooting took place. For example, the measurement data may be measurement values such as pulse rate and heart rate. The measurement data may also be numerical information such as sweating, facial expression, and degree of emotion. Alternatively, the measurement data may be measurement values such as room temperature and volume.

The user terminal 30 transmits a video set including at least one video to the server 10. The user terminal 30 also transmits at least one of a tag and measurement data to the server 10. The user terminal 30 may also transmit a video associated with a tag and measurement data to the server 10.

In one example, the user terminal 30 is equipped with a web browser and displays various screens (web pages) received from the server 10. For example, the user terminal 30 displays an input screen and accepts user input. The user terminal 30 also displays a summary screen and accepts selection operations for generating a playlist. The user terminal 30 transmits the input or selected information to the server 10. The user terminal 30 also plays the playlist received from the server 10. For example, the user terminal 30 plays the playlist using a dedicated tool or a web browser.

Figure 4 shows an example of a video window 200. The video window 200 is displayed on the user terminal 30. The video window 200 is a screen for playing a video set. For example, the video window 200 plays a video set that includes at least one video with a tag. The video window 200 also plays a video set with associated measurement data instead of or in addition to a tag. When a video set includes multiple videos, "playing a video set" refers to playing multiple synchronized videos while maintaining that synchronization. In this case, the video window 200 displays multiple scenes from the same point in time, represented by multiple videos. Tags are associated with videos by the user after the videos are photographed or recorded. Tags can be assigned using a dedicated tool on the video management system 1 or on the video window 200. Measurement data can also be associated using a dedicated tool on the video management system 1 or on the video window 200.

The video window 200 includes video frames 201-203, an operation interface 204, a title field 205, a tag field 206, a tag chart 207, a measurement data field 208, and a comment field 209.

Video frames 201-203 are frames that play back at least one video included in one video set. For example, video frames 201-203 play back each of up to three videos included in one video set. Video frames 201-203 play videos of different subjects captured at the same time. In one example, video frame 201 plays a video of a room that shows a television and its viewer. Video frame 202 plays a video of the viewer. Video frame 203 plays a video of the content being displayed on the television. These three videos are synchronized, so video frames 201-203 show the interior of the room and the viewer when the content is being displayed.

The operation interface 204 is an interface that controls the playback of the video set. For example, the operation interface 204 includes icons for performing playback operations such as returning to the previous cue position, rewinding five seconds, playing, skipping five seconds, skipping to the next cue position, and changing the playback speed. The operation interface 204 accepts playback operations from the user and controls the playback of the video set in response to those playback operations.

The title column 205 is a screen element that displays the title of the video. For example, the title column 205 displays the title corresponding to each of the videos played on the video frames 201-203. In one example, the title column 205 displays "Indoors," "Viewer," and "Content" as the titles corresponding to each of the three videos.

The tag field 206 is a screen element that displays tags used for a video set. The tag field 206 displays tags assigned to at least one video included in a video set. In one example, the tag field 206 displays tags such as "smile," "averted gaze," "eating and drinking," and "conversation" assigned to a video of a viewer.

The tag chart 207 is a screen element that shows the relationship between video and tagging in chronological order. The vertical axis of the tag chart 207 shows a list of tags assigned to the video. The horizontal axis of the tag chart 207 shows the time axis corresponding to the playback time of the video. The tag chart 207 shows the range of start and end times indicated by the tags in the form of a horizontal bar chart. The tag chart 207 may accept a tag selection operation from the user and play a video set in response to that selection operation. In one example, the tag chart 207 may accept a selection operation from the user on the horizontal bar of the "gaze deviation" tag assigned to the viewer's video. In response to that selection operation, the video window 200 may play the video set from the start time to the end time indicated by the horizontal bar of the "gaze deviation" tag.

The measurement data column 208 is a screen element that displays measurement data in chronological order. The vertical axis of the measurement data column 208 indicates the value of the measurement data. The horizontal axis of the measurement data column 208 indicates the time axis. In one example, the measurement data column 208 displays the values of the measurement data in a line graph. The measurement data column 208 may accept a selection operation of measurement data from the user and play a video set in response to the selection operation. In one example, the measurement data column 208 may accept a selection operation from the user for any value on the line graph. In response to the selection operation, the measurement data column 208 may play a video set from the start time indicated by the selected value. The measurement data column 208 may also accept a selection operation from the user for any range of values on the line graph. The video window 200 may play a video set from the start time to the end time indicated by the selected value range.

The comment field 209 is a screen element for registering and displaying comments related to a video set. The comment field 209 registers and displays comments received from a user or a system operator of the video management system 1. The comment field 209 may also register and display a playback position of a video set related to the comment. A playback position may be specified in the comment field 209 by a user or a system administrator. The video window 200 may accept a playback position specification operation from the user and play the video set in response to the specification operation.

The video window 200 described above may be used as a screen for playing a playlist. In this case, in the explanation of Figure 4, "video set" can be replaced with "playlist" and "video" can be replaced with "scene."

Figure 5 shows an example of the tally windows 300, 310. The tally windows 300, 310 are displayed on the user terminal 30. The tally windows 300, 310 correspond to tally screens. The tally windows 300, 310 display indices related to the additional information for each video set in table format. The additional information is information for identifying a scene. The rows of the tally windows 300, 310 correspond to indices related to the additional information. The columns of the tally windows 300, 310 correspond to each video set. The columns of the tally windows 300, 310 include a column related to one video set and a column related to the total for the video set.

Example (a) in Figure 5 shows the tally window 300 when the additional information is a tag. The index for a tag may be the number of times a scene corresponding to the tag appears, the total time, or a statistical value (e.g., average value) per unit time (e.g., one hour). The index for a tag is displayed as a hyperlink (hereinafter referred to as a "link") in the tally window 300. In example (a), each link indicates the number of times a scene corresponding to the tag appears. In one example, link 301 indicates that video set V contains three scenes tagged with "smile." Link 302 indicates that video sets V, W, X, Y, and Z contain a total of six scenes tagged with "smile."

Example (b) in Figure 5 shows the aggregation window 310 when the additional information is a measurement value category. In example (b), the measurement value category indicates the category of the pulse rate value, which is the measurement data. The index for the measurement value category may be the number of times the scene corresponding to the measurement data appears, the total time, a statistical value (e.g., average value) per unit time (e.g., 1 hour), or the measurement data value itself. The index for the measurement value category is shown as a link in the aggregation window 310. In example (b), each link indicates the number of times the scene corresponding to the measurement value category appears. In one example, link 311 indicates that a total of five scenes associated with the measurement value category "50-65" are included in the video sets V, W, X, Y, and Z.

The user terminal 30 accepts a selection operation from the user on a link in the aggregation window 300, 310. The user terminal 30 transmits link information related to the selected link to the server 10. The link information includes information specifying the video set and accompanying information. For example, when a link on the aggregation window 300 is selected, the link information includes the video set ID corresponding to the selected video set and the tag ID corresponding to the tag. When a link on the aggregation window 310 is selected, the link information includes the video set ID corresponding to the selected video set and the measurement value category as accompanying information.

[System Operation]
The operation of the video management system 1 and the video management method according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing an example of the operation of the video management system 1 according to this embodiment as a processing flow S1. In the processing flow S1, the database 20 is described as having stored therein video data, tag data, and measurement data in advance.

(First operation example)
In step S11, the user terminal 30 accepts user input for displaying the tabulation results. For example, the user terminal 30 displays an input screen for inputting conditions for tabulating the video data. The user terminal 30 accepts input for specifying at least one video set. The user terminal 30 may accept input of one or more tags. In one example, the user terminal 30 may accept input of the tag "smile". The user terminal 30 may accept input of a combination of the tag "smile" and the tag "eating and drinking". The user terminal 30 may accept input of value ranges for one or more measurement data. In one example, the user terminal 30 may accept input of the measurement data "pulse rate is between 50 and 100". The user terminal 30 may accept input of a combination of the measurement data "pulse rate is 50 or more" and the measurement data "room temperature is 25°C or less". The user terminal 30 may accept input of a combination of tags and measurement data. In one example, the user terminal 30 may receive an input of a combination of the tag "eating" and the measurement data "room temperature is 25°C or lower." The combination of conditions for aggregation may be a combination that satisfies both conditions, or a combination that satisfies any one of the conditions.

In step S12, the user terminal 30 sends a tally request to the server 10. The tally request is a message that includes the input content received in step S11 and causes the server 10 to perform tallying. For example, the tally request includes a video set ID. The server 10 receives the tally request from the user terminal 30.

In step S13, the aggregation unit 13 of the server 10 aggregates the data in the database 20 based on the aggregation request. The aggregation unit 13 acquires data from the database 20 based on the aggregation conditions indicated in the aggregation request. For example, the aggregation unit 13 acquires from the database 20 the video ID and video data associated with the video set ID indicated in the aggregation request. The aggregation unit 13 acquires from the database 20 tag data associated with the acquired video ID. The aggregation unit 13 acquires measurement data associated with the video set ID from the database 20. Furthermore, for each video set, the aggregation unit 13 aggregates at least one of the number of times a scene corresponding to the tag appears and the number of times a scene corresponding to the measurement data appears. The aggregation unit 13 generates the aggregation results as table data.

In step S14, the screen management unit 11 of the server 10 outputs the tally results. For example, the screen management unit 11 generates a tally screen using the table data generated in step S13. The screen management unit 11 transmits the tally screen to the user terminal 30. The user terminal 30 receives the tally screen from the server 10.

In step S15, the user terminal 30 displays the tally results. For example, the user terminal 30 displays the tally window 300 (see example (a) in Figure 5), which is a tally screen.

In step S16, the user terminal 30 accepts a selection operation on a link in the summary window. For example, the user terminal 30 accepts a selection operation from the user on link 301 in the summary window 300. This allows the user terminal 30 to specify a video set and additional information for identifying a scene.

In step S17, the user terminal 30 transmits link information corresponding to the selected link to the server 10. For example, the user terminal 30 transmits link information related to link 301 to the server 10. This link information includes a video set ID corresponding to video set V and a tag ID corresponding to the tag "smile." As a result, the server 10 acquires a request specifying a video set containing at least one video and accompanying information for identifying the scene.

In step S18, the playlist generation unit 14 of the server 10 generates a playlist based on the link information. Here, an example of a playlist generated by the playlist generation unit 14 will be described with reference to Figure 7.

The playlist generation unit 14 retrieves video data associated with video set V from the database 20 based on the video set ID in the link information. Video set V includes at least one piece of video data. The playlist generation unit 14 also retrieves tag data associated with the tag "smile" from the database 20 based on the tag ID in the link information.

The playlist generation unit 14 extracts scenes corresponding to the incidental information from at least one video. More specifically, the playlist generation unit 14 extracts scenes corresponding to the start and end times indicated by the incidental information from at least one video. The incidental information may be a tag assigned to at least one video by the user. The playlist generation unit 14 may extract scenes corresponding to the start and end times indicated by the tag from at least one video. For example, the playlist generation unit 14 extracts scenes 401, 402, and 403 corresponding to the start and end times indicated by the tag "smile" from the video data included in video set V. In other words, the tag "smile" is assigned three times to the video data included in video set V.

In one example, video set V includes three pieces of video data: "Indoors," "Viewer," and "Content." In this case, the tag "Smile" can be assigned only to the video data for "Viewer." Scene extraction is performed on video set V. In other words, the playlist generation unit 14 extracts scenes corresponding to the start and end times indicated by the tag "Smile" from the video data for "Indoors," "Viewer," and "Content."

The playlist generation unit 14 uses the video set to generate a playlist for playing scenes. For example, the playlist generation unit 14 generates playlist P, which plays extracted scenes 401, 402, and 403 in succession in order. "Playing scenes successively" means playing multiple scenes as a single video without interruption. Playlist P represents a single edited video.

Returning to FIG. 6, in step S19, the playlist generation unit 14 of the server 10 outputs a playlist. For example, the playlist generation unit 14 transmits the generated playlist P to the user terminal 30. The user terminal 30 receives the playlist P from the server 10.

In step S20, the user terminal 30 plays the playlist. For example, the user terminal 30 plays the playlist P using the video window 200.

(Second operation example)
The second operation example will be described with reference to Fig. 6. The second operation example is an operation example when a user specifies multiple video sets and tags on the tally screen. The second operation example shares steps S11 to S15 with the first operation example.

In relation to step S16, the user terminal 30 may accept a selection operation from the user on the link 302 in the summary window 300.

In relation to step S17, the user terminal 30 may send link information regarding the link 302 to the server 10. This link information includes a video set ID corresponding to each of the video sets V, X, and Y, and a tag ID corresponding to the tag "smile." This allows the server 10 to receive a request specifying multiple video sets.

In relation to step S18, the playlist generation unit 14 may generate a playlist based on link information related to link 302. Here, an example of a playlist generated by the playlist generation unit 14 will be described with reference to Figure 8.

Based on the video set ID in the link information, video data associated with each of the video sets V, X, and Y may be obtained from the database 20. Each of the video sets V, X, and Y includes at least one piece of video data. The playlist generation unit 14 may also obtain tag data associated with the tag "smile" from the database 20 based on the tag ID in the link information.

The playlist generation unit 14 may extract scenes from at least one video for each of the multiple video sets. As described above, the playlist generation unit 14 may extract scenes 401, 402, and 403 corresponding to the start time and end time indicated by the tag "smile" from the video data included in video set V. Similarly, the playlist generation unit 14 may extract scene 411 from the video data included in video set X. The playlist generation unit 14 may extract scenes 421 and 422 from the video data included in video set Y. If video set X (or video set Y) includes multiple videos, the playlist generation unit 14 may extract scenes from each of the multiple videos corresponding to the start time and end time indicated by the tag "smile", as with video set V.

The playlist generation unit 14 may use multiple video sets to generate a playlist for playing multiple scenes corresponding to the multiple video sets. The playlist generation unit 14 may use video sets V, X, and Y to generate a playlist for playing extracted scenes 401, 402, 403, 411, 421, and 422.

For example, the playlist generation unit 14 may generate a playlist for continuously playing back multiple scenes. In one example, the playlist generation unit 14 may generate a playlist Qa that continuously plays back extracted scenes 401, 402, 403, 411, 421, and 422 in order. Playlist Qa represents one edited video.

Alternatively, the playlist generation unit 14 may generate a playlist for each of multiple video sets. In one example, the playlist generation unit 14 may generate a playlist Qb for each of video sets V, X, and Y. Playlist Qb has playlists corresponding to each of video sets V, X, and Y. The playlist corresponding to video set V is a playlist for playing extracted scenes 401, 402, and 403 in succession in order. The playlist corresponding to video set X is a playlist for playing extracted scene 411. The playlist corresponding to video set Y is a playlist for playing extracted scenes 421 and 422 in succession in order. Playlist Qb shows three edited videos.

In relation to step S19, the playlist generation unit 14 may transmit the generated playlist Qa or playlist Qb to the user terminal 30. The user terminal 30 may receive the playlist Qa or playlist Qb from the server 10.

In relation to step S20, the user terminal 30 may play playlist Qa or playlist Qb using the video window 200.

(Third operation example)
A third operation example will be described with reference to FIG. 6. The third operation example is an operation example when the user specifies multiple image sets and measurement value categories on the tally screen. Compared to the first operation example, the third operation example differs in that multiple image sets are specified and that measurement value categories are specified as additional information. The third operation example shares steps S11 to S14 with the first operation example.

In relation to step S15, the user terminal 30 may display the tally window 310 (see example (b) in Figure 5), which is a tally screen.

In relation to step S16, the user terminal 30 may accept a selection operation from the user on the link 311 in the summary window 310.

In relation to step S17, the user terminal 30 may transmit link information regarding the link 311 to the server 10. This link information includes the video set IDs corresponding to the video sets V and W, respectively, and the measurement value category "50-65" as additional information.

In relation to step S18, the playlist generation unit 14 may generate a playlist based on link information related to link 311. Here, with reference to Figure 9, an example of a playlist generated by the playlist generation unit 14 will be described.

Video data associated with each of the video sets V and W may be obtained from the database 20 based on the video set ID in the link information. Each of the video sets V and W includes at least one piece of video data. In the database 20, the video set and the measurement data are synchronized. Therefore, when a measurement value category is specified by the link information, the start time and end time of the scene are automatically determined. In one example, the time when the measurement data value begins to fall within the range of the measurement value category corresponds to the start time. The time when the measurement data value falls outside the range of the measurement value category corresponds to the end time.

The playlist generation unit 14 extracts scenes from at least one video that correspond to the start time and end time indicated by the accompanying information. The accompanying information may be a measurement value category that indicates a category of measurement data values obtained at the same time as the video set. The playlist generation unit 14 may extract scenes from at least one video that correspond to the start time and end time defined by the measurement value category. For example, the playlist generation unit 14 may extract scenes 501, 502, and 503 from the video data included in video set V that correspond to the start time and end time defined by the measurement value category. Similarly, the playlist generation unit 14 may extract scenes 511 and 512 from the video data included in video set W. If video set W includes multiple videos, the playlist generation unit 14 may extract scenes from each of the multiple videos that correspond to the start time and end time defined by the measurement value category, as with video set V.

The playlist generation unit 14 may use the video sets V and W to generate a playlist for playing the extracted scenes 501, 502, 503, 511, and 512.

In one example, the playlist generation unit 14 may generate a playlist Ra that plays the extracted scenes 501, 502, 503, 511, and 512 consecutively in order. The playlist Ra represents one edited video.

In one example, the playlist generation unit 14 may generate a playlist Rb for each of the video sets V and W. The playlist Rb has a playlist corresponding to each of the video sets V and W. The playlist corresponding to the video set V is a playlist for playing the extracted scenes 501, 502, and 503 in succession in order. The playlist corresponding to the video set W is a playlist for playing the extracted scenes 511 and 512. The playlist Rb shows two edited videos.

In relation to step S19, the playlist generation unit 14 may transmit the generated playlist Ra or playlist Rb to the user terminal 30. The user terminal 30 may receive the playlist Ra or playlist Rb from the server 10.

In relation to step S20, the user terminal 30 may play playlist Ra or playlist Rb using the video window 200.

[effect]
As described above, a video management system according to one aspect of the present disclosure includes at least one processor that receives a request specifying a video set including at least one video and incidental information for identifying scenes, extracts scenes corresponding to the incidental information from the at least one video, generates a playlist for playing the scenes, and outputs the playlist.

A video management method according to one aspect of the present disclosure is executed by a video management system including at least one processor. The video management method includes the steps of: receiving a request specifying a video set including at least one video and accompanying information for identifying scenes; extracting scenes corresponding to the accompanying information from the at least one video; generating a playlist for playing the scenes; and outputting the playlist.

A video management program according to one aspect of the present disclosure causes a computer to execute the steps of: acquiring a request specifying a video set including at least one video and accompanying information for identifying a scene; extracting scenes corresponding to the accompanying information from the at least one video; generating a playlist for playing the scenes; and outputting the playlist.

In this aspect, a scene corresponding to the specified supplementary information is extracted from at least one video in the specified video set. A playlist is then generated from the extracted scenes and output. The playlist corresponds to an edited video that brings together the specified scenes in the specified video set. This provides an environment for analyzing desired scenes.

In another aspect of the video management system, at least one processor may receive a request specifying multiple video sets, extract a scene from at least one video for each of the multiple video sets, and generate a playlist for playing the multiple scenes corresponding to the multiple video sets. With this configuration, a scene corresponding to the specified supplementary information is extracted from at least one video for each of the specified multiple video sets. A playlist is then generated from the extracted multiple scenes. This provides an environment for analyzing desired scenes in multiple video sets.

In another aspect of the video management system, at least one processor may generate a playlist for consecutively playing multiple scenes. This allows desired scenes from multiple video sets to be combined into a single playlist.

In another aspect of the video management system, at least one processor may generate a playlist for each of the multiple video sets. This makes it possible to provide a playlist that collects desired scenes from the multiple video sets for each video set.

In a video management system according to another aspect, at least one processor may extract, from at least one video, a scene corresponding to the start time and end time indicated by the supplementary information. With this configuration, scenes are extracted using the start time and end time indicated by the supplementary information, making scene extraction easier.

In another aspect of the video management system, the additional information may be a tag assigned by a user to at least one video. At least one processor may extract, from at least one video, a scene corresponding to the start time and end time indicated by the tag. With this configuration, the tag assigned by the user is specified to identify the scene. The tag defines the start time and end time of the scene, making it easier to extract the scene.

In another aspect of the video management system, the additional information may be a measurement value category indicating a category of measurement data values obtained at the same time as the video set. At least one processor may extract scenes from at least one video that correspond to start and end times determined by the measurement value category. With this configuration, a measurement value category based on the measurement data is specified to identify a scene. Because the video set and the measurement data are synchronized, the start and end times of a scene are automatically determined when the measurement value category is specified. This makes it easier to extract scenes.

[Modification]
The present disclosure has been described in detail above based on the embodiments. However, the present disclosure is not limited to the above embodiments. Various modifications of the present disclosure are possible without departing from the spirit and scope of the present disclosure.

In the embodiments, the incidental information is described as one tag or one measurement value category, but various other types of incidental information are possible. For example, the incidental information may be a combination of multiple tags. In one example, the incidental information may be a combination of the tag "smile" and the tag "eating and drinking." The incidental information may also be a combination of multiple measurement value categories. In one example, the incidental information may be a combination of the measurement value category "pulse rate 50-65" and the measurement value category "room temperature below 25°C." The incidental information may be a combination of at least one tag and at least one measurement data. In one example, the incidental information may be a combination of the tag "eating and drinking" and the measurement value category "room temperature below 25°C." Incidental information that includes two or more elements (tags or measurement value categories) such as these may be incidental information that satisfies all of the elements, or may be incidental information that satisfies any of the elements.

In the embodiment, it has been described that tags are assigned by the user, but tags may also be assigned automatically by analyzing video frames. In the embodiment, it has been described that the video set includes three pieces of video data: "interior," "viewer," and "content," but the video data is not limited to these. For example, the video set may include three pieces of video data: "interior of the vehicle," "view outside the vehicle," and "console operation." In this case, tags may be "yawning," "red light," "air conditioner operation," etc. Measurement data may be "brake pressure," etc.

In the embodiment, the playlist is generated using one or more video sets, but the playlist may also be list data that collects the start and end times of scenes. In this case, the user terminal 30 may obtain video data from the database 20 based on the video ID for each scene. The user terminal 30 may play scenes that correspond to the start and end times indicated in the list data. The user terminal 30 may repeat the process of obtaining video data and playing scenes for each scene.

In this disclosure, the expression "at least one processor executes a first process, executes a second process, ... executes an nth process," or an equivalent expression, refers to a concept that includes cases where the entity executing the n processes from the first process to the nth process (i.e., the processor) changes midway through. In other words, this expression refers to a concept that includes both cases where all n processes are executed by the same processor, and cases where the processor changes among the n processes according to an arbitrary policy.

The processing steps of the method executed by at least one processor are not limited to the examples in the above embodiments. For example, some of the steps described above may be omitted, or the steps may be executed in a different order. Furthermore, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the steps described above.

When comparing the magnitude of two numerical values, either of the two criteria "greater than or equal to" and "greater than" may be used, or either of the two criteria "less than or equal to" and "under." The choice of such criteria does not change the technical significance of the process of comparing the magnitude of two numerical values.

1...Video management system, 10...Server, 11...Screen management section, 12...Data management section, 13...Counting section, 14...Playlist generation section, 20...Database, 30...User terminal.

Claims (7)

at least one processor;
the at least one processor:
receiving a request specifying a video set including at least one video and a measurement value category for identifying a scene, the measurement value category being at least one of a measurement value related to an object and a measurement value related to an environment where the video set was taken and obtained at the same time as the video set;
extracting the scene corresponding to the start time and end time defined by the measurement value division from the at least one video;
generating a playlist for playing said scenes;
outputting the playlist;
Video management system. the at least one processor:
obtaining the request specifying a plurality of the video sets;
extracting the scene from the at least one video for each of the plurality of video sets;
generating the playlist for playing a plurality of the scenes corresponding to a plurality of the video sets;
The video management system according to claim 1 . The video management system of claim 2, wherein the at least one processor generates the playlist for consecutively playing back a plurality of the scenes. The video management system of claim 2, wherein the at least one processor generates the playlist for each of the plurality of video sets. the at least one processor:
Displaying a summary window showing the indexes for the measurement value category in a table format;
acquiring a request to specify the image set and the measurement value category by a selection operation from a user on the aggregation window;
The video management system according to any one of claims 1 to 4. 1. A video management method executed by a video management system having at least one processor, comprising:
receiving a request specifying a video set including at least one video and a measurement value category for identifying a scene, the measurement value category being at least one of a measurement value related to an object and a measurement value related to an environment where the video set was taken and obtained at the same time as the video set;
extracting the scene corresponding to the start time and end time defined by the measurement value division from the at least one video;
generating a playlist for playing the scenes;
and outputting the playlist. receiving a request specifying a video set including at least one video and a measurement value category for identifying a scene, the measurement value category being at least one of a measurement value related to an object and a measurement value related to an environment where the video set was taken and obtained at the same time as the video set;
extracting the scene corresponding to the start time and end time defined by the measurement value division from the at least one video;
generating a playlist for playing the scenes;
and outputting the playlist.