JP4358723B2 - Digest video creation device, digest video creation method, digest video creation program, and computer-readable recording medium recording the program - Google Patents

Description

  The present invention relates to a digest video creation method and apparatus for creating a digest video by selecting a specific scene from input video, and a digest video creation program and program for use in realizing the digest video creation method. Digest video creation method and apparatus, and digest video creation method capable of efficiently creating a digest video by efficiently selecting a scene in a video suitable as a digest scene The present invention relates to a digest video creation program used for realizing the above and a computer-readable recording medium on which the program is recorded.

  In the present invention, each scene section in an important video that is a point in understanding the outline of the video content is defined as a digest scene, and a digest video is a collection of a plurality of digest scenes as a series of videos. Defined and described below.

  Digest video viewing has the advantage of being able to easily understand the gist of the original video in a short time, and is a video viewing format that is often used in television broadcasts and video distribution services on the Internet.

  However, in the past, the work of creating a digest video often relied on humans and was a very expensive work. In particular, for the task of finding a scene suitable as a digest, it is necessary to select various criteria according to the purpose of the video content, and specialized skills are also required.

  For example, in the case of sports videos such as baseball and soccer, important comments in the case of scenes related to scoring such as home run scenes and scenes where goals were determined, and in the case of talk program videos such as debates It is necessary to have a skill to efficiently search for scenes that are being discussed.

  Furthermore, after searching for a suitable scene as a digest scene, it is necessary to fine-tune the sections so that the searched scene sections can be accommodated within a short time predetermined by editing. In order to provide a service that distributes digest video, the content and length of the digest video must be created in a short time with quality appropriate for a broadcast service. The person's mental fatigue and burden were great.

  In order to deal with such a digest video creation cost problem, a technique for automatically creating a digest video has been proposed in the following Non-Patent Document 1 and the like.

  Non-Patent Document 1 proposes an automatic digest video creation method that focuses on sound information in a video. This is a method based on the hypothesis that a scene suitable for a digest scene has characteristics different from those of other sounds, such as an extremely high sound level or a low sound level.

Non-Patent Document 1 shows an experimental result on this hypothesis, and it has been confirmed that the technique shows very high performance particularly for sports images.
Kota Hidaka, Osamu Mizuno, Nobuya Nakajo, "Multimedia Content Summarization Technology Focusing on Speech Enhancement," IEICE Information Technology Forum FIT2002

  It is true that the digest video automatic creation method proposed in Non-Patent Document 1 is an effective technique that exhibits very high performance.

  However, the digest video creation method based on the proposed technology is semantically important, which is required when a large number of people pay a fee for viewing, as in the case of creating a digest video for a broadcast service or the like. The problem of working cost is not fully solved when creating a digest video that includes a certain scene and the contents of the original video can be easily understood. Remains.

  The proposed technology of Non-Patent Document 1 is very effective in reducing the time required to create a digest video because it can automatically create a digest video, but the scene content of the digest video to be created Since only the video features such as sound are used, the meaning of the video content is not considered.

  For this reason, the digest video automatically created by the proposed technique of Non-Patent Document 1 may not include all the digest scenes that match the intention of the video producer.

  For example, in the video of a baseball game, a home run scene is a cheering sound in a stadium or a television program, so the live sound is at a higher level than other scenes. Even the method of creating a digest scene is selected without omission. However, when a scene that does not have any particular sound level such as a pitcher or batter facial expression before pitching when full is desired to be included as a digest, such a scene is not automatically selected.

  That is, in the proposed technique of Non-Patent Document 1, as a result, apart from automatically created digest scenes, there is a manual search for other scenes suitable as digest scenes. It is necessary to adjust the scene section of the selected digest scene and the automatically selected digest scene to produce the video that fits within the determined time, and the problem that the work cost still remains remains . In particular, when a digest video must be prepared in a short time, such as in a live program or a sports program, the problem cannot be solved sufficiently only by the proposed technique of Non-Patent Document 1.

  The present invention has been made in consideration of the above points, and when creating a digest video by selecting a specific scene from the input video, the content of the scene is semantically suitable as a digest scene. It is an object of the present invention to provide a new digest video creation technique that enables the creation of a video that includes the above-mentioned scene and that is stored within a predetermined time as a digest video more efficiently than the prior art.

In order to achieve this object, the digest video creation apparatus of the present invention provides (1) presenting the input video to the creator of the digest video to perform the process of creating the digest video of the input video. by selecting the scene section to be selected in response to the presentation, a selector for selecting any one or more of scene segments in a video image have entered as a digest scene candidate likeness digest from the video input (2) (3) Compare the time length of the digest video to be created with the total time length of the digest scene candidates selected by the selection unit, and if the two do not match If the two match, the digest feature acquired by the acquisition unit is used to create a new scene segment from the input video. Select a part to add, or select a scene section to delete from the digest scene candidates and delete it to define the digest scene. On the other hand, if the two match, select in the selection part A definition unit that directly defines the digest scene candidate as a digest scene, and (4) an output unit that outputs the digest scene defined by the definition unit as a digest video of the input video.

  Here, each of the processing means described above can be realized by a computer program, and this computer program is provided by being recorded on an appropriate computer-readable recording medium or provided via a network to implement the present invention. In this case, the present invention is realized by being installed and operating on a control means such as a CPU.

In the digest video creation device of the present invention configured as described above, when a video is input , the input video is presented to the creator of the digest video, and a scene section selected in response to the presentation is selected. in, select any one or more of scene segments in a video image have entered as a digest scene candidate, further using a predetermined method, automatically the digest feature amount indicating a digest likeness from the input image get.

  Subsequently, the time length of the digest video to be created is compared with the total time length of the selected digest scene candidates.

  If it is determined from this comparison that the total time length of the selected digest scene candidates is shorter than the time length of the digest video to be created, the selection is performed using a predetermined method using the acquired digest feature amount. Select a new scene section from the input video so that the total length of time combined with the digest scene candidate is the same as the time length of the digest video to be created, and the selected new scene section And the selected digest scene candidate are defined as a digest scene.

  On the other hand, when it is determined from this comparison that the total time length of the selected digest scene candidates is longer than the time length of the digest video to be created, a predetermined method using the acquired digest feature amount is used. Select the scene section to be deleted from the selected digest scene candidates so that the total length of time subtracted from the selected digest scene candidate is the same as the time length of the digest video to be created, and the remaining The performed scene section is defined as a digest scene.

  On the other hand, if it is determined from this comparison that the total time length of the selected digest scene candidate is the same as the time length of the digest video to be created, the selected digest scene candidate is defined as a digest scene as it is.

  When the digest scene is defined in this way, the defined digest scene is output as a digest video of the input video.

In this way, according to the present invention, the user is the creator of the digest video, from the digest creation target image, it is possible to select the scene to be included as a digest video.

Thus, the user, by the hand work is semantically meaningful, and it is possible to reliably select the scene to be included in the digest video.

  Further, according to the present invention, a time length value of a digest video to be created is set in advance, and the value and a total time length of a semantically important scene section suitable as a digest video selected by an arbitrary method are obtained. Depending on the comparison result, the scene length can be exactly the same as the preset time length by adding more scene sections or conversely deleting scene sections partially from the selected scene sections. The digest scene section can be defined to be the same. Furthermore, this process can be automatically executed instantaneously by a predetermined method using digest feature values.

  Therefore, according to the present invention, when creating a digest video by selecting a specific scene from the video, it includes a scene section that is semantically important and can be determined as a digest video. Thus, it is possible to efficiently create a video that falls within a long time.

  Therefore, according to the present invention, it is possible to easily create a digest video having a quality required for a broadcast service or the like without incurring a work cost as compared with the prior art. In particular, when it is necessary to create a digest video with sufficient service quality in a short time, such as when a digest video must be created in the middle of a live program or a sports program, the burden on the video producer is reduced. It can be suppressed and brings about a great effect in terms of reducing the work cost.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 illustrates an example of a device configuration of a digest video creation device 1 configured according to the present invention.

  As shown in this figure, a digest video creation device 1 of the present invention includes a video input unit 10, a video storage unit 11, a digest scene candidate selection unit 12, a digest scene candidate storage unit 13, and a digest feature quantity acquisition unit. 14, a digest feature amount storage unit 15, a digest scene section definition unit 16, a digest video storage unit 17, and a digest video output unit 18.

  The video input unit 10 shown in FIG. 1 performs processing for inputting video and storing it in the video storage unit 11.

  Specifically, it can be realized by a video capture board for a computer, a memory on the computer, and a hard disk. For example, it can be realized by capturing a VHS format video signal while digitizing it with a video capture board and writing the captured digital data into a memory or hard disk.

  The digest scene candidate selection unit 12 shown in FIG. 1 selects any one or a plurality of scene sections in the video input by the video input unit 10 as digest scene candidates using a predetermined method, The information is stored in the digest scene candidate storage unit 13 having a data structure as shown in FIG.

  Although omitted in FIG. 2, the digest scene candidate storage unit 13 calculates and stores the total time length t of the scene sections selected by the digest scene candidate selection unit 12 by the digest scene candidate selection unit 12. There may be.

  FIG. 3 shows an example of a specific processing image of the digest scene candidate selection unit 12. Here, FIG. 3 shows a schematic diagram of an example of a scene time transition in a baseball program.

  In this processing image, it is shown that “the scene of the winning home run” and “the scene of the reverse runner touch-out” are selected as the digest scene candidates 1 and 2, respectively. At this time, a digest scene candidate can be selected by recording the start time and end time in the program of both scenes.

As a predetermined way for selecting a digest scene candidate reference video scene content in computers with video editing functions such as non-linear digital video editing machine, start the appropriate scene section digest manually Recording the point and end point is a possible means.

  The digest feature value acquisition unit 14 shown in FIG. 1 automatically acquires a digest feature value indicating the digest likeness from the video input by the video input unit 10 using a predetermined method, and performs the digest feature value. The process of storing in the storage unit 15 is performed.

  The processing function of the digest feature quantity acquisition unit 14 can be realized by mounting a predetermined logic using computer hardware and software developed on a memory. For example, it is realizable with the software on the computer which mounted the proposal technique of nonpatent literature 1.

  FIG. 4 shows a schematic diagram of the digest feature value acquired by the digest feature value acquisition unit 14.

  The digest feature value is a value or vector associated with time information in the video, and FIG. 4 is a graph showing the individual time in the video and the digest feature value corresponding to the time. is there. According to the proposed technique of Non-Patent Document 1, it is possible to select a scene portion having a high sound level as a digest and to calculate a digest feature amount as a portion having a high value as a digest feature amount. FIG. 4 shows that scenes at times T1 and T2 in the video have been determined as appropriate scenes because they have high digest feature values.

  The digest scene section definition unit 16 shown in FIG. 1 includes the video input by the video input section 10, the total time length t of the scene sections selected by the digest scene candidate selection section 12, and the start time / end of each scene section. Using the time and the digest feature quantity acquired by the digest feature quantity acquisition unit 14 to define a digest scene included in the digest video that is finally output by the digest video creation device 1 according to the present invention It is.

  5 and 6 show an example of a processing flow executed by the digest scene section definition unit 16.

  When there is a digest video creation request, the digest scene section definition unit 16 first inputs from the video storage unit 11 to the video input unit 10 in step 10 as shown in the processing flow of FIGS. Loaded video.

  Subsequently, at step 11, the digest video time length T to be finally output by the digest video creation device 1 of the present invention is input using a predetermined method. This can be realized, for example, by using software on a computer having a graphical user interface and manually input by a software user.

  Subsequently, in step 12, the total time length t of the scene section selected by the digest scene candidate selection section 12 and the start time / end time of each scene section are read from the digest scene candidate storage section 13. This can be realized by implementing logic as software on the computer memory for reading the total time length t of the scene sections selected by the digest scene candidate selection unit 12 and the start time / end time of each scene section. .

  If the total time length t is not stored in the digest scene candidate storage unit 13, the time length of each scene section is obtained according to the start time / end time of each scene section read from the digest scene candidate storage unit 13. By summing them up, the total time length t of the scene sections selected by the digest scene candidate selection unit 12 is calculated.

  Subsequently, in step 13, the digest feature value acquired by the digest feature value acquisition unit 14 is read from the digest feature value storage unit 15. This can be realized by mounting a process of reading the digest feature value acquired by the digest feature value acquisition unit 14 as software on a computer.

  Subsequently, in step 14, it is determined whether or not the time length T of the digest video input in step 11 is larger than the total time length t of the scene section read in step 12.

  When it is determined that the value of the time length T is not greater than the value of the total time length t according to this determination processing, the process proceeds to step 15 to determine whether the value of the total time length t is greater than the value of the time length T. Judging.

  When it is determined that the value of the total time length t is larger than the value of the time length T according to the determination processing in steps 14 and 15, the process proceeds to step 16 and the value of the time length T is larger than the value of the total time length t. When it is determined, the process proceeds to step 18, and when it is determined that the value of the time length T is equal to the total time length t, the process proceeds to step 20.

  Step 18 is a process to be performed when T> t, and an additional digest scene is selected from the input video using a predetermined method using the digest feature value, and this is used as a digest scene candidate. Add to define the final digest scene. In the subsequent step 19, the digest video constituted by the defined digest scene is stored in the digest video storage unit 17, and the digest scene section definition unit 16 ends the processing.

  FIG. 7 illustrates an example of the processing in step 18.

  In FIG. 7, a scene of 1 minute 30 seconds is selected as a digest scene candidate from the baseball video of FIG. 3, whereas the final digest video is produced with a length of 2 minutes. It is assumed.

  In this case, in order to adjust the time length of the final digest video to 2 minutes, for example, an existing digest scene candidate (a winning home run scene, a reverse runner touch-out scene) is selected from the time distribution of digest feature values. The range excluding the scene section is selected, and a scene section with a high digest feature value for 30 seconds is selected as an additional scene. Finally, a 2-minute video obtained by adding the scene segment 1 minute 30 seconds and the additional scene segment 30 seconds of the digest scene candidate is defined as a digest video.

  On the other hand, step 16 is a process to be performed when T <t, and a scene to be partially deleted from the scene section in the digest scene candidate using a predetermined method using the digest feature quantity. A section is selected, and a scene section in the digest scene candidate remaining after the deletion is defined as a final digest scene. In the subsequent step 17, the digest video composed of the defined digest scene is stored in the digest video storage unit 17, and the digest scene section definition unit 16 ends the processing.

  FIG. 8 illustrates an example of the processing in step 16.

  In FIG. 8, a scene for 1 minute 30 seconds is selected as a digest scene candidate from the baseball video in FIG. 3, whereas a final digest video having a length of 1 minute is produced. It is assumed.

  In this case, in order to adjust the time length of the final digest video to 1 minute, for example, a scene of an existing digest scene candidate (a winning home run scene, a reverse runner touch-out scene) in the time distribution of the digest feature value From the section range, as the scene section having a low digest feature value, 20 seconds from the winning home run scene section and 10 seconds from the reverse runner touch-out scene section are deleted. Finally, a scene section obtained by deleting a total of 30 seconds of scene sections having a low digest feature value from the scene section 1 minute 30 seconds of the digest scene candidate is defined as a digest video.

  On the other hand, step 20 is a process performed when T and t have the same value, and the final digest output by the digest video creation device 1 of the present invention as it is is output from the scene segment selected by the digest scene candidate selection unit 12. Define as a scene. Then, in the following step 21, the digest video composed of the defined digest scene is stored in the digest video storage unit 17, and the digest scene section definition unit 16 ends the processing.

  Here, the processing of step 14 to step 21 can be realized by mounting the logic of each step as software on a computer.

  In this way, upon receiving the digest video stored in the digest video storage unit 17, the digest video output unit 18 shown in FIG. 1 receives the digest video from the digest video storage unit 17 when requested to output the digest video. Processing is performed to read and output.

  In this manner, the digest scene section definition unit 16 compares the time length T of the digest video to be created with the total time length t of the scene sections selected by the digest scene candidate selection section 12 in accordance with the result of step 16. , 18, and 20, and according to the processing executed by the digest scene section definition unit 16, a scene section that is semantically important and suitable as a digest scene is preferentially included, and a desired video length is provided as a digest video. It is possible to efficiently create a digest video in a short time, thereby solving the problems of the prior art.

It is an apparatus block diagram of the digest image | video production apparatus of this invention. It is explanatory drawing of the data structure of a digest scene candidate storage part. It is explanatory drawing of the process which a digest scene candidate selection part performs. It is explanatory drawing of a digest feature-value. It is the processing flow which a digest scene area definition part performs. It is the processing flow which a digest scene area definition part performs. It is explanatory drawing of the process which a digest scene area definition part performs. It is explanatory drawing of the process which a digest scene area definition part performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digest image | video production apparatus 10 Image | video input part 11 Image | video storage part 12 Digest scene candidate selection part 13 Digest scene candidate storage part 14 Digest feature-value acquisition part 15 Digest feature-value storage part 16 Digest scene area definition part 17 Digest video storage part 18 Digest Video output section

Claims (4)

A digest video creation device for creating a digest video of an input video,
Was input image presented to the author of the digest video and selecting the scene section to be selected in response to the presentation, any one or more of scene segments in the video as a digest scene candidates enter A selection section to select;
An acquisition unit that acquires a digest feature amount indicating a digest likeness from the input video;
When the time length of the digest video to be created is compared with the total time length of the digest scene candidates selected by the selection unit, if the two do not match, the two match, Using the digest feature acquired by the acquisition unit, select and add a new scene section from the input video, or select and delete the scene section to be deleted from the digest scene candidates In the meantime, if the two match, a definition unit that defines the digest scene candidate selected by the selection unit as a digest scene as it is,
An output unit that outputs the digest scene defined by the definition unit as a digest video of the input video;
A digest video creation device. A digest video creation method for creating a digest video of an input video,
Was input image presented to the author of the digest video and selecting the scene section to be selected in response to the presentation, any one or more of scene segments in the video as a digest scene candidates enter A selection step to select;
An acquisition step of acquiring a digest feature amount indicating a digest likeness from the input video;
When the time length of the digest video to be created is compared with the total time length of the digest scene candidates selected in the selection step, and if the two do not match, the two match, Using the digest feature acquired in the acquisition step, select and add a new scene section from the input video, or select and delete the scene section to be deleted from the digest scene candidates A digest scene, and when the two match, a definition step of defining the digest scene candidate selected in the selection step as a digest scene as it is,
An output step of outputting the digest scene defined in the definition step as a digest video of the input video,
A featured digest video creation method. A digest video creation program for creating a digest video of an input video,
Was input image presented to the author of the digest video and selecting the scene section to be selected in response to the presentation, any one or more of scene segments in the video as a digest scene candidates enter A selection process to select,
An acquisition process for acquiring a digest feature amount indicating a digest likeness from the input video;
When the time length of the digest video to be created is compared with the total time length of the digest scene candidates selected in the selection process, and if the two do not match, the two match, Using the digest feature acquired in the acquisition process, select and add a new scene section from the input video, or select and delete the scene section to be deleted from the digest scene candidates In the meantime, if the two match, the definition process for defining the digest scene candidate selected in the selection process as a digest scene as it is,
A digest video creation program for causing a computer to execute an output process for outputting a digest scene defined in the definition process as a digest video of an input video. A computer-readable recording medium recording a digest video creation program for creating a digest video of an input video,
Was input image presented to the author of the digest video and selecting the scene section to be selected in response to the presentation, any one or more of scene segments in the video as a digest scene candidates enter A selection process to select,
An acquisition process for acquiring a digest feature amount indicating a digest likeness from the input video;
When the time length of the digest video to be created is compared with the total time length of the digest scene candidates selected in the selection process, and if the two do not match, the two match, Using the digest feature acquired in the acquisition process, select and add a new scene section from the input video, or select and delete the scene section to be deleted from the digest scene candidates In the meantime, when the two match, the definition process of defining the digest scene candidate selected in the selection process as a digest scene as it is,
A computer-readable recording medium on which a digest video creation program for causing a computer to execute an output process for outputting a digest scene defined in the definition process as a digest video of an input video is recorded.

Images