JP4033142B2 - Video / scenario matching method, apparatus and program - Google Patents

Description

  The present invention relates to a video / scenario matching method, apparatus, and program, and more particularly, to a video / scenario matching method, apparatus, and program for associating scenario text with each partial section (scene) of a video.

  As a first method for associating a conventional video with a scenario text, there is a method of detecting a shot change of a video and inputting a comment by cutting and pasting a comment for each shot.

In addition, as a second method, there is a method of correcting an error by specifying IN / OUT points for all sections when there is an error in the process of associating the video with the scenario text (for example, Patent Documents). 1 and 2).
JP 2003-224774 “Semi-automatic subtitle program production system” Japanese Patent Laid-Open No. 2003-224773 “Subtitle Editing Support System by Moving Boundary of Subtitles Arranged on Timeline”

  However, the first conventional method has a problem that it takes time and effort to cut and paste the text corresponding to the shot from the scenario text. Also, since the beginning of the shot does not match the beginning of the narration, it takes time until the narration is heard from the beginning of the shot. In addition, there is a problem that it is not possible to deal with images in which shot switching does not exist.

  In addition, the conventional second method has a problem that it is necessary to specify IN / OUT points for all sections in which the association is wrong, which takes time.

  The present invention has been made in view of the above points, and an object thereof is to provide a video / scenario matching method, apparatus, and program for improving the efficiency of the work of associating a partial section (scene) of a video with a scenario text. To do.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention relates to a video / scenario matching method for associating a scenario text with each partial section (scene) of a video,
The video data read from the video storage means is associated with the scenario text read from the scenario storage means (step 1).
Using the user interface with the operator,
Displaying a correspondence result between the read video data partial section and the read scenario text to prompt the operator to make a decision (step 2);
If there is an input from the operator that the correspondence is incorrect (step 3),
Whether correction is requested of the operator (step 4), a lock flag of a record having a correspondence relation between the partial section of the video data in the lock information storage means and the scenario text is turned on (step 5), and the correspondence is updated. Inquire the operator about whether (Step 6),
If there is an input from the operator to update, the association processing (step 1) is repeated, and if not, the process ends.

The present invention is a data input process of inputting video data from the video storage means and inputting scenario text delimited by a certain word length from the scenario storage means;
A scenario text analysis process for generating a speaker pattern of a scenario, which is a symbol string including a speaker name for each unit in which the scenario text is divided, and storing it in the speaker pattern storage means;
A video analysis process of dividing video data into unit sections, generating an utterance pattern that is a symbol string including a speaker name, and storing it in the utterance pattern storage means;
Matching the symbol string stored in the speaker pattern storage means with the symbol string stored in the utterance pattern storage means, generating matching information in which the line change and the time are associated, and storing the matching information in the matching information storage means The matching process;
The matching information is read from the matching information storage means, the video is acquired from the video storage means corresponding to the matching information, the scenario text is acquired from the scenario storage means, and the associated result is displayed on the display means. A matching result display process for displaying the video partial section designated by the operator;
When it is determined by the operator that the matching result is correct and the lock of the correspondence relationship is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means is turned ON,
When the operator instructs the correction of the association result, the operator inputs a keyword for the scenario to be associated, searches the scenario storage means for the scenario text corresponding to the keyword, and displays it on the display device. When the operator designates the lock of the correspondence relationship, the display indicating the lock state on the screen on the display device is performed, the lock flag of the lock storage unit is turned on, and the matching information is updated. Realignment process;
When matching of all the video data and the scenario text is completed, the process includes a matching result output process for outputting matching information.

Further, according to the present invention, in the association process,
By referring to the lock information file, the scenario text is divided at the portion corresponding to the dialogue ID corresponding to the dialogue ID whose lock flag is ON, and the video data corresponding to the time when the lock flag is ON is analyzed. Then, they are individually associated and integrated into the matching information storage means.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention is a video / scenario matching device for associating a scenario text with each partial section (scene) of a video,
Video storage means 110 for storing video data;
Scenario storage means 120 for storing a scenario text consisting of a dialogue ID, a speaker ID, a speaker name, and dialogue;
A scenario text analysis unit 140 that reads scenario text from the scenario storage unit, generates a speaker pattern of the scenario, and stores it in the speaker pattern storage unit 12;
A video analysis unit 130 that reads video data from the video storage unit 110, generates a speech pattern that is a symbol string including a speaker name by dividing the video data into unit sections, and stores the speech pattern in the speech pattern storage unit 11;
Matching the symbol string stored in the speaker pattern storage unit 12 with the symbol string stored in the utterance pattern storage unit 11 to generate matching information in which the change of the dialogue and the time are associated with each other, the matching information storage unit 13 The association means 150 stored in
The matching information is read from the matching information storage unit 13, the video corresponding to the matching information is acquired from the video storage unit 110, and the scenario text corresponding to the matching information is acquired from the scenario storage unit 120. A matching result display means 161 for displaying the received result on the display means and displaying the video partial section designated by the operator;
If the operator determines that the association result is correct and the correspondence lock is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means 14 is turned ON. In addition, when the operator gives an instruction to correct the association result, the operator inputs the keyword of the scenario to be associated, and the scenario text corresponding to the keyword is retrieved from the scenario storage unit 120 and displayed. When it is displayed on the device and the corresponding lock is designated by the operator, the lock state on the screen on the display device is displayed and the lock flag of the lock storage means 14 is turned ON. And re-matching means 162 for instructing to update the matching information;
When matching of all video data and scenario text is completed, a matching result output unit 163 that outputs matching information is provided.

In addition, the association means 150 described above
A part corresponding to the time when the lock flag is turned on by referring to the lock information file and dividing the speaker pattern at the part corresponding to the dialogue ID for which the lock flag is turned on. Includes a unit that divides the speaker pattern, associates them individually, and integrates them into the matching information storage unit.

The present invention is a video / scenario matching program for associating a scenario text with each partial section (scene) of a video,
On the computer,
A data input step of inputting video data from the video storage means and inputting scenario text delimited by a certain word length from the scenario storage means;
A scenario text analysis step of generating a speaker pattern of a scenario that is a symbol string including a speaker name for each unit of scenario text and storing it in the speaker pattern storage means;
A video analysis step of dividing the video data into unit intervals, generating an utterance pattern that is a symbol string including a speaker name, and storing the utterance pattern in the utterance pattern storage means;
Matching the symbol string stored in the speaker pattern storage means with the symbol string stored in the utterance pattern storage means, generating matching information in which the line change and the time are associated, and storing the matching information in the matching information storage means A mapping step;
The matching information is read from the matching information storage means, the video is acquired from the video storage means corresponding to the matching information, the scenario text is acquired from the scenario storage means, and the associated result is displayed on the display means. An association result display step for displaying the video partial section designated by the operator;
When it is determined by the operator that the matching result is correct and the lock of the correspondence relationship is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means is turned ON,
When the operator instructs the correction of the association result, the operator inputs a keyword for the scenario to be associated, searches the scenario storage means for the scenario text corresponding to the keyword, and displays it on the display device. When the operator designates the lock of the correspondence relationship, the display indicating the lock state on the screen on the display device is performed, the lock flag of the lock storage unit is turned on, and the matching information is updated. A realignment step;
When matching of all the video data and the scenario text is completed, a matching result output step for outputting matching information is executed.

In the above association step,
When performing the association, the lock information file is referred to, the speaker pattern is divided at the part corresponding to the dialogue ID for which the lock flag is ON, and it corresponds to the time when the lock flag is ON The speaker pattern is divided into parts, individually associated with each other, and integrated with the matching information storage unit.

  In the present invention, when associating video data with a scenario, the correspondence point instructed by the operator is set by turning on the lock flag for the association result determined to be correct by the operator. A better association can be generated through

  Also, when searching for dialogues corresponding to video by text search, the search range is the lock information immediately before (the closest one before the video) to the next lock information (the one closest to the time after the video) ). For this reason, the number of text search results is reduced, and the effort for confirming the dialogue is reduced as compared with the case where there is no lock flag.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, terms used in the following description will be described.
・ Video: Video file such as mpeg;
・ Scenario text: A text file in which the speaker name and dialogue are written in a specific format;
Scenario matching: An operation that gives a time code so that a specific line matches the content of the video and audio (performed by repeating the association operation and the correction operation);
Correlation operation: An operation for automatically associating a scenario and video / audio, and a separate program (correlation program)
-Correcting operation: An operation for manually correcting the alignment state obtained as a result of the associating operation through the GUI;
Cue (CUE): one partial video associated with a specific scenario text;
Next, the apparatus configuration of the present invention will be described.

  FIG. 3 shows the configuration of the video scenario matching apparatus in one embodiment of the present invention.

  The video scenario matching apparatus shown in the figure includes a video storage unit 110, a scenario storage unit 120, a video analysis unit 130, a scenario text analysis unit 140, an association unit 150, a control unit 160, a display unit 170, an input unit 180, and an utterance pattern. The file 11, the speaker pattern file 12, the matching information file 13, and the lock information file 14 are configured.

  The video storage unit 110 stores data of the input video file.

  The scenario storage unit 120 stores the input scenario text. An example of scenario text is shown in FIG. As shown in the figure, the scenario text is composed of a dialogue ID, a speaker ID, a speaker name, and a dialogue.

  The video analysis unit 130 reads the video data from the video storage unit 110, divides it into unit section lengths, generates an utterance pattern that is a list in which speaker names in a certain length of time section are arranged, and the utterance pattern file 11 To store. FIG. 5 shows an example of an utterance pattern. As shown in the figure, the utterance pattern 11 includes a unit section ID, a unit section start point, a unit section end point, and a speaker ID. It is assumed that the utterance pattern 11 is stored in the storage means in a file format.

  As shown in FIG. 6, the scenario text analysis unit 140 reads the scenario text from the scenario storage unit 120 and assigns a dialogue ID, which is a list of speaker names when dialogue is divided by a certain word length, to a dialogue ID. Is generated and stored in the speaker pattern file 12.

  The associating unit 150 associates the symbol string of the utterance pattern 11 and the symbol string of the speaker pattern 12 by a technique such as DP matching, and associates the line transition and time (sec) as shown in FIG. The matching information is generated and stored in the matching information file 13.

  As shown in FIG. 8, the control unit 160 acquires the scenario text of the scenario storage unit 120 and the still image of the video storage unit 110 from the dialogue ID and the playback time of the matching information in the matching information file 13 and stores them in the display device 170. On the other hand, control is performed so that the alignment result display windows are displayed side by side. In the matching control window, the scenario text and still image of the selected CUE number (initial value is the first) are displayed, and when the playback button is clicked from the input device 180, the video is displayed from the designated position. Reproduce. In addition, when the control unit 160 determines that the correspondence between the partial segment of the video reproduced from the input device 180 and the scenario segment is correct, the control unit 160 locks the lock information file 14 so that the correspondence between the dialogue and the time is fixed. When the information (the lock flag is set to ON) is added and the associating unit 150 associates the utterance pattern with the symbol string of the speaker pattern, the speaker pattern and the utterance pattern are associated with each other by DP matching. As for the dialogue ID, the reproduction time (unit section Start) is obtained from the utterance pattern, and the dialogue is created by arranging the dialogue ID and the reproduction time. At this time, since there may be a plurality of lines having the same dialogue ID, only the one with the earliest time is written to the matching information file 13.

  As shown in FIG. 9, the lock information file 14 includes a lock ID, a dialogue ID, a playback time of a locked video, and a lock flag. When a lock is instructed by the control unit 160, the lock information file 14 includes a record including a lock ID given in the locked order, a matched dialogue ID acquired from the matching information file 13, and a reproduction time of the locked video. Is added.

  The display device 170 displays a matching result display window and a matching control window in accordance with instructions from the control unit 160.

  Here, the matching result display window and the matching control window will be described.

  The matching result window is a window as shown on the right side of FIG. 9, and includes a CUE number, a lock icon, a CUE head cut image and time, a playback position cursor, a modified CUE head cut image and time, a dialogue sentence display section, and the like.

  The matching control window is a window as shown on the left side of FIG. 10, and includes a menu bar (file, tool, help), video display unit, video seek control, dialogue display unit, dialogue seek control, audio waveform display unit, and frame video. It consists of a display unit and a correction control button for the associated position. As shown in FIG. 11, the associated position correction control buttons include a link button, a lock button, and the like. The link button is a button for switching between video-side display and scenario text display synchronization / asynchronization. The lock button locks a matched pair of video and audio and scenario text. When matching is performed recursively, the locked pair is immobile. By using these buttons, the link is released for modification and the correct matching pair is selected and locked. Further, when the alignment is already performed in the initial state, processing such as locking before realignment is possible.

  Next, the operation in the above configuration will be described with reference to FIG.

  Step 101) The video analysis unit 130 inputs video from the video storage unit 110, the scenario text analysis unit 140 receives the scenario from the scenario storage unit 120, and the scenario text is a scenario consisting of a dialogue ID, a speaker ID, a speaker name, and a dialogue. Enter text.

  Step 102) In the scenario text analysis unit 140, the dialogue of the scenario text is divided by a certain word length, serial numbers are assigned, speaker names are arranged in the order of the serial numbers, and the speaker pattern file 12 is generated.

  Step 103) Further, the video analysis unit 130 generates an utterance pattern file 13 including a unit section ID, a unit section start point, a unit section end point, and a speaker ID for each time section of a certain length from the video data. .

  Step 104) The controller 160 initializes all lock flags of the lock information file 14 to OFF.

  Step 105) The matching unit 150 obtains the reproduction time of the video associated with the dialogue ID by performing DP matching on the symbol strings for each pattern of the utterance pattern file 11 and the speaker pattern file 12, and associates them. To generate the matching information file 13.

  Step 106) The control unit 160 reads the matching information file 13, the video data of the video storage unit 110, and the scenario text of the scenario storage unit 120, and displays a matching result display window and a matching control window on the display device 170 as matching results. To display.

  Step 107) When the operator operates the text seek operation button in the alignment control window (previous CUE / next CUE) to instruct the reproduction of the video partial section and / or scenario text, the control unit 160 responds to the instruction. Based on this, the partial video and / or scenario text of the designated CUE is displayed in the matching control window. At this time, the line of the CUE number is read from the matching information file 13 and the corresponding video scenario is displayed. The CUE number is matched with the line number of the matching information file when the matching information file 13 is read.

  Step 108) The operator looks at the matching control window displayed on the display device 170 to determine whether or not the correspondence between the partial section of the video being played back and the scenario segment is correct. ) Go to step 109, and if not correct (if not consistent), go to step 110.

  Step 109) If the partial section displayed in the alignment control window matches the scenario segment, the operator locks the lock button from the input device 180. As a result, the control unit 160 turns on the lock flag of the lock information file 14 and proceeds to step 107. At this time, the lock icon corresponding to each CUE number in the matching result display window is displayed in a locked manner. At this time, the control unit 160 notifies the display unit 170 of the current CUE number and lock-displays the lock icon.

  Step 110) The operator designates a search keyword from the input device 180 in the text box of the matching control window.

  Step 111) The control unit 160 searches the scenario storage unit 120 based on the keyword, and when the keyword exists, highlights the keyword in the scenario text column in the matching result display window.

  Step 112) When the operator corrects the association and turns on the lock flag, the control unit 160 obtains the associated dialogue ID and the playback time of the locked video from the matching information file 14, and sets the lock flag. Set to ON, give a lock ID, and additionally register in the lock file 14.

  Step 113) When the operator gives an instruction to associate the video and the scenario again, the process proceeds to Step 106, and the processes after Step 106 are performed again.

  As a method for creating an utterance pattern from video audio in step 103 described above, for example, described in “Automatic Speech and Speaker Recognition-Advanced Topics,” Kluwer Academic Publishers Group, 1996, ISBN 0-7923-9706-1 This can be realized by using the method described above.

  The processing in step 105 will be described in detail.

  FIG. 13 is a flowchart of processing for associating a speaker pattern with an utterance pattern according to the present invention.

  First, the associating unit 150 clears the matching information file 13 (step 1501), and then divides the speaker pattern and the utterance pattern into the same number based on the lock information file 14 (steps 1052 and 1053). At this time, the division number is the number of lines in the lock information file + 1. Next, the divided speaker pattern and the utterance pattern are individually DP-matched, the dialogue ID is obtained from the speaker pattern, and the reproduction time (unit section Start) is obtained from the utterance pattern, so that the dialogue ID and the reproduction time are obtained. Is created (step 1055). From this list, the earliest time of each dialogue ID is added to the matching information file 13 (step 1056). This process is repeated for the number of divisions (step 1054) to generate the matching information file 13.

  Next, the association operation will be described in detail.

  When the association unit 150 performs the first association, as shown in FIG. 14, the speaker pattern and the utterance pattern are matched. In this case, since all the lock information is OFF, the association is performed as it is. And output to the matching information file 13.

Also, when there is lock information, as shown in FIG.
Number of divisions = number of lines of lock information + 1
And dividing the speaker pattern based on the number of divisions. In the example of the figure, the speaker pattern pieces 1, 2, and 3 are divided. The speaker pattern piece 1 is from the head before the dialogue ID, and the speaker pattern piece 2 is from the dialogue ID 1 to the dialogue ID 3. The speaker pattern piece 3 is divided from the line ID 3 to before the line ID 18.

  For the utterance pattern, as shown in FIG. 16, referring to the lock information, the utterance pattern piece 1 is from the beginning until 0:00:00 according to the number of divisions calculated by the above formula, The utterance pattern piece 2 is divided from 0:00 to 0:07:00, and the utterance pattern piece 3 is divided from 0:07:00 to 0:15:30.

  The matching information file 13 is updated by associating the speaker pattern pieces and the utterance pattern pieces divided in FIG. 15 and FIG.

  Next, based on said operation | movement, it demonstrates concretely using FIGS. 18-23.

  Here, before describing the operation, an interface in the input device 180 and the output device 190 will be described.

  When the next CUE of video control (or previous CUE) is input from the input device 180, the next (previous) section of the video section displayed on the playback screen of the currently displayed matching control window is the matching control window. Displayed on the playback screen. At this time, if the link button of the seek operation is ON, the dialogue displayed in the text box is synchronized and transitions to the next (previous) dialogue (when OFF, there is no text change).

  When the next (previous) text of the text box control is input from the input device 180, the next (previous) line of the currently displayed line is displayed. Here, if the seek operation link button is ON, the video section is also synchronized to transition to the next (previous) section.

  When the link button is turned on and a video and text seek operation is performed, the video and text change synchronously, so it is possible to check whether the association is correct.

  If the association is wrong, the link can be turned off to listen to the audio of the video, and the corresponding search can be performed by searching for the corresponding dialogue using the text search function. At this time, the range of text search is limited by the lock information.

  In the following, it is assumed that the utterance pattern file 11 and the speaker pattern file 12 are created by the video analysis unit 130 and the scenario text analysis unit 140, and the matching information file 13 is created by the association unit 150.

  FIG. 18 is an example of a startup screen at the start of association. Here, when the operator double-clicks the program icon and starts up and clicks “File (F)” on the screen, a file selection dialog box is displayed, and a matching information file is selected. At this time, the control unit 160 turns on the link button of the matching control window and turns off the lock button (step 104). The control unit 160 associates the speaker pattern with the utterance pattern and adds them to the matching information file 13 (step 105), whereby the control unit 160 reads the matching information from the matching information file 13 and based on the dialogue ID. The dialogue in the scenario text is referenced from the scenario storage unit 120, and similarly, a cut video is generated from the video file storage unit 110 based on the time code and displayed on the matching control window (step 106). In addition, the head cut of “CUE1” is displayed on the video display part of the matching result window, and the line of “CUE1” is displayed on the dialog display part. In FIG. 19, the state where the link button is ON and the lock button is OFF is displayed as the initial state.

  In this state, an arbitrary CUE can be displayed in the matching control window by operating the video seek control or the dialogue seek control (step 107). The CUE displayed on the matching control window is highlighted with a thick frame of the same color in the matching status display window.

  In the above state, when the operator clicks the lock button at an arbitrary CUE position (steps 107 and 108), as shown in FIG. 20, the icon of the lock button changes to red (locking mark). Further, the CUE number of the locked matching result window also changes in the same manner as the lock button. The locked CUE information is written to the lock information file 14 (step 109).

  Next, after the above operation, the correspondence between arbitrary sections can be corrected. First, when a seek link button is clicked, both the seek link button and the lock button transition to an inactive state, as shown in FIG. Next, it jumps to a video at an arbitrary time using video seek control, and as shown in FIG. 22, video playback is paused by video control at that time. Repeat the seek operation and pause at the start of a specific line. A dialogue text segment having an appropriate correspondence is selected by specifying a keyword as a dialogue seek control (step 110), and the matched portion is highlighted (step 111).

  When the lock button is clicked at this time, the seek link button and the lock button are turned ON, and the time information and the dialogue ID are written in the lock information file 14 (step 112).

  It is possible to perform the alignment again after repeating the above operation and performing the lock a plurality of times.

  When the operator selects “realignment” in the tool of the menu bar shown in FIG. 23, the path of the alignment information file 13 to be newly written by the associating unit 150 is specified, and when the operator clicks the alignment button, the associating unit 150 Performs matching processing based on the lock information file 14 and overwrites the matching information file 13. The previous matching information file 13 can be left as a backup after changing the file name.

  If the video and dialogue are correctly associated after the above processing is completed, the control unit 160 outputs matching information.

  When the operator designates an operation to turn on the lock button in the above step 109, the control unit 160 adds lock information for fixing the correspondence between the dialogue and the time to the lock information file 14, and the association unit 150 speaks. When associating the symbol string of the pattern with the symbol string of the speaker pattern, the speaker pattern is divided at the portion corresponding to the dialogue ID for which the lock flag of the lock information file 14 is ON, and the lock flag is turned ON. The utterance pattern is divided at the part corresponding to the time of, and is individually associated and integrated into the matching information file 13.

  As a result, it is possible to generate a better association that passes through the corresponding points designated manually by the association unit 150.

  Also, when searching for the corresponding dialogue of the video by text search, the search range is the closest to the lock information immediately before (the time before the video) to the lock information immediately after (the time after the video) Stuff). For this reason, the number of text searches is reduced, and the effort for confirming dialogue is reduced as compared with the case where there is no lock flag. As shown in FIG. 24, when there is no lock information, it is necessary to search the entire range, but when the lock information is used, it is not necessary to search the locked range.

  It should be noted that the operations of the video analysis unit 130, scenario text analysis unit 140, association unit 150, and control unit 160 shown in FIG. 3 described above are constructed as programs, installed in a computer used as a video scenario matching device, a CPU, etc. It is possible to execute this control means.

  The built program is stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM connected to a computer used as a video scenario matching apparatus, and is installed in the computer when it is executed. It is also possible.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a system that associates video data and scenario data (text data) via a user interface.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the video scenario matching apparatus in one embodiment of this invention. It is a block diagram of the scenario text in one embodiment of this invention. It is a block diagram of the speech pattern file in one embodiment of this invention. It is a figure which shows the production | generation procedure of the speaker pattern file in one embodiment of this invention. It is an example of the consistency information file in one embodiment of this invention. It is a figure for demonstrating operation | movement of the control part in one embodiment of this invention. It is a block diagram of the lock information file in one embodiment of this invention. It is a figure for demonstrating the window displayed on the display apparatus in one embodiment of this invention. It is a figure for demonstrating the correction control button of the matching position in one embodiment of this invention. It is a flowchart of operation | movement of the video scenario matching apparatus in one embodiment of this invention. It is a flowchart of the process which matches a speaker pattern and an utterance pattern in one embodiment of this invention. It is a figure for demonstrating the initial association in one embodiment of this invention. It is FIG. (1) for demonstrating matching when there exists lock information in one embodiment of this invention. It is FIG. (2) for demonstrating matching when there exists lock information in one embodiment of this invention. It is FIG. (3) for demonstrating matching when there exists lock information in one embodiment of this invention. It is an example of the starting screen at the time of the correlation start in one embodiment of this invention. It is an example of a screen at the time of opening the matching information in one embodiment of the present invention. It is an example of a display when the operator in one embodiment of the present invention makes a determination (determined to be correct). It is an example of a display when the operator in one embodiment of the present invention makes a determination (determines that it is not correct). It is an example of correction | amendment of the correspondence in one embodiment of this invention. It is an example of a display in the case of performing realignment in one embodiment of the present invention. It is a figure for demonstrating the effect by the lock information in one embodiment of this invention.

Explanation of symbols

11 utterance pattern storage means, utterance pattern file 12 speaker pattern storage means, speaker pattern file 13 matching information storage means, matching information file 14 lock information storage means, lock information file 15 matching information 110 video storage means, video storage section 120 Scenario storage unit, scenario storage unit 130 Video analysis unit, video analysis unit 140 Scenario text analysis unit, scenario text analysis unit 150 Association unit, association unit 160 Control unit 161 Association result display unit 162 Rematching unit 163 Matching result output Means 170 Display unit 180 Input unit

Claims (7)

In the video / scenario matching method for associating scenario text with each partial section (scene) of video,
Associate the video data read from the video storage means with the scenario text read from the scenario storage means,
Using the user interface with the operator,
Displaying the correspondence result between the read video data partial section and the read scenario text, prompting the operator to make a decision,
If there is an input from the operator that the correspondence is incorrect,
Requesting the operator to make corrections, turning on a lock flag of a record having a correspondence relationship between a partial section of video data in a lock information storage means and a scenario text, and inquiring to the operator whether or not to update the correspondence;
A video / scenario matching method, characterized in that the association processing is repeated if there is an input from the operator to update. A data input process for inputting video data from the video storage means and inputting scenario text delimited by a certain word length from the scenario storage means;
A scenario text analysis step of generating a speaker pattern of a scenario that is a symbol string including a speaker name for each unit of the scenario text and storing it in the speaker pattern storage means;
A video analysis process of dividing the video data into unit sections, generating an utterance pattern that is a symbol string including a speaker name, and storing the utterance pattern in a utterance pattern storage unit;
Matching the symbol string stored in the speaker pattern storage means with the symbol string stored in the utterance pattern storage means, generating matching information in which the change of the dialogue and the time are associated with each other, the matching information storage means Storing the matching process;
The matching information is read from the matching information storage means, the video is acquired from the video storage means corresponding to the matching information, the scenario text is acquired from the scenario storage means, and the associated result is displayed. A matching result display process for displaying the video partial section designated by the operator,
When it is determined by the operator that the correspondence result is correct and the correspondence lock is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means is turned ON. West,
When the operator instructs the association result to be corrected, the operator inputs the keyword of the scenario to be associated, searches the scenario storage unit for the scenario text corresponding to the keyword, and displays it on the display device. When the operator designates the lock of the correspondence relationship, the lock state display on the screen on the display device is displayed, the lock flag of the lock storage means is turned ON, and Realignment process to update the alignment information;
2. The video / scenario matching method according to claim 1, further comprising: a matching result output process for outputting matching information when matching of all video data and scenario text is completed. In the matching process,
By referring to the lock information file, the scenario text is divided at a portion corresponding to the dialogue ID corresponding to the dialogue ID for which the lock flag is ON, and video data corresponding to the time when the lock flag is ON is obtained. The video / scenario matching method according to claim 2, wherein the video / scenario matching method is analyzed, individually associated, and integrated into the matching information storage means. A video / scenario matching device for associating scenario text with each partial section (scene) of video,
Video storage means for storing video data;
Scenario storage means for storing a scenario text composed of a dialogue ID, a speaker ID, a speaker name, and dialogue;
A scenario text analysis process of reading the scenario text from the scenario storage means, generating a speaker pattern of the scenario and storing it in the speaker pattern storage means;
A video analysis process of reading the video data from the video storage means, dividing it into unit intervals, generating a speech pattern which is a symbol string including a speaker name, and storing it in the speech pattern storage means;
Matching the symbol string stored in the speaker pattern storage means with the symbol string stored in the utterance pattern storage means, generating matching information in which the change of the dialogue and the time are associated with each other, the matching information storage means Storing association means;
Read the matching information from the matching information storage means, acquire the video corresponding to the matching information from the video storage means, and further acquire the scenario text corresponding to the matching information from the scenario storage means Display the displayed result on the display means, association result display means for displaying the video partial section designated by the operator,
When it is determined by the operator that the matching result is correct and lock of the correspondence relationship is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means is turned on. In addition, when the operator gives an instruction to correct the association result, the operator inputs a scenario keyword to be associated, and searches the scenario storage means for the scenario text corresponding to the keyword. When displayed on the display device and the corresponding lock is designated by the operator, a display indicating the lock state on the screen on the display device is performed and the lock flag of the lock storage means is turned ON. And re-matching means for instructing to update the matching information;
A video / scenario matching device, comprising: matching result output means for outputting matching information when matching of all video data and scenario text is completed. The association means includes
When performing the association, the lock information file is referred to, the speaker pattern is divided at the part corresponding to the dialogue ID for which the lock flag is ON, and it corresponds to the time when the lock flag is ON 5. The video / scenario matching apparatus according to claim 4, further comprising means for dividing the speaker pattern into parts, individually associating them, and integrating them into the matching information storage means. A video / scenario matching program for associating scenario text with each partial section (scene) of video,
On the computer,
A data input step of inputting video data from the video storage means and inputting scenario text delimited by a certain word length from the scenario storage means;
A scenario text analysis step of generating a speaker pattern of a scenario that is a symbol string including a speaker name for each unit in which the scenario text is divided and storing it in a speaker pattern storage means;
A video analysis step of dividing the video data into unit intervals, generating an utterance pattern that is a symbol string including a speaker name, and storing the utterance pattern in the utterance pattern storage means;
Matching the symbol string stored in the speaker pattern storage means with the symbol string stored in the utterance pattern storage means, generating matching information in which the change of the dialogue and the time are associated with each other, the matching information storage means A mapping step to store;
The matching information is read from the matching information storage means, the video is acquired from the video storage means corresponding to the matching information, the scenario text is acquired from the scenario storage means, and the associated result is displayed. A matching result display step for displaying the video partial section specified by the operator,
When it is determined by the operator that the correspondence result is correct and the correspondence lock is designated, the lock state on the screen on the display device is displayed and the lock flag of the lock information storage means is turned ON. West,
When the operator instructs the association result to be corrected, the operator inputs the keyword of the scenario to be associated, searches the scenario storage unit for the scenario text corresponding to the keyword, and displays it on the display device. When the operator designates the lock of the correspondence relationship, the lock state display on the screen on the display device is displayed, the lock flag of the lock storage means is turned ON, and A realignment step to update the alignment information;
A video / scenario matching program for executing a matching result output step of outputting matching information when matching of all video data and scenario text is completed. In the association step,
When performing the association, the lock information file is referred to, the speaker pattern is divided at the part corresponding to the dialogue ID for which the lock flag is ON, and it corresponds to the time when the lock flag is ON 7. The video / scenario matching program according to claim 6, wherein a speaker pattern is divided into portions, individually associated with each other, and integrated into the matching information storage means.