JP5287338B2 - Imaging device, video editing device, and video editing system - Google Patents

Description

  The present invention relates to an imaging device, a video editing device, and a video editing system that automatically extract important scenes based on operation information of a plurality of input means recorded in association with a captured video.

  In recent years, devices for recording video data and audio data as files on a randomly accessible recording medium such as an optical disk or a semiconductor memory have become common. For example, in broadcasting stations, video production companies, etc. Video editing work is performed using an editing apparatus equipped with a recording medium. Even in the case of using photographic equipment for recording on a magnetic tape medium, video editing work is often performed in the same manner as described above after the video data and audio data on the magnetic tape medium are filed.

  On the other hand, events such as lectures, weddings, and theaters are carried out according to a predetermined schedule. Traditionally, when shooting and editing events such as lectures and weddings, the area around the important part is taken along the course of the event, and the section that is particularly important from the material data taken at the time of editing (important (Scene) is being cut out.

Based on these backgrounds, a technique is disclosed that inputs scenario data including a shooting method and a title before shooting and performs shooting guidance according to the scenario data (see, for example, Patent Document 1). On the other hand, when a presentation is performed with a personal computer, a technique for transmitting switching operation information and dividing video data when switching the display of presentation materials (for example, see Patent Document 2) is disclosed. In addition, a technique (for example, refer to Patent Document 3) that extracts an important scene in which a subject to which an RFID is attached is photographed using RFID is disclosed.
JP 2006-174318 A JP 2004-236238 A JP 2007-67518 A

  However, in the conventional configuration disclosed in Patent Document 1, in order to perform guidance based on schedule information (scenario data), it is necessary to create prior information such as schedule information (scenario data) before shooting. There is a problem. In addition, the conventional configuration disclosed in Patent Document 2 divides video data based on one operation (display switching). There is a problem that it is difficult. Further, even in the conventional configuration disclosed in Patent Document 3, for example, although it is an effective technique for classification by subject, scenes other than the important scenes to be noted are also cut out, and it takes time to select important scenes. There is a problem.

  In view of the above problems, an object of the video editing system of the present invention is to perform automatic extraction (automatic segmentation) of important scenes intended by many people without requiring prior information.

  In order to solve the above-described problem, a video editing system according to the present invention generates, as recording status information, information related to a first imaging device that captures and records a video, and recording status information. A recording device comprising: a second imaging device provided with a recording status transmission means for transmitting to the receiving device; and a recording status index providing means for receiving recording status information transmitted from the second imaging device and recording it as a recording status index. A situation receiving apparatus; and an editing apparatus including a scene cutout unit that determines an important scene to be selected from video data captured by the first imaging apparatus based on a recording situation index.

  According to the video editing system of the present invention, it is possible to automatically extract an important scene from captured video data based on the recording status of each imaging device without requiring prior information such as scenario data.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a video editing system according to an embodiment of the present invention. The present embodiment includes an imaging recording apparatus that records video on a recording medium, and an editing apparatus that extracts and edits important scenes from video data recorded on the recording medium. The imaging recording apparatus is not provided with a recording status transmission means, and is distinguished between a first imaging recording apparatus for recording moving image data to be edited and a second imaging recording apparatus provided with a recording status transmission means. It is possible to provide a plurality of second imaging / recording apparatuses.

  The first imaging recording device and the second imaging recording device are either a camera recorder or a digital still camera, respectively. In the video editing system of the present embodiment, a specific example of the first imaging / recording apparatus is the camera recorder 1, and a specific example of the second imaging / recording apparatus is one each of the camera recorder 21 and the digital still camera 22. Although the configuration will be described as an example, even when the specific examples and the number of the first imaging recording device and the second imaging recording device are different from this, it can be realized by performing the same processing.

  Such video editing systems are intended for business use for video production companies, etc., such as shooting events at venues such as lectures, weddings, music concerts and performances, video editing, etc. Used for For example, video data acquired by the camera recorder 1 at a lecture or wedding venue is recorded on a recording medium that can be attached to and detached from the camera recorder 1, and this recording medium is also recorded with other video data in the editing room. Editing work is performed using the video data recorded in.

  As shown in FIG. 1, the present video editing system is roughly divided into a camera recorder 1 as a first imaging / recording device, a camera recorder 21 as a second imaging / recording device, and a digital still as a second imaging / recording device. Camera 22, recording medium 10 for recording moving image data captured by camera recorder 1, recording medium 113 for recording moving image data captured by camera recorder 21, and still image captured by digital still camera 22 A recording medium 123 for recording image data, a recording status index file, and an index file recording device 3 that records the generated recording status index file on the recording medium 150; Recording medium for recording the recording status index file 150, a configuration including the editing apparatus 2 for editing moving image data recorded on the recording medium 10 and the recording medium 113, 123. Note that the camera recorder 1 as the first image recording device, the camera recorder 21 as the second image recording device, and the digital still camera 22 as the second image recording device are collectively referred to simply as an image recording device. I will call it.

  The camera recorder 1 includes an imaging unit 100 that captures an image of a subject and outputs moving image data as video data, and a video recording unit 104 that records the captured moving image data. In addition, the imaging unit 100 images a subject and provides moving image data to the video recording unit 104. The video recording unit 104 performs video processing, and then records the moving image data as a moving image file as a video file on the recording medium 10. The video recording means 104 can also compress and encode the supplied moving image data. As a compression encoding method, for example, a widely used MPEG (Moving Picture Experts Group) -2 method, an MPEG-4 / AVC (Advanced Video Coding) method, or a business-use digital VTR (Video Tape Recorder) standard. SMPTE (Society of Motion Picture and Television Engineers) -314M (DV-Based 25M) standard. Note that since the present invention does not depend on the compression encoding method, any compression encoding method can be used. Further, when a sufficient recording capacity can be secured in the recording medium 10, the moving image data need not be subjected to compression encoding. In the present embodiment, an example will be described in which luminance signals and color difference signals of effective pixels of moving image data are sequentially arranged and recorded in an uncompressed file format (a specific example will be described later).

  The camera recorder 21 includes an imaging unit 110 for imaging a subject, a video recording unit 114 for recording moving image data captured by the imaging unit 110, and an operation input unit for an operator to input an operation. 111, and a recording status transmission unit 112 that transmits the operation content and operation time instructed from the operation input unit 111. The imaging unit 110 captures a subject and generates moving image data as video data. The video recording unit 114 records the moving image data captured by the imaging unit 110 as a moving image file on the recording medium 113. Similarly to the camera recorder 1, the camera recorder 21 does not depend on the compression encoding method, so any compression encoding method may be used. For the moving image file, an uncompressed file format is used as in the camera recorder 1 (a specific example of the file format will be described later). The operation input unit 111 is an input unit for an operator to input operations such as recording start and recording end to the camera recorder 21. The recording status transmission unit 112 uses the operation content instructed by the operator in the operation input unit 111 as the recording status information, and associates it with the operation time at that time, imaging device information, etc., to the recording status reception unit 152 in the index file recording apparatus 3. Send.

  The digital still camera 22 includes an imaging unit 120 for capturing still image data, a video recording unit 124 for recording still image data captured by the imaging unit 120, and an operation for an operator to input an operation. An operation input unit 121 and a recording status transmission unit 122 that transmits an operation content and operation time instructed from the operation input unit 121 are provided. The imaging unit 120 images a subject and generates still image data as video data. The video recording unit 124 records the still image data captured by the imaging unit 120 on the recording medium 123 as a still image file as a video file. Since the digital still camera 22 does not depend on the still image compression encoding method, any compression encoding method may be used. An example in which a still image file is used as a RAW data format that is an uncompressed file format of a still image will be described. Since these file formats are generally widely used, a detailed description thereof will be omitted. The operation input means 121 is an input means for an operator to input an operation such as releasing a shutter (taking a picture) on the digital still camera 22. The recording status transmission unit 122 uses the operation content instructed by the operator in the operation input unit 121 as the recording status information, and associates it with the operation time at that time, photographing apparatus information, etc., to the recording status receiving unit 152 in the index file recording apparatus 3. Send.

  The index file recording device 3 includes a recording status receiving unit 152 and a recording status index adding unit 151. The recording status receiving unit 152 receives recording status information, operation time, photographing device information, and the like transmitted from the camera recorder 21 and the digital still camera 22. The recording status index assigning unit 151 records the recording status information, the operation time, the photographing device information, and the like received by the recording status receiving unit 152 on the recording medium 150 as a recording status index.

  The editing device 2 includes an index interpretation unit 201, a scene cutout unit 202, a playlist creation unit 203, and a video editing unit 204. The index interpretation unit 201 is a unit that analyzes the recording status index recorded on the recording medium 150, and provides the analysis result to the scene cutout unit 202. The scene cutout unit 202 cuts out an important scene based on the analysis result of the provided recording status index, provides section information of the important scene to the playlist creation unit 203, and video editing data including the important scene as the video editing unit 204. To provide. The playlist creation means 203 creates a playlist file from the provided important scene section information. The video editing unit 204 cuts out an important scene from moving image data including the provided important scene and edits the video data (for example, cut editing). The moving image data recorded on the recording medium 10 and the recording status index recorded on the recording medium 150 are based on the operation time included in the recording status index and time information such as the shooting time included in the moving image data. To associate. Details of each means in these editing apparatuses 2 and correspondence between moving image data and a recording status index will be described later.

  In the present embodiment, the camera recorder 1, the camera recorder 21, and the digital still camera 22 are mainly used by extending the communication function with respect to the camera recorder and the digital still camera released for business use and consumer use. . Specific descriptions of communication functions necessary for the camera recorder and the digital still camera will be described later. The recording medium 10 and the recording media 113, 123, and 150 may be any randomly accessible medium such as a semiconductor memory, a hard disk drive, or an optical disk. In the present embodiment, an example in which the recording media 10 and 113, 123, and 150 are semiconductor memories will be described. The editing device 2 is realized as a computer that executes a nonlinear editing program. A non-linear editing program generally operates on a computer, takes video data from a recording medium, and performs cut editing and video processing. In the present embodiment, a general non-linear editing program is realized by operating on a computer a program in which an index interpretation unit 201, a scene cutout unit 202, and a playlist creation unit 203 are further added.

  An outline of the flow of processing from shooting to editing in the video editing system configured as described above will be described. Here, events (events) that are subject to photography, such as lectures, weddings, music concerts, presentations, theatre, and play performances, are called photography events. The state of the event to be imaged is photographed, and the edited moving image data is edited after or during the event. The present invention is most effective in the case of a shooting target event in which the shooting target is physically concentrated with respect to a large audience (operators of the imaging / recording apparatus).

  First, the imaging method of each imaging recording apparatus in the present embodiment will be described. Both imaging and recording apparatuses shall shoot the same shooting target event.

  In the present embodiment, the operators of the camera recorder 21 and the digital still camera 22 that are the second imaging / recording apparatus are assumed to be attendees and spectators of the shooting target event. In addition, it is assumed that each operator freely photographs using his / her own image recording apparatus. The video editing system according to the present embodiment automatically performs scene segmentation using a time zone in which the number of shots is large as an important scene. For example, in weddings, there are generally parts that many people consider important, such as entering the bride and groom. Preferably, each operator shoots a scene that is considered important, but even if each operator shoots without conscious of the important scene, it can be said that a section taken by many operators is also important. For example, a scene in which a large number of viewers (operators) are impressed by a climax scene or the like in a theater performance or a theater performance is an important scene. That is, in the video editing system of the present invention, scene extraction is performed from captured moving image data, with a scene (section) having a large number of recordings (multiplicity) per unit time in an event to be captured as an important scene. In addition, as a photographing method, for example, a part of a photographing target event may be photographed such as photographing an acquaintance appearing in a theater up, or a whole image of the photographing target event may be photographed.

  On the other hand, it is assumed that an operator (hereinafter referred to as a main operator) of the camera recorder 1 serving as the first imaging / recording apparatus has a role of photographing an event to be photographed. It is preferable that the main operator mainly shoots the entire image of the shooting target event and continuously shoots while the shooting target event is being performed. As for photographing with the camera recorder 1, there is no special operation or the like, and the subject may be photographed in the same manner as a general photographing form. In the present embodiment, as an example, the camera recorder 1 will be described assuming that it always keeps shooting the shooting target event. The moving image data shot by the camera recorder 1 is mainly video data used during editing.

  Next, the flow of processing at the time of shooting in each imaging recording apparatus will be described.

  First, immediately before the shooting target event starts, the main operator instructs the camera recorder 1 to start recording via an operation input means (not shown). The operation input unit notifies the video recording unit 104 of a recording start instruction. The video recording unit 104 starts recording the moving image data captured by the imaging unit 100 on the recording medium 10 based on the recording start instruction. When the shooting target event is completed, the recording process in the video recording unit 104 and the recording medium 10 is stopped according to the recording end instruction via the operation input unit of the main operator. That is, the operation input unit notifies the video recording unit 104 of a recording end instruction. The video recording unit 104 stops recording moving image data on the recording medium 10 based on the received recording end instruction. Since the process of recording moving image data from the start of recording to the end of recording is a technique that is generally widely implemented in camera recorders, a detailed description thereof will be omitted.

  On the other hand, the operator of the camera recorder 21 shoots an arbitrary scene of the shooting target event freely during the shooting target event, that is, at the operator's discretion. When shooting, the operator gives an instruction to start recording via the operation input unit 111. The operation input unit 111 notifies the video recording unit 114 and the recording status transmission unit 112 of a recording start instruction. The video recording unit 114 records moving image data on the recording medium 113 based on the recording start instruction. The recording status transmission unit 112 records the recording status information received from the operation input unit 111 (for example, information indicating that the operator has issued a recording start instruction), the operation time at that time, and the photographing device information in association with each other. It transmits to the situation receiving means 152. Similarly, when ending the recording, the operator instructs the recording end through the operation input unit 111 and notifies the video recording unit 114 and the recording status transmission unit 112 of the recording end instruction. The video recording unit 114 stops recording moving image data on the recording medium 113 based on the received recording end instruction. Further, the recording status transmission unit 112 associates the recording status information received from the operation input unit 111 (for example, information indicating a recording end instruction), the operation time at that time, and the photographing device information in the same manner as at the start of recording. Transmit to the receiving means 152.

  Here, the operation time is a time when the operation input unit 111 receives an instruction to start or end recording from the operator, and if the imaging recording apparatus is equipped with an integrated circuit (RTC) having a Real Time Clock function. It can be acquired. When an operation instruction is received by the operation input unit 111, the recording status index can be associated by acquiring the time from the RTC and holding it as a set together with the operation content and imaging device information. In order to synchronize the RTCs of the respective imaging and recording devices, for example, each of the second imaging and recording devices communicates with the first imaging and recording device before the shooting target event starts, and each RTC is subjected to the first imaging. It is desirable to perform processing synchronized with the RTC of the recording apparatus.

  The photographing device information is a device-specific identifier (DEVICE ID) of the camera recorder 21, and a character string or a number unique to each device may be assigned. For example, in the present embodiment, the device ID of the camera recorder 21 is “MV0001”. The recording status information represents information for instructing the imaging / recording unit to operate the device by operating information of the operator such as a recording start instruction and a recording end instruction.

  Further, the operator of the digital still camera 22 can freely shoot like each operator of the camera recorder 21. The difference between the camera recorder 21 and the digital still camera 22 is that the camera recorder 21 is a device that shoots moving images, whereas the digital still camera 22 is a device that shoots still images, and the shooting format is different. . Due to the difference in the photographing format, there arises an operational difference between moving image photographing in the camera recorder 21 instructing recording start and recording end and still image photographing in the digital still camera 22 instructing only the recording instruction.

  The flow of processing at the time of shooting in the digital still camera 22 is the same as that of the camera recorder 21 except for the above-described difference in shooting format. That is, when the operator instructs recording via the operation input unit 121, the operation input unit 121 notifies the video recording unit 124 and the recording status transmission unit 122 of the recording instruction. The video recording unit 124 records still image data on the recording medium 123 based on the recording instruction. Further, the recording status transmission unit 122 associates the recording status information (for example, recording instruction) received from the operation input unit 121, the operation time at that time, and the imaging device information, and transmits them to the recording status reception unit 152. Here, the device ID of the digital still camera 22 will be described as “DSC0001”.

  Next, the operation of the index file recording device 3 will be described. The recording status receiving unit 152 includes recording status information (for example, a recording start instruction, a recording end instruction, a recording instruction) transmitted from the second imaging recording apparatus, that is, the camera recorder 21 and the digital still camera 22, an operation time, The photographic device information is received and provided to the recording status index assigning means 151. The recording status index assigning unit 151 associates the supplied recording status information, the operation time, and the photographing device information as a set to obtain a recording status index. The recording status index assigning unit 151 further records the individual recording status indexes on the recording medium 150 as a recording status index file.

  FIG. 3 is a diagram showing a specific example of a recording status index file for recording the recording status index on the video recording medium 10. The recording status index file is a text file in the Comma Separated Values (CSV) format with a comma as a delimiter, and the file name is, for example, IndexList. Let txt be. The contents to be recorded (one line in FIG. 3) sequentially indicate the INDEX number, INDEX type, photographing device information, and operation time. The INDEX number is a serial number assigned in the order in which the recording status information received by the recording status receiving unit 152 is received. However, when the recording status index file is created, some items are sorted using the key and then the numbers are assigned. Good. The INDEX type represents the type of recording status index file. “POINT” corresponds to a recording instruction in the digital still camera 22, “START” corresponds to a recording start instruction in the camera recorder 21, and “STOP” corresponds to a recording end instruction in the camera recorder 21. These correspond to the example of the recording status index shown in FIG. 2 (2) later, and the explanation of FIG. 2 will be described later.

  In this embodiment, a format for recording the recording status index file in a file different from the video file will be described. However, individual recording status information is embedded and recorded in the video file recorded by the second imaging recording apparatus. You may do it. In this case, each of the second imaging / recording apparatuses records with the recording status index embedded in the video file. The same effect can be obtained by inputting this video file to the index file recording device 3, and the index file recording device 3 extracts and summarizes the recording status index from each video file to construct the recording status index file.

  Next, the correspondence between the recording status index and the moving image data shot by the camera recorder 1 and the outline of the important scene cutout (extraction) method performed by the scene cutout means 202 will be described. The scene cutout unit 202 selects, as an important scene, a time zone in which operation is performed on more image capturing and recording devices based on the degree of overlap of operation times included in the recording status index. FIG. 2 is a diagram showing a concept of a scene cutout method based on moving image data and a recording status index. FIG. 2A shows the relationship between the recording status index (INDEX) and moving image data captured by the camera recorder 1, and the concept of scene cutout processing. The horizontal axis shows the passage of time. FIG. 2 (2) is a diagram showing one specific example of the recording status index, and FIG. 2 (1) shows this content along the time axis of the moving image data.

  INDEX (01), INDEX (03), and INDEX (05) are recording status indexes transmitted by the digital still camera 22 (DEVICE ID is DSC0001). INDEX (02) and INDEX (04) are recording status indexes transmitted by the camera recorder 21 (DEVICE ID is MV0001). For example, INDEX (01) indicates that a recording instruction has been made at time 10: 05: 05: 00. INDEX (02) and INDEX (04) indicate that a recording start instruction is given at time 10: 05: 06: 0, and a recording end instruction is given at time 10: 05: 10: 00. Represent. Further, 10: 05: 05: 00 means 10:05:05 frame, and represents a hour digit, minute digit, second digit, and frame digit. This format is a method expressed as a time code on a camera recorder for business use. If there is no concept of a frame in the operation time of the digital still camera 22, for example, 00 frames may be supplemented as the digit of the time information frame.

  For example, the recording status index INDEX (01) at the time 10: 05: 05: 00 given by the reception of the recording status information from the digital still camera 22 is a moving image captured by the camera recorder 1 at the same time using the time information. The image data can be associated. Note that the shooting time of each frame of the moving image data necessary for associating the moving image data with the recording status index may be calculated from the shooting time and the number of frames of the first frame of the moving image data. The time information may be multiplexed, or the time information used in the MPEG-2 system or the like (for example, Presentation Time Stamp) may be used for the association. In the present embodiment, a moving image file is recorded in a format in which the shooting time of each frame is embedded in moving image data as in the file format shown in FIG. In FIG. 13, the moving image data in frame n is represented as moving image data (n), the shooting time in frame n is represented as shooting time information (n), and the moving image data (n) is represented as luminance (Y). Color difference (U / V) data is recorded in order (for example, a format generally called YUV422). By embedding the shooting time of each frame in the moving image data, it is possible to confirm the match between each frame shooting time of the moving image data and the operation time. Association can be performed. Similarly, INDEX (02), INDEX (03), INDEX (04), and INDEX (05) can be associated with moving image data.

  In the editing apparatus 2, the scene cutout unit 202 calculates sections (important scene candidates) that are candidates for extraction of important scenes based on each recording status index (INDEX) interpreted by the index interpretation unit 201. This is performed for the purpose of preventing deterioration in the accuracy of important scene extraction, which is caused by a mixture of a moving image (between the section / START and STOP) and a still image (point / POINT) as the INDEX type. For example, when the INDEX type is “POINT”, a certain section before and after the operation time (one frame before and after in the example of FIG. 2) is set as an important scene candidate. When the INDEX type is “START” and “STOP”, an important scene candidate is a portion between “START” and “STOP”, which is a pair of recording start / end shot with the same photographing device. In the example of FIG. 2, INDEX0001 (02) and INDEX (04) whose DEVICE IDs are both MV0001 correspond to INDEX (02) and INDEX (04) as important scene candidates.

  Next, an important point is calculated from a plurality of important scene candidates extracted by the above method, and an important scene is extracted. As shown in FIG. 2, the duplication (multiplicity) of important scene candidates is calculated, and when the multiplicity is equal to or greater than the threshold, the first frame (time) of the section where the multiplicity is equal to or greater than the threshold is extracted as an important point. In the example of FIG. 2, the important point is determined with a threshold value of 3.

  From the important point determined as described above, the moving image data of a certain section before and after is extracted as an important scene. In the example of FIG. 2, the fixed front and rear sections are set to 2 seconds before and 5 seconds after. A plurality of important points and important scenes are detected and vary depending on the multiplicity threshold. The multiplicity threshold may be determined based on the number of devices connected to the recording status receiving unit 152. When the number of connected devices is large, the multiplicity threshold may be set to a large value, and when the number of connected devices is small, the multiplicity threshold may be set to a small value. In addition, although the section length of the important scene varies depending on the time accuracy (time resolution) of the association between the recording status index and the moving image data, the association can be freely made in units of frames or seconds based on the desired section length. You may go. Further, the important point may be an arbitrary section length instead of a frame unit.

  The outline of the association between the recording status index and the moving image data and the important scene extraction method has been described above. Next, the flow of editing (scene cutout and playlist creation) processing in the editing apparatus 2 using the recording status index information will be specifically described.

  First, the index interpretation unit 201 reads a recording status index file recorded on the recording medium 150 and creates an index table (a set of recording status indexes) for managing the recording status index as shown in FIG. . This index table provides an index table to the scene cutout unit 202.

  The scene cutout unit 202 uses the index table supplied from the index interpretation unit 201 to extract (cut out) an important scene according to the algorithm described above, and provides it to the playlist creation unit 203. A specific example of the scene cutout method in the scene cutout unit 202 will be described later.

  The playlist creation unit 203 creates a playlist based on the important scene information from the scene cutout unit 202 and the video data identifier (for example, the file name of the moving image file) recorded on the recording medium 10. Here, a playlist is a list of playback sections of one or more moving image data, and a playback device (not shown) plays back playback sections registered in the playlist in order. To do. The playlist created by the playlist creating means 203 is recorded and managed as a playlist file in the recording media 10 and 150 and other recording media (not shown). FIG. 4 shows a specific example of the playlist file. The playlist file is a text file in CSV format, and the file name is, for example, PlayList. Let it be txt. In the playlist file, the file name of the moving image data, the reproduction start time in the moving image file specified by the first item, and the reproduction end time in the moving image file specified by the first item are written in order. For the playlist file shown in FIG. 4, the playback device in order MV0000 — 0000. From the moving image data associated with 10: 05: 04: 00 to the moving image data associated with 10: 05: 11: 00 in the dat file, MV0000 — 0000.0000. From the moving image data associated with 12:33:00 of the dat file to the moving image data associated with 12:33:00, MV0001_0001. From the moving image data associated with 00: 00: 10: 00 of the dat file to the moving image data associated with 00: 12: 00: 12: 00 are reproduced. A specific example of the playlist creation method in the playlist creation unit 203 will be described later.

  The video editing unit 204 reads the playlist file recorded by the playlist creation unit 203, deletes (cuts) moving image data other than the playback section according to the described playlist, joins them, and displays subtitles and video effects. By giving it, it is used for the purpose of creating a complete package medium (so-called “complete package”). Since these video editing means can be realized by a technique that is generally widely known as nonlinear editing software, a detailed description thereof will be omitted.

  In this way, only important scenes can be automatically extracted and reproduced using the recording status index information.

  The flow of processing relating to the shooting and editing method in the present embodiment has been described above, but each process and algorithm in the video editing system of the present embodiment will be specifically described.

  First, communication functions required for the index file recording device 3, the camera recorder 21, and the digital still camera 22 in the present embodiment will be described. First, the camera recorder 21 and the digital still camera 22 have a configuration in which recording status transmission means 112 and 122 are added to a general camera recorder and a digital still camera. The general second camera recorder and digital still camera have a function of recording video data (moving image and still image data) in which a captured video is associated with a shooting time on a recording medium. It is. Since these imaging and recording apparatuses are technologies that are generally widely implemented in business and consumer imaging and recording apparatuses, a detailed description thereof will be omitted.

  The transmission / reception method in the recording status transmission means 112 and 122 and the recording status reception means 152 will be specifically described. As a specific example of the transmission / reception method, there are two transmission / reception methods: a first transmission / reception method that performs transmission / reception using a wireless communication technology such as a wireless LAN (Local Area Network), and a second transmission / reception method that performs transmission / reception using a signal line. This will be described as an example. Since the present invention does not depend on the transmission / reception method, the recording status information, the operation time, and the imaging device information can be transmitted / received, or the recording status information can be transmitted / received, and the reception side (recording status receiving means 152) Any method capable of detecting or generating photographing apparatus information may be used. For example, as the operation time, the time at which the reception side receives the recording status information may be set as the operation time, and the imaging device information may be generated according to the line or port at which the reception side has received the recording status information.

  The first transmission / reception method is a method of transmitting / receiving using a wireless technology. For example, the recording status index is transmitted / received using a wireless LAN technology. In the present invention, it is only necessary that the recording status information, the operation time, and the photographing device information can be transferred as one set, and if the communication is one-way communication from the recording status transmission means 112 and 122 to the recording status reception means 152. Good. These communications can be realized in the wireless LAN technology by a known technology as disclosed in, for example, Japanese Patent Application Laid-Open No. 2003-289817. The first transmission / reception method is highly convenient for the operator and the main operator because no connection is required.

  On the other hand, the second transmission / reception method is a method that is simply realized by wire and uses one reception port (for example, a pin terminal of a microcomputer or the like) for one device. First, in the recording status receiving unit 152, photographing device information is assigned in advance to each receiving port. For the reception port, for example, a 1-bit recording status signal that becomes High during recording and becomes Low when not recording is used. The operation time may be acquired from the RTC when the recording status receiving unit 152 is equipped with an RTC and the recording status signal changes from High to Low or from Low to High. Also, when the recording status signal changes, the recording status index is created by acquiring the shooting device information assigned in advance to the receiving port where the signal has changed, and the recording status information, the operation time, and the shooting device information are associated. Is possible. On the other hand, in the recording status transmission means 112 and 122, for example, when a recording start instruction is received from the operation input means 111, the recording status signal is set to High, and when a recording end instruction is received, the recording status signal is set to Low. do it.

  Next, details of the scene cutout method in the scene cutout unit 202 and the playlist creation method in the playlist creation unit 203 will be described.

  In the present embodiment, an example of three editing (scene cutout and playlist creation) methods will be described: a main line editing method, a terminal synchronization editing method, and an imaging device-specific editing method. Here, the main line editing method is an editing method for creating a playlist for reproducing moving image data recorded on the recording medium 10 based on the recording status index file recorded on the recording medium 150. The moving image data recorded on the recording medium 10 is referred to as main line data. Further, the terminal synchronous editing method refers to moving image data that is not main line data, for example, moving image data recorded on the recording medium 113 by the camera recorder 21, based on the recording status index file recorded on the recording medium 150. This is an editing method for creating a playlist to be played. In this method, it is assumed that an operator designates a reproduction target in advance through an input means (not shown). Furthermore, the editing method for each photographic device is based on the photographic device information in the recording status index file, and the main line data is cut out using the recording status index corresponding to the scene shot only by any photographic device. This is an editing method for creating a playlist to be played. In this method, it is assumed that the operator designates the photographing device in advance through the input means. The outline of the scene cutout shown in FIG. 2 (1) described above is an example in the case of the main line editing method. In the present embodiment, a configuration including these three editing methods will be described, but it goes without saying that a configuration including a part of them may be used.

  First, a scene cutout method in the scene cutout unit 202 will be specifically described. FIG. 5 is a block diagram showing a detailed configuration of the scene cutout unit 202. The scene cutout unit 202 includes a main line cutout unit 300, a terminal-synchronized cutout unit 301, a photographing device-specific cutout unit 302, and an editing method selection unit 303 that selects them. The main line cutout unit 300, the terminal synchronous cutout unit 301, and the photographing device-specific cutout unit 302 are blocks that perform scene cutout processing corresponding to each of the editing methods described above. A specific processing flow of each clipping unit will be described later. The editing method selection means 303 is a means for selecting an editing (scene clipping) method. According to an instruction from the operator of the editing apparatus 2, the main line clipping means 300, the terminal synchronous clipping means 301, and the photographing device specific clipping means 302 One clipping means is selected from the inside. That is, as the scene cutout unit 202, any one of the three cutout units is used. Note that the clipping means may be switched for each editing target.

  FIG. 6 is a flowchart showing a basic processing flow in the scene cutout unit 202. The basic process flow is a process common to each scene cutout means, and the outline of the process is described in FIG.

  First, in S100, the scene cutout unit 202 calculates an important scene candidate for each recording situation index recorded in the recording situation index file. For example, the important scene candidate (n) in the digital still camera 22 is expressed by the following equation.

Tstart (n) ≦ important scene candidate (n) <Tstop (n)
Tstart (n) = Tindex (n) −Toffset
Tstop (n) = Tindex (n) + Toffset
Here, the start time Tstart (n) represents the start time of the important scene candidate. Tstop (n) represents the end time of the important scene candidate. Tindex (n) represents the nth recording status index (INDEX) information. Further, Toffset is a fixed section (offset) that is added before and after the operation time in order to make information indicating one time of still image shooting with a digital still camera into section information. For example, a time of about several seconds is set. You just have to. FIG. 2 shows an example in which Toffset is 1 second.

  The start time of the important scene candidate in the second camera recorder 21 is the same as the operation time when the recording status information is a recording start instruction (“START”), and the end time is recorded by the recording status information. What is necessary is just to make it the same time as the operation time in the case of an end instruction (“STOP”).

  The calculated important scene candidate is held in the scene cutout unit 202.

  Next, in S101, a cut-out scene (important scene) is extracted. The scene cutout method in S101 differs depending on each scene cutout means (main line cutout means 300, terminal synchronous cutout means 301, photographing device specific cutout means 302).

  Details of the algorithm in each scene segmentation means will be described below.

  FIG. 7 is a flowchart showing a process flow of the scene cutout method in the main line cutout means 300 and the terminal synchronous cutout means 301. The processing flow of the scene cutout method of the main line cutout unit 300 and the terminal synchronous cutout unit 301 is the same. Although the basic concept has already been described with reference to FIG. 2, it will be described in more detail here.

  First, in S200, the multiplicity of important scene candidates is calculated. Here, the multiplicity indicates the number of overlapping important scene candidates at each time as seen in the important scene candidates (A), (B), and (C) in FIG. The multiplicity may be calculated by overlapping the time (section) of each important scene candidate from the start time and end time of each important scene candidate.

  The scene cutout unit 202 calculates the number of overlaps between important scene candidates from the start time and end time of all the important scene candidates for all the sections (shooting sections) of the main line data recorded on the recording medium 10. FIG. 14 is a diagram illustrating a specific example of a method for calculating the multiplicity. First, the sampling unit on the time axis is determined. The time precision in the multiplicity calculation is determined by this sampling unit, and can be, for example, a frame unit or a second unit. The method of calculating the multiplicity by sampling in units of frames has the best time accuracy, but at the same time, the processing load for calculating the multiplicity increases. FIG. 14 will be described as an example when sampling is performed in units of seconds. However, the time code is shown up to the frame digit. FIG. 14 shows the horizontal axis as a time axis. FIG. 14A shows moving image data. For example, important scene candidates (A), (B), and (C) are calculated as shown in FIG. 14 with respect to the time axis of the moving image data. Shall. Here, the start time of the important scene candidate (A) is Tstart (A), the end time is Tstop (A), and similarly, the start time and end time of the important scene candidates (B) and (C) are respectively Tstart (B). , Tstop (B), Tstart (C), and Tstop (C).

  For example, the multiplicity calculation process is configured to hold a multiplicity calculation table in which all sections of main line data recorded on the recording medium 10 are sampled in the above-described sampling unit. The multiplicity calculation table is a multiplicity table corresponding to each important scene candidate (in FIG. 14, the multiplicity tables (A), (B), ( C) and a multiplicity result table. For example, the multiplicity table (A) is a multiplicity table corresponding to the important scene candidate (A), and the time included in the section calculated as the important scene candidate is 1 and the time not included is 0. Similarly, multiplicity tables (B) and (C) respectively corresponding to the important scene candidates (B) and (C) are calculated. Next, the multiplicity result table is calculated by adding the values of all multiplicity calculation tables at a certain time. For example, in FIG. 14, the value of the multiplicity result table corresponding to the time Tstart (A) is 1. This is because the important scene candidate corresponding to the time Tstart (A) is only the important scene candidate (A). Similarly, the value of the multiplicity result table corresponding to the time Tstart (B) is 3. The multiplicity result table can be calculated by performing the above-described processing for all the times included in all the sections of the main line data recorded in the recording medium 10. That is, the multiplicity of the desired time can be found by referring to the desired time in the multiplicity result table. In the present embodiment, the method of calculating the multiplicity using the multiplicity table has been described. However, when each multiplicity table is calculated, if a value is added directly to the multiplicity result table, the multiplicity table is calculated. This can be realized without using the severity tables (A), (B), and (C).

  Next, in S201, branch processing based on the multiplicity calculated in S200 is performed. If there is a section having a multiplicity greater than or equal to the threshold, the process proceeds to S202. If there is no section having a multiplicity greater than or equal to the threshold, the process ends. If there is no section having a multiplicity greater than or equal to the threshold value, the threshold value may be lowered and calculated again.

  In S202, the first frame of the section where the multiplicity is equal to or greater than the threshold is stored as an important point. For example, if the threshold value is 3, the section having a multiplicity of 3 or more is one section from time Tstart (B) (or Tstart (C)) to immediately before time Tstop (A), so the important point is Tstart (B ). This is 10: 05: 06: 00 with reference to FIG.

  Next, in S203, an important scene is calculated and output from each important point calculated in S202. Here, moving image data for a predetermined time before and after the important point is set as an important scene. As shown in FIG. 2, if the important point is 10: 05: 06: 00 and the fixed interval is 2 seconds before and 5 seconds after, the important scene is 10: 05: 04: 00-10: 05: 11:00. Note that the important point may be any frame as long as it falls within the interval in which the multiplicity is equal to or greater than the threshold.

  As another calculation method of the important scene, if the length of the section where the multiplicity is equal to or greater than the threshold is not less than a predetermined value, this section is regarded as an important scene as it is, and if it is less than the predetermined value, a certain section is added before and after You may make it be a section.

  FIG. 8 is a flowchart showing a flow of processing of the scene cutout method in the photographing device cutout unit 302. In S400, with respect to the important scene candidate calculated from the recording information index having only arbitrary photographing device information designated in advance by the operator, all sections of the important scene candidate are output as important scenes. In this case, the start time and end time of the important scene candidate may be simply set as the start time and end time of the important scene, respectively. For example, when the desired photographing device information is “DSC0001”, the important scene candidate (A), the important scene candidate (C), and the important scene candidate (D) become important scenes as described in the example of FIG. However, in the case of the example in FIG. 2, since there is an overlap between important scene candidates, the important scene candidates with overlap are connected to each other, and 10: 05: 04: 00 to 10: 05: 08: 00: 00, 10:05. Two sections from 05:15:00 to 10: 05: 17: 00 are important scenes.

  Next, a method for creating a playlist from an important scene in the playlist creating unit 203 will be specifically described. Also in the playlist creation method, processing differs depending on the editing method. FIG. 9 is a block diagram showing a detailed configuration of the playlist creation means 203. Similar to the scene cutout unit 202, the editing method selection unit 403 selects a playlist (appropriately abbreviated as PL) creation unit corresponding to each editing method according to the editing method. The main line PL creation means 400 corresponds to the main line editing method, the terminal synchronization PL creation means 401 corresponds to the terminal synchronization editing method, and the photographing device-specific PL creation means 402 corresponds to the photographing device-specific editing method. Here, it is assumed that the editing method selection unit 403 and the editing method selection unit 303 select the same editing method.

  First, as described in the outline description of FIG. 2, the playlist creation method in the main line PL creation unit 400 determines the playback section from the start time and end time of the important scene extracted by the main line scene cutout unit 300, and plays back. The target video data is moving image data (main line data) recorded in the recording medium 10. For example, the file name of the moving image data (main line data) recorded on the recording medium 10 is MV0000 — 0000. Assuming dat, in the example of FIG. 2, the playlist shown in FIG. 10 is created. That is, the playback section is set from the start time to the end time of each important scene, and MV0000_0000. A playlist having dat as video data to be reproduced is created.

  Next, the playlist creation method in the terminal synchronization PL creation means 401 determines the playback section from the start time and end time of the important scene detected by the terminal synchronization scene cutout means 301, and designates the playback target (specified in advance). The moving image data is recorded on the recording medium 113 (recorded by the camera recorder 21). For example, the file name of the moving image data recorded on the recording medium 113 is MV0001_0000. Assuming dat, in the example of FIG. 2, the playlist shown in FIG. 11 is created. That is, the playback section is set from the start time to the end time of the important scene, and MV0001_0000. A playlist having dat as moving image data to be reproduced is created. This terminal-synchronized editing method uses moving image data recorded on the recording medium 113 as a substitute for main line data, and is effective when, for example, the shooting state of moving image data of main line data is poor.

  Next, the playlist creation method in the photographing device-specific PL creation unit 402 determines a playback section from the start time and end time of the important scene detected by the photographing device-based clipping unit 302, and sets the playback target to the recording medium 10. The recorded moving image data (main line data) is used. For example, the file name of the moving image data recorded on the recording medium 10 is MV0000 — 0000. Assuming that dat is the desired photographing device information “DSC0001”, in the case of the example of FIG. 2, the playlist shown in FIG. 12 is created. That is, an important scene whose photographing device information is “DSC0001” is set as a playback section, and MV0000 — 0000. A playlist having dat as moving image data to be reproduced is created. This editing method for each photographing device has an advantage that the moving image data of the section captured by the digital still camera 22 can be extracted even when a still image is captured by the digital still camera 22. It should be noted that moving image data other than the main line data (for example, one captured by the camera recorder 21) may be designated as a reproduction target.

  In any playlist creation method, when the scene cutout unit 202 extracts a plurality of important scenes, they may be arranged in time order to form a playlist.

  As described above, according to the video editing system of the present embodiment, the editing efficiency can be dramatically improved, and an editing system that is highly convenient for the user can be realized.

  In the present embodiment, the configuration including the first imaging and recording apparatus and the second imaging and recording apparatus has been described. However, a configuration that does not include the first imaging and recording apparatus may be employed. That is, the camera recorder 1 may also include a recording status transmission unit and have a configuration equivalent to that of the camera recorder 21. In this case, it is necessary that at least one moving image data can be recorded in the second imaging / recording apparatus. Also, moving image data to be recorded by any one of the second imaging and recording devices is determined in advance as main line data, or is there a terminal synchronous editing method as an editing method in the editing device 2 (because there is no specific main line data)? Or, an editing method for each photographing device (reproduction target other than main line data) is selected.

  The method of recording in the text format (CSV format) when recording the recording status index file and the playlist file has been described. For example, a markup language such as an XML (extensible Markup Language) format or a text format is used. The same effect can be achieved even if it is described in a binary data format.

  Note that the index file recording device 3 may be included in the camera recorder 1. Further, the index file recording device 3 may be included in the camera recorder 1 to record the moving image data and the index file on the same recording medium. That is, the recording medium 10 and the recording medium 150 may be a single recording medium. In this case, since the moving image data to be recorded on the recording medium 10 and the index file to be recorded on the recording medium 150 are different files, the recording medium 10 and the recording medium 150 of this embodiment are read as a single recording medium. This is possible. Alternatively, the recording status index may be embedded in the moving image file recorded by the camera recorder 1 and recorded, the moving image file may be input to the editing device 2, and the recording status index may be extracted by the editing device 2. . Further, the index file recording device 3 may be included in the editing device 2.

  In addition, although the camera recorder 1, the camera recorder 21, the digital still camera 22, and the index file recording device 3 have been described as being different devices, for example, the camera recorder 1, the camera recorder 21, the digital still camera 22, and the index file recording device 3 are provided with a plurality of imaging units. Even when the same apparatus is realized as a video recording apparatus for recording video data on a recording medium, the same effect is exhibited. In the case of realizing the same device, it is preferable to use, for example, the second transmission / reception method as the recording status index transmission / reception method. Further, the same effect can be obtained even when the editing device 2 is included in the same device.

  It should be noted that the same effect can be achieved even if a device capable of simultaneously recording moving images and still images is used. In this case, when shooting a moving image, the process described with respect to the second camera recorder 21 may be performed. When shooting a still image, the process described with reference to the digital still camera 22 is performed. Can be done.

  In the present embodiment, the image capturing units 100, 110, and 120 always perform image capturing, and the video recording units 104, 114, and 124 start and end recording according to instructions from each operation input unit. However, an instruction from each operation input unit may be sent to each imaging unit, and imaging at the imaging unit may be started and ended in accordance with the start and end of recording.

  The video editing system according to the present invention drastically improves the convenience of the user in scene extraction, and is an event venue including a video editing system, a video editing device, an image photographing device, etc., applying a semiconductor memory, an optical disk, etc. It can be widely applied to other facilities.

The block diagram which shows the structure of the video editing system of this invention The figure which shows the concept of the scene cut-out method by a recording condition index The figure which shows one specific example of a recording condition index file The figure which shows one specific example of a playlist file Block diagram showing detailed configuration of scene cutout means Flow chart showing the basic processing flow in the scene cutout means The flowchart which shows the flow of the process of scene extraction in a main line edit and a terminal synchronous edit system The flowchart which shows the flow of the process of scene extraction in the edit system classified by photographing equipment Block diagram showing the detailed configuration of the playlist creation means The figure which shows an example of the play list creation result in a main line edit system The figure which shows an example of the play list creation result in a terminal synchronous edit system The figure which shows an example of the play list creation result in the edit system according to imaging | photography apparatus Conceptual diagram showing an example of a recording format of a moving image file The figure which shows one specific example of the calculation method of multiplicity

DESCRIPTION OF SYMBOLS 1 Camera recorder 2 Editing apparatus 3 Index file recording apparatus 10 Recording medium 21 Camera recorder 22 Digital still camera 100 Imaging part 104 Video recording means 110 Imaging part 111 Operation input means 112 Recording status transmission means 113 Recording medium 114 Video recording means 120 Imaging part 121 operation input means 122 recording status transmission means 123 recording medium 124 video recording means 150 recording medium 151 recording information index assignment means 152 recording status reception means 201 index interpretation means 202 scene cutout means 203 play list creation means 204 video editing means 300 main line cutout Means 301 Terminal synchronous cutout means 302 Shooting device specific cutout means 303 Editing method selection means 400 Main line PL creation means 401 Terminal synchronization PL creation means 402 Shadow equipment by PL creating means 403 editing method selection means

Claims (6)

A first imaging device for recording captured images;
A second imaging device comprising status transmission means for generating, as status information, information relating to the status of recording the captured video, and transmitting the status information to the status receiving device;
A situation receiving device comprising situation index giving means for receiving the situation information transmitted from the second imaging device and recording the situation information as a situation index;
An editing device comprising scene cutout means for determining an important scene to be selected from the video imaged by the first imaging device based on the situation index;
A video editing system. A first imaging device for recording captured images;
A second imaging device comprising status transmission means for transmitting the operation content of the own device as status information to the status receiving device together with the operation time;
A situation receiving device comprising situation index giving means for receiving the situation information and the operation time transmitted from the second imaging device and recording them as a situation index;
An editing device comprising scene cutout means for determining an important scene to be selected from video data captured by the first imaging device based on the situation index;
A video editing system. The scene clipping means, according to claim 1 or claim 2 video editing system according to select important scenes on the basis of the degree of overlapping of the operating time. The scene clipping means calculates a important scene candidates based on the operation time, the important scene shows the overlap degree of the candidate to calculate a multiplicity claim 1 or claim selecting the important scenes on the basis of the multiplicity Item 2. The video editing system according to Item 2 . The scene cutout unit includes a first scene cutout method for cutting out a scene based on all the situation indexes of the imaging apparatus, and a second scene cutout method for cutting out a scene corresponding to an arbitrary photographing device. The video editing system according to claim 1 or 2 . A video editing apparatus comprising: a scene cutout unit that receives a situation index including an operation time in an imaging apparatus and selects an important scene based on the degree of duplication of the operation time based on the situation index.

Images