JP3524007B2 - Moving image scene changing unit detecting apparatus and method, medium storing moving image scene changing unit detection control program, and moving image processing apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention stores a moving picture scene detecting apparatus and method for detecting a cut point in a moving picture and a scene turning point using a transition effect including dissolve and wipe, and a detection control program thereof. And a moving image processing apparatus using the device.

[0002]

2. Description of the Related Art FIG. 22 is a schematic diagram for explaining a general structure of a moving image. The moving image is composed of one or a plurality of shots. A shot is a scene, that is, a set of one or a plurality of frames continuously shot by a series of operations for obtaining one scene. A frame corresponds to a still image that constitutes a moving image. There are two types of portions where two consecutive shots in a moving image are converted, that is, a moving image conversion portion, that is, a so-called cut point and a transition portion that is a portion where a set of frames for a so-called transition effect is present.

The cut point is a point at which the two shots are switched between frames. What is a transition effect?
Shot conversion is an effect that gradually occurs between a plurality of frames in which the display order is continuous, and specific examples thereof include dissolve and wipe. Dissolve is an effect that the viewer can see that the succeeding shot of the two shots gradually emerges from the preceding shot of the two consecutive shots. The wipe is an effect of changing the preceding shot so that it is wiped off by the succeeding shot and moving to the succeeding shot.

23 (A) to 23 (C) are diagrams showing a moving image conversion portion in a moving image and frames in the vicinity thereof, arranged in parallel in the order of display. As shown in FIG. 23A, when two shots 11 and 12 are converted at the cut point 13, 2
After the trailing frame 14 of the preceding shot 11 of the two shots 11 and 12 is displayed, the leading frame 15 of the succeeding shot 12 of the two frames is displayed. That is, in the above case, the cut point 13 corresponds to a point between the end frame 14 of the preceding shot 11 and the start frame 15 of the subsequent shot 12. Further, the frames 14 and 15 immediately before or after the cut point 13 may be referred to as the cut point frames.

As shown in FIGS. 23 (B) and 23 (C), when two shots 21 and 22 are converted at the transition part 23, one or a plurality of them from the end in the preceding shot 21 of the two shots. After the retrospective frame 24 is displayed, one or a plurality of composite frames that form the transition portion 23 are displayed, and after the display, a frame 25 that advances one or more from the beginning in the subsequent shot 22 of the two shots is displayed. Is displayed. The plurality of composite frames are the backward frames 2 in the preceding shot 21.
4 to the end of the preceding shot 21 and one or a plurality of frames from the beginning in the succeeding shot 22 to one frame before the advanced frame 25 of the succeeding shot 22. , And are composed respectively. The method of synthesizing these frames differs depending on the specific method of the transition effect.

[0006] In recent years, in order to display a list of shots in a moving image to facilitate understanding of the content of the moving image, or to facilitate search for shots in the moving image, the present invention relates to detection of a scene change portion in the moving image. Many conventional techniques have been proposed.
JP-A-3-214364 and JP-A-6-89545
JP-A-6-319108, JP-A-8-1
Japanese Patent Laid-Open No. 39995 and Japanese Patent Laid-Open No. 4-111181 disclose a technique relating to detection of a cut point among the above-mentioned conventional techniques. Further, JP-A-7-288840 and JP-A-8-214210 disclose techniques relating to detection of the cut point and the transition portion. In these publications, moving images are sometimes called images,
Shots are sometimes referred to as scenes and cut points are sometimes referred to as scene changes.

[0007] Generally, in the prior art relating to the detection of cut points, an index relating to the correlation between two consecutive frames in a moving image is obtained while shifting the frame in the moving image, and the index becomes less than a predetermined threshold value. It is determined that there is a cut point between the two frames. JP-A-3-214
An apparatus using the scene change detection method of Japanese Patent No. 364 obtains the difference between the luminance histograms of the two frames as the index. An apparatus using the method for detecting a scene change portion disclosed in Japanese Patent Laid-Open No. 6-89545 uses the above-mentioned 2 as the index.
Of the plurality of pixels that respectively configure one frame, the numbers of pixels that are different from each other in both frames are used. JP-A-6-
In the scene change detecting device of Japanese Patent No. 319108, the distance corresponding to the error in the distribution of the feature vector between the two frames is less than a predetermined threshold value among the plurality of pixels forming each of the two frames as the index. Use the number of pixels. An apparatus using the scene change point detection method disclosed in Japanese Patent Laid-Open No. 8-139995 uses, as the index, a χ square distribution of a pixel value histogram between the two frames. An apparatus using the conversion point detection method disclosed in JP-A-4-111181 uses the rate of change of the correlation as the index in order to prevent erroneous detection of a cut point.

An apparatus using the image change point detecting method disclosed in Japanese Patent Application Laid-Open No. 7-288840 includes a scene changing section that uses dissolves and similar effects in order to detect a plurality of types of moving picture changing sections in one moving picture. , A scene change section using wipes and similar effects, a scene change section caused by camera movement, and a cut point are sequentially detected for one moving image. JP-A-8-214
The image cut point detection device of Japanese Patent Laid-Open No. 210 obtains the inter-image distances between frames of all combinations in a section consisting of a plurality of consecutive frames in a moving image and stores them as a table, and the inter-image distances in the table. Then, the rate of change of the frames in the section is obtained, and it is determined whether the section is the moving image conversion unit based on the magnitude relationship between the rate of change and a predetermined threshold value. In the publication, the inter-image distance corresponds to the degree of difference between the combined frames.

[0009]

The correlation between any two consecutive frames in a moving image including a transition part is
Regardless of the position of the frame in the moving image, it is often sufficiently higher than the correlation between the two frames that sandwich the cut point. Therefore, it is difficult to detect the transition part based on the magnitude relationship between the correlation between two consecutive frames and the threshold value, using the devices for detecting the cut points disclosed in the above-mentioned publications.

Further, in the apparatus using the method disclosed in Japanese Patent Laid-Open No. 7-288840, the plurality of processings applied to the moving images detect only the moving image converting sections respectively using the transition effects of different kinds. Therefore, if there are a plurality of types of moving picture conversion units to be detected, the apparatus needs to perform a process specific to the detection of each type of moving picture conversion unit on each moving picture. As a result, the number of processes to be performed on a moving image increases as the types of moving image converting units to be detected increase, and thus the entire process related to detecting a moving image converting unit becomes complicated. Further, since the apparatus needs to be provided with each of the means for the plurality of processes, the apparatus becomes complicated.

Further, in the apparatus disclosed in Japanese Patent Laid-Open No. 8-214210, at the time of creating the table of the inter-image distances between all the frames in the section, at least when the section is shifted by one frame in the moving image, It is necessary to newly calculate as many inter-image distances as there are frames in the section. Therefore, the device takes a very long time to perform the calculation regarding the detection of the moving image conversion unit.

An object of the present invention is to store a moving picture conversion unit detecting apparatus and method and a moving picture conversion unit detection control program capable of easily detecting a moving picture conversion unit using the transition effect regardless of the type of the transition effect. And a moving image processing device using the moving image conversion unit detection device.

[0013]

According to a first aspect of the present invention, a moving image composed of a plurality of shots should be stored, and each shot should have the same or similar features and be sequentially displayed over time. A moving image storage unit each composed of a plurality of frames, each frame being divided into one or a plurality of regions, and at least one color representing each region in each frame and the appearance of the color in each region. And a moving image, which detects the frequency and outputs the combination of the detected color and the appearance frequency of all the regions in each frame as the characteristic data indicating the characteristic of each frame. A plurality of second frames each of which is separated from the first frame by a plurality of predetermined reference numbers of the first frame, which is one of the frames, in the display order of the frames in the moving image. And a plurality of dissociation frame dissimilarities representing the degree of dissimilarity between the dissimilarity frames and the dissimilarity dissimilarity calculation means for calculating the dissimilarity dissimilarity dissimilarities between the first and second frames based on the characteristic data, The display order in the moving image is adjacent based on the characteristics detected by the characteristics detection unit and the characteristics detection unit that respectively detect the characteristics of the change in the dissociation degree between separated frames corresponding to the second frame according to the display order. It is a moving picture scene change part detection device characterized by including a transition detection means for detecting, from within a moving picture, a combined frame having a transition effect in which power frames are overlapped and combined.

According to the first aspect of the present invention, the moving picture scene change section detecting device detects a characteristic of change based on a display order of the dissimilarity between the separated frames, and based on the characteristic, the moving characteristic is detected from within the moving picture. Detect composite frame. As a result, the device can detect the scene change part using the transition effect in the composite frame, that is, the moving image, regardless of the type of the transition effect. Further, the apparatus can reduce the amount of calculation for detecting the scene change portion to be smaller than the amount of calculation of the apparatus for detecting the scene change portion of the prior art, and the accuracy of the detection It can be improved over prior art devices. Further, the moving picture scene change unit detection device further includes the feature data generation means, and the separation frame dissimilarity degree calculation means operates as described above. The frequency of appearance of a color in a single region is equivalent to the area of that color portion in the region. As a result, the difference between the separated frames is 2
It does not change to hypersensitivity or insensitivity in response to the change in the difference between the two frames, and appropriately responds to the change. As a result, the feature of the change of the detached frame is less likely to collapse, and therefore, the moving image scene conversion unit detection device of the first invention can further improve the detection accuracy of the scene conversion unit using the transition effect.

A second aspect of the present invention stores a moving image composed of a plurality of shots, and each shot is composed of a plurality of frames which have the same or similar characteristics and which should be sequentially displayed with the passage of time. And a plurality of second frames that are separated from the first frame by a plurality of reference numbers that are predetermined in the display order of the frames in the moving image of the first frame that is any one frame in the moving image. Dissociation-frame dissimilarity calculation means for respectively calculating a plurality of dissociation-frame dissimilarities, each of which represents the degree of difference in characteristics from the frame, by shifting the first frame.
The change pattern associated with the display order of the dissociation degrees between the separated frames of the plurality of frames in which the display order is continuous determines whether or not each of the change patterns corresponds to a plurality of predetermined reference patterns, and The display order in the moving image should be adjacent based on the characteristics detected by the characteristics detection unit that detects the characteristics of the change in the dissimilarity between separated frames corresponding to two frames according to the display order, and the characteristics detected by the characteristics detection unit. A transition detection unit that detects a composite frame having a transition effect, which is composed by overlapping frames, in a moving image, and compares the composite frame with a change pattern of the dissimilarity between dissociated frames of any one of the reference patterns. The first reference pattern to be done,
And the second reference pattern to be compared with the change pattern of the dissimilarity between dissimilarity frames, the reference number of which is larger than the number of dissimilarity between dissociated frames at the time of calculation,
And that the shape of the graph showing the second reference pattern is convex, and the height and width of the graph of the first reference pattern are less than the height and width of the graph of the second reference pattern, respectively. It is a featured moving image scene transition detection device.

According to the second aspect of the invention, the moving picture scene change section detecting device detects a characteristic of change based on a display order of the dissimilarity between the separated frames, and based on the characteristic, the moving characteristic is detected from within the moving picture. Detect composite frame. As a result, the device can detect the scene change part using the transition effect in the composite frame, that is, the moving image, regardless of the type of the transition effect. Further, the apparatus can reduce the amount of calculation for detecting the scene change portion to be smaller than the amount of calculation of the apparatus for detecting the scene change portion of the prior art, and the accuracy of the detection It can be improved over prior art devices. Further, in the moving picture scene change unit detection device, the characteristic detection means operates as described above. As a result, the characteristic detection unit outputs, as the characteristic, a comparison result of the change pattern of the dissimilarity between the separated frames and the plurality of reference patterns. That is, the moving image scene change unit detection device of the second invention can easily detect the scene change unit from the moving image based on the change patterns of the dissimilarity degrees between the separated frames.

A third aspect of the present invention stores a moving image composed of a plurality of shots, and each shot is composed of a plurality of frames having the same or similar features and which should be sequentially displayed over time. And a plurality of second frames, which are separated from the first frame by a plurality of reference numbers, which are predetermined in the display order of the frames in the moving image, of the first frame which is any one frame in the moving image. A separation frame difference degree calculation means for calculating a plurality of separation frame difference degrees, each of which indicates a degree of difference in characteristics from a frame, by shifting the first frame;
Approximate expression calculating means for respectively obtaining a mathematical expression of a graph that approximates the change pattern of the dissociation degree between the separated frames of the plurality of frames in which the display order is continuous, and based on the parameters determined based on the obtained plurality of mathematical expressions ,
It is determined whether or not the change pattern approximated by the graph of each equation matches each of the reference patterns, and the change degree according to the display order of the dissociation degree between the separated frames corresponding to each of the second frames of the frame is determined. Based on the characteristics detected by the characteristics detecting unit and the characteristics detected by the characteristics detecting unit, a composite frame having a transition effect, which is composed by overlapping frames whose display order in the moving image should be adjacent to each other, is displayed in the moving image. And a transition detecting unit for detecting the transition from the moving image scene changing unit.

According to the third aspect of the invention, the moving picture scene change section detecting device detects the characteristic of the change based on the display order of the dissimilarity between the separated frames, and based on the characteristic, the characteristic of the change is detected from the moving picture. Detect composite frame. As a result, the device can detect the scene change part using the transition effect in the composite frame, that is, the moving image, regardless of the type of the transition effect. Further, the apparatus can reduce the amount of calculation for detecting the scene change portion to be smaller than the amount of calculation of the apparatus for detecting the scene change portion of the prior art, and the accuracy of the detection It can be improved over prior art devices. Further, the moving image scene change unit detection device further includes the approximate expression calculation unit, and the characteristic detection unit is configured to operate as described above. As a result, the change pattern of the dissimilarity between dissociated frames and the reference pattern can be robustly compared with the variation of the dissimilarity between separated frames caused by noise in each frame. As a result, the moving image scene change unit detection device according to the third aspect of the present invention can further improve the detection accuracy of the scene change unit using the transition effect.

The moving picture scene change portion detecting device of the fourth invention is
The mathematical formula is a quadratic function, and the parameter is a coefficient of the calculated quadratic function.

According to a fourth aspect of the present invention, in the moving image / scene changing portion detecting device of the third aspect of the invention, the mathematical formulas and parameters are as described above. Since the graph of the quadratic function has one convex portion, it is easy to approximate the reference pattern. Further, since the formula of the quadratic function has a simple structure, it is easy to extract parameters relating to the center, width, position and height of the convex portion of the graph. As a result, the characteristic detecting means in the apparatus of the fourth aspect of the invention can easily compare the change patterns with the reference patterns.

The moving picture scene change unit detecting device of the fifth invention is
The formula is a function including a sine function, and the parameters are a phase and a gain of the sine function in the calculated function.

According to a fifth aspect of the present invention, in the moving image scene changing portion detecting device of the third aspect of the invention, the mathematical formulas and parameters are as described above. Since the half cycle portion of the graph of the function including the sine function has one convex portion, it is easy to approximate the reference pattern. Further, since the formula of the function has a simple structure, it is easy to extract the parameters regarding the center, width, position and height of the convex portion of the graph. As a result, the characteristic detecting means in the apparatus of the fifth aspect of the present invention can easily compare the change patterns with the reference patterns.

According to a sixth aspect of the present invention, a moving image composed of a plurality of shots is stored, and each of the shots is composed of a plurality of frames having the same or similar characteristics and which should be sequentially displayed over time. And a plurality of second frames, which are separated from the first frame by a plurality of reference numbers, which are predetermined in the display order of the frames in the moving image, of the first frame which is any one frame in the moving image. A separation frame difference degree calculation means for calculating a plurality of separation frame difference degrees, each of which indicates a degree of difference in characteristics from a frame, by shifting the first frame;
Based on the characteristics detected by the characteristics detection unit and the characteristics detected by the characteristics detection unit, the characteristics of the change in the dissimilarity between separated frames corresponding to the respective second frames of the frame in the display order are detected. Transition detection means for detecting a composite frame having a transition effect, which is composed by overlapping frames whose display order should be adjacent to each other, and a transition detection means for each frame in the video, and only one display order of each frame. The adjacent frame dissimilarity calculating means for calculating the dissimilarity between adjacent frames indicating the difference between the frames in different display orders and the respective frames, and the dissimilarity between adjacent frames of each frame, The degree of difference between adjacent frames of the remaining frames excluding the respective frames of the plurality of frames including the consecutive display order is determined in advance. An adjacent frame difference correction unit that subtracts the sum of products of a plurality of weighting factors, and a frame in which the difference between adjacent frames after the subtraction is equal to or more than a predetermined cut point detection threshold value is detected from the moving image. Based on the detected frame and the frame detection means,
A cut point detecting unit that detects a cut point in the moving image, and each weighting coefficient to be multiplied by the dissimilarity between the adjacent frames of each of the residual frames is calculated from among the dissimilarities between the adjacent frames of the residual frames. The moving image scene change unit detection device is characterized in that the difference between adjacent frames is smaller than the difference between adjacent frames and is greater than or equal to a weighting coefficient.

According to the sixth aspect of the present invention, the moving picture scene change section detection device detects a change characteristic based on the display order of the dissimilarity degrees between the respective separated frames, and based on the characteristic, the change is detected from within the moving picture. Detect composite frame. As a result, the device can detect the scene change part using the transition effect in the composite frame, that is, the moving image, regardless of the type of the transition effect. Further, the apparatus can reduce the amount of calculation for detecting the scene change portion to be smaller than the amount of calculation of the apparatus for detecting the scene change portion of the prior art, and the accuracy of the detection It can be improved over prior art devices. Further, the moving image / scene changing unit detecting device is configured to include the above-mentioned means relating to the cut point detection. By the correction, the component in each adjacent frame difference degree due to the difference between successive frames in each shot due to the movement of the camera and the subject at the time of shooting each shot in the moving image is reduced. As a result, the moving image / scene changing portion detection device according to the sixth aspect of the present invention can improve the detection accuracy of the cut point.

A seventh aspect of the present invention stores a moving image composed of a plurality of shots, and each of the shots is composed of a plurality of frames having the same or similar features and which should be sequentially displayed with the passage of time. And a plurality of second frames, which are separated from the first frame by a plurality of reference numbers, which are predetermined in the display order of the frames in the moving image, of the first frame which is any one frame in the moving image. A separation frame difference degree calculation means for calculating a plurality of separation frame difference degrees, each of which indicates a degree of difference in characteristics from a frame, by shifting the first frame;
Based on the characteristics detected by the characteristics detection unit and the characteristics detected by the characteristics detection unit, the characteristics of the change in the dissimilarity between separated frames corresponding to the respective second frames of the frame in the display order are detected. Transition detection means for detecting a composite frame having a transition effect, which is composed by overlapping frames whose display order should be adjacent to each other, and a transition detection means for each frame in the video, and only one display order of each frame. Adjacent frame dissimilarity calculation means for respectively calculating dissimilarity between adjacent frames indicating different features of frames in different display orders, and cut inspection in which the dissimilarity between adjacent frames is predetermined from within the moving image. Of all the frames that are equal to or more than the threshold for use, the separation of any one of the plurality of frames including the frames and having a continuous arrangement order is performed. It is characterized by including frame detection means for detecting frames whose inter-frame dissimilarities are all equal to or greater than a predetermined threshold value, and cut point detection means for detecting cut points in the moving image based on the detected frames. It is a moving picture scene change part detection device.

According to the seventh invention, the moving picture scene change section detecting device detects the characteristic of the change based on the display order of the dissimilarity between the separated frames, and based on the characteristic, the characteristic of the change is detected from within the moving picture. Detect composite frame. As a result, the device can detect the scene change part using the transition effect in the composite frame, that is, the moving image, regardless of the type of the transition effect. Further, the apparatus can reduce the amount of calculation for detecting the scene change portion to be smaller than the amount of calculation of the apparatus for detecting the scene change portion of the prior art, and the accuracy of the detection It can be improved over prior art devices. Further, the moving image / scene changing unit detecting device is configured to include the above-mentioned means relating to the cut point detection. As a result, when the difference between adjacent frames of two consecutive frames in each shot in the moving image is equal to or more than the cut point detection threshold value, the frame detection unit includes a plurality of consecutive frames including the two frames. Whether or not to detect the two frames is determined according to the magnitude relationship between the dissociation between frames and the threshold value. The increase in the dissociation frame dissimilarity occurs, for example, when the flash is hit only when one of two consecutive frames in a single shot is shot. As a result, the moving picture scene change unit detection device of the seventh aspect of the present invention can prevent erroneous detection of the cut point due to the influence of the flash, so that the detection accuracy of the cut point can be improved.

According to an eighth aspect of the present invention, a display means capable of displaying a frame, the moving image scene changing portion detecting device according to any one of claims 1 to 7, and the moving image scene changing portion detecting device are detected. Based on the position of the composite frame in the video,
A moving image processing apparatus comprising: a frame extracting unit that extracts at least one frame from each shot in the moving image; and a display control unit that displays the extracted frame on the display unit. is there.

According to an eighth aspect of the present invention, the moving image processing apparatus includes the moving image scene change unit detecting device according to any one of the first to seventh aspects of the invention. As a result, the moving picture processing device can automatically recognize each shot in the moving picture based on the scene change section detected by the detection device, and extract and display the frame of each shot. You can As a result, the moving image processing apparatus can reduce the load required to detect the scene change unit and improve the moving image processing accuracy when processing a moving image in shot units.

In a ninth aspect of the invention, the display control means displays all the extracted frames on the display means in order from the earliest display order in the moving image with the lapse of time. It is characterized by

According to a ninth invention, in the moving picture processing device of the eighth invention, the display control means operates as described above. As a result, the moving image processing apparatus of the ninth invention displays on the display means a moving image composed of the plurality of extracted frames, that is, a moving image for quick viewing of the original moving image from which the frames are extracted. You can As a result, the operator of the moving picture processing apparatus can grasp the contents of the original moving picture properly and in a short time.

In the moving image processing apparatus of the tenth invention, the display control means reduces the extracted frames, and causes the display means to display a combined frame obtained by combining the reduced frames. Characterize.

According to a tenth invention, in the moving picture processing device of the eighth invention, the display control means operates as described above. As a result, the moving image processing apparatus according to the tenth aspect of the invention is to combine the extracted reduced frames of the plurality of frames, that is, a frame for listing each shot of the original moving image from which the frames are extracted,
It can be displayed on the display means. As a result, the operator of the moving picture processing apparatus can grasp the contents of the original moving picture properly and in a short time.

An eleventh aspect of the invention is a process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed with the passage of time. Each frame is divided into one or a plurality of regions, at least one color representing each region in each frame and the appearance frequency of the color in each region are detected, and each frame is detected. A process of outputting a combination of detected colors and appearance frequencies of all regions as feature data indicating the feature of each frame, and a process of outputting one of the frames in the moving image.
A plurality of dissociation frame-to-frame dissimilarity degrees, each of which represents the degree of difference in characteristics between a plurality of reference frames that are predetermined from a plurality of reference number of frames in the display order of the frames in the moving image, A process of shifting the first frame based on the characteristic data to perform a calculation, and a process of detecting a change characteristic of the dissimilarity between separated frames corresponding to each of the second frames in the display order. And, based on the detected characteristics, a process of detecting, from within the moving image, a combined frame having a transition effect in which frames that should be adjacent in the display order in the moving image overlap and are combined. This is a scene change part detection method. According to an eleventh aspect, in the moving picture scene change section detecting method, a characteristic of change based on a display order of the plurality of dissociation degrees between the separated frames is detected, and based on the characteristic, the composite frame is detected from within the moving picture. To detect. As a result, when the method is used, the scene change part using the transition effect in the composite frame, that is, the moving image can be detected regardless of the type of the transition effect. Further, when the method is used, the amount of calculation for detecting the scene change portion can be made smaller than that of the method for detecting the conventional scene change portion, and the accuracy of the detection can be improved. It can be improved over the conventional method. Further, the moving image scene change unit detection method further includes a process of outputting the characteristic data, and based on the characteristic data,
The dissimilarity between separated frames is calculated. The frequency of appearance of a color in a single region is equivalent to the area of that color portion in the region. As a result, the dissimilarity between the separated frames does not change to hypersensitivity and insensitivity in response to the change in the difference between the two frames, and appropriately responds to the change. As a result, since the characteristic of the change of the detached frame is less likely to be broken, the moving image scene changing portion detection method of the eleventh invention is
The detection accuracy of the scene change part using the transition effect,
It can be further improved.

A twelfth aspect of the invention is a process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed over time, A plurality of reference numbers of the first frames, which is one of the frames in the moving image, are displayed in the display order of the frames in the moving image.
A process of calculating a plurality of dissociation degrees between separated frames, each of which represents a degree of difference in characteristics from a plurality of second frames respectively separated from the frame, respectively, by shifting the first frame, and a plurality of consecutive display orders. Between the separated frames corresponding to each of the second frames of the frames, it is determined whether or not a change pattern of the dissimilarity between the separated frames of the frames according to the display order matches a plurality of predetermined reference patterns. Based on the process of detecting the respective characteristics of the change of the dissimilarity according to the display order and the detected characteristics, a composite frame having a transition effect in which the frames that should be adjacent in the display order in the moving image are composited is created. , A process of detecting from within the moving image, of any one of the reference patterns, First reference pattern to be compared with the standardized pattern, and a second reference pattern to be compared with the change pattern of the dissociation frame dissimilarity in which the reference number is larger than that of any one of the dissociation frame dissimilarities at the time of calculation. Indicates that the shapes of the graphs showing the first and second reference patterns are convex, and the height and width of the graph of the first reference pattern are the height and width of the graph of the second reference pattern, respectively. It is a method for detecting a moving image scene change portion characterized by being less than. According to a twelfth aspect, in the moving picture scene change section detecting method, a characteristic of change based on a display order of the dissimilarity degrees between the plurality of separated frames is detected, and based on the characteristic, the composite frame is detected from within the moving picture. To detect. As a result, when the method is used, the scene change part using the transition effect in the composite frame, that is, the moving image can be detected regardless of the type of the transition effect. Further, when the method is used, the amount of calculation for detecting the scene change portion can be made smaller than that of the method for detecting the conventional scene change portion, and the accuracy of the detection can be improved. It can be improved over the conventional method. In the process of detecting the change characteristics based on the display order of the dissimilarity degrees between the plurality of separated frames in the moving image scene transition detection method, the dissimilarity degrees between the separated frames of the plurality of frames in the display order are detected. It is adapted to judge whether or not the change patterns associated with the display order of No. 1 match with a plurality of predetermined reference patterns. As a result, in the process of detecting the characteristics of changes based on the display order of the plurality of dissociation frames between dissimilarity frames, the result of comparison between the change patterns of the dissociation frames between dissociation frames and the plurality of reference patterns is used as the characteristics. Output. That is, the moving image scene changing portion detecting method of the twelfth invention is
The scene change unit can be easily detected from the moving image based on the change pattern of the dissimilarity degrees between the separated frames.

A thirteenth aspect of the invention is a process of storing a moving image composed of a plurality of shots each having a plurality of frames which have the same or similar characteristics and are to be sequentially displayed over time, A plurality of reference numbers of the first frames, which is one of the frames in the moving image, are displayed in the display order of the frames in the moving image.
A process of calculating a plurality of dissociation degrees between separated frames, each of which represents a degree of difference in characteristics from a plurality of second frames respectively separated from the frame, respectively, by shifting the first frame, and a plurality of consecutive display orders. Based on the process of obtaining the formulas of the graphs that respectively approximate the change patterns of the dissociation degrees between the frames and the parameters determined based on the obtained plurality of formulas, the changes approximated by the graphs of the formulas A process of determining whether or not a pattern matches each of the reference patterns and detecting a characteristic of a change in the dissimilarity between separated frames corresponding to each of the second frames in the display order, Based on the detected characteristics, it has a transition effect in which frames that should be adjacent in the display order in the moving image are overlapped and combined. The formed frame, which is a moving scene change portion detection method which comprises a process of detecting from the video. According to a thirteenth aspect of the present invention, in the method of detecting a moving image scene change portion, a characteristic of a change based on a display order of the dissimilarity degrees between the plurality of separated frames is detected, and based on the characteristic, the composite frame is detected from within the moving image. To detect. As a result, when the method is used, the scene change part using the transition effect in the composite frame, that is, the moving image can be detected regardless of the type of the transition effect. Further, when the method is used, the amount of calculation for detecting the scene change portion can be made smaller than that of the method for detecting the conventional scene change portion, and the accuracy of the detection can be improved. It can be improved over the conventional method. Further, in the moving picture scene transition part detecting method, the method further includes a step of obtaining a mathematical expression of a graph that respectively approximates a change pattern of the dissimilarity between separated frames of a plurality of frames in which the display order is continuous, and each of the frames. In the process of respectively detecting the characteristics of the change in the dissociation degree between separated frames corresponding to the second frame in the display order, approximation is made with a graph of each mathematical expression based on the parameters determined based on the plurality of mathematical expressions obtained. It is adapted to judge whether or not the change pattern to be formed matches each of the reference patterns. As a result, the change pattern of the dissimilarity between dissociated frames and the reference pattern can be robustly compared with the variation of the dissimilarity between separated frames caused by noise in each frame. As a result, the moving image scene change part detection method of the thirteenth invention can further improve the detection accuracy of the scene change part using the transition effect.

In the moving image scene changing portion detecting method of the fourteenth invention, the mathematical expression is a quadratic function, and the parameter is
It is characterized in that it is a coefficient of the calculated quadratic function. According to a fourteenth aspect of the present invention, in the moving image scene change portion detection method according to the thirteenth invention, the mathematical formula and the parameters are as described above. Since the graph of the quadratic function has one convex portion, it is easy to approximate the reference pattern. Further, since the formula of the quadratic function has a simple structure, it is easy to extract parameters relating to the center, width, position and height of the convex portion of the graph. As a result, in the process of detecting the characteristic in the method of the fourteenth aspect, the change patterns can be easily compared with the reference patterns.

In the moving picture scene change part detecting method of the fifteenth invention, the mathematical expression is a function including a sine function, and the parameters are a phase and a gain of the sine function in the calculated function. And According to a fifteenth aspect of the present invention, in the moving picture scene change portion detection method of the thirteenth invention, the formula and the parameters are as described above. Since the half cycle portion of the graph of the function including the sine function has one convex portion, it is easy to approximate the reference pattern. Also, since the formula of the function is simple in construction, the center of the convex portion of the graph, the width,
It is easy to extract parameters related to position and height. As a result, in the process of detecting the characteristic in the method of the fifteenth invention, it is possible to easily compare the change patterns with the reference patterns.

A sixteenth aspect of the invention is a process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed with the passage of time. A plurality of reference numbers of the first frames, which is one of the frames in the moving image, are displayed in the display order of the frames in the moving image.
A process of calculating a plurality of dissimilarity degrees between separated frames, each of which represents a degree of difference in characteristics from a plurality of second frames separated from the frame, by respectively shifting the first frame, and each of the second frames of the frame. A process of detecting a characteristic of a change in the dissociation degree between disjoint frames corresponding to the display order, and a transition synthesized based on the detected characteristic, in which frames that should be adjacent in the display order in the moving image overlap each other. A process of detecting a composite frame having an effect from a moving image, and for each frame in the moving image, a display order different from the display order of each frame and a difference in characteristics between the respective frames are shown. Based on the processing of calculating the dissimilarity between adjacent frames and the dissimilarity between adjacent frames of each frame, the display order including each frame is continuous. A process of subtracting the sum of products of the difference between adjacent frames of the remaining frames excluding each frame of the plurality of frames and a plurality of predetermined weighting factors, and the difference between adjacent frames after the subtraction are A frame having a predetermined cut point detection threshold value or more is detected from within the moving image, and a process of detecting a cut point within the moving image based on the detected frame is included, and the remaining frames are adjacent to each other. Each of the weighting factors to be multiplied to the inter-frame dissimilarity is equal to or more than the weighting factor to be multiplied to the adjacent frame dissimilarity smaller than the adjacent frame dissimilarity among the adjacent frame dissimilarities of the residual frame. A method for detecting a moving image scene transition part, which is characterized in that: First
According to a sixth aspect of the present invention, in the moving picture scene change section detecting method, a characteristic of a change based on a display order of the dissimilarity degrees between the plurality of separated frames is detected, and based on the characteristic, the composite frame is extracted from the moving picture. To detect. As a result, when the method is used, the scene change part using the transition effect in the composite frame, that is, the moving image can be detected regardless of the type of the transition effect. Further, when the method is used, the amount of calculation for detecting the scene change portion can be made smaller than that of the method for detecting the conventional scene change portion, and the accuracy of the detection can be improved. It can be improved over the conventional method. Further, the moving picture scene change portion detection method is configured to include the above-mentioned processing relating to cut point detection. By the correction, the component in each adjacent frame difference degree due to the difference between successive frames in each shot due to the movement of the camera and the subject at the time of shooting each shot in the moving image is reduced. As a result, the moving-image / scene-changing-portion detecting method according to the sixteenth aspect of the present invention can improve the accuracy of detecting cut points.

A seventeenth aspect of the present invention is a process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed with the passage of time. A plurality of reference numbers of the first frames, which is one of the frames in the moving image, are displayed in the display order of the frames in the moving image.
A process of calculating a plurality of dissimilarity degrees between separated frames, each of which represents a degree of difference in characteristics from a plurality of second frames separated from the frame, by respectively shifting the first frame, and each of the second frames of the frame. A process of detecting a characteristic of a change in the dissociation degree between disjoint frames corresponding to the display order, and a transition synthesized based on the detected characteristic, in which frames that should be adjacent in the display order in the moving image overlap each other. A process of detecting a composite frame having an effect from a moving image, and for each frame in the moving image, a display order different from the display order of each frame and a difference in characteristics between the respective frames are shown. The process of calculating the dissimilarity between adjacent frames is performed, and if the dissimilarity between adjacent frames is equal to or greater than a predetermined cut point detection threshold value from within the moving image. Out of the frame, one of each of a plurality of frames arranged order include respective frames are consecutive 1
A moving image scene characterized by including a process of detecting frames in which the dissimilarity between two separated frames is all equal to or greater than a predetermined threshold value, and a process of detecting a cut point in the moving image based on the detected frames. It is a conversion part detection method. According to a seventeenth aspect of the present invention, in the moving picture scene change section detection method, a characteristic of change based on a display order of the dissimilarity degrees between the plurality of separated frames is detected, and based on the characteristic, the composite frame is included in the moving picture. To detect. As a result, when the method is used, the scene change part using the transition effect in the composite frame, that is, the moving image can be detected regardless of the type of the transition effect. Further, when the method is used, the amount of calculation for detecting the scene change portion can be made smaller than that of the method for detecting the conventional scene change portion, and the accuracy of the detection can be improved. It can be improved over the conventional method. Further, the moving picture scene change portion detection method is configured to include the above-mentioned processing relating to cut point detection. As a result, when the dissimilarity between adjacent frames of two consecutive frames in each shot in the moving image is equal to or higher than the cut point detection threshold value, the dissimilarity between separated frames of a plurality of consecutive frames including the two frames. Whether or not to detect the two frames is determined according to the magnitude relationship between the degree and the threshold value. The increase in the dissociation frame dissimilarity occurs, for example, when the flash is hit only when one of two consecutive frames in a single shot is shot. As a result, the moving image scene changing portion detection method according to the seventeenth aspect of the present invention can prevent erroneous detection of the cut point due to the influence of the flash, thereby improving the detection accuracy of the cut point.

An eighteenth invention is a medium for storing a moving picture scene changing portion detection control program for causing a computer to execute the moving picture scene changing portion detecting method according to any one of claims 11 to 17. According to the eighteenth invention,
The medium includes the moving image / scene changing portion detection control program. When the program is installed in a computer and executed, the computer operates in the same manner as the moving picture scene change unit detection device of any one of the first to seventh inventions. As a result, the computer can detect the scene change portion using the transition effect in the moving image regardless of the type of the transition effect, and the computer, like the device according to the first to seventh inventions, and It is possible to reduce the amount of calculation required to detect the scene change unit and improve the accuracy of the detection.

[0041]

[0042]

FIG. 1 is a block diagram showing a functional configuration of a moving picture / scene changing portion detecting device 50 according to a first embodiment of the present invention. The moving image scene transition detection device (hereinafter, abbreviated as “first transition detection device”) 50 of the first embodiment functionally includes a frame expansion unit 51, a frame storage unit 52, and an in-frame feature extraction. Unit 53, in-frame feature storage unit 54, dissimilarity calculation unit 55, dissimilarity storage unit 5
6, a characteristic shape detection unit 57, a cut detection unit 58, a detection result output unit 59, and a control circuit 60. Control circuit 60
Controls the operation timing of the other units 51 to 59 in the first conversion unit detection device 50.

It is assumed that the moving image to be processed, that is, the target moving image, is composed of a plurality of shots sequentially connected through the scene changing unit, as in the case described with reference to FIG. Each of the shots is a set of one or a plurality of frames to be sequentially displayed with the passage of time. The scene change part in the target moving image may be the cut point described in FIG. 23A, and may be the transition part described in FIGS. 23B and 23C, that is, for the shot conversion using the transition effect. It may be a set of composite frames. Each frame corresponds to a still image for composing a moving image. Data indicating each frame of the target moving image having such a configuration is sequentially input to the first conversion unit detection device 50 in accordance with a predetermined display order over time.

The data indicating each frame of the inputted target moving image is given to the frame expanding section 51 in the given order. The frame expansion unit 51 converts the data form of each frame into a form suitable for processing in the conversion unit detection device 31. In the following explanation,
It is assumed that the frame indicated by the data after the form conversion is a still image configured by arranging a plurality of pixels in a matrix, and the data of each frame after the form conversion includes all the pixels forming the frame. It is assumed that it is a set of pixel data sets respectively indicating the color and the luminance, that is, so-called grid data.

That is, for example, the frame expanding section 51
If the data of each frame at the time of input is analog data, the data is converted from analog to digital and then converted to grid data, and if the data of each frame at the time of input is compressed digital data, After expanding the data, it is converted to grid data. In the first conversion unit detection device 31, each frame in the target moving image is handled in the form of the grid-shaped data, and the target moving image is handled as a set of grid-shaped data of all the frames forming the moving image. In the following description, they will be referred to as "frames" and "target moving images" regardless of the form handled.

Each frame in the target moving image after the form conversion is
The frames are sequentially stored in the frame storage unit 52. Hereinafter, of all the frames forming the target moving image, the n-th frame from the beginning in the display order is referred to as the n-th frame. n
Is an arbitrary integer greater than or equal to 0 and less than the total number N of frames of the target image. That is, it is assumed that the target moving image is composed of the 0th frame to the (N-1) th frame. It is assumed that the frame storage unit 52 can store at least one frame after the form conversion.

The in-frame feature extraction unit 53 obtains the in-frame feature of the frame based on the frame in the frame storage unit 52. In-frame feature Rn of the nth frame
Is a set of quantitative values used to distinguish the nth frame from other frames in the target moving image. The subscript n is an integer of 0 or more and N-1 or less.

The in-frame feature storage unit 54 stores a predetermined number NF of feature buffers BF [0] to BF connected in cascade.
[NF-1] is provided, and one in-frame feature Rn is stored in each feature buffer BF [f]. NF is 1
It is an arbitrary integer of not less than N and not more than N, and f is an arbitrary integer of not less than 0 and not more than NF-1. That is, the in-frame feature storage unit 54 stores at least three feature buffers BF [f].
Equipped with. Each feature buffer BF [f] stores the in-frame feature of the frame in the display order in the target moving image as the subscript f increases.

The dissimilarity calculation unit 55 uses the intra-frame feature storage.
Multiple in-frame features R in section 54 ~ R_{n-NF + 1}Using
Difference between multiple frames for each frame in the target video
Respectively. One of the frames in the nth frame
The inter-arm dissimilarity D (n, n-Lx) is the n-th frame,
A predetermined inter-frame distance Lx from the n-th frame
Each different from each other apart from each other
It is quantified. x is an arbitrary integer of 1 or more
It The greater the difference between frames, the earlier the nth frame
The difference with other frames is great. Interframe distance and
Is the other frame to be compared with the reference nth frame.
It is the difference in the display order from the game. Basic distance between frames
In the transition part you are trying to detect
It may be less than or equal to the total number of frames.
Typically less than the total number of frames that make up a single shot.
It

In the present embodiment, regarding the nth frame,
Dissimilarity between the nth frame and a frame whose display order is earlier than the nth frame by the distances L1 to L3 between the first to third frames, that is, the n-L1, the n-L2, and the n-L3 frames. The first to third frame dissimilarities D (n, n-L1), D (n, n-L2), D
It is assumed that each of (n, n-L3) is obtained. At this time, the first inter-frame distance L1 is 1, the second inter-frame distance L2 is an integer of 2 or more, the third inter-frame difference L3 is larger than the second inter-frame distance L3, and the transition portion is Is less than or equal to the total number of frames. That is, the first frame difference D (n, n-L1)
Is the difference between two consecutive frames in the target video,
That is, it is the degree of difference between adjacent frames, and the second and third
Inter-frame dissimilarity D (n, n-L2), D (n, n-L)
3) is a dissimilarity between two frames arranged with at least one other frame sandwiched in the target moving image, that is, a dissimilarity between separated frames. In the present embodiment, a moving image for 1 second is composed of 30 frames, the transition effect is about 1 second (30 frames), the second inter-frame distance L2 is 10, and the third inter-frame distance L3 is 30. .

The dissimilarity storage unit 56 has the same number of buffer groups each including a predetermined number of dissimilarity buffers connected in cascade as the number of inter-frame dissimilarities for one frame. That is, in the present embodiment, the difference storage unit 5
Reference numeral 6 designates a first to a third buffer group, that is, a first to a third difference buffers BDL1 [0] to BDL1 [ND-1], BDL2 each having a predetermined number ND connected in cascade.
[0] to BDL2 [ND-1], BDL3 [0] to BD
L3 [ND-1], and each difference degree buffer B
DL1 [d], BDL2 [d], and BDL3 [d] are each one 1st to 3rd interframe difference D (n, nL)
1), D (n, n-L2) and D (n, n-L3) are stored. ND is an arbitrary integer of 1 or more and N or less, and d is an arbitrary integer of 0 or more and ND-1 or less. Each of the first to third difference degree buffers BDL1 [d] to BDL3
In [d], the larger the subscript d, the more the degree of difference between the first to third frames of the frames in the display order in the target moving image is stored.

The cut detector 58 at least detects the first interframe difference D of a plurality of frames in the difference storage 56.
Using (n, n-L1), the cut point is detected from within the target moving image. The feature shape detection unit 57 uses at least the second and third inter-frame dissimilarities D (n, n-L2) and D (n, n-L3) of the plurality of frames in the dissimilarity storage unit 56 to determine the target moving image. Detect the transition part from inside.
The detection result output unit 59 uses the cut / feature shape detection unit 5
Based on the detection results of Nos. 8 and 57, it is output as a detection result whether at least one of the cut point or the transition portion, that is, the scene change portion is present in the target moving image.

Generally, when two arbitrary frames are frames within the same shot, the shorter the distance between the two frames is, the smaller the difference between the two frames is, and the distance between the frames is decreased. Is longer, the difference becomes larger. This is for the following reason. In the above case,
The shorter the inter-frame distance is, the shorter the time interval between the photographing times of the two respective frames is, and thus the deviation of the relative positional relationship between the subject and the camera between the two photographing times becomes smaller. As a result, since the subject appears in substantially the same position in each of the two frames, the correlation between the two frames is high. On the contrary, in the above case, the longer the inter-frame distance is, the longer the time interval between the photographing times of the two frames is, and thus the deviation of the positional relationship between the two photographing times becomes large. As a result, the correlation between the two frames is low. The higher the correlation between the two frames,
The difference between the frames between the two frames is low.

Specific transition effects include, for example, dissolve, wipe, slide, push, stretch, and peel. Dissolve causes a subsequent shot to stand out from within the preceding shot when switching between the preceding shot and the following shot. The wipe causes the preceding shot to be wiped off by the succeeding shot when the switching is performed. In the slide, when the switching is performed, the subsequent shot is slid and inserted into the preceding shot. The push inserts the subsequent shot so as to push out the preceding shot at the time of the switching. In the stretch, when the switching is performed, the preceding shot and the succeeding shot are reduced and arranged, and the area ratio of each shot is changed. Switch. In addition to the specific transition effect, for example, one of two shots is three-dimensionally deformed and moved. For wipes, slides, pushes and stretches, the shot may be deformed in any of the top, bottom, left, and right directions of the screen of the display device. You can come from anywhere.

FIG. 2 is a schematic diagram in which a transition part in a target moving image and frames in shots before and after the part are arranged in display order. Two shots SA and SB are connected via a transition part TR composed of a plurality of composite frames. Transition part TR
The frames C (1) to C (NN) in the two frames A and 2 are a plurality of frames A from a frame that is earlier than the end of the preceding shot SA of the two shots by a predetermined number NN to the end frame and 2 A plurality of frames B from the first frame of the succeeding shot SB of the one shot to a frame whose display order is later than the first frame by a predetermined number NN are synthesized based on Expression 1.

C (n) = α (n) B + (1-α (n)) A n = 1,2,3, ..., NN ... (1) 0 <α (n) <1 ... (2) α (N) = n ÷ (NN + 1) (3) In Expression 1, “NN” is the number of frames used in the scene changing part using the transition effect, and the coefficient “α”.
(N) ”is defined by equations 2 and 3. The number NN of frames is not specified, and any numerical value may be used. Specifically, for example, when the transition part uses the dissolve effect, the colors of the corresponding pixels of the frames A and B of the two shots SA and SB described above are mixed according to the relationship of Expression 1. Corresponding pixels refer to pixels that overlap each other when two frames are superposed with their coordinate systems matched.
Further, for example, when the transition portion T uses the wipe effect, the area ratio of the frames A and B of the two shots SA and SB occupying on the screen of the display device changes according to the relation of the expression 1.

The relationship between the transition effect and the dissimilarity between frames in the target moving image, that is, the change in the display order of the dissimilarity between the second and third frames in the present embodiment, and the transition effect will be described below.

FIG. 3 shows the dissimilarities D (n, n-L2) and D (n, n-L3) between the second and third frames of the frames in and near the transition part in the target moving image,
It is a graph plotted according to the display order of the frame. Graphs GD1 and GD2 respectively show changes according to the display order of the dissimilarities D (n, n-L2) and D (n, n-L3) between the second and third frames. In addition, in FIG.
It is assumed that the plurality of frames in the preceding shot and the plurality of frames in the following shot have almost no difference. Further, in the calculation of the dissimilarity between frames, the nth frame to be calculated is referred to as a frame of interest, and the frame whose display order is earlier than the nth frame by the distance between frames is referred to as a comparison frame.

The section WA of the graphs GD1 and GD2
When calculating the dissimilarity between separated frames in 1 and WA2, both the focus and the comparison frames are the frames in the preceding shot. In this case, the dissimilarity between the separated frames remains relatively low, and the change in the dissimilarity between the separated frames in the display order is small. Section WB of graphs GD1 and GD2
1, when calculating the dissimilarity between separated frames in WB2, only the frame of interest is the frame in the transition part,
The comparative frame is the frame in the preceding shot. In this case, as the display order of the frame of interest increases, the frame of interest becomes more distant from the preceding shot. Therefore, the larger the display order of the frame of interest, the larger the dissimilarity between separated frames.

Section WC of graphs GD1 and GD2
When calculating the dissimilarity between disjoint frames in 1 and WC2, both the focus frame and the comparison frame are frames in the transition part, or both frames are frames of two shots sandwiching the transition part.
That is, in the latter case, only the frame of interest is the frame in the preceding shot, and only the comparison frame is the frame in the subsequent shot. In these cases, the dissimilarity between the separated frames maintains a relatively high value, and the change in the dissimilarity in the display order is small.

Section WD of graphs GD1 and GD2
When calculating the dissimilarity between separated frames in 1 and WD2, only the focused frame is the frame in the subsequent shot, and the comparison frame is the frame in the transition part. In this case, as the display order of the frame of interest becomes larger, the frame of interest gets closer to the subsequent shot. Therefore, as the display order of the frame of interest becomes larger, the dissimilarity between separated frames becomes smaller. Sections WE1 and WE2 of the graphs GD1 and GD2
At the time of calculating the dissimilarity between disjoint frames, the focus and comparison frames are both the frames in the subsequent shot. In this case, the dissimilarity between the separated frames remains relatively low, and the change in the dissimilarity between the separated frames in the display order is small.

Further, based on the graphs GD1 and GD2, the start points of the sections WB1 and WB2, that is, the point TB at which the difference between the second and third frames starts to increase, are substantially equal to each other, and the sections WC1, WD1, and WE1 At the beginning,
That is, the graph G of the dissimilarity between the second frames after the increase starts
It can be seen that the start points of the sections WC2, WD2, and WE2, that is, the change points of the graph of the degree of difference between the third frames after the start of increase are later than the change points of D1.
Further, based on the graphs GD1 and GD2, the section W is more likely than the second interframe difference D (n, n-L2) in the section WC1.
It can be seen that the third interframe difference D (n, n−L3) in C2 is larger and the section WC2 is shorter than the section WC1.

That is, the graphs GD1 and GD2 respectively showing the change patterns according to the display order of the second and third frame dissimilarities of the transition part in the target moving image and the frames in the vicinity thereof include the convex parts GP1 and GP2, respectively. And the convex portion GP of both graphs GD1 and GD2
1, the starting points of GP2 are substantially equal to each other, and the convex portion GP1 of the former graph GD1 is smaller in width and height than the convex portion GP2 of the latter graph GP2.

The convex portion in each change pattern of the separation frame dissimilarity may occur due to the relative movement between the camera and the subject at the time of frame photographing. When the convex portion in the change pattern of the dissimilarity between the separated frames is caused by the target frame being the transition portion in the target moving image and a frame in the vicinity thereof, these convex portions Satisfies the above relationship. The relationship of the change patterns of the dissimilarity degree between the plurality of separated frames as described above, regardless of the specific effect of the transition effect used in the transition part,
apply.

FIG. 4 shows the actual first to first frames of the transition portion TR in and around the transition portion TR in the target moving image when the dissolve effect is used as the transition effect.
Dissimilarity between third frames D (n, n-L1), D (n, n)
3 is a graph in which −L2) and D (n, n−L3) are plotted in the display order of the frame. FIG. 5 (A)-
(C) in the graph of FIG. 4, the first to third frame dissimilarity calculation when the target frame is the first frame K0 of the transition part TR, the frame in the transition part TR, and the frame in the subsequent shot It is a schematic diagram for showing the comparative frame for. Figure 5
The number lines in (A) to (C) correspond to the frames in the transition portion TR and in the vicinity thereof arranged in the display order, and each scale of the number line corresponds to each frame.

As shown in the graph GL1, the first frame dissimilarity when the target frame is in the transition portion TR, that is, the change pattern of the adjacent frame dissimilarity, the target frame has the preceding and succeeding shots SA and S.
It is almost equal to the change pattern of the degree of difference between the first frames when it is within B. In the latter case, a slight change in the degree of difference between the first frames occurs due to a change in the relative positional relationship between the subject and the camera at the time of shooting each frame. That is, the first inter-frame dissimilarity maintains substantially the same value regardless of where the frame of interest is in the preceding and subsequent shots and in the transition portion. This is because the scene change part using the transition effect, that is, the transition part gradually changes the shot over a plurality of frames, so that the difference between two consecutive frames is small.

As shown in the graph GL2, the degree of dissimilarity between the second frames gradually increases after the target frame becomes the leading frame K0 of the transition part TR, and the target frame is the frame K1 at the center of the transition part TR. In the above case, it becomes almost maximum, and further decreases from the above case until the frame of interest exceeds the end frame KT of the transition portion TR to become the frame K2 in the subsequent shot. As described above, the change pattern of the second frame dissimilarity includes a convex portion due to at least one of the target frame and the comparison frame being in the transition portion TR. This is for the following reason.

As shown in FIGS. 5 (A) to 5 (C), while the frame of interest changes from the first frame to the central frames K0 to K1 in the transition part TR, the transition part TR is located between the frames of interest and comparison. More frames will be included gradually. Therefore, the influence of the scene change using the transition effect on the degree of difference between the second frames gradually increases, and thus the degree of difference between the second frames gradually increases. When the frame of interest is in the vicinity of the frame K1 in the central portion, the frames between the frames of interest and comparison are all frames within the transition part TR, and therefore the degree of difference between the second frames is maximized. Furthermore, the frame of interest is the central portion within the transition portion TR.
While changing to the frames K1 and K2 in the subsequent shot, the frames in the transition part TR are gradually not included between the focus and comparison frames. Therefore, the influence of the scene change using the transition effect on the degree of difference between the second frames gradually becomes smaller, and the degree of difference between the second frames gradually becomes smaller.

As shown in the graph GL3, the degree of difference between the third frames gradually increases after the time when the target frame becomes the head frame K0 of the transition portion TR,
It becomes almost maximum when the frame of interest becomes the frame K2 in the subsequent shot, and gradually becomes smaller thereafter. As described above, the change pattern of the third inter-frame difference degree includes the convex portion. The convex portion of the third inter-frame difference degree change pattern is similar to the convex portion of the second inter-frame difference degree change pattern, and the former convex portion is larger than the latter convex portion. This is for the following reason.

As shown in FIGS. 5A to 5C, as the inter-frame distance increases, the two shots SA and S in which the frame of interest and the comparison frame sandwich the transition portion TR.
These shots SA, because it is likely to be the frame in B,
The influence of SB conversion has a great influence on the degree of difference between the focus frame and the comparison frame. As a result, the maximum value of the inter-frame dissimilarity of the convex portion increases as the inter-frame distance increases. Further, as shown in FIGS. 5A to 5C, the longer the inter-frame distance, the later the display order of the target frame that may include at least a part of the transition portion TR between the target and comparison frames becomes. As a result, the width of the convex portion increases as the inter-frame distance increases.

Based on the above, the conversion unit detection device 31
Can easily determine whether or not there is a scene change part using the transition effect in the target moving image, based on the shape feature of the change pattern according to the display order of the dissimilarity degrees between the separated frames. Further, the rate of change in the degree of difference between the second and third frames of the convex portion is equal to or greater than the amount of change in the degree of difference between the second and third frames caused by the change in the relative positional relationship during the frame shooting. . Therefore, when the judgment is made based on the geometrical characteristics, the convex portions GP2 and G of the graphs GL2 and GL3 of the dissociation degree between the separated frames are actually used.
Curves CL2 and CL3 obtained by smoothing P3, for example
Based on, it is good to judge the presence or absence of the scene change part using the transition effect.

FIG. 6 shows the actual first to third frame dissimilarity D of the cut points in the target moving image and the frames in the vicinity thereof.
(N, n-L1), D (n, n-L2), D (n, n-)
L3) are graphs GL1 to GL3 plotted according to the display order of the frames. As shown in the graph GL1, the first inter-frame dissimilarity becomes greater than or equal to the predetermined threshold value TC only when the focused frame is the frame NC at the cut point. The threshold value TC is sufficiently larger than the degree of difference between the first frames when the frame of interest is a frame in the preceding and subsequent shots. That is, the conversion unit detection device 31 can detect the cut point in the target moving image based on at least the magnitude relationship between the difference between adjacent frames and the predetermined threshold value TC.

As shown in the graph GL2, the degree of difference between the second frames becomes the maximum value equal to or more than the threshold value TC when the target frame is the frame of the cut point, and the target frame is the frame in the subsequent shot. When the display order of the frame becomes slower, it becomes smaller gradually. As shown in the graph GL3, the third inter-frame dissimilarity becomes the maximum value equal to or more than the threshold value TC when the focused frame is the frame of the cut point, and when the focused frame is the frame in the subsequent shot, A value equal to or greater than the threshold value TC is maintained within a range corresponding to the third inter-frame distance. Note that, for example, the larger the inter-frame distance, the larger the change due to the change in the relative positional relationship between the subject and the camera in the degree of difference between two frames in a single shot. The dissimilarity may not increase rapidly at the cut point NC at the same time.

FIG. 7 is a flow chart for explaining the main operation of the scene change part detection processing in the first change part detection device 50. For example, when at least one frame of the target moving image after the form conversion is stored in the frame storage unit 52, the process proceeds from step A1 to step A2. Further, it is assumed that the frame expansion unit 51 sequentially performs frame form conversion of the target moving image in parallel with the processing of steps A1 to A13.

In step A2, the control circuit 60 defines a counter that serves as a standard for defining the display order of the input frames, that is, a target frame to be processed for the time being, of all the frames in the target moving image. Counter n
Is initialized to 0, the control circuit 60 executes step A3
Then, set the scene change flag FC indicating the presence or absence of the scene change unit to "F.
Set to "ALSE". The scene change flag FC set to FALSE indicates that the scene change section is not detected based on the frame of interest.

In step A4, the control circuit 60 gives the in-frame feature extraction section 53 the frame of interest defined by the counter n, ie, the nth frame, out of the frames in the frame storage section 52. In-frame feature extraction unit 53
In step A4, the in-frame feature Rn of the focused frame is extracted based on the focused frame. The in-frame feature Rn of the frame of interest may be, for example, a histogram showing the frequency of appearance of pixels relating to the brightness or color of the frame of interest. It may be the sum total.

After the extraction, the control circuit 60 executes step A5.
Then, the in-frame feature Rn of the extracted target frame is
It is stored in the in-frame feature storage unit 54. In particular,
The in-frame feature extraction unit 53 causes the latest in-frame feature R _n
Then, the in-frame feature storage unit 54 determines that the in-frame feature R in the feature buffer BF [NF-1] at the end.
_{n-1- (NF-1)} is discarded, and the in-frame features R _{n-1 to} R _{n-1 in} each of the feature buffers BF [0] to BF [NF-2] one before the beginning to the end. _{- (NF-2)} were respectively transferred in front of the feature buffer BF [1] ~BF [NF- 1] of the respective feature buffer, the start the latest frame in the feature R _n in the feature buffer BF [0] of Remember. As a result, as shown in Table 1, the intra-frame characteristic storage unit 54 stores the n-NF + 1 to
The n-th frame in the feature of the frame R _{n- (NF-1)} for storing to R _n.

[0078]

[Table 1]

The in-frame feature storage unit 54 stores at least the in-frame feature R of a plurality of frames from the frame of interest to the frame whose display order is old by the third inter-frame distance L3.
Remember _n-L3 . This is when the calculation of inter-frame dissimilarity in difference calculating unit 55, the frame in the feature R _n-L1 old frame display order by interframe distance L1~L3 than interest frame, R _n-L2, R _{n- This is because L3} is needed.

After storing, the control circuit 60 controls the dissimilarity calculator 5
5, based on the stored contents of the in-frame feature storage unit 54,
The degree of difference D (n, n) between the first to third frames of the frame of interest
-L1), D (n, n-L2), D (n, n-L3) are obtained. That is, the dissimilarity calculation unit 55
In-frame feature Rn in the first feature buffer BF [0]
And the first to third inter-frame distance L1 from the head buffer
~ Feature buffer BF [L [L3 that has the same number of subscripts as L3]
1], BF [L2], BF [L3], the degree of difference D (BF [0], B from the intra-frame features Rn _-L1 , Rn _-L2 , Rn _-L3 )
F [L1]), D (BF [0], BF [L2]), D
(BF [0], BF [L3]) is calculated.

For example, if the intra-frame feature of each frame is a histogram of the luminance or color of the frame, the sum of absolute values of the differences of the histogram shown in Expression 4 or the χ of the histogram shown in Expression 5 is used as the inter-frame difference degree. Square test is required. When the intra-frame feature of each frame is obtained based on the above-described reduced frame, the sum of absolute values of differences in color or luminance for corresponding pixels of two frames is obtained as the inter-frame difference degree. In the following equation, “H (n, i)” and “H (x, i)” are
The frequency of appearance of the pixel of the i-th color or luminance in the n-th and x-th frames is shown. x is 0 or more and N or less and n
Is an integer other than. The i-th color or luminance corresponds to the scale on the horizontal axis in the histogram.

D (n, x) = Σ _i | H (n, i) -H (x, i) | (4) D (n, x) = Σ _i | H (n, i) -H ( x, i) | ÷ H (x, i) (5) After the calculation, the control circuit 60 calculates the difference D (n, n) between the first to third frames of the obtained target frame in step A7.
-L1), D (n, n-L2), D (n, n-L3)
Are stored in the first to third buffer groups in the difference storage unit 56, respectively. Specifically, if the dissimilarity calculation unit 55 obtains the first inter-frame dissimilarity D (n, n−L1) of the target frame, the dissimilarity storage unit 56 determines that the dissimilarity at the end of the first buffer group. Dissimilarity D between the first frames in the buffer BDL1 [ND-1] D (n-1- (ND-1), n-1-
(ND-1) -L1) is discarded, and each difference buffer BDL1 [0] to BDL1 [ND one before the beginning to the end
-2] the first inter-frame difference D (n-1, n-1-)
L1) to D (n-1- (ND-2), n-1- (ND-
2) -L1) are respectively transferred to the difference buffers BDL1 [1] to BDL1 [ND-1] in the preceding stage of the respective difference buffers, and the first difference frame BDL1 [0] is used as the first frame of the target frame. Difference D (n, n-L1)
Memorize The specific behavior of each of the difference buffers BDL2 [d] and BDL3 [d] in the second and third buffer groups is as follows.
Equal to [d]. As a result, as shown in Tables 2 to 4, respectively, in the first to third buffer groups, the n- (ND-
1) to the degree of difference between the first to third frames of the nth frame are stored.

[0083]

[Table 2]

[0084]

[Table 3]

[0085]

[Table 4]

After storing, the control circuit 60 causes the step A9
Then, the cut detection unit 58 and the dissimilarity storage unit 56 store the predetermined number B in the display order from the frame of interest based on the stored contents.
It is determined whether or not there is a cut point between the n-BC frame which is earlier by C and the frame adjacent to the frame. Specifically, the cut detection unit 58, as shown in Expression 6, is the BCth difference buffer BDL1 in the first buffer group.
Dissimilarity D between first frames in [BC] (n-BC, n-
BC-L1) exceeds a predetermined cut detection threshold value TC, and as shown in Expression 7, a plurality of difference degree buffers BDL2 from the head to the BCth in the second buffer group
[0] to second frame difference D in BDL2 [BC]
(N, n-L2) to D (n-BC, n-BC-L1)
, Respectively, determines whether or not the cut detection threshold TC is exceeded. The predetermined value BC is, for example, half of the second inter-frame distance L2 (L2 / 2).

BDL1 [BC]> TC and ... (6) BDL2 [i]> TC (i = 0, 1, 2, ..., BC) (7) Only when all the above conditions are satisfied, the n-BC It is determined that there is a cut point between the frame and the adjacent frame. In this case, the control device 60 sets the scene change flag FC to TRUE. The scene change flag FC in which TRUE is set indicates that there is a scene change section in the target moving image. If at least one of Expressions 6 and 7 is not satisfied, it is determined that there is no cut point between the n-BC frame and the adjacent frame. In this case, the scene change flag FC remains FALSE.

After detecting the cut point, the control circuit 60 determines in step A10 whether the scene change flag FC is TRUE. If yes, go to step A10 to step 12; otherwise, step A10.
To Step A11.

The control unit 60 causes the characteristic shape detecting section 57 to detect the transition portion in step A11. Schematically, the characteristic shape detection unit 57 includes, in the plurality of second and third frame dissimilarity change patterns stored in the dissimilarity storage unit 56, a characteristic shape associated with the above-described transition portion, that is, Check if there is a convex part.

In FIG. 8, the difference levels BDL2 [0] and BDL3 [0] at the beginning of the second and third buffer groups store the difference between the second and third frames of the last frame of the transition part. In the state, the difference degree between the second and third frames in the second and third buffer groups is set to the difference degree buffer BDL2 in which the difference degree is stored.
It is a graph plotted according to the buffer number j of [j] and BDL3 [j]. The graph of FIG. 8 is line-symmetric with the portions of the frames K0 to K2 of the graph of FIG. 4 described above about an imaginary axis parallel to the vertical axis. Characteristic shape detector 5
7 determines whether the transition part having the latest frame of interest at the end exists in the target video.
Then, it is detected whether the stored contents of the third buffer group are in the state shown in FIG.

Specifically, the characteristic shape detecting section 57 is
And a predetermined section j of the third buffer group, that is, a plurality of consecutive difference buffers BDL2 [0] to BDL2 [I], BDL3 [0] in the second and third buffer groups.
~ It is determined whether or not the degree of difference between the second and third frames in BDL3 [I] satisfies all of the following eight conditions.
In the formulas in the table below, the reference numeral of the dissimilarity buffer indicates the dissimilarity between frames in the buffer. See also "TL
“2” and “TL3” are predetermined threshold values, “ε” is a predetermined allowable error, and “I / 2” is a half value of the number I of the last buffer of the section j. Furthermore, section j, "HL1", and "HL2" are expressed by the following equations 8 to 1.
It is defined by 0, and the allowable error ε is set by Expression 11, for example. Further, as shown in Expression 12, the threshold TL3 to be compared with the value HL3 corresponding to the height of the variation pattern of the third inter-frame dissimilarity in the section j is the variation pattern of the second inter-frame dissimilarity in the section j. Is a value equal to or higher than a threshold value TL2 that is compared with a value HL2 corresponding to the height of.

[0092]

[Table 5]

J = 0, 1, 2, ..., I (8) HL2 = BDL2 [I / 2] -max (BDL2 [0], BDL2 [I]) (9) HL3 = BDL3 [0]- BDL3 [I]) (10) ε = HL2 / 4 (11) TL2 ≦ TL3 (12) Under the conditions 1, 2, and 5, the inter-frame dissimilarities in the plurality of dissimilarity buffers in the section are Whether it is a decreasing or increasing tendency may be actually judged as follows. For example, if the inter-frame dissimilarity in the section varies smoothly as shown by the solid line in FIG. 8, the dissimilarity is calculated from the inter-frame dissimilarity in any one disparity buffer in the section. The difference obtained by subtracting the frame dissimilarity in the dissimilarity buffer next to the buffer is obtained while changing the arbitrary dissimilarity buffer within the section, and the sign of each of the obtained plurality of differences is positive or negative. Then, the tendency of the dissimilarity is determined. Further, as indicated by the broken line in FIG. 8, when the dissimilarity does not change smoothly, the intra-frame dissimilarity in the plurality of dissimilarity buffers in the section is smoothed before the tendency is determined. As a result of the smoothing, the inter-frame dissimilarity changes smoothly as shown in FIG. 9, and therefore the tendency may be determined by the above procedure using the smoothed inter-frame dissimilarity. The above is the procedure for determining the changing tendency of the inter-frame difference degree.

Referring again to FIG. Only when all of the above eight conditions are satisfied, it is determined that the transition part having the latest focused frame at the end is in the target moving image.
In this case, the control device 60 sets the scene change flag FC to TRU.
Set E. If at least one of the eight conditions is not satisfied, it is determined that there is no transition part. In this case, the scene change flag FC is FALSE
To be kept.

At step A12, the control circuit 60 determines whether or not there is a cut point or a scene change which is a transition part in the target moving image. Specifically, the control circuit 6
0 indicates that the scene change flag is “TRUE” or “FAL
It is judged whether it is "SE". In the former case, the control circuit 60 outputs, as a detection result, data indicating that there is a scene change section. In the latter case, the control circuit 60
As a detection result, data indicating that there is no scene change unit is output. After outputting the data, the control circuit 60 adds 1 to the counter n to update it, and after updating, returns to step A3.

By the above processing, the first conversion section detection device 50 determines whether or not there is a transition part in the target moving image, regardless of the specific method of the transition effect.
It can be easily judged. Further, the first conversion unit detection device 50 outputs data indicating only the presence / absence of the scene conversion unit, but the output data is not limited to this, and may be other items related to the scene conversion unit. For example, the data indicating where the frame corresponding to the scene change portion is in the target moving image may be the data.

The judgments of the equations 6 and 7 in step A9 are based on the following reasons. Based on the graph of FIG. 6 described above, when the focus and comparison frames are two frames in a single shot, the inter-frame difference degree is sufficiently smaller than the cut point detection threshold value TC, and the focus and comparison frames are It can be seen that in the case of the frames in each of the two shots before and after the cut point, the inter-frame difference degree is sufficiently larger than the cut point detection threshold value TC. This is because in the latter case, the shots are converted at the positions of these frames in the target moving image. Therefore, it can be basically determined that there is a cut point between a frame having a difference between adjacent frames that is equal to or greater than the threshold value TC and a frame adjacent to the frame.

If, for example, a flash is used during the shooting of a single shot, if the frame affected by the flash and the frame not affected by the flash are continuous, the degree of difference between adjacent frames between these two frames is determined. But,
It may be equal to or more than the threshold value TC. In this case, 2
Since two frames sandwiching one frame are frames within a single shot, the dissimilarity between the separated frames of the two frames is expected to be less than the threshold value TC. That is, when the condition of Expression 6 is satisfied by the degree of difference between adjacent frames of the frame affected by the flash, the condition of Expression 7 is expected not to be satisfied for all variables i. As a result, if the condition of formula 7 is used in addition to the condition of formula 6, if the difference between two consecutive frames in a single shot is large, it is erroneously recognized that there is a cut point between the two frames. Can be prevented.

The dissimilarity between separated frames used for the condition judgment of the expression 7 is preferably such that the distance between frames at the time of calculation is as small as possible, for example, the distance between frames is about 2 or more and 9 or less. This is because the larger the inter-frame distance between the two frames in a single shot, the lower the correlation between the two frames, that is, the greater the dissimilarity between the separated frames of the two frames, so that the above-mentioned misidentification can be prevented. Because it will be difficult. The above is the description of Expressions 6 and 7.

The target moving image is realized by, for example, a moving image obtained by photographing a subject with a video camera, a movie, or a moving image broadcast by analog or digital television broadcasting. Each scene change part in the target moving image is a transition part which is a set of composite frames for a scene change using a so-called cut point or a transition effect, as in the case described with reference to FIG. And Each frame of the target moving image is not limited to a so-called frame in television broadcasting, and may be any frame as long as it is a still image for forming a moving image. The data format of the target moving image may be digital data or analog data at the time of input to the first conversion unit detection device 50. Further, the data indicating the target moving image may be compressed.

A single shot is, for example, in one series of shooting operations using a shooting device capable of shooting moving images.
It is configured by at least a part of all the frames captured from the start of capturing to the end of capturing. The photographing device is, for example, a video camera. That is, a single shot may be composed of all the above-mentioned frames, or may be composed of the remaining frames except some of the above-mentioned frames by the editing process after shooting. In general, two arbitrary frames among a plurality of frames in a single shot have a higher correlation as the inter-frame distance between the two frames becomes smaller, and become uncorrelated as the inter-frame distance increases. become.

The first conversion section detecting device 50 electrically includes a frame expanding section 51, a central processing circuit, and the above-mentioned storage sections 52, 54 and 56. The in-frame feature extraction unit 53, the dissimilarity calculation unit 55, the feature shape detection unit 57, the cut detection unit 58, the detection result output unit 59, and the control circuit 60 are realized by the arithmetic processing of the central arithmetic processing circuit. It is a virtual part. Further, in this case, each of the storage units 52, 54 and 56 may be realized by an individual storage device or may be realized as a storage area set in one storage device. Further, in this case, the frame expansion unit 51 may also be realized by the arithmetic processing of the central arithmetic processing circuit. Furthermore, the first conversion unit detection device 5
0 may realize at least one of the above-mentioned units 53, 55, 57 to 60 by an existing circuit separate from the central processing circuit.

Further, instead of the target moving image, a moving image obtained by sequentially displaying a plurality of frames extracted from the target moving image at regular intervals over time may be used. In this case, the target moving image is the first conversion unit detection device 50.
It is not known where the cut point and the transition portion are in the target moving image before the processing in step 1. Therefore, it is preferable that the plurality of frames are extracted from the target moving image at equal intervals. At this time, the correlation between frames becomes smaller as the inter-frame distance becomes larger even for a plurality of frames in a single shot. It is preferable to extract 10 frames or more and 15 frames or less per 30 frames. That is, it is preferable to extract one frame every 2-3 frames from the target moving image.

In the present embodiment, the second interframe distance is 10 and the third interframe distance is 30. This is because the length of the transition part used in a general video is about 1 second, so it is intended to detect the transition part of about 1 second in a video composed of shots of 1 second or more. is there. That is, the first conversion unit detection device can basically detect a transition portion having a length equal to or greater than the third interframe distance from within the moving image.

Further, in the present embodiment, the distance between two frames is used to determine the degree of difference between two separated frames for one frame. The number of dissociation degrees between separated frames required for one frame is not limited to two, and may be three or more. For example, when the fourth inter-frame distance longer than the third inter-frame distance is further used, the third and the third inter-frame distances are almost the same as the transition portion detection based on the characteristics of the change patterns of the second and third inter-frame difference degrees. Based on the characteristics of the change pattern of the degree of difference between the four frames, it is possible to detect a transition portion having a larger number of used frames than the former transition portion. Further, for example, when a fourth inter-frame distance different from the second and third inter-frame dissimilarities is used, the relationship between the change patterns of the second to fourth inter-frame dissimilarities in the transition part is previously determined. It may be determined in advance that it is determined that there is a transition portion in the target moving image when the change patterns of these three inter-frame dissimilarities have the above relationship. In this case, the detection accuracy of the transition part is improved.

The moving picture / scene changing portion detecting apparatus according to the second embodiment of the present invention will be described below. The moving picture scene change unit detection device of the second embodiment (hereinafter abbreviated as “second change unit detection device”) is different from the first change unit detection device 50 in the in-frame feature extraction unit and the difference calculation. The specific calculation method of the intra-frame feature and the inter-frame dissimilarity in each section is different, and the others are the same. Therefore, in the structure and operation of the second conversion unit detection device, the first conversion unit detection device 50
The description of the same structure and operation as the above is omitted. Further, among the components in the second conversion unit detection device, the same components as those of the first conversion unit detection device 50 are designated by the same reference numerals.

FIG. 10 is a schematic diagram for explaining a schematic detection method of the intra-frame feature Rn and the inter-frame difference degree in the second conversion unit detection device.

The in-frame feature extraction unit of the second conversion unit detection device first determines a predetermined number MMA of the frame FA in order to obtain the in-frame feature of any one frame FA.
The region is divided into X regions, and then, from each region, at least one color representing the region and the frequency of appearance of pixels of the color in the region are obtained. As a result, the in-frame feature R _FA of the frame FA includes the representative colors of all the areas in the frame FA and the appearance frequency of the color. In order to store the color number and the appearance frequency of each of these representative colors, each feature buffer of the intra-frame feature storage unit 54 stores M
MAX × PMAX representative color regions Rn [0] [p]. C
OL-R _FA [MMAX-1] [p]. COL (p = 1,
2, ..., MP-1) and the same number of appearance frequency regions R _FA [0]
[P]. FRQ to R _FA [MMAX-1] [p]. FRQ
With. The frame division number MMAX is, for example, 1
2, a single frame is divided into a matrix of 3 rows and 4 columns. Further, the upper limit PMAX of the representative color extracted from the single region is, for example, 26 when the number of colors acquired by the pixel is 26.
In the case of 5 colors, it is 8, for example. Such in-frame feature R _FA is stored in the feature buffer in the in-frame feature storage unit 54 in the table format shown in FIG. 10, for example.

The dissimilarity calculation section of the second conversion section detection device, based on the in-frame characteristics R _{FA of the} target frame _FA and the in-frame characteristics R _FB of the comparison frame FB, will be described later with reference to FIG.
The degree of difference between the first to third frames is calculated using the calculation method shown in. The inter-frame dissimilarity is roughly the intra-frame feature R _FA of both frames FA and FB,
This corresponds to the sum of the products of the difference in the appearance frequency between the representative colors in the R _FB that have close inter-color distances and the representative color of the close colors. The characteristic shape and the cut point detection units 57 and 58 of the second conversion unit detection device are the plurality of first detection units obtained by the above procedure.
~ Using the degree of dissimilarity between the third frames, each scene change part in the moving image is detected, and the detection result output part 59 is caused to output the detection result.

FIG. 11 is a flow chart for explaining a concrete method of extracting the in-frame feature in the in-frame feature extraction unit of the second conversion unit detection device. In the description of FIG. 11, the in-frame feature extraction unit is abbreviated as “extraction unit”. The calculation of FIG. 11 is executed by replacing the calculation of step A5 in the entire detection process of the scene changing unit described in FIG. That is, in step A4 of FIG. 7, at least at the time when the frame of interest to be processed, that is, the nth frame is stored in the frame storage unit 52, the control circuit 60 causes the extraction unit to extract the in-frame feature of the nth frame. Start processing. As a result, the process proceeds from step B1 to step B2.

In step B2, the extraction section performs preprocessing for feature extraction on the nth frame. Specifically, the pre-processing is processing for reducing the number of pixels and the number of colors in the nth frame. For example, in order to reduce the frame size, the extraction unit periodically spreads the pixels among all the pixels forming the nth frame to be processed. Thereby, for example, the n-th frame of 640 × 480 dots is reduced to 80 × 60 dots. Further, for example, the extraction unit uses a so-called uniform quantization method in order to reduce the number of colors. As a result, in the single pixel data set in the original frame, for example, when data for indicating red, green, and blue is allocated 8 bits each, that is, 258 gray levels are allocated, the frame after preprocessing is performed. In the single pixel data set in, 3 bits are allocated to the data for indicating red and green, and 2 bits are allocated to the data for indicating blue. That is, the color acquired by a single pixel is reduced from approximately 16 million colors to 256 colors.

After the pre-processing, the extracting section carries out step B3.
Then, the n-th frame is divided into a predetermined number of MMAX areas, and the area number counter m is initialized to zero. For example, it is assumed that the nth frame is divided into a matrix as shown in FIG. Further, it is assumed that area numbers 0 to MMAX-1 are given to all areas in the raster order, for example. The area number counter m is for defining the area to be processed in the processing of steps B5 to B11, and has an area number equal to the numerical value stored in the counter, that is, the m-th area.
It is assumed that the area is a processing target area. After the initialization, in step B4, the extraction unit determines whether the count value of the area number counter m is less than the number MMAX of all areas. If so, the process proceeds from step B4 to step B5.

In step B5, the extraction section creates a color histogram for the region to be processed, that is, the mth region in the nth frame. The histogram shows the appearance frequency of each of the plurality of colors acquired by the pixel in the m-th area. After the creation, the extraction unit performs step B6.
Then, the color number counter p is initialized to 0. After the initialization, in step B7, the extraction unit determines whether or not the count value of the color number counter p is less than the predetermined maximum extracted color number PMAX. If so, step B7 to step B
Go to 8.

In step B8, the extraction unit searches the histogram for the p + 1-th color C _{p + 1} of all the colors acquired by the pixel, which has the highest appearance frequency in the m-th region. "P + 1" is the current color number counter p
Is a number one greater than the value of. After searching, the extraction unit
In step B9, the color number of the retrieved color C _{p + 1} and the appearance frequency HC [C _{p + 1} ] of the color in the m-th area are stored in the first feature buffer in the intra-frame feature storage unit 54. Representative color and appearance frequency region Rn [m] for storing the p + 1th representative color of the m region and its appearance frequency, respectively
[P]. COL, Rn [m] [p]. Store in FRQ. After the storage, the extraction unit adds 1 to the value of the color number counter p to update it in step B10, and returns from step B10 to step B7. As described above, in the present embodiment, of all the colors acquired by the pixels, the color having the higher appearance frequency in the area is used as the representative color of the area. this is,
This is because the color with the higher appearance frequency in the area is considered to be stable as a characteristic of the area.

The processing of steps B7 to B10 is repeated until the value of the color number counter p reaches or exceeds the maximum extracted color number PMAX. As a result, the representative colors of the predetermined number PMAX detected from the m-th area and their appearance frequencies are shown in Table 6
, The representative color for the m-th region and the appearance frequency region Rn [m] [0]. COL to Rn [m] [PMAX-
1]. COL, Rn [m] [0]. FRQ to Rn [m]
[PMAX-1]. Each is stored in FRQ. Also, as shown in Table 6, the representative colors and the appearance frequencies of these colors are as follows.
It is associated with the value of the color number counter p at the time when it is obtained.
As a result, the representative color of the m-th region and its frequency are briefly summarized.

[0116]

[Table 6]

If it is determined in step B7 that the count value of the color number counter p is equal to or greater than the predetermined maximum extracted color number PMAX, the process proceeds from step B6 to step B11. The storing and extracting unit adds 1 to the value of the area number counter m to update it in step B10, and returns from step B11 to step B4. As a result, the processing of steps B4 to B11 is repeated until the area number counter m reaches or exceeds the maximum area number MMAX. As a result, the 0th to the MMA
The representative color and the appearance frequency of the X-1 region are obtained, respectively. When it is determined in step B4 that the count value of the area number counter m is equal to or larger than the predetermined number MMAX, the process proceeds from step B4 to step B12, and the processing operation of the flowchart ends.

In the processing for the feature extraction, the preprocessing is performed on the nth frame to be processed in step B2 for the following reason. Generally, the correlation between two adjacent pixels in a single frame is high, so that the information contained in that frame is redundant. Therefore, when the size of the frame is reduced, that is, when the number of pixels forming a single frame is reduced, the tendency of the information in the frame after the reduction is the same as that of the information in the original frame before the reduction. Often almost equal. For example, VGA size, that is, multiple frames of 640 × 480 dots are set to 80 × 60.
When the dots are reduced to the respective dots, the tendency of the change in the correlation and the dissimilarity between the plurality of frames after the reduction is substantially equal to the tendency of the change in the correlation and the dissimilarity between the plurality of original frames. Therefore, when the number of pixels of the frame to be processed is reduced in advance before the feature extraction, the memory capacity required for storing the frame is reduced as compared with the case where the frame is not reduced. Further, in the above case, the number of accesses to the portion in the memory in which each pixel in the frame is stored for feature extraction of the reduced frame is smaller than that in the case where the frame is not reduced. The computational complexity for is reduced than if the frame is not reduced. When the number of colors of pixels in the frame is reduced, the memory capacity and the amount of calculation in feature extraction are reduced for the same reason as when the number of pixels in the frame is reduced. By reducing the number of pixels and the number of colors in a single frame in this manner, it becomes easier to handle information in the frame. The above is the reason why the frame is preprocessed.

FIG. 13 is a flow chart for explaining a concrete method of extracting the inter-frame dissimilarity in the dissimilarity calculation section of the second conversion section detection device. In the description of FIG. 13, the difference calculation unit is abbreviated as “calculation unit”. Figure 1
Since the calculation No. 3 can be applied to each of the calculation of the first to third inter-frame dissimilarities, in the following description, only one of these three inter-frame dissimilarities will be described. Any one inter-frame dissimilarity is abbreviated as "inter-frame dissimilarity".

The arithmetic operation of FIG. 13 is executed by replacing the arithmetic operation of step A7 in the entire detection processing of the scene changing unit described with reference to FIG. That is, step A6 in FIG.
In, at least the intra-frame features R _n and R of the n-th and n-Lx frames which are the focus and comparison frames
_{When n-Lx} is stored in the intra _- frame feature storage unit 54,
The control circuit 60 causes the calculation unit to start calculation of the inter-frame difference degree. As a result, the process proceeds from step C1 to step C2.

The calculation section initializes a variable DSUM for storing the inter-frame difference degree to 0 in step C2, and initializes a region number counter m to 0 in step C3. After the initialization, the calculation unit determines in step C4 whether or not the count value of the region number counter m is less than the number MMAX of all regions. If so, step C
It progresses from 4 to step C5. The calculation unit performs step C
At 5, the color number counter p is initialized to 0. After the initialization, in step C6, the extraction unit determines whether or not the count value of the color number counter p is less than a predetermined maximum extracted color number PMAX. If so, the process proceeds from step C6 to step C7. Among the in-frame features Rn of the frame of interest, the variables m and p are the current region and the color number counter m,
Representative color and appearance frequency region Rn that are respectively equal to the value of p
[M] [p]. COL, Rn [m] [p]. FRQ is
It becomes the representative color and appearance frequency region of the current processing target in steps C7 to C14.

In step C7, the calculating unit calculates the appearance frequency region Rn [m] [p]. F
It is determined whether the appearance frequency currently stored in the RQ is 0 or more. If so, proceed from step C7 to step C8. The calculation unit, in step C8,
Among all the current appearance frequencies for the m-th region in the in _- frame feature R _{n -LX} of the comparison frame, the most representative color of the current processing target among all the representative colors in the m-th region Search for an effective appearance frequency that has a similar color appearance frequency. The m-th area is an area having the same number as the value of the area number counter m among all the areas in the comparison frame R _{n -LX} . The variable q is an arbitrary integer of 0 or more and PMAX-1 or less. The effective appearance frequency is the appearance frequency R _n-LX
[M] [q]. FRQ is 0 or more.

That is to say, the calculation unit stores in the feature buffer storing the intra-frame feature R _{n -LX} of the comparison frame,
Representative color area R _n-LX [m] [0]. COL ~
Rn _-LX [m] [PMAX-1]. Among the representative colors in COL, the representative color region Rn [m] [p].
A representative color region R _n-LX [m] [q]. _Which stores a color closest to the representative color C _{p + 1} in COL. COL is searched as a representative color area to be processed at the present time of the comparison frame, and all the appearance frequency areas R for the m-th area in the feature buffer are searched.
_n-LX [m] [0]. FRQ to Rn _-LX [m] [PMAX-
1]. From among the appearance frequencies currently stored in FRQ, the representative color region Rn _-LX [m] [q]. COL which is an appearance frequency region corresponding to COL and stores an appearance frequency of 0 or more. R _n-LX [m] [q]. The FRQ is searched for as the appearance frequency region of the representative color of the current processing target of the comparison frame.

After the search, the calculation unit, in step C9,
Determine if a valid variable q exists. The variable q is not valid because the appearance frequency region Rn _-LX [m] [0] _.m for the m-th region in the in _- frame feature Rn _-LX of the comparison frame in the search process of step C7. FRQ ~ R
_n-LX [m] [PMAX-1]. This is a case where the stored contents of FRQ are all 0, and in this case, the variable q becomes indefinite. That is, in this case, the representative color and the appearance frequency region of the current treatment target are not determined. If the variable q is valid, the process proceeds from step C9 to step C10.

The calculation section calculates the equation 13 in step C10.
As shown in FIG. 5, the appearance frequency regions Rn [m] [p]. FR
Q, Rn _-LX [m] [q]. The smaller value FMIN is obtained from the appearance frequencies in FRQ. After the selection, in steps C11 and C12, the calculation unit calculates the appearance frequency regions Rn [m] [p]. FRQ,
Rn _-LX [m] [q]. The smaller appearance frequency FMIN is subtracted from the stored contents of the FRQ and updated. As a result, one of the two appearance frequency areas becomes 0, and the other stores the difference between the two appearance frequencies.

FMIN = min (Rn [m] [p] .FRQ, Rn _-LX [m] [q] .FRQ) (13) Rn [m] [p]. FRQ = Rn [m] [p]. FRQ-FMIN (14) Rn _-LX [m] [q]. FRQ = Rn _-LX [m] [q]. FRQ-FMIN (15) After updating, the calculation unit calculates the color of the representative color in step C13 based on the color number of the representative color of the current processing target of the focus and comparison frames, as shown in Expression 16. The distance d is calculated. The inter-color distance d is a numerical value that quantitatively shows the difference between the two colors. For example, two colors C1 and C2 are coordinates (R1, G1, B1), (R2, G in the RGB color space.
2, B2) respectively, the inter-color distance d is the coordinates (R1, G1, B1), (R
2, G2, B2) as the numerical values indicated by the bits respectively assigned to red, green, and blue in the pixel data set of the pixels of the colors C1, C2, as shown in Expression 17. , Is defined.

D = Rn [m] [p] .COL and Rn _-LX [m] [q] .COL difference (16) d = | R1-R2 | + | G1-G2 | + | B1 -B2 | (17) After the calculation, the calculation unit calculates the value of the dissimilarity variable DSUM at the current time, and the appearance of the representative color of the current processing target of the focus and comparison frames after the update. The product of the larger value of the stored contents of the frequency area and the inter-color distance d is added and updated. That is, the product of the inter-color distance and the difference between the two appearance frequencies is added to the current difference degree variable. After updating, step C14 to step C15
Return to. As a result, the processing of steps C7 to C14 is performed in the appearance frequency region R of the representative color of the current frame that is the processing target.
n [m] [p]. The process is repeated until the stored content of FRQ becomes 0 or the variable q becomes invalid.

DSUM = DSUM + max (Rn [m] [p] .FRQ, Rn _-LX [m] [q] .FRQ) × d (18) In step C7, the appearance frequency region Rn [m] [p]. ． F
When it is determined that the stored content of RQ has become 0 or less, the process proceeds from step C7 to step C15. Also step C
When it is determined in 9 that the variable q is not valid, the process proceeds from step C9 to step C15. The calculation unit adds 1 to the count value of the color number counter p to update it in step C15, and returns from step C15 to step C6. As a result, the processing in steps C6 to C15 is performed until the count value of the color number counter p reaches a predetermined value PMAX, that is, the stored contents of all the appearance frequency areas for the m-th area of the attention or comparison frame are 0 or less. It is repeated until.

When it is determined in step C6 that the count value of the color number counter p is equal to or more than the value PMAX, the process proceeds from step C6 to step C16. In step C16, the calculator adds 1 to the count value of the area number counter m to update the count value, and returns from step C16 to step C4.
As a result, the processes of steps C4 to C16 are repeated until the count value of the area number counter m reaches the predetermined division number MMAX. As a result, the processes of steps C4 to C16 are performed on all the regions of the focus and comparison frames. When it is determined in step C4 that the count value of the area number counter m is equal to or greater than the division number MMAX,
The process proceeds from step C4 to step C17, and the processing operation of the flowchart is finished. As a result, the value stored in the dissimilarity variable DSAM at the end of the flowchart of FIG. 13 is output as the inter-frame dissimilarity between the focus and comparison frames.

As described above, the second conversion section detection device 31 determines the color representing each of a plurality of areas obtained by dividing the nth frame and each area as the intra-frame feature Rn of the nth frame. The appearance frequency of the pixel of the color in the inside is used, and the inter-frame difference degree of the nth frame is calculated by the method shown in FIG. This is for the following reason.

When the sum of the differences in color or brightness of the corresponding pixels in two frames, that is, the so-called inter-frame difference is used as the intra-frame characteristic, the position of the subject in the two frames within the frame is very small. If there is a deviation, the inter-frame difference between the two frames changes significantly. That is, in the above case, the degree of difference between frames is hypersensitive to a minute difference between two frames. When the difference between the pixel histograms of two frames is used as the in-frame feature, the feature regarding the position of the pixel in the frame is lost from the in-frame feature. Therefore, for example, if the two frames are displaced only in the position of the subject imaged inside, the degree of difference between the frames tends to be high. That is, in the above case, the degree of difference between frames becomes insensitive to the subtle difference between the two frames. Further, in either of the above two cases, the inter-frame dissimilarity is liable to sensitively react to changes in the hue of two frames and the luminance of each pixel of the frames.

If the first inter-frame dissimilarity reacts to the frame dissimilarity as described above, the first inter-frame dissimilarity may increase above the threshold value even in frames other than the cut point. It becomes difficult to improve the accuracy of detecting the cut points using the degree of dissimilarity. Also, if the difference between the second and third frames reacts to the difference between frames as described above, a pattern peculiar to the transition part of the change patterns of the difference between the second and third frames is changed to the frame. The variation of the dissimilarity related to the difference may overlap, and the variation of the dissimilarity may cancel the peculiar pattern. As a result, it becomes difficult to detect the pattern peculiar to the transition part, and thus it becomes difficult to improve the detection accuracy of the transition part using the inter-frame difference degree.

The in-frame feature of this embodiment is obtained as described above. When calculating the in-frame feature, the frame to be calculated is divided into a plurality of areas. As a result,
Based on the number of divisions of the frame, the degree to which the motion of the subject in the frame is absorbed by the characteristics of each region can be adjusted. For example, the smaller the number of divisions, the easier the movement in the frame is absorbed by the features of each region,
The larger the number of divisions, the less likely the movement in the frame is absorbed by the features of each region. That is, the greater the number of divisions, the easier the movement of the subject is reflected in the characteristics of each region.

The greater the number of divisions, the closer the difference between the features of the corresponding areas of the two frames becomes to the difference in color and luminance between the corresponding pixels. The smaller the number of divisions, the greater the difference in the characteristics of the regions. It is close to the difference in characteristics between the frames.

The inter-frame difference degree of this embodiment is
Basically, it is the sum of the products of the difference in appearance frequency between representative colors having similar colors among the representative colors in the corresponding regions of the focus and comparison frames and the inter-color distance of the colors. As described above, by using the difference between the close colors of the representative colors, the inter-frame difference degree can absorb the minute difference in the brightness and the hue between the two frames, so that the difference between the frames is sensitive to the difference. do not do.

On the basis of these reasons, when the intra-frame feature and inter-frame dissimilarity of this embodiment are used, the intra-frame feature and inter-frame dissimilarity are hypersensitive or excessive with respect to the difference between two frames. It can be prevented from becoming insensitive. As a result, the characteristic shapes of the cut points and the transition portions clearly appear in the change pattern of the inter-frame dissimilarity, so that the detection accuracy of the cut points and the transition portions can be improved. In the detection of the in-frame feature described above, the frequency of appearance of a pixel of a certain color in the area is equivalent to the area occupied by the pixel of the color in the area. Therefore, the in-frame feature may be detected using the area indicated by the pixel instead of the appearance frequency.

A moving picture / scene changing portion detecting apparatus according to the third embodiment of the present invention will be described below. Compared with the first conversion unit detection device 50, the moving image scene conversion unit detection device of the third embodiment (hereinafter abbreviated as “third conversion unit detection device”) has a specific cut point in the cut detection unit. The detection method is different, and the others are the same. Therefore, of the structure and operation of the third conversion unit detection device, the description of the same structure and operation as the first conversion unit detection device 50 will be omitted. Further, among the components in the third conversion portion detection device, the same components as those of the conversion portion detection device 31 of the first embodiment are designated by the same reference numerals.

A method of detecting a cut point in the cut detecting unit of the third conversion unit detecting device will be schematically described below. The predetermined cut point detection threshold value TC is generally equal to or less than the dissimilarity between adjacent frames between two consecutive frames that sandwich the cut point, and the adjacent frame between two consecutive frames within a single shot. It is set so as to be larger than the dissimilarity. As a result, the cut point can be detected based on the magnitude relationship between the difference between adjacent frames and the threshold value TC.

However, as described above, when the relative positional change between the subject and the camera is large in a single shot, even if there are two consecutive frames in the shot due to the positional change, the adjacent frames are adjacent to each other. The degree of disparity may become greater than the threshold value TC. Further, even in a frame within a single shot, the inter-frame dissimilarity between two frames arranged with a plurality of frames interposed therebetween tends to be large due to the influence of the position change. For example, if the single shot is edited and a part of the frames is deleted, the two frames arranged with the two frames sandwiched therebetween become continuous frames. The difference between adjacent frames may be equal to or more than the threshold value TC. As described above, when the dissimilarity between two adjacent frames in a single shot is equal to or greater than the threshold value TC, two consecutive frames in a single shot are recorded.
It may be falsely detected that there is a cut point between two frames.

FIG. 14 is a graph showing changes in the dissimilarity between adjacent frames of each frame in a moving image according to the frame order. In FIG. 14, at least a plurality of frames within the range WFA are within a single shot, and the range WF
The plurality of frames in B include two frames sandwiching the cut point. It is assumed that the positional change between the camera and the subject in the shot shown within the range WFA is extremely severe. Based on FIG. 14, when the dissimilarity between adjacent frames between two frames in a single shot is equal to or more than the threshold value based on the position change, the other two adjacent to the two frames are detected.
It can be seen that the inter-frame dissimilarity of one frame is also above the threshold value. On the contrary, based on FIG.
When the dissimilarity between adjacent frames between two frames is equal to or more than the threshold value based on the cut point being sandwiched, the dissimilarity difference between adjacent frames of other two frames near the two frames is always less than the threshold value. I know that That is, the dissimilarity between the adjacent frames of the two frames sandwiching the cut point is higher than the dissimilarity between the adjacent frames of the other two frames around the cut point.

Therefore, the cut detection unit of the third conversion unit detection device, prior to the detection of the cut point, determines the degree of difference L1 [0] to L1 [N-1] between adjacent frames of each frame in the moving image,
Each of the frames is corrected using the inter-frame dissimilarity of a plurality of frames before and after each frame. As a result of this, FIG.
As shown in FIG. 7, only the dissimilarity between adjacent frames due to the position change within the range WFA is suppressed, and the dissimilarity between adjacent frames due to the cut point is maintained as it is. The cut point detection unit detects the cut point in the moving image by using the degree of difference between the adjacent frames thus corrected.

FIG. 16 is a flow chart for explaining a specific method of detecting a cut point in the cut point detection unit of the third conversion unit detection device. In the description of FIG.
The cut point detection unit is abbreviated as “detection unit”. The calculation of FIG. 16 is executed by substituting the calculation of step A9 in the entire detection processing of the scene changing unit described in FIG. That is, in step A8 of FIG. 7, after the difference level between the first to third frames of the frame of interest, that is, the nth frame is stored in the difference level buffer, the control circuit 60
The detection unit is caused to start the detection processing of the cut points. As a result, the process proceeds from step D1 to step D2.

In steps D2 to D4, the detecting unit selects the n-BC frame whose display order is earlier than the target frame by the predetermined number BC as the target frame for detecting the cut point, and selects the target frame. The first difference between frames, that is, the difference between adjacent frames is corrected.

That is, the detecting section, in step D2,
BCth difference buffer BDL1 in the first buffer group
A predetermined number L arranged in front of and behind [BC]
L-by-L buffers, that is, BC- in the first buffer group
The dissimilarities between the first frames in the LLth to BC-1th and BC + 1th to BC + LLth buffers are rearranged in descending order. “2LL” is a number that is twice the predetermined number LL and is less than or equal to the second interframe distance L2. The degree of difference between the 2LL first frames after the rearrangement is determined by the array variable CS.
It is stored in each of [0] to CS [2LL-1]. The larger the variable k is, the more the array variable CS [k] becomes 2LL.
The larger one of the first inter-frame differences is stored. Here, the variable k is an integer of 0 or more and 2LL-1 or less. As a result, array variables CS [0] to CS [2LL-
1] stores the first inter-frame dissimilarity of LL frames before and after the target frame in descending order.

After the storage, the detection unit obtains the correction amount R for correcting the first inter-frame difference degree of the target frame, that is, the n-BC frame, based on Expression 19. That is, the correction amount R is the sum of the products of the first interframe dissimilarity of the 2LL frames before and after the target frame and the predetermined 2LL weight coefficients α _{0 to} α _2LL-1 . In the following formula, “CS [k]” is the kth array variable CS [k]
The degree of difference between the first frames is shown.

[0146] At least one of all the weighting factors α _{0 to} α _2LL-1 is a factor exceeding 0. The correction amount R is for suppressing the degree of difference between the first frames which is increased due to the positional change between the subject and the camera in a single shot. Therefore, of all the weighting factors α _{0 to} α _2LL-1 , the variable k
The larger the value, that is, the array variable CS whose order is earlier
It is preferable that the value to be multiplied by [k] is large. For example, as shown in Expression 20, the leading weighting coefficient α ₀ is 1, and all the remaining weighting coefficients α _{1 to} α _2LL-1
May be zero.

[0147]

[Equation 1]

After the correction amount is calculated, the detecting section carries out step D
In 4, the corrected first inter-frame difference CR of the target frame is calculated as shown in Expression 21. The corrected first inter-frame dissimilarity CR is calculated from the original first inter-frame dissimilarity L1 (n-BC, n-BC-1) of the frame of interest as follows:
This is a value obtained by subtracting the correction amount R calculated in step D3. In Expression 19, “BDL1 [BC]” is the first inter-frame dissimilarity stored in the BC-th dissimilarity buffer in the first buffer group, that is, the original first of the frame of interest.
This is the difference between frames. CR = BDL1 [BC] -R (21) As described above, when the weighting factors α _{0 to} α _2LL-1 are determined in this way, the first inter-frame difference degree of the _target frame is caused by the cut point. The correction amount R is larger when the first frame dissimilarity is increasing due to the position change than when it is increasing. As a result, the corrected first interframe difference CR in the former case is almost equal to the original first interframe difference L1 (n-BC, n-BC-1),
In the latter case, the corrected first interframe difference CR is the original first interframe difference L1 (n-BC, n-BC-1).
Will be much smaller than As a result, the change of the corrected first inter-frame difference degree of each frame in the moving image according to the frame arrangement order is as shown in FIG. 15 described above. That is, the correction processing in steps D2 to D4 suppresses the degree of difference between the first frames in the section WFA, which has increased due to the position change, and is smaller than the original value.

Referring again to FIG. After the correction process,
In step D5, the detection unit corrects the first frame difference CR after the corrected target frame is calculated in step D4.
Is greater than or equal to a predetermined cut point detection threshold value TC. Only when the corrected first inter-frame difference CR is equal to or larger than the threshold value TC, it is determined that there is a cut point between the target frame, that is, the n-BC frame and the adjacent frame. In this case, the control device 60 proceeds from step D5 to step D6, sets the scene change flag FC to TRUE, and ends the processing operation of the flowchart in step D7. When the corrected first inter-frame difference CR is less than the threshold TC, the n-Bth
It is determined that there is no cut point between the C frame and the frame next to it. In this case, steps D5 to D6
Is skipped and the process proceeds to step D7 to end the processing operation of the flowchart. As a result, the scene change flag FC is F.
It is kept as ALSE.

As described above, the cut detecting unit of the third conversion unit detecting device determines the first frame difference degree of the target frame based on the first frame difference degree of the other plurality of frames before and after the target frame. The presence or absence of the cut point is determined using the corrected first frame difference degree. As a result, the cut detection unit can suppress erroneous detection of a cut point due to a positional change between the subject and the camera within a single shot, that is, so-called movement of the subject. Therefore, the 3rd conversion part detection apparatus can improve the detection accuracy of the said cut detection part.

A moving picture / scene changing portion detecting apparatus according to the fourth embodiment of the present invention will be described below. Compared with the first conversion unit detection device 50, the moving image scene conversion unit detection device of the fourth exemplary embodiment (hereinafter abbreviated as “fourth conversion unit detection device”) is more specific than the transition part in the characteristic shape detection unit. The different detection methods are different, and the others are the same. Therefore, of the structure and operation of the fourth conversion unit detection device, description of the same structure and operation as the first conversion unit detection device 50 will be omitted. Also, of the components in the fourth conversion unit detection device, the same components as those of the first conversion unit detection device 50 are designated by the same reference numerals. In the description of the fourth conversion unit detection device, the characteristic shape detection unit may be abbreviated as “detection unit”.

The characteristic shape detection unit of the fourth conversion unit detection device is
In general, a change in the degree of dissimilarity between the second and third frames of the target frame and a plurality of adjacent frames is approximated by a quadratic function represented by Expressions 22 and 23, and two approximation expressions are used. The positional relationship and shape of two parabolas, which are graphs 22 and 23, are expressed by the approximate expression 22,
Judgment is made based on the coefficient of 23 to check whether the frame of interest and the frames in the vicinity thereof form the transition part. In the approximate expressions 22 and 23, a _L2 , b _L2 , c
_L2 , a _L3 , b _L3 , and c _L3 are coefficients, and x and y are variables.

F _L2 (x) = a _L2 (x−b _L2 ) ² + c _L2 = 0 (22) f _L3 (x) = a _L3 (x−b _L3 ) ₂ + c _L3 = 0 (23) In the present embodiment, the so-called least square method is used to calculate the coefficient. That is, the detection unit finds the solution of the simultaneous equations of Equation 24 by defining the error vector e _i = 0. The variable i is a variable for defining a section in which the inter-frame dissimilarity is approximated by the approximate expressions 22 and 23, that is, an observation section, among all the frames in the moving image. The variable i is specifically an integer that is greater than or equal to a predetermined minimum constant i0 and less than or equal to a predetermined maximum constant imax. In other words, the approximate expressions 22 and 23 are the frames having the degree of difference between the second and third frames of the i frames from the frame that is traced back i frames from the frame of interest to the frame of interest, that is, from the (i) th frame to the nth frame. Approximate the change pattern according to the order of arrangement. The minimum constant i0 is predetermined based on the minimum length of the transition part, that is, the minimum number of frames forming the transition part, and is 5 in the present embodiment.

A _i X = D _i + e _i (i = i0, i0 + 1, ..., imax) (24) In Expression 24, “A _i ” is i + 1 row 3 column (3 × (i + 1)) shown in Expression 25. Is the matrix of. “X” is, as shown in Expression 26, three unknown coefficients a, of the quadratic expression shown in Expression 27.
It is a coefficient vector having three elements in which b and c are arranged. "D
_As shown in Expression 28, “ _i ” is a vector with the number of elements i + 1 in which the observed values d _{0 to} d _i , that is, the values in the difference buffers BD [0] to BD [i] are arranged. The three unknown coefficients a, b, c of the equation 27 and the three unknown coefficients a _L2 of the equation 22,
The relationship with b _L2 and c _L2 is as shown in Expressions 29 to 31. Further, the relationship between the three unknown coefficients a, b, and c of the equation 27 and the three unknown coefficients a _L3 , b _L3 , and c _L3 of the equation 23 is as follows.
31 unknown coefficients a _L2 , b _L2 , and c _L2 are set to unknown coefficients a _L3 ,
It is the same as the one replaced with b _L3 and c _L3 .

[0155]

[Equation 2]

A _L2 = a (29) b _L2 = -b ÷ (2a) (30) c _L2 = c-ab ² (31) Solving the coefficient vector X in Expression 23 using the least squares method. That is, the error vector e _i is defined as 0, and
This is equivalent to performing the matrix calculation shown in. “A _i ^T ” is a transposed matrix of the matrix A _i , and “(A _i ^T A _i ) ⁻¹ ” is an inverse matrix of a matrix which is a product of the transposed matrix A _i ^T and the matrix A _i .
Here, the matrix “(A _i ^T A _i ) ⁻¹ A _i ^T ” is a matrix of i + 1 rows and 3 columns ((i + 1) × 3), and is uniquely determined when the variable i is determined. Therefore, as shown in Expression 33, in the following description, the matrix “(A _i ^T A _i ) ⁻¹ A _i ^T ” is replaced with the matrix S _i . Further, the error e _i between the approximate parabola obtained by the method of least squares and the actual change pattern of the difference is
It is calculated based on Expression 34.

X = (A _i ^T A _i ) ⁻¹ A _i ^T D ⁱ (32) S _i = (A _i ^T A _i ) ⁻¹ A _i ^T (i = i0, i0 + 1, ..., imax) ... (33) e _i = A _i x−D _i (34) FIG. 17 is a flowchart for explaining a specific method for detecting the transition part in the characteristic shape detection part of the fourth conversion part detection device. The calculation of FIG. 17 is performed in step A1 of the entire detection processing of the scene changing unit described in FIG.
It is executed by substituting the operation of 1. That is, in step A10 of FIG. 7, the scene change flag is set to FAL.
If it is determined to be SE, the process proceeds from step E1 to step E2, and the control device 60 causes the detection unit to start the transition part detection process.

In step E2, the detecting section calculates the approximate expression 2
A matrix A _i0 for use in calculating the coefficients 0, 21;
A _{i0 + 1} , ..., A _imax , S _i0 , S _{i0 + 1} , ..., S _imax are prepared. As described above, each matrix A _i , S _i is uniquely determined if the variable i is determined, and therefore the elements of the matrix A _i , S _i can be calculated in advance. As a result, the detection unit can reduce the amount of calculation in subsequent calculations. After the matrix calculation, the detection unit substitutes the minimum constant i0 into the counter i in order to initialize the counter i prepared in advance. After that, the current value of the counter i is used as the value of the variable i in the processes of steps E4 to E11. After the initialization, the detection unit performs step E4.
Then, the current value of the counter i is the maximum value ima determined in advance.
It is determined whether it is less than x. If so, the process proceeds from step E4 to step E5.

In step E5, the detection section performs a simple shape check of the degree of difference between the second and third frames of the observation section. In the simple shape check, a change pattern according to a plurality of frames from the n-th frame to the n-th frame, that is, a change pattern according to the arrangement order of the frames of the second and third frames of the observation section is apparently transitioned. This is a check for judging whether or not a predetermined pattern peculiar to the part, that is, a convex shape is formed. Specifically, the detection unit calculates the difference between the second and third frames in the difference buffer at the present time by using the equations 35 to 37.
It is determined whether or not the condition shown in is satisfied. In the following formula, BDL
2 [i] and BDL3 [i] indicate the inter-frame dissimilarity stored in the i-th dissimilarity buffer in each of the second and third buffer groups. “THL2” and “THL3” are predetermined threshold values. If Expressions 35 to 37 are all satisfied, the process proceeds from Step E5 to Step E6.

BDL2 [i / 2] -BDL2 [0]> THL2 (35) BDL2 [i / 2] -BDL2 [i]> THL2 (36) BDL3 [0] -BDL3 [i]> THL3 ((35) 37) In step E6, the detection unit uses the i-th observation value at the present time in the dissimilarity buffer in the second buffer group, that is, the inter-second-frame dissimilarity of the frames in the observation period, As shown in, the second coefficient vector X _L2 is calculated. Based on the unknown coefficients a, b, and c obtained as a result and the above-mentioned equations 29 to 31, the unknown coefficient a _L2 of the approximate equation 22 of the second interframe dissimilarity of the frames in the observation section,
b _L2 and c _L2 are obtained. After the calculation, in step E7, the detection unit calculates the approximate expression 22 of the second interframe dissimilarity and the actual change pattern of the second interframe dissimilarity of the frame of the observation section based on the expression 39. And the error vector e _L2 of Further, the detection unit is step E
8 and E9, as shown in equations 40 and 41, step E
6 and E7, the unknown coefficient a _L3 of the approximate expression 23 of the third inter-frame dissimilarity degree of the frames in the observation section,
An error vector e _L3 between b _L3 and c _L3 and the approximate expression 23 and the actual degree of difference between the third frames of the observation section is obtained. At this time, it is preferable that the matrices D _L2 i, D _L3 i are calculated prior to the calculations of the expressions 38, 39; 40, 41, and the calculation results are used for the calculations of the expressions 38, 39; 40, 41, respectively. .

X _L2 = S _i D _L2 i (38) e _L2 = A _i X _L2- D _L2 i (39) X _L3 = S _i D _L3 i (40) e _L3 = A _i X _L3 − D _L3 i (41) After the calculation, the detection unit calculates the coefficients a _L2 , b _L2 , c _L2 , a _L3 , and b of the approximate expressions 22 and 23 obtained in steps E6 to E9.
_{Using L3} , c _L3 and the error vectors e _L2 , e _L3 , it is determined whether or not the shapes of the graphs of the above two approximate expressions 22 and 23 are predetermined shapes peculiar to the transition part. Specifically, the detection unit calculates the coefficients a _L2 , b _L2 , c _L2 , a _L3 , b _L3 , and c _L3 of the approximate expressions 22 and 23 and the error vectors e _L2 and e _L3 by the following conditional expressions 42 to 47. It is determined whether or not the condition is satisfied. In the following formula, "i / 2" is the variable i
Is half the value of. “Ε1” is a constant indicating the allowable error of the unknown constant b _L2 , and is, for example, a value (ε
= I / 8). “H _L2 ” and “H _L3 ” are the heights of parabolas that are graphs of the approximate expressions 22 and 23 of the dissimilarity between the second and third frames, and are specifically defined by expressions 48 and 49. "Xx" is the solution of the equation f _L2 (i) = f _L3 (x) at x> 0. “Ε2” is a constant indicating an allowable error, and is, for example, a value of 1/10 of the variable i (ε = i /
10). “| E _L2 |” and “| e _L3 |” are average values of the sizes of the elements of the error vectors e _L2 and e _L3 . "T
“E” is a predetermined threshold value. "Max (c _L3 , f _L3
(0)) ”is the larger value of the unknown coefficient c _L3 and the value of the approximate expression f _L3 (x) when the variable x is 0.

A _L2 <0 and a _L3 <0 (42) i / 2-ε1 <b _L2 <i / 2 + ε1 (43) b _L3 <b _L2 (44) Th <H _L2 <H _L3 ... (45) | i-xx | <ε2 ... (46) | e _L2 | <Te, and | e _L3 | <TE ... (47) _HL2 = c _L2 -f _L2 (0) ... (48) _HL3 = max (c _L3 , f _L3 (0)) − f _L3 (i) (49) The expression 42 is satisfied when the graphs of the approximate expressions 22 and 23 are both parabola convex upwards. Expression 43 is satisfied when the graph of the approximate expression 22 of the degree of difference between the second frames, that is, when the center line of the first parabola is approximately the center of the observation range. Expression 44 is an approximate expression 2 of the degree of difference between the third frames.
3 is satisfied, that is, the center line of the second parabola is closer to the origin of the coordinate system than the center line of the first parabola. Expression 45 is satisfied when the height of the second parabola is higher than the height of the first parabola, and the heights of both parabolas are equal to or higher than a predetermined height Th. Expression 46 is satisfied when the parabolas of both are close to each other near x = i on the coordinate system. Expression 47 is the approximate expression 22, 2
3 is satisfied when the error between 3 and the actual observed value, that is, the actual difference between the second and third frames is sufficiently small. That is, when all the conditional expressions 42 to 47 are satisfied,
It is expected that the change pattern of the degree of difference between the second and third frames approximated by the approximate expressions 22 and 23 within the observation section is a pattern peculiar to the transition part as shown in FIG.

As a result, the detection section carries out step E11.
Then, the conditional expression 42-
It is determined whether all 47 are satisfied. All conditional expressions 4
When 2 to 47 are satisfied, it is determined that the transition effect is occurring in the observation section. In this case, the control circuit 60 proceeds from step E11 to step E12.
Sets "TRUE" to the scene change flag Fc,
At step E14, the processing operation of the flowchart ends.

When at least one of the conditional expressions 42 to 47 is not satisfied, it is determined that the transition effect does not occur in the observation section. In this case, the process proceeds from step E11 to step E13, and the counter i is incremented by 1 and updated. As a result, the observation section is expanded by one frame. After updating, step E13 to step E4
Return to. The operations of steps E3 to E11 are the same as those of step E1.
Repeat until a transition effect is detected at 1.

In the simple shape check of step E5, if at least one of conditional expressions 35 to 37 is not satisfied, the second and third frames of the observation section are checked.
It is expected that the change pattern of the dissimilarity between frames is not convex. In this case, steps E5 to E1
After step 3, the process returns to step E4. When the value of the counter i becomes equal to or larger than the constant imax in step E4, the process proceeds from step E4 to step E14, and the processing operation of the flowchart ends. In this case, the scene change flag Fc is kept as "FALSE".

As described above, the characteristic shape detection unit of the fourth conversion unit detection device approximates the change pattern of the dissimilarity between the second and third frames within the observation period by a parabola. As a result, when the dissimilarity changes due to noise superimposed on the frame or a large change in the positional relationship between the camera and the subject at the time of shooting a single shot, the feature shape detection unit It is possible to judge robustly against a change whether or not a scene change using a transition effect has occurred. Further, the detecting unit performs a simple shape check of the change pattern before calculating and determining the approximate expression. Thereby, the detection unit
When it is clear that the change pattern of the inter-frame dissimilarity of the observation section is not a pattern peculiar to the transition part, the calculation and determination of the approximate expression regarding the observation section can be omitted. As a result, the processing amount of the detection unit is reduced.

A moving picture / scene changing portion detecting apparatus according to the fifth embodiment of the present invention will be described below. The moving picture scene change unit detection device of the fourth embodiment (hereinafter abbreviated as “fifth change unit detection device”) is more specific than the first change unit detection device 50 in the transition portion of the characteristic shape detection unit. The different detection methods are different, and the others are the same. Therefore, of the structure and operation of the fifth conversion unit detection device, description of the same structure and operation as the first conversion unit detection device 50 will be omitted. Further, of the components in the fifth conversion unit detection device, the same components as those of the first conversion unit detection device 50 are designated by the same reference numerals. In the description of the fifth conversion unit detection device, the characteristic shape detection unit may be abbreviated as “detection unit”.

The characteristic shape detection unit of the fifth conversion unit detection device is
In general, the change patterns according to the order of arrangement of the frames of the second and third inter-frame dissimilarities of the frame of interest and a plurality of frames in the vicinity thereof are respectively approximated by an equation including a sine function, and a graph of the approximation equation The mutual positional relationship and shape are judged based on the coefficient of the approximate expression, and it is checked whether or not the frame of interest and the frames in the vicinity thereof form a transition part. Specifically, the n-th in the moving image
With the range from the i-th frame to the n-th frame as the observation range, the change pattern of the second inter-frame difference degree of the frames in the observation range is approximated by a half-sine wave of one cycle as shown in FIG. As shown in FIG. 19, the change pattern of the dissimilarity between the third frames of the observation range frame is as follows.
It is approximated by a cosine wave of 1/4 of one cycle. Variable i
Is a variable for defining the observation section in the moving image, and is equivalent to the variable i described in the fourth embodiment. That is, the variable i is an integer equal to or larger than the minimum constant i0 and equal to or smaller than the maximum constant imax, and the minimum constant i0 is 5 in the present embodiment. Further, in the present embodiment, in order to calculate the approximate expression, the FFT is used as a conversion method for converting the inter-frame dissimilarity of the frames in the observation section into frequency domain data.
(Fast Fourier Transform) is used.

FIG. 20 is a flow chart for explaining a specific method of detecting a transition part in the characteristic shape detecting section of the fifth conversion section detecting device. The operation of FIG. 20 is the same as that of the whole detection process of the scene changing unit described in FIG.
This is executed by replacing the calculation in step A11. That is, in step A10 of FIG. 7, when it is determined that the scene change flag is FALSE, the process proceeds from step F1 to step F2, and the control device 60 causes the detection unit to start the transition part detection process.

In step F2, the detecting section substitutes the minimum constant i0 into the counter i to initialize the counter i prepared in advance. After that, steps F3 to F7
In the process of, the current value of the counter i is used as the value of the variable i. After the initialization, in step F3, the detection unit determines that the current value of the counter i is the maximum constant imax.
It is determined whether it is less than. If so, the process proceeds from step E4 to step E5. In step F4, the detection unit performs a simple shape check of the degree of difference between the second and third frames in the observation section. Since the simple shape check of step F4 is the same as the simple shape check of step E5 of FIG. 17 of the fourth embodiment, description thereof will be omitted. If it is determined in step F4 that all of the simple shape check conditional expressions 35 to 37 are satisfied, the process proceeds from step F4 to step F5.

In step F5, the detector detects the second and third frame dissimilarities of all the frames in the observation section,
First and second time series data groups DL2 and DL3 for FFT calculation are created based on equations 50 to 52 and 53 to 57, respectively. First and second time series data groups DL2, D
L3 is a data element DL2 [0] to DL2 [N-1], DL3 forming each time series data group DL2, DL3.
The number of [0] to DL3 [N-1] is a power of 2, and the data is equivalent to one cycle of sine wave and cosine wave. Where N is Equation 58
As shown in, it is assumed that it is a power of 2 and is larger than the value of 4 times the variable i. k is an integer.

[0172]

[Equation 3]

After the data is created, the detecting section carries out step F.
At 6, FFT is performed on the first and second time series data groups DL2 and DL3. The FFT calculation method is the same as a general FFT calculation method. As a result of the FFT operation, the first and second complex data groups FL2
[0] to FL2 [N-1], FL3 [0] to FL3 [N
-1] is obtained respectively.

After the calculation, the detecting section discriminates the waveform showing the change pattern of the original time series data groups DL2 and DL3 based on the first and second complex number data groups in step F7. Specifically, the detection unit determines whether the following seven conditional expressions 59 to 65 are satisfied based on the first and second complex number data groups. In the following equation, “| FL2 [j] |” and “| FL3 [j] |” are
The gain of the complex number data FL2 [j], FL3 [j] is shown. "Ph (FL2 [j])", "Ph (FL3
[J]) ”is the complex number data FL2 [j], FL3
The phase of [j] is shown in units of degrees. "T
L ”and“ TH ”are constants that are set in advance,
“Ε” is a permissible error, which is, for example, 5 degrees.

| FL2 [1] |> TL (59) | FL2 [j] |> TH (j = 2, 3, ..., N-2) (60) 90-ε <Ph (FL2 [1] ) <90 + ε ... (61) | FL3 [1] |> TL ... (62) | FL3 [j] |> TH (j = 2, 3, ..., N-2) (63) −ε <Ph (FL3 [1]) <+ ε ... (64) | FL3 [1] |> | FL2 [1] | ... (65) The conditional expressions 59, 60, 61, and 63 are the shapes of the graphs of the first and second time series data. This is for checking whether is close to a sine wave and a cosine wave, respectively. In particular,
The conditional expressions 59 and 62 are satisfied when the amplitude of the waveform that is the graph of the first and second time-series data is equal to or larger than the constant TL, and the conditional expressions 60 and 63 correspond to the first noise component of the waveform. This holds when the magnitudes of the high frequency components of the first and second time series data groups have a sufficiently small effect on the entire data group. Equations 61 and 64 are obtained when the phases of the waveforms of the graphs of the first and second time-series data groups are the sine wave and cosine wave shown in FIGS.
To establish. The conditional expression 65 is the second and third in the observation section.
This holds when the change pattern of the inter-frame dissimilarity has the magnitude relationship shown in FIG. That is, all conditional expressions 59 to 65
When is satisfied, the change pattern in the observation section of the degree of difference between the second and third frames approximated by the approximation formula including the sine function is a pattern peculiar to the transition part as shown in FIG. It is expected to be.

As a result, the detecting section, in step F8,
By the determination process of step F7, it is determined whether or not all conditional expressions 59 to 65 are satisfied. All conditional expressions 59-6
When 5 is satisfied, it is determined that the transition effect is occurring in the observation section. In this case, the process proceeds from step F8 to step F9, and the control circuit 60 sets the scene change flag Fc to "TRUE", and then the step F
At 11, the processing operation of the flowchart ends.

When at least one of the conditional expressions 59 to 65 is not satisfied, it is determined that the transition effect does not occur in the observation section. In this case, the process proceeds from step F8 to step F10 and the counter i is incremented by 1.
Add and update. As a result, the observation section is expanded by one frame. After updating, the process returns from step F10 to step F3. The operations of steps F3 to F10 are repeated until the transition effect is detected in step F8.

Further, in the simple shape check of step F4, if at least one of the conditional expressions 35 to 37 is not satisfied, the second and third frames of the observation section are satisfied.
It is expected that the change pattern of the dissimilarity between frames is not convex. In this case, steps F4 to F1
After 0, the process returns to step F3. When the value of the counter i becomes equal to or larger than the maximum constant imax in step F3, the process proceeds from step F3 to step F11, and the processing operation of the flowchart is finished. In this case, the scene change flag Fc is kept as "FALSE".

As described above, the characteristic shape detection unit of the fifth conversion unit detection apparatus approximates the change pattern of the degree of difference between the second and third frames within the observation period by a function including a sine function. The cosine wave can be replaced with a sine wave that is delayed or advanced in phase. As a result, when the dissimilarity changes due to noise superimposed on the frame or a large change in the positional relationship between the camera and the subject at the time of shooting a single shot, the feature shape detection unit It is possible to judge robustly against a change whether or not a scene change using a transition effect has occurred.

In the detection process of FIG. 20, step F5
The reason for creating the time series data groups DL2 and DL3 as described above is as follows. Generally, the data group to which FFT is applied is based on a detailed calculation method of the FFT.
It is preferred to include a power of n data elements. Also F
The complex number data obtained as a result of the FT is the DC component of the Fourier function showing the graph of the original time series data group, the frequency component of the first cycle, the frequency component of the second round, ... The gain, that is, the gain of each frequency component of the time series data group is shown. Therefore, when FFT is applied to a data group corresponding to a sine wave or a cosine wave that is an integral multiple of one cycle, only the gain of the frequency component of the portion corresponding to the cycle of the data group increases.

In the present embodiment, the waveform to be obtained based on the degree of difference between the second and third frames of all the frames in the observation section is the half cycle of the sine wave as shown in FIGS. And a part for a quarter cycle of the cosine wave. Further, the number of degrees of difference between the second and third frames of all the frames in the observation section is not necessarily a power of two. Based on these facts, when the FFT is directly applied to the second and third frame dissimilarities of all the frames in the observation section, the obtained complex number data group is subjected to the change pattern of the second and third frame dissimilarities. Since the shape feature does not appear clearly, it becomes difficult to determine the shape of the change pattern.

For this reason, as described with reference to FIG. 20, the detection unit determines the first and the first based on the difference degrees between the second and third frames of all the frames of the observation section and the equations 50 to 52 and 53 to 57. Two time series data groups are created, and FFT is applied to the first and second time series data groups. As a result, if the observation section is the transition part, the first and second time series data become data for one cycle of the sine wave and the cosine wave. Therefore, the detecting unit determines whether the first and second complex number data are close to the calculation result obtained by performing the FFT on the data of one cycle of the sine wave and the cosine wave, as shown in Expressions 59 to 64. By investigating, it is possible to easily determine whether or not the observation section is a transition part. The above is the reason why the time series data is created and the reason why the transition effect can be determined by the determination in step F7.

The order of various constants used in the first to fifth embodiments will be described below. This description is defined as follows. It is assumed that each pixel data set in the frame data after the form conversion is so-called RGB data. That is, each pixel data set (R, G, B) is
It is composed of three data R, G, and B indicating the brightness of red, green, and blue, and each data is 0.
It is assumed that the value is Imax or less. Further, it is assumed that one frame is divided into mmax areas. Furthermore, pixel data sets (R1, G1, B1), (R2, G
2, B2), the inter-color distance d between two pixels is defined by the following equation 66.

D = | R1-R2 | + | G1-G2 | + | B1-B2 | (66) Pixel data of all pixels in the i-th region in one of the two frames Set (R1, G1, B
1) is (0, 0, 0), and the pixel data set (R2, G2, B2) of all the pixels in the i-th area in the other frame of the two frames is (Imax, I
max, Imax), as shown in Equation 67,
The difference Di between these two areas is maximized. "Si"
Is the area in the i-th region, that is, the number of pixels in the region. When the dissimilarity Di of all the regions in the two frames is the maximum, the inter-frame dissimilarity D of the two frames is the maximum value Dm as shown in Expression 68.
It becomes ax. “SF” indicates the area of the frame and is equal to the sum of the areas of all the regions in the frame.

[0185] The maximum value Imax of the RGB data and the area S of the frame
F depends on the number of gradations of a pixel and the size of a moving image, respectively. Therefore, as shown in Expression 69, the maximum value Dmax of the inter-frame dissimilarity is normalized. Further, various parameters used in the present embodiment are respectively normalized in accordance with the normalization of the maximum value Dmax of the inter-frame difference degree.

DmaxA = Dmax ÷ (3Imax × SF) = 1.0 (69) The maximum value DmaxA after normalization of the inter-frame dissimilarity
Parameter corresponding to the maximum value DmaxA =
The parameters after normalization in the case of 1 are as follows. The cut point detection threshold value TC is a value of 0.02 or more and 0.04 or less. Step A1 of FIG. 7 of the first embodiment
The thresholds TL2 and TL3 in the conditional expression for detecting the transition part in 1 are 0.02, respectively. Steps E5 and F of FIGS. 17 and 20 of the fourth and fifth embodiments.
Simple shape check thresholds THL2 and THL3 in No. 4
Are 0.02 respectively. FIG. 1 of the fourth embodiment
The threshold value TE of the error vector when determining the shape of the parabola in step E10 of 7 is 0.006. The threshold value TE is an average value because it changes depending on the number of samples. The thresholds TL and TH at the time of the FFT result determination in step F4 of FIG. 20 of the fifth embodiment are 0.02, respectively.

[0187] The moving picture scene change unit detection devices of the first to fifth embodiments are examples of the moving picture scene change unit detection device of the present invention, and if the main operations are the same, they are carried out in various other forms. be able to. In particular, the detailed operation of each unit is not limited to this and may be realized by other operations as long as the same processing result is obtained. In addition, a new moving picture scene changing unit is provided by appropriately combining the moving picture scene changing unit detection devices of the second to fourth embodiments or the moving picture scene changing unit detection devices of the second, third and fifth embodiments. It may be a detection device.

Further, these moving image scene changing portion detecting devices store software for causing the central processing circuit to perform the above-described detecting operation of the scene changing portion in the moving image in a computer readable storage medium. It may be realized by installing the software on a computer capable of inputting and storing a moving image. This storage medium includes a CD-ROM and a floppy disk.

Further, in the fifth and fourth embodiments, in order to detect the transition portion, the change patterns of the second and third frame dissimilarities are respectively determined by a function including a triangular function and a quadratic function. It is close. The change pattern is not limited to these functions and may be approximated by other mathematical expressions. At this time, the function including the trigonometric function and the quadratic function include the convex portion in the graph,
Moreover, the parameters indicating the center, width, position, height, etc. of the convex portion can be easily extracted. Therefore, when a function including a trigonometric function and a quadratic function are used for the approximation, approximation of the change pattern and shape determination are facilitated, which is preferable.

An application example of the moving picture / scene changing portion detecting apparatus of the present invention will be described below. FIG. 21 is a block diagram showing a functional configuration of a moving image quick-viewing device 32 including the moving image scene changing portion detecting device of the present invention. The moving image watching device 32 is provided in the moving image storage unit 3.
4 and the moving image reproducing unit 35. The moving image storage unit 34 and the moving image reproducing unit 35 can be detachably attached with a medium 37 capable of storing moving images. The moving image storage unit 34 stores a moving image in addition to any one device (hereinafter, referred to as “converting unit detection device”) 31 of the moving image scene change unit detection devices of the first to fifth embodiments. The control unit 40, the detected time code storage unit 41, and the time code table storage control unit 42 are included. The moving image reproducing unit 35 includes a moving image reading unit 43, a moving image display control unit 44, a display unit 45, a time code table reading unit 46,
A read time code storage unit 47, a quick-view moving image creation unit 48,
The quick-view moving image storage unit 49 is included.

The moving image to be processed, that is, the target moving image is the same as that described in FIG. Data indicating each frame of such a target moving image is sequentially input to the moving image quick-reading device 32 as time passes. Although the target moving image and each frame in the moving image are handled in the form of data in the moving image watching device, they are referred to as “target moving image” and “frame” in the following description regardless of the handled form.

Prior to the input of the target moving image, the medium 37 is attached to the moving image storage section 33. Video recording controller 40
Causes the sequentially input frames in the target moving image to be sequentially stored in the medium 37 in a predetermined storage format. The conversion unit detection device 31 sequentially detects a scene conversion unit in the target moving image based on each frame of the target image that is sequentially input. Detection time code storage unit 41
Every time the scene changing unit is detected by the changing unit detecting device 31, a time code indicating the timing of occurrence of a frame in the scene changing unit or a frame close to the screen changing unit in the target moving image is sequentially stored. To do. As a result, the detected time code storage unit 41 includes the conversion unit detection device 3
When 1 finishes processing all the frames from the beginning to the end of the target moving image, the time codes of all the scene change parts detected from the target moving image are stored in, for example, a table format. A set of time codes of all the scene change parts is called a time code table. The time code table recording control unit 42 stores the time code table in the medium 37 after all the frames forming one target moving image are stored in the medium 37. After the storage, the medium 37 is removed from the moving image storage unit 33.

The moving picture reproducing section 34 reproduces the target moving picture stored in the medium 37 and has a quick look at the target moving picture. The quick view of the target moving image is a process of extracting and displaying at least one frame from each shot in the target moving image so that the operator can easily understand the content of the target moving image. A medium 37 in which the target moving image and the time code table are stored prior to the reproduction and the quick-view of the target moving image.
Is attached to the moving image reproducing unit 34.

When the target moving image is reproduced, after loading the medium, the moving image reading unit 43 reads the target moving image from the medium 37 and gives it to the moving image display control unit 44. The moving image display control unit 44 sequentially gives each frame of the given target moving image to the display unit 45 in the display order in a displayable data format with the lapse of time to display the frames. As a result, the target moving image is displayed on the display unit 45.

When the target moving image is previewed, the time code table reading unit 46 reads the time code table of the target moving image from the medium 37 and stores it in the read time code storage unit 47 after the medium is loaded. Quick-view movie creation section 48
Selects at least one frame from each shot of the target moving image based on the time code table in the read time code storage unit 47, and reads all selected frames from the medium 37 by the moving image reading unit 43. Let For example, the quick-view moving image creating unit 48 causes the medium 37 to read all the frames indicated by all the time codes in the time code table in the target moving image.
All the read frames are given from the moving image reading unit 43 to the quick-view moving image creating unit 48. After obtaining the frames, the quick-view moving image creating section 48 stores the given frames in the quick-view moving image storage section 49 in the same shape.
After the storage, the quick-view video creating unit 48 displays the video display control unit 44.
First, all the frames in the quick-view moving image storage unit 49 are sequentially displayed on the display unit 45 with a predetermined time interval in order from the earliest display order in the target moving image at predetermined time intervals. As a result, the result display unit 45 displays the quick-view moving image corresponding to the fast-forwarding of the target moving image. The medium 37 is realized by, for example, a VTR tape or an optical disc. Medium 37
If the optical disc is an optical disc, access to the optical disc is generally extremely fast, and thus the read frame may be stored in the quick-view moving image storage unit as it is.

Further, the quick-view moving image creating section 48 reduces all of the given frames after obtaining the frames,
All the frames after the reduction may be combined to create a frame for a list, and the frames may be stored in the quick-view moving image storage unit 49. In this case, after the frames are stored, the quick-view moving image creating unit 48 causes the moving image display control unit 44 to continue to display the list frames in the quick-view moving image storage unit 49 on the display unit 45 within a predetermined period. In addition, when a plurality of frames for the list are created, the frames for each list are sequentially displayed for a predetermined period. A list moving image corresponding to the list of each shot of the target moving image is displayed on the result display unit 45. In this case, the list moving image looks like a still image to the viewer.

As described above, the moving image watching device 32 using the conversion unit detecting device 31 of the present invention uses at least one frame extracted from each shot in the target moving image to obtain a quick moving image of the target moving image. And create a list video. On the other hand, in general, the prior art moving image watching device creates a quick watching movie and a list moving image using frames extracted every predetermined number of shots regardless of shots. As a result, when the operator views the quick-view moving images and the list moving images generated by the moving-picture watching device 32 according to the present embodiment, the operator sees the quick-view moving images and the list moving images generated by the conventional moving-picture watching device. Rather, the content of the target moving image can be grasped reliably and easily. As a result, the moving image watching device 32 of the present embodiment allows the operator to quickly grasp the content of the target moving image, as compared with the moving image watching device of the related art.
In addition, the moving image viewing device of the present embodiment allows the operator to easily search for a desired shot in the target moving image by the operator.

Further, for example, the conversion portion detecting device of the present invention is
It may be used in a moving image editing apparatus for editing the target moving image. In this case, the moving image editing apparatus can easily grasp the shots in the target moving image based on the detection result of the conversion unit detection device before editing the target moving image. You can As a result, the moving image editing apparatus can efficiently edit the target moving image.

Furthermore, for example, the conversion unit detecting device of the present invention may be used in a moving picture coding apparatus for coding the target moving picture. Generally, when encoding video,
Since the difference between the frames before and after the cut point in the moving image is large, the code amount is likely to increase, and the frame in the transition portion using the dissolve effect in the moving image has difficulty in motion prediction, so the code amount is increased. It's easy to do. In this case, the moving picture coding apparatus can easily grasp the shots in the target moving picture based on the detection result of the conversion unit detecting apparatus before the coding of the target moving picture. Also, it is possible to detect the transition portion and perform appropriate processing for suppressing the code amount of the portion. As a result, the moving picture coding apparatus can suppress the code amount of the target image after coding.

As described above, the moving picture scene change unit detecting apparatus of the present invention is not limited to the above-described various processes for the target moving picture,
It may be used in a device for any processing on a target moving image. As a result, the shot of the target image can be taken into consideration when performing the process on the target image, so that the process can be performed accurately.

[0201]

As described above, according to the first aspect of the present invention, the moving picture scene change unit detection apparatus determines the characteristic of the change in the dissimilarity between the plurality of separated frames of the frames in the moving picture based on the display order of the frames. Then, based on the characteristic, the scene change part using the transition effect is detected in the moving image. As a result, the device can accurately detect the scene change part in the moving image regardless of the type of transition effect. Further, the moving picture scene change unit detection device calculates the dissimilarity between separated frames of each frame based on the representative colors of a plurality of regions obtained by dividing each frame and the appearance frequencies thereof. As a result, the moving image scene change unit detection device can further improve the detection accuracy of the scene change unit.

According to the second aspect of the invention, the moving picture scene change unit detecting device detects the characteristic of the change in the dissimilarity between the plurality of separated frames of the frame in the moving picture based on the display order of the frame, and the characteristic is detected. Based on the above, the scene change part using the transition effect is detected in the moving image. As a result, the device can accurately detect the scene change part in the moving image regardless of the type of transition effect. Further, the moving picture scene change unit detection device outputs, as the characteristic, a comparison result of the change pattern of the dissimilarity between separated frames and a plurality of reference patterns. That is, the moving picture scene change unit detection device can easily detect the scene change section from the moving picture based on the change patterns of the dissimilarity degrees between the separated frames.

According to the third aspect of the invention, the moving picture scene change unit detecting apparatus detects the characteristic of the change in the dissimilarity between the plurality of separated frames of the frame in the moving picture based on the display order of the frame, and the characteristic is detected. Based on the above, the scene change part using the transition effect is detected in the moving image. As a result, the device can accurately detect the scene change part in the moving image regardless of the type of transition effect. Further, the moving image scene change unit detection device obtains an approximate expression that approximates the change pattern of the dissimilarity degrees between the separated frames, and obtains a comparison result of the change pattern and the reference pattern based on the approximate expression. As a result, the moving image scene change unit detection device can further improve the detection accuracy of the scene change unit. Further, according to the fourth and fifth inventions,
The approximate expression is a function including a quadratic function or a trigonometric function. As a result, the moving picture / scene changing unit detection apparatus can easily compare the change patterns with the reference patterns.

According to the sixth aspect of the present invention, the moving picture scene change unit detection device detects the characteristic of the change in the dissimilarity between a plurality of separated frames of the frame in the moving picture based on the display order of the frame, and the characteristic is detected. Based on the above, the scene change part using the transition effect is detected in the moving image. As a result, the device can accurately detect the scene change part in the moving image regardless of the type of transition effect. Further, the moving image scene change unit detection device further detects a cut point in the moving image based on the degree of difference between adjacent frames of each frame in the moving image, and determines each adjacent frame difference degree of each shot in the moving image. The correction is performed so that the component caused by the movement of the camera and the subject at the time of shooting is reduced.
As a result, the moving picture scene changer detection device can detect the scene changer that is the cut point in addition to the scene changer using the transition effect, and can improve the detection accuracy of the cut point. .

According to the seventh aspect of the invention, the moving picture scene change unit detecting device detects the characteristic of the change in the dissimilarity between a plurality of separated frames of the moving picture based on the display order of the frame, and Based on the above, the scene change part using the transition effect is detected in the moving image. As a result, the device can accurately detect the scene change part in the moving image regardless of the type of transition effect. Further, the moving image scene change unit detection device further detects a cut point in the moving image based on the degree of difference between adjacent frames of each frame in the moving image, and further, separates two consecutive frames in each shot in the moving image. It is determined that there is a cut point only when the inter-frame dissimilarity is equal to or higher than the cut point detection threshold value and the discrete inter-frame dissimilarity of a plurality of consecutive frames including the frame is equal to or higher than the threshold value. As a result, the moving image scene change unit detection device can detect a scene change unit that is a cut point, in addition to the scene change unit using the transition effect. Further, since the above device can prevent erroneous detection of the cut point due to the influence of the flash, it is possible to improve the detection accuracy of the cut point.

According to an eighth aspect of the present invention, a moving picture processing device comprises the moving picture scene changing part detecting device according to any one of the first to seventh inventions, and the detection result of the moving picture scene changing part detecting device is obtained. The input moving image is processed based on. As a result, the moving image processing apparatus can reduce the load required to detect the scene change unit and improve the moving image processing accuracy when processing a moving image in shot units.

According to the ninth invention, the moving picture processing device extracts a plurality of frames from the moving picture based on the detection result and sequentially displays the frames. Still further, according to the tenth invention, the moving picture processing device extracts a plurality of frames from the moving picture based on the detection result, reduces and combines the respective frames, and the resultant combination is obtained. Display the frame. These results,
The operator of the moving picture processing device can grasp the content of the original moving picture in a proper and short time.

According to the eleventh aspect of the present invention, in the moving image scene changing portion detecting method, the characteristic of the change in the dissimilarity between the plurality of separated frames of the frame in the moving image based on the display order of the frame is detected, and the characteristic is detected. Based on the above, the scene change part using the transition effect is detected in the moving image. As a result, when the method is used, it is possible to accurately detect the scene change part from the moving image regardless of the type of transition effect. Further, in the moving picture scene change portion detection method, the dissimilarity between separated frames of each frame is calculated based on the representative colors of a plurality of areas obtained by dividing each frame and the appearance frequencies thereof. As a result, when the method is used, the detection accuracy of the scene change unit can be further improved. Further, according to the twelfth invention, in the moving picture scene change part detecting method, a characteristic of a change in the dissimilarity between a plurality of separated frames of a frame in the moving picture based on the display order of the frame is detected, and based on the characteristic. , A scene change part using a transition effect is detected in the moving image. As a result, when the method is used, it is possible to accurately detect the scene change part from the moving image regardless of the type of transition effect.
Further, in the moving picture scene change portion detection method, as the characteristic, a comparison result of the change pattern of the dissimilarity between separated frames and a plurality of reference patterns is output. That is, when the method is used, the scene change part can be easily detected from the moving image based on the change patterns of the dissimilarity degrees between the separated frames. Further, according to the thirteenth invention, in the moving picture scene change part detecting method, the characteristic of the change in the dissimilarity between the plurality of separated frames of the frame in the moving picture is detected based on the display order of the frame, and based on the characteristic. , A scene change part using a transition effect is detected in the moving image. As a result, if the method is used,
Regardless of the type of transition effect, it is possible to accurately detect the scene change section from the moving image. In addition, the moving image scene change portion detection method obtains an approximate expression that approximates the change pattern of the dissimilarity degrees between the separated frames, and obtains the result of comparison between the change pattern and the reference pattern based on the approximate expression. As a result, when the method is used, the detection accuracy of the scene change unit can be further improved. Further, according to the fourteenth and fifteenth inventions, the approximate expression is a function including a quadratic function or a trigonometric function. As a result, when the method is used, the change patterns can be easily compared with the reference patterns. Further, according to the sixteenth invention, in the moving picture scene change part detecting method, the characteristic of the change in the dissimilarity between the plurality of separated frames of the moving picture based on the display order of the frames is detected, and based on the characteristic. , A scene change part using a transition effect is detected in the moving image. As a result, when the method is used, it is possible to accurately detect the scene change part from the moving image regardless of the type of transition effect. Further, in the moving image scene change detection method, based on the dissimilarity between adjacent frames of each frame in the moving image, the cut point in the moving image is further detected, and the adjacent frame dissimilarity is calculated for each shot in the moving image. The correction is performed so that the component caused by the movement of the camera and the subject at the time of shooting is reduced. As a result, when the method is used, it is possible to detect a scene change part that is a cut point in addition to the scene change part using the transition effect,
In addition, it is possible to improve the detection accuracy of the cut points.
Further, according to the seventeenth invention, in the moving picture scene change portion detecting method, a characteristic of a change in the dissimilarity between the plurality of separated frames of the moving picture based on the display order of the frames is detected, and based on the characteristic. , A scene change part using a transition effect is detected in the moving image. As a result, when the method is used, it is possible to accurately detect the scene change part from the moving image regardless of the type of transition effect. Further, in the moving picture scene change part detecting method, a cut point in the moving picture is further detected based on the degree of difference between adjacent frames of each frame in the moving picture, and further, two continuous frames in each shot in the moving picture are discretized. It is determined that there is a cut point only when the inter-frame dissimilarity is equal to or higher than the cut point detection threshold value and the discrete inter-frame dissimilarity of a plurality of consecutive frames including the frame is equal to or higher than the threshold value. As a result, when the above method is used, in addition to the scene change part using the transition effect, the scene change part which is the cut point can be detected. Further, in the above method, it is possible to prevent erroneous detection of the cut point due to the influence of the flash, so that it is possible to improve the detection accuracy of the cut point.

[0209] According to the eighteenth invention, the medium is
The program includes the moving picture scene change part detection control program for causing the computer to operate in the same manner as the moving picture scene change part detecting device of the first to seventh inventions when installed and executed in the computer. As a result of these, when the program is executed by a computer, the scene change unit can be accurately detected from the moving image regardless of the type of transition effect, and the amount of calculation required for the detection of the scene change unit. Can be reduced and the accuracy of the detection can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of a moving image / scene changing unit detection device 31 according to a first embodiment of the present invention.

FIG. 2 is a diagram showing transition parts in a moving image and frames in the vicinity thereof, arranged in display order.

FIG. 3 is a graph schematically showing a change in the dissimilarity between separated frames of a transition part and a frame in the vicinity thereof in a moving image according to the order of arrangement of frames.

FIG. 4 is a graph specifically showing a change in the dissimilarity between the first to third frames of the transition part in the moving image and the frames in the vicinity thereof according to the arrangement order of the frames.

FIG. 5 is a diagram showing a relationship between a comparison frame and a transition part in a case where a frame of interest exists in a preceding shot, a transition part, and a subsequent shot in calculating a difference between frames.

FIG. 6 is a graph specifically showing a change in the dissimilarity between the first to third frames of the cut point in the moving image and the frames in the vicinity thereof according to the order of arrangement of the frames.

FIG. 7 is a moving image / scene changing unit detection device 3 according to the first embodiment.
6 is a flowchart for explaining a process of detecting a scene change part in a moving image in No. 1 in FIG.

8 is a diagram illustrating a difference buffer between second and third frames stored in a plurality of difference buffers in second and third buffer groups in a difference storage unit 56 in the detection process of FIG. 3 is a graph showing the relationship with the buffer number of.

9 is a difference buffer between the second and third frames stored in the plurality of difference buffers in the second and third buffer groups in the difference storage unit 56 and the difference buffer in the detection process of FIG. 7. 3 is a graph showing the relationship with the buffer number of.

FIG. 10 is a diagram schematically illustrating a method of extracting a frame feature of each frame in a moving image and a method of calculating an inter-frame difference degree in the moving image scene change unit detection device according to the second embodiment of the present invention. is there.

FIG. 11 is a diagram for explaining a method of extracting a frame feature of an arbitrary one frame in a frame feature extraction unit in the moving picture scene change unit detection device of the second exemplary embodiment.

FIG. 12 is a diagram showing the frame feature extraction method of FIG.
It is a figure for demonstrating the division | segmentation state of a frame.

FIG. 13 is a diagram for explaining a method of calculating an inter-frame dissimilarity between arbitrary two frames in a dissimilarity calculation unit in the moving picture scene change unit detection apparatus according to the second embodiment.

FIG. 14 is a first diagram before a correction of a cut point and a frame in the vicinity thereof in order to explain a method of detecting the cut point in the cut point detection unit of the moving picture / scene conversion unit detection device according to the third embodiment of the present invention. 7 is a graph showing a change in inter-frame dissimilarity according to the arrangement order of frames.

FIG. 15 is a diagram for explaining a method of detecting a cut point in the cut point detection unit of the moving picture scene change unit detection apparatus according to the third embodiment, in which the corrected first frame difference between the cut point and the frames in the vicinity thereof. It is a graph which shows the change according to the arrangement order of the degree frame.

FIG. 16 is a flowchart for explaining a method of detecting cut points in a cut point detection unit of the moving picture / scene conversion unit detection device according to the third embodiment.

FIG. 17 is a flowchart for explaining a method of detecting a transition part in the characteristic shape detection unit of the moving picture / scene conversion unit detection device according to the fourth embodiment of the present invention.

FIG. 18 shows a waveform to approximate a change pattern of a second inter-frame dissimilarity in a method of detecting a transition part in the characteristic shape detecting section of the moving picture / scene changing section detecting apparatus according to the fifth embodiment of the present invention. It is a graph.

FIG. 19 is a graph showing a waveform for approximating a change pattern of a third inter-frame difference degree in the method of detecting a transition part in the characteristic shape detection unit of the moving picture / scene conversion unit detection device of the fifth exemplary embodiment.

FIG. 20 is a flow chart for explaining a method of detecting a transition part in the feature shape detection unit of the moving picture / scene conversion unit detection device of the fifth exemplary embodiment.

FIG. 21 is a block diagram showing an electrical configuration of a moving image viewing device 32 including the moving image scene change unit detection device of the present invention.

FIG. 22 is a schematic diagram showing the structure of a moving image.

FIG. 23 is a diagram in which frames at a cut point in a moving image, a transition part using a dissolve and a wipe, and frames in the vicinity thereof are arranged in the frame display order.

[Explanation of symbols]

31 Video scene change unit detection device 51 Frame development section 53 In-frame feature extraction unit 55 Dissimilarity calculator 57 Characteristic Shape Detector 58 Cut detector 59 Detection result output section 60 control circuit

Claims (18)

(57) [Claims] 1. A moving picture memory for storing a moving picture composed of a plurality of shots, wherein each shot has a same or similar characteristic and is composed of a plurality of frames to be sequentially displayed with the passage of time. Means for dividing each frame into one or a plurality of areas, detecting at least one color representing each area in each frame and the frequency of appearance of the color in each area, Characteristic data generating means for outputting a combination of detected colors and appearance frequencies of all regions in the frame as characteristic data indicating the characteristic of each frame; and one of the frames in the moving image. The degree of difference in characteristics between one frame and a plurality of second frames that are respectively separated from the first frame by a plurality of predetermined reference numbers in the display order of the frames in the moving image is calculated. And a separation frame inter-dissimilarity calculation unit that calculates the plurality of dissimilarity degrees between the separated frames, which are calculated by shifting the first frame based on the characteristic data, and a separation frame corresponding to each second frame of the frames. The characteristics of the change of the degree of inter-dissipation according to the display order, characteristic detecting means for respectively detecting, based on the characteristics detected by the characteristic detecting means,
A moving picture scene changing unit detection device, comprising: a transition detection unit that detects a composite frame having a transition effect, which is composed by superimposing frames that should be adjacent in display order in a moving picture, from a moving picture. 2. A moving picture memory for storing a moving picture composed of a plurality of shots, wherein each shot has the same or similar characteristics and is composed of a plurality of frames to be sequentially displayed over time. And a plurality of second frames that are separated from the first frame by a plurality of reference numbers of a first frame, which is one of the frames in the moving image, that is predetermined in the display order of the frames in the moving image. Between the plurality of separated frames which calculate the difference between the separated frames respectively indicating the degree of difference between the separated frames, and between the separated frame difference calculating means for calculating the difference between the separated first frames and the plurality of separated frames of the plurality of frames whose display order is continuous. It is judged whether or not the change pattern of the dissimilarity degree according to the display order matches each of a plurality of predetermined reference patterns, and The characteristics of change according the display order of the separating inter-frame dissimilarity corresponding to the second frame, and the characteristic detecting means for detecting each, based on the detected characteristic by the characteristic detecting means,
A transition detection unit that detects a composite frame having a transition effect that is composed by overlapping frames that should be adjacent to each other in the moving order in the moving image; and in any one of the reference patterns, the separation of any one of the reference patterns. The first reference pattern to be compared with the change pattern of the interframe dissimilarity, and the change pattern of the disengagement frame dissimilarity, in which the reference number at the time of calculation is larger than any one of the dissociation frame dissimilarities, are compared. The second reference pattern to be used has a convex shape in the graph showing each of the first and second reference patterns, and the height and width of the graph of the first reference pattern are each the graph of the second reference pattern. An apparatus for detecting a moving image scene transition part, characterized in that it is less than the height and width of 3. A moving picture memory for storing a moving picture composed of a plurality of shots, wherein each shot has a same or similar characteristic and is composed of a plurality of frames to be sequentially displayed over time. And a plurality of second frames that are separated from the first frame by a plurality of reference numbers of a first frame, which is one of the frames in the moving image, that is predetermined in the display order of the frames in the moving image. Dissimilarity between dissimilarity frames that respectively express the degree of dissimilarity between dissimilarity frames, and dissimilarity dissimilarity between dissimilarity frames between a plurality of frames whose display order is continuous, The approximation formula calculating means for obtaining the respective mathematical formulas of the graph that respectively approximates the degree change pattern, and Based on the parameter, it is determined whether or not the change pattern approximated by the graph of each equation matches each of the reference patterns, and the difference degree between the separated frames corresponding to each of the second frames of the frame is determined. The characteristics of the change according to the display order, based on the characteristics detected by the characteristic detection unit, the characteristic detection unit, respectively,
A moving picture scene changing unit detection device, comprising: a transition detection unit that detects a composite frame having a transition effect, which is composed by superimposing frames that should be adjacent in display order in a moving picture, from a moving picture. 4. The apparatus according to claim 3, wherein the mathematical formula is a quadratic function, and the parameter is a coefficient of the calculated quadratic function. 5. The moving image scene conversion unit according to claim 3, wherein the mathematical expression is a function including a sine function, and the parameters are a phase and a gain of the sine function in the calculated function. Detection device. 6. A moving picture memory for storing a moving picture composed of a plurality of shots, wherein each of the shots has a same or similar characteristic and is composed of a plurality of frames to be sequentially displayed with the passage of time. And a plurality of second frames that are separated from the first frame by a plurality of reference numbers of a first frame, which is one of the frames in the moving image, that is predetermined in the display order of the frames in the moving image. Dissimilarity difference calculation means for calculating a plurality of dissociation frame dissimilarities representing the degree of dissimilarity between the dissimilarity frames, and dissimilarity dissimilarity differences corresponding to each of the second frames of the frames. Characteristics of the change according to the display order of the degree, based on the characteristics detected by the characteristic detection unit, the characteristic detection unit,
A transition detection unit that detects a composite frame having a transition effect in which a frame that should be adjacent in the display order in the moving image is overlapped, and a transition detection unit that detects, from each frame in the moving image, a display order of the respective frames. From adjacent frame difference calculation means for calculating the difference between adjacent frames indicating the difference in feature between the frames having only one different display order and the respective frames, and from the difference between adjacent frames of each frame, Difference between adjacent frames in which the sum of products of the difference between adjacent frames of the remaining frames excluding the respective frames out of the plurality of frames including the respective frames and having a continuous display order and a plurality of predetermined weighting factors are respectively subtracted From the moving image, a degree correction means and Each of the weighting factors, which includes a frame detection unit for outputting and a cut point detection unit for detecting a cut point in the moving image based on the detected frame, and which is multiplied by a difference between adjacent frames of each of the remaining frames. Is a weight coefficient multiplied by a dissimilarity between adjacent frames smaller than the dissimilarity between adjacent frames among the dissimilarities between adjacent frames of the remaining frames, or more. . 7. A moving picture memory for storing a moving picture composed of a plurality of shots, each of the shots having the same or similar characteristics and each of which is composed of a plurality of frames to be sequentially displayed with the passage of time. And a plurality of second frames that are separated from the first frame by a plurality of reference numbers of a first frame, which is one of the frames in the moving image, that is predetermined in the display order of the frames in the moving image. Dissimilarity difference calculation means for calculating a plurality of dissociation frame dissimilarities representing the degree of dissimilarity between the dissimilarity frames, and dissimilarity dissimilarity differences corresponding to each of the second frames of the frames. Characteristics of the change according to the display order of the degree, based on the characteristics detected by the characteristic detection unit, the characteristic detection unit,
A transition detection unit that detects a composite frame having a transition effect in which a frame that should be adjacent in the display order in the moving image is overlapped, and a transition detection unit that detects, from each frame in the moving image, a display order of the respective frames. An adjacent frame difference degree calculating unit that calculates the difference degree between adjacent frames indicating the difference in feature between each frame having only one different display order and each frame, and the difference degree between adjacent frames is calculated in advance from the moving image. Among all the frames having the cut point detection threshold or more to be defined, the dissimilarity between separated frames of any one of the plurality of frames including the respective frames and having a continuous arrangement order is a frame having a threshold value or more, Frame detecting means for detecting, and cut point detecting means for detecting a cut point in the moving image based on the detected frame. Video scene change portion detection apparatus characterized by. 8. A display means capable of displaying a frame, the moving picture scene changer detection device according to claim 1, and a moving picture of a composite frame detected by the moving picture scene changer detection device. A frame extraction unit that extracts at least one frame from each shot in the moving image based on a position within the frame, and a display control unit that displays the extracted frame on the display unit. Video processing device. 9. The display control means causes the display means to display all of the extracted frames in order from the earliest display order in the moving image over time. The moving image processing apparatus according to claim 8. 10. The display control means reduces the extracted frames, and causes the display means to display a combined frame obtained by combining the reduced frames. Video processing device. 11. A process of storing a moving image composed of a plurality of shots each of which has the same or similar characteristics and which should be sequentially displayed over time, and a process of storing each frame. It is divided into one or a plurality of regions, at least one color representing each region in each frame and the frequency of appearance of the color in each region are respectively detected, and all the regions in each frame are detected. A process of outputting a combination of the detected color and the appearance frequency as feature data indicating the feature of each frame, and a process of outputting a first frame, which is one of the frames in the movie, of a frame in the movie. Differences between a plurality of separated frames, each of which represents a degree of difference in characteristics from a plurality of second frames that are separated from the first frame by a plurality of reference numbers that are predetermined in display order Based on the characteristic data, the processing of calculating the first frame by shifting the first frame, and the characteristic of change in the dissimilarity between separated frames corresponding to the second frames of the frame according to the display order are respectively detected. And a process of detecting a composite frame having a transition effect in which a frame in which a display order in the moving image should be adjacent to each other is overlapped, from the moving image, based on the detected characteristic. Method for detecting moving picture scene transition part. 12. A process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed over time, and a process of storing the moving image. The degree of difference in the characteristics of the first frame, which is any one of the frames, from the plurality of reference frames, which are separated from the first frame by a plurality of predetermined reference numbers in the display order of the frames in the moving image, respectively. A process of calculating a plurality of dissociation degrees between separated frames by respectively shifting the first frame, and a change pattern associated with the display order of the dissimilarities between separated frames of a plurality of frames in which the display order is continuous are predetermined. Differences between separated frames corresponding to the respective second frames of the frames are determined by determining whether or not they match a plurality of reference patterns. The process of detecting the respective characteristics of the change according to the display order of, and, based on the detected characteristics, a composite frame having a transition effect in which the frames that should be adjacent in the display order in the moving image are combined, A first reference pattern to be compared with the change pattern of the dissimilarity degree between the separated frames, and a difference between the separated frames, among all the reference patterns. The second reference pattern to be compared with the change pattern of the dissimilarity between separated frames, in which the reference number of times when calculating is larger than the degree, has a convex shape in the graph showing each of the first and second reference patterns, Further, the height and width of the graph of the first reference pattern are less than the height and width of the graph of the second reference pattern, respectively. A method for detecting a video scene transition part. 13. A process of storing a moving image composed of a plurality of shots each of which has the same or similar features and which should be sequentially displayed with the passage of time, and The degree of difference in the characteristics of the first frame, which is any one of the frames, from the plurality of reference frames, which are separated from the first frame by a plurality of predetermined reference numbers in the display order of the frames in the moving image, respectively. A process of calculating the dissimilarity between the separated frames by shifting the first frame, and a mathematical expression of a graph approximating the change pattern of the dissimilarity between the separated frames of the plurality of frames in which the display order is continuous, respectively. Based on the processing to be obtained and the parameters determined based on the plurality of obtained mathematical formulas, the variables approximated by the graphs of the respective mathematical formulas are obtained. A process of determining whether or not a coded pattern matches each of the reference patterns, and detecting a characteristic of a change in the dissimilarity between dissociated frames corresponding to each of the second frames in the display order, A moving image scene conversion process for detecting, from the moving image, a combined frame having a transition effect in which frames that should be adjacent in the display order in the moving image are overlapped and combined based on the detected characteristics. Part detection method. 14. The method according to claim 13, wherein the formula is a quadratic function, and the parameter is a coefficient of the calculated quadratic function. 15. The moving image scene conversion unit according to claim 13, wherein the mathematical expression is a function including a sine function, and the parameters are a phase and a gain of the sine function in the calculated function. Detection method. 16. A process of storing a moving image composed of a plurality of shots, each of which has the same or similar features and which should be sequentially displayed over time, and a process of storing the moving image. The degree of difference in the characteristics of the first frame, which is any one of the frames, from the plurality of second frames that are respectively separated from the first frame by a plurality of predetermined reference numbers in the display order of the frames in the moving image. A process of calculating a plurality of dissimilarity between separated frames by shifting the first frame, and a characteristic of a change in the dissimilarity between separated frames corresponding to each of the second frames in the display order are detected. Based on the detected characteristics, the transition effect that is composed by overlapping the frames that should be adjacent in the display order in the moving image is combined. A process of detecting a composite frame having a result from a moving image, and showing, for each frame in the moving image, a difference between the display order of each frame and a frame in a display order different by one and the feature of each frame From the processing of calculating the dissimilarity between adjacent frames, and from the dissimilarity between adjacent frames of each frame, the adjacency of the remaining frames excluding the respective frames among a plurality of frames including the frames and having a continuous display order From the moving image, a process of subtracting the sum of the products of the dissimilarity between frames and a plurality of predetermined weighting factors, and a frame in which the dissimilarity between adjacent frames after the subtraction is equal to or greater than a predetermined cut point detection threshold value from the moving image A process of detecting, and a process of detecting a cut point in the moving image based on the detected frame. Each of the weighting factors to be calculated is equal to or more than a weighting factor by which the dissimilarity between adjacent frames, which is smaller than the dissimilarity between adjacent frames among the dissimilarities between adjacent frames of the residual frame, is multiplied. Video scene transition detection method. 17. A process of storing a moving image composed of a plurality of shots each of which has the same or similar characteristics and which should be sequentially displayed over time, and a process of storing the moving image. The degree of difference in the characteristics of the first frame, which is any one of the frames, from the plurality of second frames that are respectively separated from the first frame by a plurality of predetermined reference numbers in the display order of the frames in the moving image. A process of calculating a plurality of dissimilarity between separated frames by shifting the first frame, and a characteristic of a change in the dissimilarity between separated frames corresponding to each of the second frames in the display order are detected. Based on the detected characteristics, the transition effect that is composed by overlapping the frames that should be adjacent in the display order in the moving image is combined. A process of detecting a composite frame having a result from a moving image, and showing, for each frame in the moving image, a difference between the display order of each frame and a frame in a display order different by one and the feature of each frame A process of calculating the dissimilarity between adjacent frames, and a plurality of consecutive frames in which the dissimilarity between adjacent frames is equal to or higher than a predetermined cut point detection threshold value in the moving image and includes the respective frames And a process of detecting a frame in which the dissimilarity between separated frames of any one of the frames is all equal to or greater than a predetermined threshold value, and a process of detecting a cut point in the moving image based on the detected frame. A method for detecting a moving image scene transition part, which is characterized by the above. 18. A medium for storing a moving picture scene change detection control program for causing a computer to execute the moving picture scene change detection method according to any one of claims 11 to 17.