JP4656199B2 - Image processing apparatus, thumbnail movie creation method, and thumbnail movie creation program - Google Patents

Description

  The present invention relates to various image processing apparatuses such as a PC (Personal Computer) and a PDA (Personal Digital Assistant) in addition to a digital camera and a movie camera that can shoot moving images, for example. The present invention relates to an image processing apparatus, a thumbnail moving image creating method and a thumbnail moving image creating program used in the apparatus.

  2. Description of the Related Art Conventionally, image processing apparatuses that handle moving images, such as digital cameras that can shoot moving images, are known that create thumbnail images using one frame in moving image data (see, for example, Patent Documents 1 and 2). A thumbnail image is an image that is used as an index when finding a head or grasping image contents, and is also called an index image or a preview image.

  Usually, in this type of image processing apparatus, an image (still image) positioned at the head is automatically selected from moving image data in which each image is continuous in time series, and is used as a thumbnail image. However, the image at the head position is not necessarily the optimum image as a thumbnail image. Therefore, Patent Document 1 proposes that the user can arbitrarily set an optimal image as a thumbnail image from moving image data. Japanese Patent Application Laid-Open No. 2004-228620 proposes that an optimum thumbnail image corresponding to a scene change or the like can be created together with creation of a free thumbnail image by user selection.

JP 2001-169231 A

Japanese Patent Laid-Open No. 2000-278641

  However, in Patent Documents 1 and 2 and the like, an image used as a thumbnail (index) is a still image corresponding to one frame in moving image data. For this reason, this kind of thumbnail image lacks dynamic elements, and there is a problem that it is not possible to immediately know which scene was shot by just looking at that one frame.

  The present invention has been made in view of the above points, and includes an image processing apparatus, a thumbnail moving image generating method, and a thumbnail moving image generating program that have dynamic elements and can generate an optimal thumbnail corresponding to a scene change. The purpose is to provide.

The invention of claim 1 wherein the time series and the moving image storage means for storing moving image data is still image data continuous, the two adjacent in time-series in the moving image data stored in the video storage unit still A motion amount detection means for detecting a motion amount that is a dissimilarity between the images in time series, and a time point when the motion amount detected in time series by the motion amount detection means becomes greater than a predetermined threshold. From the moving image data, a scene turning point detecting means for detecting a certain scene turning point, a cutting time determining means for determining a cutting time according to the amount of motion detected by the motion amount detecting means at the scene turning point , wherein as the base point scene turning point scene turning points detected by the detecting means, a partial moving a moving image data cutout time period determined by the cutout time determining means, cuts out the dynamic A cutout section, as a base point a scene turning point detected by the scene turning point detecting means, wherein to said cut time determined by the cut-out time determining means elapses, said to prohibit the detection of a new scene turning point A cut-out control means for controlling the scene turning point detection means; and a storage control means for storing the partial video clipped by the video cut-out means in the video storage means in association with the video data as a thumbnail video. Features.

According to a second aspect of the invention, the cut time determining means, said higher amount of motion is large is detected by the motion amount detecting means definitive scene turning point, characterized in that to increase the cut time to determine.

The invention according to claim 3 is a thumbnail moving image creation method of an image processing apparatus provided with moving image storage means for storing moving image data which is still image data that is continuous in time series, and is stored in the moving image storage means. the dissimilarity in a motion amount between two still images adjacent in a time series in the moving image data, when the motion amount detection step of sequentially detect, are detected in time series in the motion amount detection step A scene turning point detecting step for detecting a scene turning point when the amount of movement becomes larger than a predetermined threshold , and a cutout time is set according to the amount of movement detected in the motion amount detecting step at the scene turning point. and cut out the time determination step of determining, from the moving image data, as a base point detected scene turning point in the scene turning point detecting step, the cut time determining scan The Tsu partial moving a cut time of the video data determined in the flop, and video clipping step of cutting out, as a base point the detected scene turning point in the scene turning point detecting step, determined by the cut time determining step A cutout control step for controlling the scene turning point detection step so as to prohibit detection of a new scene turning point until the cutout time elapses , and the partial video cut out in the moving image cutout step as the thumbnail video. And a storage control step of storing in the moving image storage means in association with data .

According to a fourth aspect of the present invention, there is provided a moving image storage means for storing moving image data, which is still image data continuous in time series, and two adjacent images in time series in the moving image data stored in the moving image storage means. A motion amount detecting means for detecting a motion amount which is a dissimilarity between the still images in a time series, and a time point when the motion amount detected in a time series by the motion amount detecting means is larger than a predetermined threshold value. A scene turning point detecting means for detecting a scene turning point, a cutting time determining means for determining a cutting time according to the amount of motion detected by the motion amount detecting means at the scene turning point, from the video data, as a base point scene turning point detected by the scene turning point detecting means, a partial moving a moving image data cutout time period determined by the cutout time determining means, switching Video cut means issuing said as a base point a scene turning point detected by the scene turning point detecting means, wherein to said cut time determined by the cut-out time determining means has elapsed, so as to inhibit the detection of a new scene turning point cut control means for controlling the scene turning point detecting means, a cut out portion moving in the moving cut step, that function as storage control means for storing in said video storage means in association with the moving image data as a thumbnail video Features.

According to the present invention, it is possible to create an optimal thumbnail having dynamic elements and corresponding to a scene change.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus of the present invention includes all apparatuses that handle moving image data such as a PDA (Personal Digital Assistant) and a personal computer in addition to a digital camera having a moving image shooting function. Here, a configuration as a general computer is shown.

  As shown in FIG. 1, the image processing apparatus according to the present embodiment includes a CPU 11, an input device 12, a display device 13, a storage device 14, a communication device 15, an interface 16, and an imaging device 17.

  The CPU 11 controls the entire apparatus. By reading a program stored in the storage device 14, the CPU 11 executes various processes according to procedures described in the program. The input device 12 is an input device including, for example, a keyboard, a mouse, a touch panel, a pen, and the like, and is used for inputting various data and giving instructions. The display device 13 is a display device such as an LCD (Liquid Crystal Display), for example, and displays various data on the screen.

  The storage device 14 can be configured by various memory devices such as a ROM, a RAM, a CD-ROM, a DVD-ROM, and a memory card, and stores various types of information necessary for the device. The storage device 14 includes a program storage area 14a, a moving picture data storage area 14b, a thumbnail moving picture storage area 14c, a thumbnail composite moving picture storage area 14d, and the like.

  Various programs including a program for realizing the present invention are stored in the program storage area 14a. The moving image data storage area 14b stores moving image data shot by the imaging device 17. The thumbnail moving image storage area 14c stores a plurality of thumbnail moving images created based on the moving image data stored in the moving image data storage area 14b. This thumbnail moving image is extracted from moving image data, and is not a still image but has a predetermined time length. The thumbnail composite video storage area 14d stores a thumbnail composite video created by combining a plurality of thumbnail videos.

  The communication device 15 performs data communication with an external terminal via a network. The network includes a communication line such as a wired or wireless LAN (Local Area Network) in addition to the Internet. An imaging device 17 having a moving image shooting function is detachably connected to the interface 16. The moving image data shot by the imaging device 17 is stored in the moving image data storage area 14b of the storage device 14.

  FIG. 2 shows an example of moving image data shot by the imaging device 17.

  The moving image data is composed of a plurality of still images (frames) continuous in time series. In the images of the scenes before and after adjacent on the time axis, the higher the correlation between the two images (the higher the similarity), the lower the amount of image movement between the two. Conversely, the lower the correlation (the lower the similarity), the higher the amount of image motion between the two. Therefore, it can be determined that the scene has changed greatly when the amount of movement is equal to or greater than a predetermined value. The part where the scene has changed greatly is called a scene turning point.

  In the example of FIG. 2, among the moving image data composed of the images a1... An, b1... Bn, c1. The correlation is low and the amount of motion is high. When this amount of motion is greater than or equal to a predetermined value, this is detected as a scene turning point.

  Next, a processing operation as the first embodiment of the present apparatus will be described.

  Now, description will be made assuming that a moving image file having moving image data shot by the imaging device 17 is stored in the moving image data storage area 14b of the storage device 14 and a thumbnail moving image is created from the moving image data.

  FIG. 3 is a flowchart showing the overall flow of image processing in the first embodiment of the present apparatus. Each process shown in this flowchart is stored in the apparatus in the form of a program, and is executed by the CPU 11 reading this program.

  When a moving image file for thumbnail creation is specified through the input device 12, the CPU 11 first reads out the moving image data in the specified moving image file from the moving image data storage area 14b of the storage device 14, and stores the moving image data. A scene turning point is detected (step A11). The scene turning point is a portion where the scene has changed greatly, and is determined by the amount of motion of the image as described above.

  The scene turning point detection process will be described in detail with reference to FIG. FIG. 4 is a flowchart showing the flow of the scene turning point detection process in the first embodiment of the present apparatus.

  As shown in FIG. 4, the CPU 11 extracts a luminance signal from the images of the preceding and succeeding scenes on the time axis (step B11), and the amount of motion representing the correlation between the images at each position based on the luminance signal. Is obtained (step B12). As described above, the higher the correlation between the two images (the higher the similarity), the lower the amount of movement of the images between them, and the lower the correlation (the lower the similarity), The amount of movement is high.

  Here, the amount of motion is obtained from the luminance signal of the image, but the amount of motion may be obtained using other information such as a color signal and a color difference signal. Furthermore, the amount of motion is one index indicating the degree of scene change, and if there is other information indicating the degree of scene change, that information may be used.

  When the movement amount for each position in both images is obtained, the CPU 11 obtains a total value, an average value, or a mean square value of these values (step B13), and then uses that value as the movement amount S of the image between them. It compares with the set threshold value Th (step B14). As a result, if the motion amount S is lower than the threshold Th (No in step B14), the CPU 11 determines that there is no scene change (step B15). On the other hand, when the motion amount S is equal to or greater than the threshold value Th (Yes in Step B14), the CPU 11 determines that there is a scene change (Step B16).

  Similarly, the CPU 11 sequentially checks the images of the scenes before and after the moving image data in chronological order to determine whether or not there is a scene change. As a result, in the example of the moving image data shown in FIG. 2, the area between the image an and the image b1, and the image bn and the image c1 are detected as scene turning points.

  Returning to FIG. 3, when the scene turning point in the moving image data is detected (Yes in step A12), the CPU 11 determines the cutout time of the partial moving image used as the thumbnail moving image (step A13). This cut-out time is determined based on the amount of movement S when it is determined in the scene turning point detection process that there is a scene change. In this case, as shown in FIG. 5 (a), a method of proportionally determining the extraction time (time length of the thumbnail video) according to the amount of movement S at that time, and as shown in FIG. 5 (b) There is a method of determining the extraction time (time length of the thumbnail video) in a stepwise manner according to the amount S. Either method may be applied, and the method may be arbitrarily selected.

  Note that when the amount of motion S between the images of the moving image data fluctuates in the short period near the threshold Th that is a guideline for scene change, the scene turning point every time the amount of motion S exceeds the threshold Th. As a result, a large number of similar thumbnail videos are created. Therefore, in order to avoid such a problem, when the motion amount S first becomes equal to or greater than the threshold value Th, a scene turning point detection operation is performed for a period from that point to at least the extraction time corresponding to the motion amount S. It is preferable to add control to prohibit.

  Returning to FIG. 3, when the cut-out time is determined, the CPU 11 performs cut-out processing of a portion to be used as a thumbnail moving image according to the cut-out time based on the scene turning point of the moving image data (step A14). As a clipping method in this case, as shown in FIG. 6, for example, when the clipping time is T, a moving image corresponding to the clipping time T is clipped from a scene before and after the scene turning point at a predetermined ratio (n: m). There are a method and a method of extracting a moving image for the extraction time T from the scene after the scene turning point. Either of these methods may be applied, and the method may be arbitrarily selected. .

  In this way, when a portion to be used as a thumbnail moving image is cut out from the moving image data, the CPU 11 compresses the cut partial moving image according to a predetermined encoding method such as MPEG (Motion Picture Expert Group) as necessary, A moving image for thumbnail (index) is created by reducing the vertical and horizontal sizes to a predetermined size (step A15). Then, the CPU 11 stores the thumbnail moving image in the thumbnail moving image storage area 14c of the storage device 14 in association with the moving image data (step A16).

  In this way, a portion where the scene changes greatly is found in the moving image data, and a thumbnail moving image is created by cutting out that portion. In this case, if there are a large number of scene change points, that is, scene turning points, a corresponding number of thumbnail videos are created in the thumbnail video storage area 14c. These thumbnail videos have different time lengths. The length depends on the degree of scene change (specifically, the amount of motion of the image), and the longer the moving image, the longer the thumbnail video is created.

  FIG. 7 shows an example of a thumbnail video.

  Assume that four thumbnail moving images Va, Vb, Vc, and Vd are created from moving image data as shown in FIG. These thumbnail videos Va, Vb, Vc, Vd have different time lengths. Here, when these thumbnail videos Va, Vb, Vc, Vd are displayed in chronological order, for example, as shown in FIG. 5B, the thumbnail video Va is 3 seconds, the thumbnail video Vb is 2 seconds, and the thumbnail video Vc. Is 1 second, and the thumbnail video Vd is 3 seconds, so that the moving image of each scene is displayed for each time.

  These thumbnail videos correspond to scene changes and have dynamic elements unlike still images. Therefore, it is possible to immediately determine which scene is captured when each thumbnail video is displayed. Moreover, since it is displayed for a long time when the scene changes greatly, it is possible to easily grasp the entire contents from these thumbnail videos.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  In the first embodiment, when the motion amount S is greater than or equal to the threshold Th, it is determined that there is a scene change, and the cut-out time is determined according to the motion amount S at that time, so that the cut-out is based on the scene change point. Cut out partial videos to be used as thumbnail videos according to time. On the other hand, the second embodiment is characterized in that a partial moving image is cut out with a period from when it is determined that there is a scene change based on the amount of motion S until the amount of motion S falls below a predetermined value. And

  This will be specifically described with reference to FIG.

  Now, as shown in FIG. 8A, there is moving image data including a plurality of images a1, b1,..., Cn continuous in time series, and the motion amount S of the moving image data is shown in FIG. It shall have the following characteristics.

  Here, two threshold values of Th1 and Th2 are prepared. The threshold value Th1 is for detecting a scene turning point, and corresponds to the threshold value Th in the first embodiment. The threshold value Th2 is used to detect the cutout end point of the partial moving image, and is set lower than the threshold value Th1.

  When there is a large change between the image a1 and the image b1, and the movement amount S at that time becomes equal to or greater than the threshold value Th1, extraction of a partial moving image used as a thumbnail moving image at that time is started. In the first embodiment, the extraction time is determined according to the amount of movement S at this time. On the other hand, in the second embodiment, the partial moving image is continuously cut out until the motion amount S becomes equal to or less than the threshold Th2. In this example, since the motion amount S is greater than or equal to the threshold Th1 from the image b1 to the image c1, the clipping is performed as it is. Then, when the motion amount S becomes equal to or less than the threshold value Th2 from the image c1 to the image cn, the extraction operation is ended with this as the extraction end point. In this way, the partial video clipped from the video data is used as a thumbnail video.

  Hereinafter, the processing operation as the second embodiment will be described in detail with reference to FIG.

  FIG. 9 is a flowchart showing the overall flow of image processing in the second embodiment of the present apparatus. Each process shown in this flowchart is stored in the apparatus in the form of a program, and is executed by the CPU 11 reading this program.

  When a moving image file for thumbnail creation is specified through the input device 12, the CPU 11 first reads out the moving image data in the specified moving image file from the moving image data storage area 14b of the storage device 14, and stores the moving image data. A scene turning point is detected (step C11). The scene turning point is a portion where the scene has changed greatly, and is determined according to the amount of motion of the image as described above.

  The scene turning point detection process is the same as that in the first embodiment. That is, as described with reference to FIG. 4, the amount of motion S between them is obtained based on the luminance signals of the images of the preceding and following scenes adjacent on the time axis, and the amount of motion S and a preset threshold Th (here In this case, when the amount of motion S is equal to or greater than the threshold Th1, it is detected that there is a scene change. Note that the motion amount may be obtained using other information such as a color signal and a color difference signal in addition to the luminance signal. Furthermore, the amount of motion is one index indicating the degree of scene change, and if there is other information indicating the degree of scene change, that information may be used.

  When a scene turning point in the moving image data is detected (Yes in step C12), the CPU 11 starts to extract a partial moving image used as a thumbnail moving image from the moving image data at the scene turning point (step C13). As described with reference to FIG. 8, this cutting operation is continued until the motion amount S becomes equal to or less than the threshold value Th2 (Th2 <Th1) (steps C14 and C15). When the motion amount S becomes equal to or less than the threshold Th2 (Yes in Step C15), the CPU 11 ends the partial moving image cutting operation at that time (Step C16).

  In this way, when a portion to be used as a thumbnail moving image is cut out from the moving image data, the CPU 11 compresses the cut partial moving image according to a predetermined encoding method such as MPEG (Motion Picture Expert Group) as necessary, A moving image for thumbnail (index) is created by reducing the vertical and horizontal sizes to a predetermined size (step C17). Then, the CPU 11 stores the thumbnail moving image in the thumbnail moving image storage area 14c of the storage device 14 in association with the moving image data (step C18).

  As described above, the first moving image data can be found by continuously finding out a portion of the moving image data where the scene changes greatly, and until the degree of the scene change becomes a predetermined value or less. Similar to the embodiment, it is possible to create thumbnails based on moving images, that is, thumbnail moving images. And by doing this, it is possible to adaptively cut out the video from the scene where the change of the scene was detected to the scene where a series of intense movements fit, and the video cut out as a thumbnail is well organized, The contents can be easily understood.

  In this case, if there are a large number of scene change points, that is, scene turning points, a corresponding number of thumbnail videos are created in the thumbnail video storage area 14c. These thumbnail videos have different time lengths. In the second embodiment, the length depends on the continuity of the scene change, and the longer the scene change time is, the longer the thumbnail video is created.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  The third embodiment is characterized in that a plurality of thumbnail videos are combined to create a single composite image (referred to as a thumbnail composite video), and the time of each thumbnail video constituting the thumbnail video is adjusted. To do.

  This will be specifically described with reference to FIG.

  Assume that the thumbnail videos Va, Vb, Vc, and Vd obtained from the video data are inserted into a 4-split screen and displayed as a single thumbnail composite video. In this case, the thumbnail video Va is 3 seconds, the thumbnail video Vb is 2 seconds, the thumbnail video Vc is 1 second, and the thumbnail video Vd is 3 seconds, each having a different time. Therefore, when a thumbnail composite video is displayed by simply combining these thumbnail videos Va, Vb, Vc, and Vd, there are portions that end earlier and portions that end later.

  Therefore, in order to eliminate such a difference in display time, the thumbnail moving images Va, Vb, Vc, and Vd constituting the thumbnail combined moving image are adjusted to the same time by performing frame thinning processing and interpolation processing.

  As a method for adjusting the time, there are a first method for taking the least common multiple of the time of each thumbnail video, a second method for taking the greatest common divisor, and a third method for adjusting to a preset time. In the first method of taking the least common multiple, for example, when the time of each thumbnail video is 1 second, 2 seconds, 3 seconds, and 4 seconds, the length of the thumbnail composite video is 12 seconds that is the least common multiple of these. To match. Further, in the second method of taking the greatest common divisor, for example, when the time of each thumbnail video is 2 seconds, 4 seconds, 6 seconds, and 12 seconds, the length of the thumbnail composite movie is set to the greatest common divisor. Set to 2 seconds. Further, in the third method of adjusting to a preset time, the length of the thumbnail composite video is adjusted to a fixed time such as 5 seconds, regardless of the time of each thumbnail video.

  Here, if the time is adjusted to a reference time (referred to as a combined thumbnail time Tc), a thumbnail video having a time longer than the combined thumbnail time Tc corresponds to a difference from the combined thumbnail time Tc. For a thumbnail video having a time shorter than the combined thumbnail time Tc, a frame corresponding to the difference from the combined thumbnail time Tc is applied. Interpolation processing (processing that extends time by interpolating frames) is performed to unify the time of each thumbnail video. In the example of FIG. 10, the combined thumbnail time is 1 second. Accordingly, frame thinning processing for time reduction is performed on the thumbnail videos Va, Vb, and Vd except the thumbnail video Vc. As a result, when a thumbnail composite video combining these is displayed on a 4-split screen, the thumbnail video Va, Vb, Vc, Vd therein is displayed for the same time (1 second).

  In the example of FIG. 10, a thumbnail composite video is created using four thumbnail videos, but the number is not particularly limited, and all thumbnail videos obtained from video data are used. Alternatively, any number of thumbnail videos among them may be used. Furthermore, the arrangement of each thumbnail video is not limited to a matrix. In short, any form may be used as long as a plurality of thumbnail videos can be combined and displayed as a single thumbnail composite video.

  Hereinafter, the processing operation as the third embodiment of the present apparatus will be described in detail with reference to FIG. Here, a description will be given on the assumption that all thumbnail videos obtained from the video data are sequentially inserted into the divided screen to create one thumbnail composite video.

  FIG. 11 is a flowchart showing the overall flow of image processing in the third embodiment of the present apparatus. Each process shown in this flowchart is stored in the apparatus in the form of a program, and is executed by the CPU 11 reading this program.

  When a moving image file for thumbnail creation is specified through the input device 12, the CPU 11 first reads out the moving image data in the specified moving image file from the moving image data storage area 14b of the storage device 14, and stores the moving image data. A scene turning point is detected (step D11). The scene turning point is a portion where the scene has changed greatly, and is determined according to the amount of motion of the image as described above. Since this scene turning point detection process is the same as that in the first embodiment, description thereof is omitted.

  When a scene turning point in the moving image data is detected (Yes in step D12), the CPU 11 cuts out a partial moving image used as a thumbnail moving image based on the scene turning point (step D13). As a method for extracting the partial moving image, there are a method as described in the first embodiment and a method as described in the second embodiment, and either method may be used. The partial video clipped here is stored as one thumbnail video in association with the video data in the thumbnail video storage area 14c of the storage device 14 (step D14). At that time, the image is compressed by the MPEG method or the like as necessary, and the vertical and horizontal sizes are reduced to a predetermined size.

  In this way, scene turning points are detected in order from the beginning of the moving image data, partial moving images are cut out at the scene turning points, and are sequentially stored as thumbnail moving images in the thumbnail moving image storage area 14c.

  When the video data is processed to the end (Yes in step D15), the CPU 11 creates a single thumbnail composite video by combining a plurality of thumbnail videos stored in the thumbnail video storage area 14c in the following procedure.

  That is, the CPU 11 first determines the combined thumbnail time Tc (step D16). The composite thumbnail time Tc is a time (reference time) for displaying each thumbnail video included in the thumbnail composite video in common, and is determined by taking the least common multiple of the time of each thumbnail video as described above. There are a first method, a second method that takes the greatest common divisor, and a third method that adjusts to a preset time. Any of these methods may be applied, and each method is arbitrarily selected. You may be able to do it.

  When the combined thumbnail time Tc is determined in this way, the CPU 11 processes each thumbnail moving image in time series. First, the CPU 11 takes out the first thumbnail video from the thumbnail video storage area 14c (step D17), and compares the time of the thumbnail video with the combined thumbnail time Tc (steps D18a and D18b).

  Here, if the time of the thumbnail video is the same as the combined thumbnail time Tc (Yes in step D18a), the CPU 11 inserts the thumbnail video as it is in a predetermined position of the divided screen (step D21). Specifically, the thumbnail moving image is expanded in the thumbnail moving image storage area 14d of the storage device 14 so as to be displayed by being inserted into a predetermined position of the divided screen. The split screen is a screen for individually displaying each thumbnail video constituting the thumbnail composite video, and for example, as shown in FIG. 10, the thumbnail video is arranged in a matrix and displayed as a list. It has become.

  On the other hand, if the thumbnail video time is different from the composite thumbnail time Tc (No in step D18a), and if the thumbnail video time is longer than the composite thumbnail time Tc (Yes in step D18b), the thumbnail video time is shortened. Thus, it is necessary to match the combined thumbnail time Tc. Therefore, the CPU 11 thins out frames corresponding to the difference from the combined thumbnail time Tc from the thumbnail video (step D19).

  Conversely, if the thumbnail video time is shorter than the composite thumbnail time Tc (No in step D18b), it is necessary to extend the thumbnail video time to match the composite thumbnail time Tc. Therefore, the CPU 11 interpolates frames corresponding to the difference from the combined thumbnail time Tc between the frames of the thumbnail video (step D20).

  As the frame thinning or frame interpolation processing, for example, there is a method of determining the frame number thinning or interpolation ratio from the ratio of the time of the thumbnail video currently being processed and the combined thumbnail time Tc. It is done. That is, for example, when the time of the thumbnail video currently being processed is 5 seconds and the combined thumbnail time Tc is 2 seconds, 3 frames are thinned out for 5 thumbnail videos, leaving 2 frames. , And so on.

  At this time, from the viewpoint of generating a thumbnail video without losing the continuity of the video as much as possible, it is desirable that the frames to be thinned out or interpolated should be equally spaced as much as possible. It is desirable to avoid processing such as. When thinning out or interpolating evenly in this way, one frame is thinned out or interpolated from the ratio of the time of the thumbnail video currently being processed and the combined thumbnail time Tc as described above. Find out if you need to do it and process it accordingly.

  As described above, in order to adjust the time of the thumbnail video to the combined thumbnail time Tc, after performing an editing process such as frame thinning or interpolation on the thumbnail video, the CPU 11 converts the edited thumbnail video to a predetermined position on the divided screen. (Step D21).

  Thereafter, similarly, until the processing of all thumbnail moving images is completed (No in step D22), the CPU 11 sequentially extracts the thumbnail moving images from the thumbnail moving image storage area 14c in chronological order, and repeats the processing of steps D18a to D21. In this way, each thumbnail video is inserted into each position of the divided screen to create one thumbnail composite video. In this case, since the time of each thumbnail video is adjusted to the composite thumbnail time Tc, when the thumbnail composite video is displayed, a portion that ends earlier and a portion that ends later are created. Each is displayed during the same time (combined thumbnail time Tc). Therefore, the user can view it as a single moving image without a feeling of strangeness.

  In the process shown in the flowchart of FIG. 11, the thumbnail composite video is created using all the thumbnail videos extracted from the video data, but some of them are optionally used. It is also possible to create a thumbnail composite video.

  Further, the moving image data may be a monochrome image or a color image, and a thumbnail moving image obtained from the moving image data and a thumbnail combined moving image obtained by combining the thumbnail moving images are not limited to a monochrome image / color image.

  Further, the moving image data to be processed by the present invention may be directly captured from a digital camera or a movie camera, or may be obtained from a recording medium such as a CD-ROM or DVD-ROM, or may be transmitted through a network or the like. It may be provided from an external terminal.

  Furthermore, the method described in each of the embodiments described above is a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc., as a program that can be executed by a computer. The program itself can be applied to various apparatuses, or the program itself can be transmitted through a transmission medium such as a network to be applied to various apparatuses. A computer that implements this apparatus reads a program recorded on a recording medium or a program provided via a transmission medium, and performs the above-described processing by controlling operations by this program.

1 is a block diagram showing a configuration of an image processing apparatus according to a first embodiment of the present invention. The figure which shows an example of the moving image data image | photographed with the imaging device used for the said image processing apparatus. 6 is a flowchart for explaining the overall flow of image processing in the first embodiment. The flowchart for demonstrating the flow of the scene turning point detection process in 1st Embodiment. The figure which shows the relationship between the moving amount | distance of the moving image in 1st Embodiment, and extraction time. The figure for demonstrating the extraction method of the partial moving image used as a thumbnail moving image in 1st Embodiment. The figure which shows an example of the thumbnail moving image in 1st Embodiment. The figure for demonstrating the cutting-out method of the partial moving image used as a thumbnail moving image in the 2nd Embodiment of this invention. 10 is a flowchart for explaining the overall flow of image processing in the second embodiment. The figure for demonstrating creation of the thumbnail synthetic | combination moving image which combines the some thumbnail moving image in the 3rd Embodiment of this invention. 10 is a flowchart for explaining the overall flow of image processing in the third embodiment.

Explanation of symbols

11 ... CPU
DESCRIPTION OF SYMBOLS 12 ... Input device 13 ... Display device 14 ... Storage device 14a ... Program storage area 14b ... Movie data storage area 14c ... Thumbnail movie storage area 14d ... Thumbnail composition movie storage area 15 ... Communication device 16 ... Interface 17 ... Imaging device

Claims (4)

Moving image storage means for storing moving image data which is still image data continuous in time series;
A motion amount detecting means for the motion amount is a dissimilarity between the two still images adjacent in a time series in the moving image data stored in the video storage means, time series to detect,
A scene turning point detecting means for detecting a scene turning point when the amount of motion detected in time series by the movement amount detecting means becomes larger than a predetermined threshold;
Extraction time determining means for determining the extraction time according to the amount of motion detected by the motion amount detection means at the scene turning point;
From the moving picture data, wherein as a base point scene turning point scene turning points detected by the detecting means, a partial moving a moving image data cutout time period determined by the cutout time determining means, moving cutout means for cutting out and ,
As a base point a scene turning point detected by the scene turning point detecting means, said cutout to said cutout time determined elapses by the time determination means detects the scene turning point so as to inhibit the detection of a new scene turning point Cutting control means for controlling the means;
An image processing apparatus comprising: a storage control unit configured to store the partial moving image cut out by the moving image cutting unit as a thumbnail moving image in association with the moving image data in the moving image storage unit. The cut time determining means, said higher amount of motion is large is detected by the motion amount detecting means definitive scene turning point, determining the image processing apparatus according to claim 1, wherein the longer a cut time. A thumbnail video creation method for an image processing apparatus provided with video storage means for storing video data which is still image data continuous in time series,
A motion amount detection step of a motion amount, time series to detect a dissimilarity between two still images adjacent in a time series in the moving image data stored in said video storage means,
A scene turning point detecting step for detecting a scene turning point at which the amount of motion detected in time series in this motion amount detecting step becomes larger than a predetermined threshold;
A cutout time determining step for determining a cutout time according to the amount of motion detected in the motion amount detection step at the scene turning point;
From the moving picture data, wherein as a base point scene turning point detecting scene turning points detected in step, a partial moving a moving image data cutout time period determined in the cut time determination step, the moving image cutout step of cutting out ,
As a base point a scene turning points detected in the scene turning point detecting step, the cut time to determine the cut time determined in step elapses, detects the scene turning point so as to inhibit the detection of a new scene turning point and clipping control step of controlling the steps,
And a storage control step of storing the partial video clipped in the video cutout step in the video storage means in association with the video data as a thumbnail video. Computer
Moving image storage means for storing moving image data which is still image data continuous in time series,
The motion amount is dissimilarity between the two still images adjacent in a time series in the moving image data stored in the video storage means, time series detected motion amount detecting means,
A scene turning point detecting means for detecting a scene turning point at which the amount of movement detected in time series by the movement amount detecting means becomes larger than a predetermined threshold;
Extraction time determining means for determining the extraction time according to the amount of motion detected by the motion amount detection means at the scene turning point;
Wherein the moving image data, wherein as a base point scene turning point scene turning points detected by the detecting means, a partial moving a moving image data cutout time period determined by the cutout time determining means cuts out video cut means,
As a base point a scene turning point detected by the scene turning point detecting means, said cutout to said cutout time determined elapses by the time determination means detects the scene turning point so as to inhibit the detection of a new scene turning point Cutting control means for controlling the means,
A thumbnail video creation program that functions as a storage control unit that stores the partial video clipped in the video cutout step as a thumbnail video in association with the video data and stored in the video storage unit.

Images