JP4193552B2 - Image data recording / reproducing apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
According to the present invention, when moving image data is continuously recorded, still image data is stored non-continuously, so that when moving image data is reproduced and displayed, the still image data at a desired timing is high-definition. about the image data recording and reproducing apparatus and methods were to be produced as a state.
[0002]
[Prior art]
In recent years, digital video cameras capable of recording moving image data and still image data have been developed and actually put into practical use. The moving image data and still image data recorded by this camera can be reproduced and displayed at any time later.
[0003]
Incidentally, Patent Document 1 discloses that moving image data and still image data are generated from a single image sensor at non-identical timing. In Japanese Patent Laid-Open No. 2004-260, high-definition still image data is recorded when a still image recording instruction is issued during moving image recording while making the system common. Furthermore, in Patent Document 3, moving image data and still image data are recorded simultaneously, and from the moving images, what fully expresses the contents of the moving images is automatically extracted as a representative image. Has been. Furthermore, in Patent Document 4, the search efficiency at the time of reproduction is enhanced by superimposing each link information in moving image data and still image data.
[0004]
[Patent Document 1]
JP-A-8-298644 [Patent Document 2]
JP-A-10-108121 [Patent Document 3]
JP 7-23322 A [Patent Document 4]
JP 2000-69407 A
[Problems to be solved by the invention]
However, in the simultaneous recording of moving image data and still image data so far, still image data is recorded mainly at a request timing from the operator while moving image data is being recorded. Accordingly, when reproducing and displaying moving image data, even if there is a still image display request from the operator, still image data at that timing is not necessarily recorded in advance, and this is why A still image cannot always be obtained.
[0006]
Therefore, when a still image at a certain timing is required for printing or the like, there is only a method of converting the moving image data (frame data) at that timing as it is to still image data. It is a fact. However, even if moving image data is converted into still image data as it is, the number of pixels of moving image data is generally smaller than that of still image data. Therefore, even if the moving image data is converted into still image data by an image density conversion circuit or the like. Even if it is converted into image data, it is merely the appearance that the number of pixels is the same as that of still image data, and it cannot be said that it is sufficient for applications such as printing.
[0007]
An object of the present invention is to provide an image data recording / reproducing apparatus and method that can easily generate still image data at a desired timing in a high-definition state when reproducing and displaying moving image data.
[0008]
[Means for Solving the Problems]
According to the present invention, an image capturing unit that captures an image of a subject, and image information from the image capturing unit are digitally image-processed so that a pair of moving image data and still image data with higher definition than the moving image data are obtained. When the still image data is recorded discontinuously based on the image data generating means to be generated, the recording timing generating means for generating the still image data recording timing, and the still image data recording timing from the recording timing generating means. In addition, the moving image data paired with the still image data is accompanied by link information to the still image data as frame information, and image data recording for recording continuous moving image data and discontinuous still image data. And a request for reproduction of still image data as a frame during reading and reproduction of moving image data from the image data recording means , Moving image data corresponding to the timing of the reproduction request, moving image data with link information to still image data located near the time of the moving image data, and accompanying the moving image data Reproduction request still image data generation means for generating still image data related to the reproduction request based on still image data specified by the link information, and the reproduction request still image data generation means includes a still image related to the reproduction request. When generating data, a motion vector, which is information indicating a moving direction and a moving distance of a subject included in the moving image displayed by the moving image data by image recognition of the moving image data, is calculated, and the stillness specified by the link information is calculated. A still image in which the subject included in the still image displayed as image data is moved based on the motion vector is generated, and the subject is moved It is intended to interpolate the data lack portion by utilizing the difference information of the moving image data in.
[0009]
As described above, when moving image data is continuously recorded, still image data is recorded discontinuously, and there is a request for reproducing still image data at a certain timing during the reproduction of moving image data. In this case, among the still image data recorded in advance, the still image data closest in time in terms of timing, the moving image data paired with the still image data, and the moving image corresponding to the timing Still image data at that timing is generated from the image data. This still image data has the same number of pixels as the original still image data, but its definition is slightly reduced, but compared to still image data obtained by converting the number of pixels of moving image data, It will be obtained as a finer one.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described below with reference to FIGS.
First, the image data recording / reproducing apparatus according to the present invention will be described. FIG. As shown in the figure, the photographic lens 11 and an imaging device (comprising a CCD image sensor, a CMOS image sensor, etc. for converting an optical image collected by the photographic lens 11 into an electrical signal, etc., are more effective than the number of effective pixels of the NTSC system or PAL system. 12 having a large number of pixels), a camera signal processing block 13, a moving image recording / reproducing block 14, a still image recording / reproducing block 15, an image processing circuit 16, a display circuit 17, and a user interface block 18.
[0011]
Among the above components, the internal configuration of the main components will be described. The camera signal processing block 13 includes an A / D converter 131 that converts an analog signal output from the image sensor 12 into a digital signal, A digital signal processing circuit 132 that performs white balance adjustment, gamma correction, and the like on the digital data output from the D converter 131 is included. The moving image recording / reproducing block 14 also performs an image density conversion circuit 141 that performs a thinning process on the moving image data from the digital signal processing circuit 132 and performs an interpolation process on the read compressed moving image data. The moving image data from the circuit 141 is compressed, and the moving image compression / decompression circuit 142 that decompresses the read compressed moving image data, and the compressed moving image data from the moving image compression / decompression circuit 142 are recorded. On the other hand, a recording medium 143 from which compressed moving image data is read out during reproduction of moving image data.
[0012]
The still image recording / reproducing block 15 performs a compression process on the still image data from the digital signal processing circuit 132, and a still image compression / decompression circuit 151 that performs a decompression process on the read compressed still image data. The compressed still image data from the decompression circuit 151 is recorded, and the recording medium 152 from which the compressed still image data is read when the still image data is reproduced is configured. The display circuit 17 is also for confirming display of image data related to recording / reproduction.
[0013]
Further, the moving image controller 181 in the user interface block 18 is for an operator to instruct start / stop of moving image data recording and start / stop of reproduction, and the still image controller 182 also stores moving image data. During reproduction, the operator instructs acquisition of still image data, and a still image acquisition button is provided. Furthermore, the image processing circuit 16 automatically generates timing for recording still image data by monitoring still image data through image processing when moving image data is continuously recorded. When reproducing moving image data, image processing for generating still image data is performed, or image data is selectively output to the display circuit 17.
[0014]
First, the recording operation in the image data recording / reproducing apparatus will be described. When the recording start button on the moving image controller 181 is pressed by the operator, continuous recording of moving image data as a frame is started. . The condensed image obtained by the photographing lens 11 is converted into an analog signal by the image pickup device 12, and further converted into a digital signal by the A / D converter 131, and further, white balance adjustment and gamma correction are performed by the digital signal processing circuit 132. Processing is performed. The moving image data obtained in this manner is subjected to thinning processing by the image density conversion circuit 141, thereby obtaining the effective number of pixels in the NTSC system or the PAL system. Thereafter, the moving image compression / decompression circuit The data is compressed on the recording medium 142 and recorded on the recording medium 143.
[0015]
The moving image data before image density conversion output from the digital signal processing circuit 132 is also high-definition still image data as a frame at the same time, but in parallel with the continuous recording of the above moving image data, The processing circuit 16 constantly monitors the still image data. When it is determined by this monitoring that still image data should be recorded, such as when there are many image changes, the still image compression / decompression circuit 151 is put into an operable state, so that the still image data is compressed. In addition, it is recorded on the recording medium 152. In this way, in a state where continuous recording of moving image data and non-continuous recording of still image data are being performed, in order to stop these recordings, the operator presses the recording stop button on the moving image controller 181. It only has to be operated.
[0016]
As described above, moving image data is recorded continuously and still image data is recorded discontinuously. FIG. 2 shows the relationship between moving image data and still image data in this case. . As shown in the figure, for each moving image data 21 recorded continuously, the still image data paired with the moving image data 21 is not continuously recorded, and the still image in the image processing circuit 16 is not recorded. High-definition still image data 22 and 23 from the digital signal processing circuit 132 are recorded sporadically only when it is determined that still image data should be recorded as a result of data monitoring, for example, there are many image changes. The In other words, after the still image data 22 is recorded, the still image data is not recorded continuously. As a result of monitoring the still image data, a result of image data difference, motion detection, or the like. Is still greater than a certain threshold with respect to the still image data 22, that is, when the image change exceeds a certain level, still image data 23 is recorded.
[0017]
Still image data may also be recorded at fixed arbitrary time intervals as an easier method. Furthermore, it is also conceivable to record still image data by combining recording based on a monitoring result of still image data and recording at an arbitrary time interval. In any case, the moving image data recorded in this way has link information to the still image data recorded at the same time as the frame information. More specifically, file names 211 and 212 of still image data recorded at the same time are recorded in the subcode area of the moving image data. Therefore, it is known whether or not a pair of still image data is simultaneously recorded for each piece of continuously recorded moving image data depending on whether or not the file name is recorded.
[0018]
Next, the playback operation in the image data recording / playback apparatus will be described. When the playback start button on the moving image controller 181 is pressed by the operator, the playback of the moving image data is started. The compressed moving image data recorded on the recording medium 143 is sequentially read from the recording medium 143, is expanded by the moving image compression / expansion circuit 142, and is further interpolated by the image density conversion circuit 141. Thus, the same image data having the same number of pixels as the still image data can be obtained. The moving image data obtained in this way is then sequentially processed by the image processing device 16 and displayed on the display circuit 17. Incidentally, in this example, the number of display pixels on the display circuit 17 is the same as the number of pixels of the still image data. Therefore, the moving image data is displayed after interpolation processing, but this is not essential. In other words, the moving image data expanded by the moving image compression / decompression circuit 142 may be displayed on the display circuit 17 via the image processing circuit 16 as it is.
[0019]
Now, when the moving image data is continuously reproduced and displayed on the display circuit 17, the best scene will be discovered and the operator wishes to obtain the frame as still image data. A still image acquisition button on the still image controller 182 is pressed. With this as an opportunity, the image processing circuit 16 performs processing for generating requested still image data from the operator. This generation process will be described. In this example, assuming that the still image acquisition button is pressed at the timing 31 shown in FIG. 3, the generation process procedure of the requested still image data is as follows.
[0020]
That is, as a relatively easy method, a method of generating from only the difference value can be considered. In the case of this method, moving image data 33 corresponding to the timing 31, moving image data 32 immediately before it and having link information to still image data, and the moving image data 32 are recorded at the same timing. Still image data 24 is used to generate requested still image data 26. More specifically, FIG. 4 shows a generation flow in one example. In this case, first, the image processing circuit 16 generates difference data by subtracting the moving image data 32 from the moving image data 33 as the difference data of the moving image data 32 and 33 (step 41). Thereafter, the difference data is interpolated to the number of pixels of the still image data to be applied to the still image data 24 (step 42). The result of this interpolation process may be added to the still image data 24 (step 43). As a result, the requested still image data 26 that matches the request timing from the operator is obtained. It can be said that the required still image data 26 obtained in this way maintains a high-definition state for an area where the difference data is small.
[0021]
By the way, in the above, moving image data immediately before and having link information to still image data is used, but moving image data having link information to still image data is used immediately after. May be. More specifically, as shown in FIG. 3, the difference data is calculated by subtracting the moving image data 34 from the moving image data 33 as the difference data of the moving image data 33 and 34, and then the difference data. May be added to the still image data 25 after interpolation processing to the number of pixels of the still image data. As a result, as in the previous case, high-definition required still image data 26 is obtained. Of course, these two methods may be combined, and a pair of moving image data and still image data recorded at a time position away from the timing 31 instead of immediately before and immediately after may be used. It is possible.
[0022]
Next, the generation method using the motion vector of an arbitrary subject existing in the moving image data will be described. Similar to the above method, the moving image data 33 corresponding to the timing and the immediately preceding still image data The moving image data 32 having link information to and the still image data 24 recorded at the same timing as the moving image data 32 are used to generate the requested still image data 26. More specifically, it is as shown in FIG.
[0023]
That is, the image processing circuit 16 calculates a motion vector by performing image recognition on the two pieces of moving image data 32 and 33. As a result, the subject at the position 35 has moved to the position 36, in other words, the position 35 of the subject and the moving direction / distance 51 are obtained. In order to apply this to the still image data 24, conversion is performed so as to correspond to the angle of view of the still image data 24, and the subject position 52 and the moving direction / distance 53 are obtained. The requested still image data 26 is obtained by applying the data thus obtained to the still image data 24. In this case, there is no region data corresponding to the position 52 in the requested still image data 26, but the portion that cannot be compensated by the motion vector is obtained from still image data and moving image data captured at other timings. As a result of applying the interpolation to the still image data 24 by interpolating in the same manner or using the difference data, the required still image data 26 matching the request timing from the operator is obtained. It can be said that the required still image data 26 obtained in this way maintains a high-definition state for an area where the difference data is small.
[0024]
In the above description, moving image data immediately before and having link information to still image data is used. However, moving image data having link information to still image data is used immediately after. May be. If it demonstrates concretely, as shown in FIG. 3, the motion vector from the moving image data 33 to the moving image data 34 will be calculated. As a result, it is obtained that the subject at the position 37 has moved from the position 36, and for the part that cannot be compensated by the motion vector, the difference data is used and this is applied to the still image data 25. As a result, high-definition requested still image data 26 that matches the request timing from the operator is obtained. Also in this method, the above two methods may be combined, and the moving image data and still image data as a pair recorded at a time position away from the timing 31 instead of immediately before and immediately after are recorded. It can also be used.
[0025]
Further, apart from the above method, it is also possible to use by detecting that an arbitrary subject in the moving image data is enlarged or reduced, or detecting that it is rotated. In any case, the requested still image data 26 obtained by the method as described above can be displayed on the display circuit 17, and since this is high-definition still image data, it is displayed in an enlarged manner. However, the subject can be clearly confirmed. On the other hand, the requested still image data 26 can be recorded on the recording medium 152 as being compressed by the still image compression / decompression circuit 151. As described above, if the recording is performed on the recording medium 152, the required still image data 26 itself has high definition, so that an image quality suitable for printing or the like can be easily obtained.
[0026]
In the above description, the still image data recording timing is automatically generated by constantly monitoring still image data. However, it can be automatically generated by constantly monitoring moving image data. . Specifically, when moving image data is recorded, the output moving image data from the image density conversion circuit 141 is monitored by the image processing circuit 16. Even in this case, as in the previous case, when it is determined that still image data should be recorded, such as when there are many image changes, the still image compression / decompression circuit 151 is put into an operable state, thereby Data is recorded on the recording medium 152 in a compressed state.
[0027]
When a still image recording button is provided on the still image controller 182, not only still image data is automatically recorded during recording of moving image data but also a still image recording button intentionally by the operator himself. It is conceivable that still image data is recorded intentionally at an arbitrary timing by pressing the button. The requested still image data generated by a request from the operator when reproducing moving image data is obtained by image processing using difference data, motion vectors, etc., when the timing is a frame in which still image data is not recorded. Therefore, it is generally difficult to obtain high-quality high-definition still image data. However, if there is a frame that the operator may request later when recording moving image data, if it is recorded in advance as still image data, high-quality still image data can be obtained more easily. Can be obtained. Further, if the frame requested by the operator is in the vicinity thereof, it is conceivable that the interpolation accuracy can be improved. For example, this is the case where the still image data recording timing is slightly deviated, that is, the shutter timing is slightly deviated.
[0028]
Here, referring to recording of moving image data and still image data on the recording media 143 and 152, preferably, the recording media 143 and 152 are the same recording medium. When the same recording medium is used, it is also conceivable to record still image data itself instead of link information in the subcode area shown in FIG. For example, when a tape is used as a recording medium, moving image data may be recorded in the moving image area of the tape format, and still image data may be recorded in the subcode area.
[0029]
Furthermore, when moving image data and still image data are recorded on the same recording medium, it is conceivable that these image data are recorded as being in the same file. FIG. 6 shows an example file format. As illustrated, the file 61 includes management information 611, moving image data 612, and still image data 613 as elements. By defining the format so that the management information 611 has link information between the moving image data 612 and the still image data 613, the moving image data and still image data generated by the present invention are handled as one file. Is possible.
[0030]
Finally, reproduction of high-definition moving images will be described. In the above description, high-definition required still image data is generated each time for an arbitrary frame. However, when required still image data is generated for all frames, high-definition moving image playback is performed. It is clear that is possible.
[0031]
【The invention's effect】
There is provided an image data recording / reproducing apparatus and method capable of easily generating still image data at a desired timing in a high-definition state when reproducing and displaying moving image data.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic configuration of an example of an image data recording / reproducing apparatus of the present invention.
FIG. 2 is a diagram illustrating a relationship between moving image data and still image data when moving image data and still image data are recorded continuously and discontinuously, respectively.
FIG. 3 is a diagram for explaining a method of generating still image data requested by an operator.
FIG. 4 is a diagram illustrating a generation flow in an example of requested still image data using a difference between moving images.
FIG. 5 is a diagram for explaining a method of generating a requested still image using a motion vector.
FIG. 6 is a diagram showing an example file format when moving image data and still image data are recorded as the same file on the same recording medium.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Shooting lens, 12 ... Image sensor, 131 ... A / D converter, 132 ... Digital signal processing circuit, 141 ... Image density conversion circuit, 142 ... Moving image compression / decompression circuit, 143, 152 ... Recording medium, 151 ... Still image compression / decompression circuit, 16 ... image processing circuit, 17 ... display circuit, 181 ... moving image controller, 182 ... still image controller

Claims (8)

Imaging means for imaging a subject;
Image data generating means for sequentially generating moving image data as a pair and still image data with higher definition than the moving image data by digitally image processing image information from the imaging means;
Recording timing generation means for generating still image data recording timing;
When still image data is recorded discontinuously based on the still image data recording timing from the recording timing generation means, the moving image data paired with the still image data includes the still image data as frame information. Image data recording means for recording continuous moving image data and non-continuous still image data with link information to image data;
When there is a reproduction request for still image data as a frame during reading and reproduction of moving image data from the image data recording means, the moving image data corresponding to the timing of the reproduction request and the time of the moving image data Still image data related to a reproduction request based on moving image data with link information to still image data located in the vicinity, and still image data specified by link information accompanying the moving image data and a playback request still image data generating means for generating,
The reproduction request still image data generation means, when generating the still image data related to the reproduction request, the moving direction and moving distance of a subject included in the moving image displayed by the moving image data by image recognition for the moving image data A motion vector, which is information indicating the image, is calculated, a still image in which the subject included in the still image displayed by the still image data specified by the link information is moved based on the motion vector is generated, and the subject is moved. A recording / reproducing apparatus that interpolates a data lacking portion in a still image that has been made using difference information of moving image data . The image data recording / reproducing apparatus according to claim 1,
In the recording timing generation means, automatic generation based on at least one of the still image data and the moving image data, automatic generation generated as an arbitrary time interval, while the still image acquisition button is being operated, An image data recording / reproducing apparatus in which still image data recording timing is generated by one or more combination generations among manual generations continuously generated. The image data recording / reproducing apparatus according to claim 1,
An image data recording / reproducing apparatus in which the image data recording means is the same recording medium, and moving image data and still image data are recorded on the same recording medium. The image data recording / reproducing apparatus according to claim 3 .
An image data recording / reproducing apparatus in which the moving image data and still image data are recorded as being in the same file. A pair of moving image data and still image data with higher definition than the moving image data are sequentially generated from the image information when the subject is photographed, and the still image data is generated based on the still image data recording timing. When recorded non-continuously, the moving image data paired with the still image data is accompanied by link information to the still image data as frame information, and the continuous moving image data and the non-continuous data. Image data generation / recording step for recording still image data;
When there is a reproduction request for still image data as a frame during reproduction of moving image data, the moving image data corresponding to the timing of the reproduction request is located near the time of the moving image data. Still image data generation for generating still image data related to a reproduction request based on moving image data accompanied by link information to still image data and still image data specified by link information accompanying the moving image data And including steps,
The reproduction request still image data generation means, when generating the still image data related to the reproduction request, the moving direction and moving distance of a subject included in the moving image displayed by the moving image data by image recognition for the moving image data A motion vector, which is information indicating the image, is calculated, a still image in which the subject included in the still image displayed by the still image data specified by the link information is moved based on the motion vector is generated, and the subject is moved. An image data recording / reproducing method for interpolating a data lacking portion in a still image that has been made using difference information of moving image data . The image data recording / reproducing method according to claim 5 ,
The still image data recording timing includes automatic generation based on at least one monitoring result of still image data and moving image data, automatic generation generated as an arbitrary time interval, and operation of a still image acquisition button. An image data recording / reproducing method generated by any one or more combination generation among continuously generated manual generations. The image data recording / reproducing method according to claim 5 ,
An image data recording / reproducing method in which the moving image data and still image data are recorded on the same recording medium. The image data recording / reproducing method according to claim 7 , wherein
An image data recording / reproducing method in which the moving image data and still image data are recorded in the same file.

Images