JP5320186B2 - Recording apparatus and reproducing apparatus - Google Patents

Description

  The present invention relates to a recording apparatus and a reproducing apparatus. For example, the shooting information instructed on the photographing device side is recorded together with the recorded video data, and the reproduction device performs image processing on the reproduced image based on the photographing information, thereby realizing further higher image quality. The present invention relates to a system, a method, and a device that can be used.

  As background art in this technical field, for example, there is JP-A-2000-350079 (Patent Document 1). In this publication, as a problem, “in a television camera having an electronic zoom function, it was applied to a video signal output from the television camera in parallel with the video signal stored in the memory. "The same electronic zoom processing is more easily performed on the video signal from the memory", and as a means for solving the problem, "interpolation processing for recording the video signal and enlarging the video signal to the electronic zoom is performed. The electronic zoom information indicating the electronic zoom magnification and / or the electronic zoom target range involved in the interpolation processing is recorded in association with the recorded video signal ”(see summary).

  Another background art in this technical field is, for example, Japanese Patent Application Laid-Open No. 01-146476 (Patent Document 2). In this publication, there is a problem that “a television camera becomes heavy and large in order to achieve high sensitivity. Also, according to the special camera system, a standard television signal such as NTSC can be easily taken out. In addition, a special device was required to realize a TV camera with a long exposure function, and it was generally difficult to implement " A possible image sensor and a storage device that stores an image signal for one field or one frame in response to an output of the image sensor, and the image sensor performs an exposure operation continuously for a plurality of field periods. Is a television camera device configured to output the same video signal from the storage device.

Japanese Unexamined Patent Publication No. 2000-350079 (first page, FIG. 1) JP-A-01-146476 (first page, FIG. 1)

  In the technique described in Patent Document 1, zoom processing is performed using the same scaler inside the television, and zoom processing with higher image quality cannot be performed. In addition, since the recording unit and the reproduction unit are integrated, the recording data cannot be switched according to the function of the reproduction unit.

  In the technique described in Patent Document 2, the same video signal is output during high-sensitivity recording using a slow shutter, and smooth motion between frames cannot be expressed.

  All or part of the above problems are solved or improved by the invention described in the claims.

  For example, when a video camera is connected to a playback device and a moving image is viewed, information associated with the shooting is recorded together with the video encoded data at the time of shooting with the video camera, and information associated with the shooting is used on the playback device side. By performing image processing of moving images, it is possible to reproduce higher quality images.

  For example, if zoom magnification information is considered as an example of information associated with shooting, a higher-quality scaler function mounted on a playback device is used to perform zoom processing based on the zoom magnification information. Enables playback and display of zoom images.

  For example, when shutter speed information is considered as an example of information accompanying shooting, the same frame image is detected based on the shutter speed information, and the same frame is detected using a high-quality frame interpolation function installed in a playback device. By replacing the image with a frame interpolation image, it is possible to reproduce and display a smooth moving image even during high-sensitivity shooting using a slow shutter.

  According to the present invention, it is possible to realize high image quality of a moving image recording / reproducing apparatus. For example, the shooting information instructed on the photographing device side is recorded together with the recorded video data, and the reproduction device performs image processing on the reproduced image based on the photographing information, thereby realizing further higher image quality. I can do it.

It is an example of the system which actualizes this example Examples of shooting information and image processing in FIG. It is another example of the system which actualizes this example It is another example of the system which actualizes this example It is an example of the reproduction | regeneration control information storage system in FIG. It is an example of the reproduction | regeneration control information in FIG. It is an example of the reproduction | regeneration control information in FIG. FIG. 6B is an example of zoom using FIG. It is an example of the operation | movement flow in FIG. It is an example of the operation | movement flow by the side of the reproduction apparatus in FIG. It is a detailed example of the image processing part in FIG. FIG. 1 is an example of a zoomed moving image. FIG. 1 shows another example of a zoomed moving image. It is another example of the system which actualizes this example FIG. 12 shows an example of a method for displaying a recorded image on a monitor. It is another example of the system which actualizes this example It is another example of the system which actualizes this example It is an example of the operation | movement flow which actualizes this Example at the time of the data output according to the kind of reproducing | regenerating apparatus, and the reproduction | regeneration operation | movement with a reproducing | regenerating apparatus. It is a detailed example of the camera input part 10 in FIG. It is a detailed example of the video encoding part 11a in FIG. It is a detailed example of the video encoding part 11b in FIG. It is a detailed example of the image processing part 21a in FIG. FIG. 1 is an example of a moving image that has undergone frame interpolation processing. It is another detailed example of the image processing part 21a in FIG. It is an example of the icon display flow to the monitor according to the image processing in FIG. It is another example of the icon display flow to the monitor according to the image processing in FIG. It is an example of the image processing in the image processing part in FIG. 1, and the icon display flow to the monitor accompanying it. It is another example of the system which actualizes this example

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a typical example of a moving image recording / reproducing system 1 according to this embodiment. Hereinafter, FIG. 1 will be described.

  The recording device 2 is a device that records a moving image. The recording device 2 is a block that captures and encodes the imaging light 100 and records it in the storage unit 13 together with shooting information that can be acquired during moving image shooting.

  As an example of the shooting information 103, there are zoom magnification information, shutter speed information, and other information indicating image characteristics at the time of moving image shooting. As an example of the zoom magnification information, there are a zoom magnification accompanying a zoom operation by a user, a zoom time interval, a zoom speed indicating a zoom magnification change rate during the zoom time interval, and the like. As an example of the shutter speed information, there are a shutter speed value that is changed according to illuminance at the time of moving image shooting and a shutter speed update time. As another example of the shutter speed information, there are period information and change rate information when the shutter speed is changed at a predetermined change rate during a predetermined period. Other examples of information indicating image characteristics include a contrast value at a predetermined time, a contrast value change rate during a predetermined time period, and information indicating a change in contrast.

  Further, this block is a block that generates a moving image by video decoding processing of recorded video encoded data, and outputs a moving image 108 together with reproduction control information generated based on shooting information.

  The reproduction control information may be information including shooting information or information generated based on shooting information. The reproduction control information may be converted into a command according to the output format of the recording device 2 or may be added to the moving image. An example of the output format is HDMI. In this case, there is a command based on HDMI-CEC as an example of the command.

  The playback device 3 is a block that receives the moving image output from the recording device 2 and the playback control information 108, performs image processing on the moving image using the playback control information, and then displays it on a monitor.

  As an example of means for transmitting moving images and playback control information 108 from the recording device 2 to the playback device 3, connection using HDMI, USB, LAN, etc., optical media such as Blu-ray disc, DVD disc, hard disk, and hologram There are connection via a semiconductor memory such as a recording medium, built-in SRAM, SDRAM, flash memory, and SD card.

  The moving image may be displayed on the monitor 23 inside the reproduction apparatus 3 or may be displayed on an externally connected monitor.

  The recording device 2 includes a camera input unit 10, a video encoding unit 11a, a recording unit 12a, a storage unit 13, a video decoding unit 14a, an output unit 15a, and a user interface unit 16.

  The user interface unit 16 is a block that inputs a user instruction 114 for instructing a recording method to the recording apparatus 2 from the user at the time of shooting, and supplies user instruction information 115 to the camera input unit 10 in accordance with the user instruction 114. Examples of the user instruction 114 include a zoom magnification instruction, a shutter speed instruction, an image quality correction instruction for a digital input image, and a shooting mode (for example, sports mode, night view mode, auto shooting mode) instruction, and the like. Examples of image quality correction include contrast correction and gamma correction.

  The camera input unit 10 is a block that captures the imaging light 100 and generates a digital input image 101 and shooting information 103 in accordance with user instruction information 115 from the user interface unit 16.

  The camera input unit 10 may have various sensors such as an illuminance sensor, an inclination sensor, and an acceleration sensor, and image processing may be performed on a digital input image using information obtained from these various sensors. Further, the shooting information 103 may be generated based on information obtained from various sensors and image processing information for a digital input image.

  The video encoding unit 11 a is a block that performs encoding processing of the digital input image 101 and generates video encoded data 102. As an example of the encoding processing method, MPEG2, H.264, or the like is used. H.264, Motion JPEG, JPEG, and the like. Also, the video encoded data may remain uncompressed. Further, this is a block for generating shooting information 103 including information that can be extracted by the encoding process. Examples of the shooting information 103 include a motion vector, intra / inter, coding rate, picture type, frame of the same image, frame in which a scene change has occurred, frame rate, resolution, and other information indicating image characteristics.

  The recording unit 12 a is a block that receives the encoded video data 102 and the shooting information 103 and records the recorded data 104 in the storage unit 13. The shooting information 103 is information including shooting information instructed from the user interface unit 16, shooting information generated by the camera input unit 10, and shooting information extracted by the video encoding unit 11a. The recording unit 12 a is a block that reads recorded recording data 104 from the storage unit 13 and outputs video encoded data 105 and shooting information 107.

  The video decoding unit 14 a is a block that reads the recording data 104 from the storage unit 13, performs a decoding process on the video encoded data 105 therein, and generates a moving image 106.

  The output unit 15 a is a block that generates reproduction control information based on the shooting information 107, converts the protocol according to the external connection method together with the moving image 106, and outputs it as moving image and reproduction control information 108. The output unit 15a may perform digital image processing on the moving image. Examples of digital image processing include resolution conversion, IP conversion, YUV / RGB format conversion, and color accuracy improvement.

  Examples of external connection methods include connection using HDMI, USB, LAN, etc., optical media such as Blu-ray disc, DVD disc, hard disk, etc., hologram recording media, built-in SRAM, SDRAM, flash memory, SD card, etc. There are connections through memory. As an example of a method for transmitting reproduction control information from the recording apparatus 2 to the reproduction apparatus 3, there is HDMI-CEC.

  The playback apparatus 3 includes an input unit 20a, an image processing unit 21a, a playback image display unit 22, a monitor 23, and a user interface 24.

  The user interface unit 24 is a block that supplies user instruction information 117 for digital image processing to the image processing unit 21a. As an example of the user instruction information 117, there are a zoom process for the moving image 110 and an instruction for an image correction process. Also, a playback mode (sport mode, cinema mode, indoor mode, auto playback mode, etc.) instruction, a user instruction for a case where the playback control information and the user instruction indicate the same kind of image processing, or a mode instruction that prioritizes the playback control information, etc. Good. This block may operate in response to an external user instruction 116 or may generate user instruction information 117 based on the user instruction 116.

  The input unit 20a is a block that receives the moving image and reproduction control information 108 output from the recording device 2 and generates a moving image and reproduction control information 110 that has been subjected to protocol conversion processing according to the external connection method.

  The image processing unit 21a is a block that performs digital image processing of a moving image based on reproduction control information. Examples of digital image processing include zoom processing, interpolation frame generation and frame data replacement processing, image quality correction processing such as contrast correction and gamma correction, partial screen cutout processing, and frame rate conversion processing. Furthermore, it is a block for outputting a moving image 112 subjected to image processing.

  Further, this block performs digital image processing on a moving image in accordance with user instruction information 117 from the user interface unit 24. When the image processing content is the same in the reproduction control information 108 and the user instruction information 117, the image processing content may be determined by the image processing unit 21a according to the reproduction control information 108 and the user instruction information 117.

  The reproduction image display unit 22 is a block that displays the moving image 112 on the monitor 23 after performing resolution conversion processing in accordance with the resolution of the monitor 23. The target region for resolution conversion may be for the moving image 112 or may be for a partial region of the moving image 112. Further, the resolution conversion process may use the circuit of the image processing unit 21a.

  FIG. 2 shows a specific example of the shooting information 103 in FIG. 1 and an example of specific image processing using the shooting information 103. Hereinafter, FIG. 2 will be described.

  An example of the shooting information 103 is zoom magnification. The image processing unit 21a in the playback device 3 performs zoom processing on the moving image 110 in accordance with the zoom magnification information, thereby generating a high-quality zoom image using a higher-quality zoom processing circuit on the playback device side 3. Can be generated. This embodiment may be combined with the embodiment described in FIG.

  Another example of the shooting information 103 is a shutter speed. In some digital cameras and camcorders, when shooting a moving image in a low-light environment, the shutter speed is reduced to improve the sensitivity at the time of shooting. For example, consider a case where the normal shutter speed is 1/60 seconds and the shutter speed in a low illumination environment is 1/30 seconds. In this case, the frame image is normally updated every 1/60 seconds, but in a low illumination environment, the frame image is updated to 1/30 seconds. On the other hand, since the frame image is encoded every 1/60 seconds, two consecutive identical frame images are encoded in a low-light environment, and a video with a frame rate of 1/2 is visually recognized. Become a statue. Therefore, based on the shutter speed information, two consecutive identical frame images A and B are detected in units of 1/60 seconds, and a frame interpolation image predicted from the preceding and following frame images for one frame image B By generating B ′ and replacing it with the original frame image B, it is possible to generate continuous moving images having different 1/60 seconds, and visually realize a double frame rate.

  FIG. 21 shows an example of the frame interpolation process. Frame images 1000 and 1001 in number 2 are the same frame image. A frame image 1011 in number 3 is an interpolated frame image 1011 generated by interpolating each pixel of the frame images 1000 and 1002 before and after the frame image 1001. In number 4, a visually smooth moving image is realized by replacing the original frame image 1001 with the interpolated frame image 1011.

  In the frame interpolation process, the previous and next frames may be used, or only the previous or subsequent frame may be used. Further, any number of frames before or after the interpolation may be used, and the frame image may not be a frame image immediately before or after.

  Another example of the shooting information 103 is image quality correction information. Usually, on the digital camera or camcorder side, encoding processing is performed after performing digital image quality correction processing such as gamma correction and contrast correction according to the surrounding environment and shooting mode. On the other hand, the playback apparatus 3 side usually has a higher quality image processing circuit than an image processing circuit mounted on a mobile device such as a digital camera or a camcorder.

Therefore, the image quality correction processing for the moving image is not performed on the digital camera or the camcorder side, and the moving image before the image quality correction processing is encoded, and the information related to the image quality correction processing is reproduced together with the moving image as reproduction control information. Consider the case of supplying to the device 3 side. In this case, since the image quality correction processing based on the reproduction control information can be performed on the playback device 3 side, higher quality image quality correction processing is performed compared to the case of image quality correction on the mobile device side such as a digital camera or a camcorder. Can be realized.
In addition, when image quality correction processing is performed on the mobile device side such as a digital camera or a camcorder, the image quality correction on the moving image is also performed on the playback device side, and as a result, the image quality adjusted on the mobile device side may not be realized. Therefore, by supplying the image quality correction information performed on the mobile device side as playback control information to the playback device 3 side, it is possible to realize image quality correction processing that takes into account the content corrected on the mobile device side on the playback device 3 side.

  FIG. 3 shows another typical example of the moving image recording / reproducing system 1 according to the present embodiment. Hereinafter, FIG. 3 will be described.

  The recording device 2 generates a stream 308 composed of reproduction control information including shooting information and video encoded data. The playback apparatus 3 receives the stream 308, performs decoding processing on the encoded video data, digital image processing based on the playback control information, and displays a moving image on the monitor 23.

  The recording device 2 includes a camera input unit 10, a video encoding unit 11b, a recording unit 12b, a storage unit 13, an output unit 15b, and a user interface unit 16. The camera input unit 10, the storage unit 13, and the user interface unit 16 have the same configuration as the block shown in FIG.

  The video encoding unit 11b is a block that performs encoding processing of the digital input image 101 and generates a stream 302 including video encoded data. As an example of the encoding processing method, MPEG2, H.264, or the like is used. H.264, Motion JPEG, JPEG, and the like. Also, the video encoded data may remain uncompressed. Also, shooting information 103 supplied from the user interface 16 or the camera input unit 10 and shooting information including information that can be extracted by the encoding process can be included in the stream 302 as reproduction control information. An example of various types of shooting information is the same as the description in FIG.

  As an example of means for including playback control information in a stream, means for storing playback control information in user data in the stream, means for storing in PSI information in the stream, and storage in management information associated with video encoded data There is means to do.

  FIG. 5 shows an example of PSI information in which reproduction control information is stored. In this example, PMT (Program Map Table) is used as PSI, and reproduction control information is stored in the Private_date_indicator area. FIG. 6 is an example showing the details of Private_date_indicator. For example, as shown in FIG. 6A, zoom magnification information (Frame_Zoom_Info) and time information (Time_Stamp_Info) of a frame to be zoomed can be stored as a pair of information. Further, as shown in FIG. 6B, the time information (Time_Stamp_Start_Info) of the zoom start frame, the time information (Time_Stamp_End_Info) of the zoom end frame, and zoom magnification information (Zoom magnification information (Time_Stamp_End_Info)) that is performed during the period from the zoom start frame to the zoom end frame. Frame_Zoom_Info) may be stored as pair information. In this case, the zoom processing performed in the period from the zoom start frame to the zoom end frame may be performed so that the zoom magnification changes linearly, or the zoom magnification may be changed according to a predetermined function. . FIG. 6C shows an example of zooming with respect to a moving image when the means of FIG. 6B is used. In this example, frame number # 01 is zoom start frame 600, frame number # 04 is zoom end number 601, and zoom magnification information is tripled. It is assumed that the zoom process changes linearly. In this case, the frame number # 02 during zooming is a frame 621 obtained by zooming the frame image 611 by 1.67 times. Frame number # 03 in the middle of zooming is a frame 622 obtained by zooming the frame image 612 by 2.33 times. Frame number # 04, which is the final zoom frame, is a frame 623 obtained by zooming the frame image 613 three times. As long as there is no new zoom instruction, the frames after frame number # 05 are frames obtained by performing the 3x zoom process on the pre-zoom frame image of the same frame number as in frame number # 04.

  The recording unit 12 b is a block that receives the stream 302 and records it as recording data 304 in the storage unit 13. Further, the block is a block that reads out the recording data 304 recorded in the storage unit 13 and supplies the stream 306 to the output unit 15b.

  The output unit 15b is a block that inputs the stream 306, converts the protocol according to the external connection method, and outputs the stream 308. The output unit 15b may perform digital image processing on the moving image.

  Examples of external connection methods include connection using HDMI, USB, LAN, etc., optical media such as Blu-ray disc, DVD disc, hard disk, etc., hologram recording media, built-in SRAM, SDRAM, flash memory, SD card, etc. There are connections through memory.

  The playback device 3 includes an input unit 20b, a video decoding unit 25, an image processing unit 21b, a playback image display unit 22, a monitor 23, and a user interface 24. The reproduced image display unit 22, the monitor 23, and the user interface 24 have the same configuration as the block shown in FIG.

  The input unit 20b is a block that receives the stream 308 and generates a stream 310 that has been subjected to protocol conversion processing according to the external connection method.

  The video decoding unit 25 is a block that performs decoding processing of the stream 310 and extracts reproduction control information from the stream.

  The image processing unit 21b is a block that receives a moving image and reproduction control information 311 and performs digital image processing of the moving image based on the reproduction control information. An example of digital image processing is the processing shown in FIG. Furthermore, it is a block for outputting a moving image 112 subjected to digital image processing. Further, this block performs digital image processing on the moving image 311 in accordance with user instruction information 117 from the user interface unit 24. When the image processing content is the same in the reproduction control information 311 and the user instruction information 117, the image processing content may be determined by the image processing unit 21b in accordance with the reproduction control information 311 and the user instruction information 117.

  FIG. 4 shows another typical example of the moving image recording / reproducing system 1 according to the present embodiment. Hereinafter, FIG. 4 will be described.

  The recording device 2 generates a stream composed of reproduction control information including shooting information and video encoded data, and records the stream on the SD card 408. The playback device 3 reads the stream recorded on the SD card 408, performs decoding processing on the encoded video data, digital image processing based on the playback control information, and displays a moving image on the monitor 23.

  The recording device 2 includes a camera input unit 10, a video encoding unit 11 b, a memory I / O unit 26, and a user interface unit 16. The camera input unit 10, the video encoding unit 11b, and the user interface unit 16 have the same configuration as the block shown in FIG.

  The memory I / O unit 26 is a block that receives the stream 302 and writes it to the SD card.

  The playback device 3 includes a memory I / O unit 27, a video decoding unit 25, an image processing unit 21b, a playback image display unit 22, a monitor 23, and a user interface 24. The video decoding unit 25, the image processing unit 21b, the reproduced image display unit 22, the monitor 23, and the user interface 24 have the same configuration as the block shown in FIG.

  The memory I / O unit 27 is a module that reads the stream 310 from the SD card 408.

  FIG. 7 is an example of an operation flow on the recording device 2 side when zoom magnification is used as shooting information in the embodiment shown in FIG.

  In step S <b> 701, it is confirmed whether a zoom instruction is given as a user instruction 114 from the user to the user interface unit 16. If there is a zoom instruction, it is checked in S702 whether the zoom magnification specified by the zoom instruction exceeds the maximum optical zoom magnification. If the maximum optical zoom magnification is exceeded, in step S703, the camera input unit 10 is instructed to perform optical zoom processing at the maximum optical zoom magnification. In S704, the video encoding unit 11a performs encoding processing on the digital input image zoomed at the maximum optical zoom magnification generated by the camera input unit, generates video encoded data, and supplies the encoded data to the recording unit 12a. . In step S705, shooting information 103 such as zoom magnification information and corresponding frame information is supplied to the recording unit 12a. Next, in step S706, recording data 104 including the encoded video data 102 and the shooting information 103 is generated. In step S707, the recording data 104 is written in the storage unit 13.

  On the other hand, if the zoom magnification designated by the zoom instruction does not exceed the maximum optical zoom magnification in S702, the camera input unit 10 is instructed to perform optical zoom processing at the optical zoom magnification designated by the zoom instruction in S708. . In step S <b> 709, the video encoding unit 11 performs encoding processing on the zoomed digital input image generated by the camera input unit 10 to generate video encoded data 102. Next, in S710, recording data including the encoded video data 102 is generated, and the recording data 104 is written in the storage unit 13 in S707.

  If there is no zoom instruction in S701, a digital input image 101 is generated based on the imaging light 100 input to the camera input unit 10 in S711, and a video encoding process is performed in the video encoding unit 11a in S709. To generate video encoded data 102.

  FIG. 7 is an example of an operation flow diagram on the playback device 3 side when playback control information including zoom magnification is used in the embodiment shown in FIG.

  In step S801, zoom magnification information is extracted from the reproduction control information 110. In step S802, it is determined whether zoom processing is necessary for the moving image 110. If zoom processing is necessary, the image processing unit 21a performs zoom processing based on the zoom magnification information in S803, and generates a moving image 112 after zoom processing. In step S803, the zoomed moving image is displayed on the monitor 23 via the reproduced image display unit 22. If it is determined in S802 that zoom processing is unnecessary, the process of S804 is performed.

  FIG. 9 shows a specific example of the image processing unit 21a or 21b. The zoom magnification information extraction unit 28 is a block that extracts the zoom magnification information 900 from the input moving image and the reproduction control information 110. The zoom processing unit 29 is a block that extracts a moving image from the moving image and reproduction control information 110, performs a zoom process using the zoom magnification information 900, and generates a zoomed moving image 112. Examples of the zoom magnification information 900 include zoom magnification information, optical zoom maximum magnification information, zoom start and end frame information, zoom start and end time information, target zoom magnification, zoom speed information, zoom transition function, and the like. As an example of zoom processing, there is an example in which zoom-in processing or zoom-out processing of a moving image is performed according to the zoom magnification. As another example, there is an example in which zoom processing is performed only for a zoom that exceeds the optical zoom magnification. As another example, there is an example in which zoom processing is performed on each frame according to the zoom speed until the zoom target is reached from the frame corresponding to the zoom start timing.

  FIG. 10 is an example of an image image in the system shown in FIG. 1, FIG. 3, and FIG. 4 in which the shooting information is zoom magnification information, N times zoom recording is performed, and the recorded data is reproduced and displayed.

  The image image column and the zoom magnification column for the image image before zooming indicate the zoom magnification of the image image at the input / output position of each block shown in the processing unit column and the image image with respect to the image image before zooming. In this embodiment, when the zoom instruction 115 is given, the recording apparatus 2 does not perform zoom processing, performs encoding processing on the digital input image 101 before zooming, and generates the encoded video data 102 together with zoom magnification information. Recording is performed as recording data 104, or a stream including zoom magnification information is generated and written to the SD card.

  Therefore, the image image 1000b at the output of the camera input unit 10 before zooming becomes the image image 1000a before zooming. When the recording device 2 is connected to the playback device 3 and displayed on the monitor 23 on the playback device 3 side, the image processing unit 21a or 21b inputs the moving image 110 or 311 before zooming, and zooms according to the zoom magnification information. The processing unit 29 performs N times zoom processing to generate a zoomed moving image 112. The moving image 112 causes the monitor 23 to display a reproduction image 113 after N times zooming via the reproduction image display unit 22.

  By using this method, the zoom processing circuit on the playback device 3 side can be used instead of the zoom processing circuit on the recording device 2 side. If a video camera is used as the recording device 2 and a stationary large TV is used as the playback device 3, and a higher-performance zoom processing circuit is mounted on the playback device 3, the use of this embodiment will increase the quality. A zoom image can be displayed.

  In the embodiment shown in FIGS. 1, 3 and 4, FIG. 11 uses the shooting information as zoom magnification information, performs N times zoom recording on the image image before zooming, and reproduces the recorded data. It is an example of the image image in the system to display.

  The image image column and the zoom magnification column for the image image before zooming indicate the zoom magnification of the image image at the input / output position of each block shown in the processing unit column and the image image with respect to the image image before zooming. In this embodiment, when the zoom instruction 115 is given, the recording apparatus 2 does not perform zoom processing, performs encoding processing on the digital input image 101 before zooming, and generates the encoded video data 102 together with zoom magnification information. Recording is performed as recording data 104, or a stream including zoom magnification information is generated and written to the SD card.

  In this embodiment, when the zoom instruction 114 is issued, the digital input image 101 subjected to zoom processing up to the maximum optical magnification is generated in the camera input unit 10 on the recording apparatus 2 side, and the digital encoding is performed in the video encoding unit 11a or b. An encoding process is performed on the input image 101 and the generated video encoded data 102 is recorded in the storage unit 13 as recording data 104 together with zoom magnification information exceeding the optical zoom maximum magnification, or a stream including the zoom magnification information is generated. Write to the SD card.

  Therefore, the image 1100b at the output of the camera input unit 10 is an image zoomed to M times which is the maximum optical zoom magnification. The image image 1100c at the input of the image processing unit 21a or 21b is an image image that has been zoomed M times. The zoom processing unit 29 performs zoom processing up to N times to generate a zoomed moving image 112. The moving image 112 becomes an image 1100d. The moving image 112 causes the monitor 23 to display the moving image 113 after N times zooming via the reproduced image display unit 22. The moving image 113 is an image image 1100e.

  By using this method, zoom processing of an optical zoom area capable of high-quality zoom processing is performed by the camera input unit 10, and zoom processing of a digital zoom area in which image quality degradation occurs due to digital zoom processing is as follows: Instead of the zoom processing circuit on the recording device 2 side, the zoom processing circuit on the playback device 3 side can be used. Considering a video camera as the recording device 2 and a stationary large-sized TV as the playback device 3, and a higher performance zoom processing circuit is mounted on the playback device 3, the use of this embodiment will result in higher quality. A zoom image can be displayed.

  FIG. 12 shows an example of a system for displaying the digital input image being recorded on the monitor 31 on the recording apparatus 3 side in the embodiment shown in FIG.

  The recording apparatus 2 includes a camera input unit 10, a video encoding unit 11a, a recording unit 12a, a storage unit 13, a video decoding unit 14a, an output unit 15a, a user interface unit 16, an image processing unit 30a, and a monitor 31. . The camera input unit 10, video encoding unit 11a, recording unit 12a, storage unit 13, video decoding unit 14a, output unit 15a, and user interface unit 16 have the same configuration as the block shown in FIG.

  The image processing unit 30 a is a block that performs image processing on the digital input image 101 based on the shooting information 103 and generates a moving image. In addition, this block is a block that performs moving image scaling processing in accordance with the resolution of the monitor 31 and supplies the processed moving image 1200 to the monitor 31. As an example of image processing, when the shooting information 103 is zoom magnification information, the digital input image 101 is subjected to zoom processing according to the zoom magnification to generate a moving image. The monitor displays a moving image after zooming in accordance with the zoom magnification information.

  FIG. 13 is an example of an operation flow in this system.

  In step S1300, zoom magnification information is extracted from the shooting information 103, and in step S1301, it is determined whether zoom processing is necessary for the moving image. When zoom processing is necessary, in S1302, the image processing unit 30a performs zoom processing on the digital input image 101 based on the zoom magnification information and scaling processing in accordance with the resolution of the monitor 31, and after zooming. A moving image is displayed on the monitor 31. If it is determined in S1300 or S1301 that the zoom process for the moving image is unnecessary, S1303 is performed.

  FIG. 14 shows a digital image in which the recording apparatus 2 further uses the reproduction control information when the reproduction apparatus 3 that cannot perform the digital image processing using the reproduction control information is connected in the embodiment shown in FIG. It is an example of the system which can perform a process.

  The recording device 2 includes a camera input unit 10, a video encoding unit 11a, a recording unit 12a, a storage unit 13, a video decoding unit 14a, an output unit 15c, a user interface unit 16, and an image processing unit 30b. The camera input unit 10, video encoding unit 11a, recording unit 12a, storage unit 13, video decoding unit 14a, and user interface unit 16 have the same configuration as the block shown in FIG.

  The image processing unit 30 b is a block that performs image processing on the moving image 106 based on the shooting information 107 and generates a moving image 1400. As an example of image processing, when the shooting information 107 is zoom magnification information, the moving image 101 is zoomed according to the zoom magnification information to generate a moving image 1400.

  When the playback device 3 connected to the recording device 2 supports image processing using the playback control information, the output unit 15c generates playback control information based on the shooting information 107, and together with the moving image 106, the protocol After the conversion, it is output as a moving image and reproduction control information 1408. On the other hand, if the playback device 3 connected to the recording device 2 does not support image processing using playback control information, the moving image 1400 is converted into a protocol and then output as a moving image 1408.

  The determination as to whether or not the playback device 3 supports image processing using playback control information may be realized by communication using a command or the like according to a connection protocol between the recording device 2 and the playback device 3 or connected. The type of the playback device 3 being used may be set on the recording device 2 side from the user side using the user instruction 114, and the setting status may be notified to the image processing unit 30b or the output unit 15c as user instruction information 115. Examples of communication using commands and the like include HDMI-CEC and Display Port.

  In this embodiment, when the playback device 3 does not support image processing using the shooting information added by the recording device 2, transcoding processing including image processing based on shooting information is performed, and image processing based on shooting information is performed. The encoded video data is generated and output to the playback device 3 via the output unit.

  FIG. 15 shows a case where the recording apparatus 2 further uses the digital image processing using the reproduction control information when the reproduction apparatus 3 not compatible with the digital image processing using the reproduction control information is connected in the embodiment shown in FIG. An example of a system capable of performing video re-encoding processing or trans-coding processing, generating video encoded data subjected to digital image processing based on playback control information, and outputting the video encoded data to the playback device 3 via the output unit 15b. It is.

  The recording device 2 includes a camera input unit 10, a video encoding unit 11b, a recording unit 12b, a storage unit 13, a video decoding unit 14a, an output unit 15b, a user interface unit 16, and an image processing unit 30c. The camera input unit 10, the recording unit 12b, the storage unit 13, and the user interface unit 16 have the same configuration as the block shown in FIG. The recording unit 14a has the same configuration as the block shown in FIG.

  The image processing unit 30b is a block that performs image processing on the moving image 1500 based on the user instruction information 115, the shooting information 103, or the reproduction control information 1500 associated with the moving image, and generates a moving image 1501. As an example of image processing, when the shooting information 103 or the reproduction control information 1500 is zoom magnification information, the moving image 1500 is zoomed according to the zoom magnification information to generate a moving image 1501.

The video encoding unit 11b performs the same encoding process as the video encoding unit 11a shown in FIG. 1 or 3 on the moving image 1501 in addition to the digital input image 101, and includes video encoded data. This is a block for generating a stream 302.
When the playback device 3 connected to the recording device 2 supports image processing using the playback control information, the output unit 15b converts the stream 306 including the playback control information into a protocol and outputs the stream 308. On the other hand, if the playback device 3 connected to the recording device 2 is not compatible with image processing using playback control information, the image processing unit 30c performs image processing and then the video encoding unit 11b performs re-encoding. The stream 306 generated by performing the conversion processing is subjected to protocol conversion, and then output as a stream 308.

  The determination as to whether or not the playback device 3 supports image processing using playback control information may be realized by communication using a command or the like according to a connection protocol between the recording device 2 and the playback device 3 or connected. The type of the playback device 3 being used may be set from the user side to the recording apparatus 2 side using the user instruction 114, and the setting state may be notified to the image processing unit 30c or the output unit 15b as user instruction information 115. Examples of communication using commands and the like include HDMI-CEC and Display Port.

  The recording apparatus 2 may have any one or all of the functions of the recording apparatuses shown in FIGS. 1, 3, 14, and 15.

  The playback device 3 may have any or all of the functions of the playback devices shown in FIGS. 1, 3, 14, and 15.

  FIG. 16 shows the operation flow of the recording device 2 according to the type of the playback device 3 to be connected and the playback and display on the playback device 3 in the embodiment shown in FIGS. 1, 3, 14 and 15. It is an example of the figure which shows the operation | movement flow.

  In S1601, it is confirmed whether or not the playback device 3 connected to the recording device 2 is the playback device shown in FIG. If it is compatible, it is checked in step S1602 if the playback device 3 supports stream input. If it is compatible, in step S1610, the output unit 15 outputs a stream 308 including reproduction control information. In the playback device 3, the stream 308 including the playback control information is input in S1611, and in S1612, the video decoding unit 60 performs a decoding process to generate a moving image. In step S1605, it is determined whether digital processing for the moving image is necessary based on the reproduction control information. If the digital image processing image is necessary, the digital image processing is performed in S1650, and if the digital image processing is not necessary, the digital image processing is not performed in S1651. Finally, in S1652, the moving image after digital image processing is displayed on the monitor 23 via the reproduced image display unit 22.

  If the stream input is not supported in S1602, the video decoding unit 13 performs a decoding process in S1620 to generate a moving image, and the playback control information and the moving image are output to the playback device 3 in S1621. In S1622, the reproduction control information and the moving image are input through the input unit 13, and the processing from S1605 is performed.

  If the playback device 3 does not support image processing using the playback control information in S1601, the playback device 3 confirms whether or not the playback device 3 supports stream input in S1603. If it is compatible, transcoding processing including digital image processing is performed on the recording device 2 side in S1630, a digital image processed stream is generated, and the stream is output from the output unit in S1631.

  If the playback device 3 does not support stream input in S1603, in S1640, video recording processing is once performed on the recording device 2 side to generate a moving image. In step S1604, it is determined whether the moving image should be subjected to digital image processing based on the reproduction control information. Depending on the necessity of digital image processing, the digital image processing in S1541 or the digital image processing in S1642 is not performed, and the moving image after digital image processing is supplied to the playback device 3 side via the output unit 13 in S1643. To do.

FIG. 17 is a diagram showing a detailed example of the camera input unit 10 in FIG. 1 or FIG. Hereinafter, FIG. 17 will be described.
The camera input unit 10 includes a frame synchronization generation unit, an identical frame detection unit, a lens unit, and an A / D conversion unit.

  The lens unit is a block for inputting the imaging light 100.

  The frame synchronization generation unit is a block that generates a frame capture timing 1700 that is captured as an image of one frame from the imaging light.

  The same frame detection unit is a block that detects the same frame based on the user instruction information 115 and the frame capture timing 1700 that are the basis of imaging information such as zoom magnification information and shutter speed information. For example, consider a case where the recording apparatus 2 performs encoding on a frame image (frame 1, frame 2, frame 3, frame 4, frame 5,...) Every 1/30 seconds. Here, when the shutter speed is designated as 1/15 seconds by the user instruction information 115 for a certain period, the same frame detection unit can determine that two identical frames exist during that period. That is, it can be determined that frame 1 and frame 2 are the same frame image, and frame 3 and frame 4 are the same frame image. The detected same frame information is output as shooting information 103.

  The imaging sensor and A / D conversion unit captures the imaging light 1701 captured by the lens unit as analog image data in accordance with shutter speed information and frame capture timing 1700, and further converts it into digital data to generate a digital input image 101. It is a block.

FIG. 18 is a diagram showing a detailed example of the video encoding unit 11a in FIG. Hereinafter, FIG. 18 will be described.
The encoding processing unit is a block that performs video encoding processing of the digital input image 101 and generates video encoded data 102. Further, this block supplies the encoded information obtained by the encoding processing unit to the same frame detection unit.

  The same frame detection unit is a block that determines whether or not consecutive frames are the same frame image based on the encoding information supplied from the encoding processing unit, and outputs the same as the imaging information 103.

  Examples of the encoding information include motion vector information, image data information, and image data difference information.

FIG. 19 is a diagram showing a detailed example of the video encoding unit 11b in FIG. Hereinafter, FIG. 19 will be described.
The encoding processing unit is a block that performs video encoding processing of the digital input image 101, generates video encoded data 102, and generates a stream 302 including the same frame information supplied from the same frame detecting unit. Further, this block supplies the encoded information obtained by the encoding processing unit to the same frame detection unit.

  The same frame detection unit determines whether or not consecutive frames are the same frame image based on the encoding information supplied from the encoding processing unit, and supplies the same frame information 1901 to the encoding processing unit It is.

  FIG. 20 shows a detailed example of the image processing unit 21a shown in FIG.

  The reproduction control information extraction unit 40 is a block that extracts shutter speed information 2002 from the reproduction control information 2000 out of the reproduction control information and the moving image 110.

  The same frame determination unit calculates a frame rate from the shutter speed information 2002, checks whether the preceding and following images (frame images) are the same image, and generates the same frame information 2003 indicating the same frame image. It is a block.

  The interpolation frame generation unit 42 is a block that inputs the moving image 2001 out of the reproduction control information and the moving image 110 and generates the interpolation frame 2004 using the same frame information 2003. Further, this is a block that generates interpolation frame presence / absence information 2004 that is related information of the interpolation frame 2004. Examples of the interpolation frame presence / absence information include an interpolation frame number and an interpolation frame presence / absence flag associated with each frame.

  When N identical images are consecutive, the interpolation frame generation unit 42 generates M interpolation frames 2004 out of the N images. M is equal to or smaller than N, and may be 0.

  The interpolation frame replacement unit 43 is a block that, when N identical images are consecutive in the moving image 2001, replaces L images among them with the interpolation frame 2004, and generates a moving image 112 after frame complementation. is there. L is a value smaller than M and may be 0. The replacement frame is determined using the same frame information 2003, interpolation frame presence / absence information attached to the interpolation frame 2004, and the like.

  21 and FIG. 3, in the embodiment shown in FIG. 1 and FIG. 3, when the shooting information is shutter speed information, the shutter speed is slow, and the digital input images 101 generated by the camera input unit 10 are a plurality of continuous images, It is an example of the system which produces | generates and displays the interpolation frame image with respect to one part.

  The image image indicates the image image at the input / output position of each block indicated in the image image extraction position column. This embodiment is a case of slow shutter photography with a shutter speed ½. The image image shows an example of five consecutive reproduced images, where the image 1000 and the image 1001 are the same image, and the image 1002 and the image 1003 are the same image.

The image processing unit 21 of the playback device 3 generates an interpolated image 1011 for the image 1001 using the image 1000 and the image 1002. Similarly, by using the image 1002 and the image 1004, an interpolation image 1013 for the image 1003 is generated. The interpolation images 1011 and 1013 are replaced with the reproduction images 1001 and 1003, and are used as output images of the image processing unit. The reproduction image replaced with the interpolated image is displayed on the reproduction side monitor 23.

  When shooting using a video camera in a dark place, the amount of light is small and the shooting sensitivity decreases. In such cases, high-sensitivity shooting is realized by reducing the shutter speed. In this case, a plurality of identical images are generated, the frame rate feeling for the reproduced image is attenuated, and the same effect as frame dropping is obtained.

  By using this method, it is possible to reproduce and display encoded video data shot in a dark place at a higher frame rate.

  FIG. 22 shows a detailed example of the image processing unit 21a.

  The reproduction control information extraction unit 50 extracts image correction information 2202 and 2203 such as contrast correction and gamma correction performed on the recording device 2 side from the input reproduction control information and the reproduction control information 2200 in the moving image 110. It is a block.

  The contrast correction unit 51 is a block that performs contrast correction on the reproduction control information and the moving image 2201 in the moving image 110 based on the image correction information 2202, and generates a contrast-corrected moving image 2204.

  The gamma correction unit 52 is a block that performs gamma correction on the moving image 2204 based on the image correction information 2203 and generates a moving image 112 after gamma correction.

  By using this embodiment, for example, the image quality (picture making) adjusted on the video camera side which is an example of the recording device 2 can be reproduced as it is on a television which is an example of the playback device 3. In addition, it is possible to avoid deterioration in image quality due to redundant correction of the same image quality correction items on the video camera side and the television side.

  FIG. 23 shows a case where an icon indicating that zooming is being performed is displayed on the monitor 23 of the playback device 3 when zooming of a moving image is instructed during recording based on the playback control information in FIG. It is an example of an operation | movement flow.

  In S2301, it is determined from the reproduction control information whether the moving images 110 and 311 are in the zoom region. As an example of the zoom area, there are a digital zoom area that performs digital zoom processing on a moving image based on the reproduction control information in the image processing units 21a and 21b, and an optical zoom area that is performed by the camera input unit 10 on the recording apparatus 2 side.

  If the moving images 110 and 311 are zoom regions, an icon indicating that zoom processing is being performed is displayed on the monitor 23 via the reproduced image display unit 22 in S2302.

  When the moving images 110 and 311 are not in the zoom region, an icon indicating that zoom processing is being performed is not displayed on the monitor 23 via the reproduced image display unit 22.

  By using this embodiment, the viewer can recognize that the moving image being viewed is being digitally zoomed. The icon may include zoom magnification.

  FIG. 24 is an operation flow for displaying an icon indicating that the frame complementing process is being performed on the monitor 23 of the playback apparatus 3 when the frame interpolation process is performed based on the playback control information in FIG. 1 or FIG. It is an example.

  In S2401, it is determined from the reproduction control information whether or not the moving images 110 and 311 are undergoing complement processing. If the moving images 110 and 311 are in the complementing process, an icon indicating that the complementing process is in progress is displayed on the monitor 23 via the reproduced image display unit 22 in S2402.

  When the moving images 110 and 311 are not being complemented, an icon indicating that the complementing processing is being performed is not displayed on the monitor 23 via the reproduced image display unit 22.

  By using this embodiment, it is possible to make the viewer recognize that the moving image being viewed is being complemented. The icon may include information related to complement processing. One example of information related to the complement processing is shutter speed.

  FIG. 25 shows one embodiment in which the playback apparatus 3 performs digital zoom processing of a moving image to be played in accordance with a user instruction 116.

  In the embodiment shown in FIG. 23, when the playback device 3 automatically performs digital zoom processing based on the zoom magnification information, the viewer looks at the icon during digital zoom and gives a zoom-out instruction to the playback device 3, thereby zooming in. You can watch out images. Zoom-out allows viewing with high quality image quality up to the recorded optical zoom maximum magnification.

  In S2501, the necessity of digital zoom processing for the moving image is confirmed based on the reproduction control information including the zoom magnification information. If digital zoom processing is necessary based on the zoom magnification information, the presence / absence of a digital zoom instruction by the user instruction 116 is confirmed in S2502. When there is a zoom instruction from the user, the image processing unit 21a or 21b performs digital zoom processing according to the user instruction 116, not the reproduction control information. When there is no zoom instruction from the user, the image processing unit 21a or 21b performs digital zoom processing according to the reproduction control information. In step S <b> 2504, a moving image is displayed on the monitor 23 via the reproduced image display unit 22.

  If it is determined in step S2501 that digital zoom processing based on the reproduction control information is not necessary, in step S2506, the presence / absence of a zoom instruction from the user is confirmed. If there is a zoom instruction from the user, the processing after S2503 is performed. If there is no zoom instruction from the user, the processing after S2504 is performed.

  Further, the playback device 3 may display an icon indicating that digital zoom is possible on the monitor 23 without performing the digital zoom process based on the playback control information. In this case, the viewer can view the zoom image by viewing the icon indicating that digital zoom is possible and issuing a zoom instruction to the playback device 3.

  26 shows a system in which, in the embodiment shown in FIG. 1, the output unit 60 can further generate reproduction instruction information 2600 based on the user instruction information 117 on the reproduction apparatus 3 side and supply it to the recording apparatus 2. It is an example.

  An example of a system in which the recording apparatus 2 side can input the reproduction instruction information 2600 via the input unit 61 and record the reproduction instruction information 2601 as additional information in association with the video encoded data in the storage unit 13. It is.

  In the output unit 60 and the input unit 61, protocol conversion may be performed according to a transmission path protocol connecting the recording device 2 and the reproduction device 3, and information necessary for recording in the storage unit 13 from the reproduction instruction information May be extracted only.

  A storage unit may be provided in the playback device 3, and moving images, playback control information, and playback instruction information may be recorded in the storage unit in the playback device 2. The storage unit has an area for recording data, and examples thereof include optical media such as Blu-ray discs, DVD discs, and hard disks, hologram recording media, built-in SRAM, SDRAM, flash memory, and semiconductor memory such as SD cards. .

  By using this embodiment, the image processing content for the moving image instructed by the user at the time of reproduction is added to the recording apparatus 2 side together with the corresponding video encoded data, and the video encoded data recorded in the storage unit 13 is recorded later. When reproducing, it is possible to reproduce the image processing for the moving image instructed by the user during the previous reproduction. Examples of image processing include zoom processing and OSD display. As an example of the OSD display, there is a case where an OSD selected by a user from among decorative OSD groups prepared in advance is displayed together with a moving image.

  Although the present embodiment has been described by taking moving image shooting as an example, still image shooting may be performed.

DESCRIPTION OF SYMBOLS 1 Moving image recording / reproducing system 2 Recording apparatus 3 Reproducing apparatus 10 Camera input part 11a Video encoding part 12a Recording part 13 Storage part 14a Video decoding part 15a Output part 16 User interface part 20a Input part 21a Image processing part 22 Reproduction image display Section 23 Monitor 24 User interface section

Claims (6)

An input unit for inputting moving image data including moving image and reproduction control information;
An image processing unit that performs image processing of the moving image using the reproduction control information;
A replay image display unit for displaying a moving image after image processing on a monitor;
With
The reproduction control information is shutter speed information for an input moving image,
The image processing unit extracts the same frame image as the previous image based on the shutter speed information, and generates a frame interpolation image from the previous or subsequent frame image for the extracted frame image, A reproduction apparatus characterized by replacing an extracted frame image with a frame interpolation image . The playback apparatus according to claim 1, wherein the image processing unit generates the frame interpolation image by using both the previous frame image and the subsequent frame image for the extracted frame image. . 3. The playback apparatus according to claim 1, wherein the playback image display unit displays an icon on a monitor during a period in which the image processing unit performs frame interpolation processing. The moving image data, reproducing apparatus according to any one of claims 1 to 3, characterized in that it is constituted by the reproduction control information and the non-compressed video. The moving image data is stream data including reproduction control information;
Reproducing apparatus according to any one of claims 1 to 3, further comprising a decoding unit that performs decoding processing of the stream data. A memory I / O unit;
The memory I / O is reproducing apparatus according to any one of claims 1 to 5, characterized in that to supply to the input unit reads the record data recorded in the removable memory.

Images