JP4296376B2 - How to get video - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention , Dynamic In particular, it is possible to generate a video thumbnail, which is a reduced video image from the main video image, and send it to a mobile device for display. Movement The present invention relates to an image acquisition method.
[0002]
[Prior art]
An image data storage device that easily retrieves desired image data from a plurality of stored image data and provides the image data receiving device with data for easily retrieving desired image data. It is shown in the gazette of Kaihei 10-164483.
[0003]
In the image data transmitting apparatus 1 disclosed in the publication, first, a still image (index image α) having a low resolution is captured, and then a moving image or a still image (detailed image β) having a specified resolution is captured. Store in the same file. When one file is specified from a plurality of files stored in the image transmitting apparatus 1 and the detailed image data β is to be received by the receiving apparatus 2, the image transmitting apparatus 1 transmits all index images α from each file. . In the image receiving device 2, the contents of each file can be confirmed accurately by displaying a list of 12 received index images α. The image receiving apparatus 2 is configured to receive the detailed image data β from the image transmitting apparatus 1 by designating a file desired to be transmitted.
[0004]
Japanese Laid-Open Patent Publication No. 11-3346 discloses a moving image file management apparatus that realizes visual management of a plurality of moving image files by storing a representative scene characterizing a moving image near the head of the moving image file. It is shown.
[0005]
The file management unit of the moving image file management apparatus instructs the representative scene registration unit to register a representative scene for a moving image file whose representative scene is not registered. The representative scene registration unit automatically detects a representative scene candidate by a scene change detection unit that detects a scene change or the like by a representative scene candidate creation unit while reproducing a moving image file, and characterizes the moving image file. Is stored. The file management unit is composed of a file list display unit that takes out the representative scene stored in the representative scene registration unit from the file and displays a list of general operating system file management information and moving image specific information together with thumbnail display. Has been.
[0006]
In Japanese Patent Laid-Open No. 11-284948, a plurality of thumbnails are created for one image data. At this time, a plurality of frames T1, T2,. A thumbnail creation method and a thumbnail display method are shown in which a plurality of thumbnails are used or a single frame T1 is taken out and a plurality of thumbnails having different sizes are created.
[0007]
[Problems to be solved by the invention]
However, in the image data storage device disclosed in Japanese Patent Application Laid-Open No. 10-164483, when transferring a moving image or a still image, a header image of each image is created at the transfer source and transmitted in advance to the transfer destination. First, the headline image is confirmed, the image to be acquired is determined, and image transmission is started. Therefore, when the user selects a moving image, the user can only select the image by browsing the still image header image created from the moving image. As the scene changes from moment to moment, it is not clear what kind of image is included later, so it is not enough for the user to use only the still image thumbnail when selecting the desired moving image. I have to say that. Therefore, the operation of the user browsing only the still image thumbnails and selecting a moving image has a problem that it is difficult to select a desired moving image.
[0008]
In addition, the moving image file management apparatus disclosed in Japanese Patent Application Laid-Open No. 11-3346 is used for checking the contents of a moving image by sticking a still image thumbnail to the moving image as a heading image representing the moving image. . Therefore, similarly to the above-described invention, there has been a problem that the user cannot select a desired moving image.
[0009]
In addition, the thumbnail creation method and the thumbnail display method disclosed in Japanese Patent Application Laid-Open No. 11-284948 detect a moving image scene change as a moving image heading image and extract a thumbnail from a frame corresponding to the image immediately after the scene change. Since the invention is automatically generated, it is somewhat easy for the user to select a moving image. However, in the case where the scene change portion is not successfully detected, there has been a problem that the user cannot select a desired image, as in the above-described inventions.
[0010]
When a user operates a communication device such as a PDA (Personal Data Assistance) or a mobile phone to view the contents of multiple image files recorded in another communication device such as an electronic camera (when using the pull model) ) Is limited to the display capability (number of pixels and gradation) of the display means and the recording capacity of the image data in the portable communication device operated by the user, so that the received image data is suitable for display. It was necessary to implement the process of converting to
[0011]
In general, the communication speed of a short-range communication interface provided in a portable communication device operated by a user never has a sufficient communication speed for transferring a moving image. Therefore, in the conventional communication interface, when moving images are transferred for displaying a list, it takes time to transfer the images, and there is a problem that the usability is poor. In addition, a moving image takes a long time to transfer because the number of frames of the image is large. However, when the user confirms the content of the moving image, it is preferable to browse and select the moving image directly.
[0012]
Conventionally, when transmitting a moving image to a communication device such as a portable terminal, the reproduction environment of the moving image (such as the number of pixels of the display image, the gradation, the frame rate, etc.) Since the environmental conditions of the entire system such as the effective communication speed of information are not taken into account, there is a waste of transmitting a moving image having an information amount far larger than the actually required image information amount.
[0015]
Book The invention obtains a moving image which can confirm a desired moving image main image with a minimum amount of data communication by using a still image thumbnail and a moving image thumbnail corresponding to the moving image main image and acquire the confirmed moving image main image. It aims to provide a method.
[0023]
[Means for Solving the Problems]
In order to achieve the above object, the present invention communicates between a first communication device and a second communication device, and from the first communication device in response to a moving image acquisition request from the second communication device. In the moving image acquisition method for acquiring a moving image main image by communication, the first communication device adds a still image thumbnail and a moving image thumbnail to the moving image main image. With a common number of pixels The second communication device creates a request for a still image thumbnail for the moving image main image from the first communication device, acquires the still image thumbnail from the first communication device, and the still image thumbnail. And the second communication device selects one still image thumbnail based on the displayed still image thumbnail, and the moving image corresponding to the selected still image thumbnail is displayed on the first communication device. Requesting a thumbnail, obtaining a moving image thumbnail from the first communication device, and displaying the moving image thumbnail as a moving image, wherein the second communication device performs the above-described still image thumbnail or moving image thumbnail based on the displayed still image thumbnail or moving image thumbnail It is characterized in that a desired moving image main image is requested from the first communication device, and the moving image main image is acquired from the first communication device.
The second communication device that requests a desired moving image main image from the first communication device first requests the first communication device for a still image thumbnail corresponding to the moving image main image, and acquires the still image thumbnail. And display. Subsequently, when confirming whether the desired moving image main image is a still image thumbnail, the moving image thumbnail corresponding to the still image thumbnail is requested to the first communication device, and the moving image thumbnail is acquired and displayed as a moving image. In this way, the moving image main image is confirmed and the desired moving image main image is surely acquired. If a desired moving picture main image can be specified from still picture thumbnails, the moving picture main image may be acquired immediately in relation to the still picture thumbnail.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
In the following, according to the accompanying drawings, Movement A preferred embodiment of the image acquisition method will be described in detail.
[0025]
FIG. 1 is an external perspective view of an electronic camera which is an embodiment of an image processing apparatus according to the present invention.
[0026]
As shown in the figure, an electronic camera 10 that is one form of an image processing apparatus includes a lens barrel 100 that houses a photographing lens that forms an image of a subject on an imaging unit, and activation of the functions of the electronic camera 10. And a power switch 105 for instructing to stop, a release button 106 for instructing a user to record images and recording audio, and a mode switching dial 107 for setting various modes provided in the electronic camera 10. .
[0027]
The electronic camera 10 also has a cancel button 109 operated when canceling a desired item or image from the items displayed on the display means 168, and an instruction to move the cursor to the desired item and select and execute that item. The execution button 110 for inputting the menu item, the menu button 111 for instructing the display unit 168 to display the menu of the setting item, and the cross button 113 used for moving the cursor displayed on the display unit 168 are provided. Yes.
[0028]
The electronic camera 10 also serves as a communication port for performing wired communication and an antenna 156 for transmitting and receiving carrier waves and data when transmitting and receiving information to and from an external communication device (such as a portable terminal) by wireless communication. A communication connector 162; display means 168 for displaying captured images and information relating to processing and operation of the electronic camera 10; an image signal connector 173 that is an output port for outputting a display image signal to an external device; A removable recording medium 177 for recording information such as sound is provided.
[0029]
In the example shown in the figure, the image captured by the user is reproduced and displayed on the display unit 168, and information on the processing mode currently set in the electronic camera 10 and compression of the image at the time of imaging are also displayed. The rate information and the frame number display 104 of the image are displayed.
[0030]
FIG. 2 is a block diagram of a signal processing system of an electronic camera which is an embodiment of the image processing apparatus according to the present invention.
[0031]
An image processing unit of an electronic camera 10 that is an embodiment of an image processing apparatus including an imaging unit includes an imaging unit 150 that forms an image of a subject on a light receiving surface, performs photoelectric conversion, and outputs the image signal, and an image size. Change processing, sharpness correction, contour processing, contrast correction, and the like are performed on the analog signal, and an A / D converter 154 that converts the analog image signal into digital image data is provided. Yes.
[0032]
The communication means of the electronic camera 10 that transmits and receives information to and from other communication devices includes an antenna 156 that transmits and receives information such as image data on a carrier wave such as radio waves, and image data acquired from information processing means and memory. Transmission / reception means 157 (transmission means, reception means) for transmitting / receiving other information to / from other communication devices, and transmitting / receiving information such as image data to / from other communication apparatuses via a communication cable. The communication connector 162 is configured.
[0033]
Further, the electronic camera 10 includes a compression / expansion unit 167 for compressing or decompressing information such as image data by a method shown in JPEG-2000 or the like, and decompressing / decompressing the compressed image data. OSD 169 for on-screen display of various characters and message data to be displayed together with images, frame memory 171 for temporarily storing image data for display, composite signal for display of digital image data, etc. A D / A converter 172 for converting the image signal to the image signal and an image signal connector 173 (not shown in FIG. 2) for outputting a display signal to an external display device are provided.
[0034]
In addition, the electronic camera 10 includes an input unit 174 and an input unit 174 including a power switch 105, a release button 106, a mode switching dial 107, a cancel button 109, an execution button 110, a menu button 111, a cross button 113, and the like. I / O 175 that inputs and outputs input information and display information such as LEDs, a recording medium mounting unit 178 that detachably mounts the recording medium 177, and image data and the like on the recording medium 177 A recording medium interface 179 for recording and reading information is provided. The recording medium 177 is a detachable recording medium typified by a semiconductor such as a memory card, a magnetic recording type typified by MO, etc., an optical recording type, or the like.
[0035]
The electronic camera 10 controls the entire electronic camera 10 and controls image data sampling timing control, image data recording control, communication control, image and audio recording control, display control, image processing control, and the like. Information processing means 180 (moving image thumbnail generating means, still image thumbnail generating means, image recording means) and information such as various constants for adjustment that can be rewritten and stored even after the power is cut off A non-volatile memory 182, a ROM 183 storing unique identification information such as the model name and serial number of the electronic camera 10, an operation program, various constants, display information, and the like, and a work area when executing the program And a RAM 184 which is a high-speed readable / writable recording means.
[0036]
Further, the electronic camera 10 includes a calendar clock 190 that keeps time, a timer 190A that counts time based on an instruction from the information processing unit 180, and a light emitting unit 193 that emits light to the subject to compensate for the shortage of light amount during imaging. A light emission control means 194 for adjusting the light emission timing of the light emission means 193 and the amount of light emitted is provided.
[0037]
As shown in the figure, the information processing means 180 of the electronic camera 10 and its peripheral circuits are connected by a bus 199, and can transmit information to each other at a high speed. Each peripheral circuit can be controlled based on the processing program.
[0038]
The information processing means 180 can set a time for the timer 190A to monitor a communication timeout and measure a transfer rate or a transfer time necessary for transmitting / receiving information such as a still image thumbnail. ing.
[0039]
The information processing means 180 (moving image thumbnail generating means) is a process for reducing the number of pixels or gradation of moving images with respect to moving image main images or captured moving images recorded in recording means such as the recording medium 177 or the RAM 184. It is possible to generate a video thumbnail that has been implemented. Further, the information processing unit 180 (moving image thumbnail generating unit) can generate a moving image thumbnail that has been subjected to processing for reducing the number of frames of a moving image.
[0040]
Further, the information processing unit 180 (still image thumbnail generating unit) performs still image reduction processing on the representative image such as the first frame of the moving image main image or the first frame after the scene change to reduce the number of pixels or gradation. Image thumbnails can be generated.
[0041]
Further, the information processing unit 180 (image recording unit) can record the moving image main image, the generated moving image thumbnail and the still image thumbnail on the recording medium 177 in association with each other. In addition, the information processing unit 180 can output an instruction to transmit / receive to the other communication device such as a mobile terminal by associating the moving image main image and the moving image thumbnail with the transmitting / receiving unit 157 (transmitting unit). .
[0042]
The transmission / reception unit 157 (reception unit) can receive moving image thumbnail information related to the number of pixels or gradation of the moving image thumbnail or the frame rate from another communication device such as a portable terminal. The transmission / reception unit 157 (reception unit) can receive still image thumbnail information related to the number of pixels or the gradation of the still image thumbnail from another communication device such as a portable terminal.
[0043]
Further, the information processing means 180 (moving image thumbnail generating means) is based on the received moving image thumbnail information, with respect to the moving picture main image recorded in the recording means such as the recording medium 177 or the RAM 184, It is possible to generate a moving image thumbnail that has been subjected to processing for reducing gradation. Further, the information processing means 180 can generate a moving image thumbnail that has been subjected to the process of reducing the number of moving image frames based on the received moving image thumbnail information. Further, the information processing means 180 (still image thumbnail generating means) applies the number of pixels to the representative image of the moving image main image recorded in the recording means such as the recording medium 177 or the RAM 184 based on the received still image thumbnail information. Alternatively, it is possible to generate a still image thumbnail that has been subjected to processing for reducing gradation.
[0044]
FIG. 3 is a diagram showing a display of various setting modes provided on the mode switching dial 107.
[0045]
According to the figure, the mode switching dial 107 is recorded with a communication mode (indicated as “Wireless” in FIG. 3) in which wireless communication is performed with other communication devices such as a personal computer and a mobile terminal existing in the vicinity. A playback mode (represented as Play in FIG. 3), a still image capture mode (represented as Cam in FIG. 3) for capturing a still image, and a moving image capture for capturing a movie A mode (denoted as Mov in FIG. 3), a setup mode (denoted as Setup in FIG. 3) for setting the date, photographing mode, number of photographing pixels, auto power-off time setting, alarm sound volume, etc. Is described. When the user operates the mode switching dial 107, the processing mode of the electronic camera 10 can be set to a predetermined mode.
[0046]
An imaging process of the electronic camera 10 configured as described above will be described.
[0047]
The subject image to be imaged is formed on the light receiving surface of the imaging unit 150 by the imaging lens, and the formed subject image is photoelectrically converted and output to the image processing unit 153. The image processing means 153 performs image processing such as correlated double sampling, amplification, and noise reduction processing on the image signal, and outputs it to the A / D converter 154 to convert it into digital data.
[0048]
The image data converted into the digital data is subjected to image processing such as image white balance, gamma correction, YC conversion, zoom processing, pixel number conversion processing and the like according to an instruction from the information processing means 180, and is then temporarily stored in the RAM 184. The
[0049]
The information processing means 180 transfers the display image data to the frame memory 171. The image data stored in the frame memory 171 is sequentially transmitted to the D / A converter 172 and displayed on the display unit 168 together with information such as characters generated from the OSD 169.
[0050]
When the user presses the release button 106 for imaging instruction provided on the input unit 174, the information processing unit 180 enters a mode for imaging a subject. When capturing a still image, generally, a two-stage release button 106 is used, the first stage is used for imaging preparation such as AE and AF adjustment, and the second stage is used as an actual imaging instruction. use. When a moving image is captured, the moving image may be captured only while the release button 106 is being pressed, or each time the release button 106 is pressed while the release button 106 is set to a double action. May be instructed to start / stop imaging.
[0051]
When the image is recorded on the recording medium 177, the information processing unit 180 sequentially reads out the image stored in the RAM 184, and performs image white balance, gamma correction, YC conversion, zoom processing, pixel number conversion processing, and the like. After the processing, the image is transferred to the compression / decompression unit 167 and the image is compressed. The information processing means 180 associates the compressed image data with the attached information such as the imaging conditions when the image is captured, generates one image file, and once again enters the FILE work area in the RAM 184 again. Memorize temporarily. Thereafter, the image file is converted into image data for recording.
[0052]
When an image file is recorded on the recording medium 177, the image file is output to the recording medium interface 179. The recording medium interface 179 performs processing for sequentially recording the acquired image files on the recording medium 177 via the recording medium mounting unit 178. At this time, the information processing means 180 may generate a still image thumbnail and a moving image thumbnail, and perform processing of associating with the main image and recording it on the recording medium 177.
[0053]
When outputting the image file to another communication device via the transmission / reception unit 157, the information processing unit 180 performs a process of outputting the generated image file to the transmission / reception unit 157. The transmission / reception unit 157 that has acquired the image file starts communication connection with another communication device such as a personal computer or a portable terminal via the communication connector 162, and performs processing for transmitting the image file.
[0054]
In the above description, the image processing apparatus according to the present invention has been described as an electronic camera including an imaging unit. However, the present invention is not limited to this, and the image processing apparatus externally transmits an image. Even a personal computer that performs image processing and recording of images obtained from the above apparatus can achieve the object of the present invention.
[0055]
FIG. 4 is an explanatory diagram showing a tree display of image files and their directories recorded on recording means such as the recording medium 177.
[0056]
As shown in the figure, the information recording area of the recording medium 177 is provided with a root directory “Root”, and the root directory includes a directory “001 Vacation” generated by the user, a directory “002Birthday”, and Is provided.
[0057]
In the “001 Vacation” directory, a group of still image files such as “DSCF0001.JPG” and a group of moving image files such as “DSCF0004.AVI” are recorded.
[0058]
In the “002 Birthday” directory, a still image file group such as “DSCF0001.JPG” and a moving image file group such as “DSCF0002.AVI” are recorded.
[0059]
In the above embodiment, an extension such as “.JPG” or “.AVI” can be added to the end of the file name to identify whether the file is a still image file or a moving image file. I have to. Further, as shown in the figure, an image file is specified by creating a 7-digit frame number composed of a 3-digit directory number (001 or 002) and a 4-digit file number (0001, 0002, etc.). Image files can be managed. Note that the file name is automatically assigned a file name that is sequentially numbered for both moving images and still images unless otherwise specified by the user.
[0060]
FIG. 5 shows the structure of an image file in which still images are recorded.
[0061]
As shown in the figure, in the image file, a portion in which additional information related to the still image main image is described and a still image thumbnail (160 × 120 pixel reduced image) of the still image main image are recorded. A portion and a portion where a still image main image is recorded are provided. The still image main image is JPEG compressed image data having a pixel number of, for example, 1280 × 960. On the other hand, the still image thumbnail is JPEG compressed image data having a pixel number of 160 × 120, for example, and is obtained by spatially reducing still image main image data.
[0062]
FIG. 6 shows the structure of an image file in which moving images are recorded.
[0063]
As shown in the figure, the image file includes a portion in which additional information related to the moving image is described, and a portion in which a still image thumbnail (160 × 120 pixel size) of a representative image of the moving image is recorded. A portion where the moving image is recorded (portion where a plurality of 640 × 480 pixel still images are recorded) and a portion where a moving image thumbnail is recorded (the moving image main image is 160 × 120 pixel size, etc.) The number of frames may be smaller than that of the main moving image). Note that the number of pixels of the thumbnail is not limited to the 160 × 120 size described above, and may be recorded after being converted to a number of 100 × 100 pixels or the like in accordance with the display capability on the mobile terminal 30 side. .
[0064]
In addition to the shooting information similar to the information recorded in the auxiliary information part of the image file of the still image, the recording part of the auxiliary information of the image file is specific to the moving image such as the number of pixels, the frame rate, the total number of frames, etc. This information is also recorded.
[0065]
As described above, since the still image thumbnail and the moving image thumbnail are JPEG compressed data having a pixel number of 160 × 120, both thumbnails can be handled in common. The still image thumbnail of the moving image file is obtained by spatially reducing a frame representing the content of the representative image of the moving image main image. In the case of an electronic still camera, generally, the moving image shooting time that can be recorded at one time is not so long, so that it is possible to create a still image thumbnail of a moving image with the first frame or scene change portion as a representative frame.
[0066]
FIG. 7 shows an external view of a mobile terminal according to the present invention.
[0067]
As shown in the figure, the mobile terminal 30 displays an image and other various information, and at the same time, a display means 368 configured by a touch panel on which a user inputs information, a power switch 305, the mobile terminal 30 A power supply pilot lamp is provided to notify the user that is operating.
[0068]
The user instructs (tap) the icon button or menu displayed on the display unit 368 with the attached touch pen, or uses the character input recognition unit assigned to the character input area of the display unit 368 to execute the command. It is possible to input characters and characters.
[0069]
FIG. 8 shows a block diagram of a signal processing system of the mobile terminal 30 according to the present invention.
[0070]
As shown in the figure, the information transmission / reception unit of the mobile terminal 30 includes an antenna 356 that performs short-distance wireless communication with a communication device such as the electronic camera 10 and data in a predetermined format for wireless transmission / reception of information. And a transmission / reception means 357 (transmission means, reception means) for converting.
[0071]
Further, the portable terminal 30 includes a display unit 368 for displaying information such as images and characters, a frame memory 371 for temporarily storing image data for display, and a display for recording recorded in the frame memory 371. The user designates the display control means 372 for reading out the image data, converting the image data into a display image signal suitable for display by the display means 368, and the icon displayed on the display means 368. A touch panel 369 that reads information indicating the character information and character information input by the user, and a touch panel control unit 370 that outputs information input by the user via the touch panel 369 to the information processing unit 380.
[0072]
Also, the mobile terminal 30 reads various information input by the user via the input unit 374 such as the power switch 305 and transmits the information to the information processing unit 380 described later, or based on an instruction acquired from the information processing unit 380. And an I / O 375 for outputting a display command to a notification means such as a pilot lamp (LED) of the power source.
[0073]
In addition, the mobile terminal 30 is provided with a recording medium mounting unit 378 for detachably mounting the recording medium 377 and a recording medium interface 379 for recording and reading information such as image data with respect to the recording medium 377. ing. The recording medium 377 is a detachable recording medium similar to the recording medium 177.
[0074]
Further, the mobile terminal 30 is processed by an information processing means (CPU) 380 that controls the entire mobile terminal 30, a ROM that stores a program for operating the information processing means 380 and various constants, and an information processing means 380. Information such as connection information used when performing transmission / reception of information with the electronic camera 10 and various constants relating to the processing of the mobile terminal 30 and the memory 381 constituted by the RAM which is a recording means serving as a work area when executing Are provided, a non-volatile memory 382 that records time, a calendar clock 390 that keeps time, a timer 390A that counts time, a battery 391A that supplies power for operating the mobile terminal 30, and a power source 391.
[0075]
As shown in the figure, the information processing means 380 in the portable terminal 30 and each peripheral circuit including the I / O 375, the memory 381, the calendar clock 390, etc. are connected by a bus 399, and the information processing means 380 It is possible to control the circuits and to control each peripheral circuit based on a processing program executed by the information processing means 380.
[0076]
The short-distance wireless communication means is communication means using light such as radio waves, ultrasonic waves, and infrared rays as a carrier wave. When radio waves are used, it may be based on wireless communication specifications such as a wireless LAN (local area network) and Bluetooth.
[0077]
In the above description, the mobile terminal 30 has been described. However, a communication terminal such as a mobile phone or an electronic notebook may be used as long as the communication device includes a recording unit, a communication unit, and the like.
[0078]
The transmission / reception unit 357 (transmission unit) can transmit the moving image thumbnail information regarding the number of pixels or the gradation or the frame rate of the moving image thumbnail to another communication device. The transmission / reception unit 357 (reception unit) can receive a still image main image, a moving image main image, a moving image thumbnail, and a still image thumbnail from another communication device.
[0079]
The display unit 368 can display the still image main image, the moving image main image, the moving image thumbnail, and the still image thumbnail received by the transmission / reception unit 357 from another communication device.
[0080]
The portable terminal 30 receives an image file from the electronic camera 10 via the transmission / reception unit 357, reads out a still image or a moving image recorded in the received image file, performs expansion processing, and displays the display unit. 368 can be displayed. Generally, unlike the electronic camera 10, the mobile terminal 30 is not a specialized device that handles images, and therefore does not include a hardware image decompression processing circuit or image processing circuit. Therefore, since the image processing is performed based on the software processing performed by the information processing means 380, the image reproduction processing speed is low. In particular, in the reproduction processing of a moving image with a heavy image processing load, the image processing that can be reproduced is limited in terms of the number of pixels, the frame rate, the gradation, and the like.
[0081]
For example, as a real-time reproduction capability of stream image data of a moving image (moving image received via the transmission / reception unit 357) received via the wireless communication interface in the mobile terminal 30, a floor of 160 × 120 pixels having 8 bits. For example, there is a limit of 20 fps (20 frames per second) at the maximum. When the moving image data received by the portable terminal 30 is reproduced, pre-processing is performed at the time of reproduction, decompression processing is performed, and after matching with the number of reproduction pixels of the own model, reproduction display is started.
[0082]
9 and 10 are diagrams illustrating a flow of communication processing performed between the electronic camera 10 and the mobile terminal 30. FIG.
[0083]
As shown in the figure, in S10 “wireless communication mode”, the user operates the mode switching dial 107 provided in the electronic camera 10 to set the mode to “Wireless”. Then, the processing mode of the electronic camera 10 is set to a wireless communication standby mode (step S10).
[0084]
On the other hand, the mobile terminal 30 is also set to a wireless communication mode for performing wireless communication. Then, the processing performed by the information processing means 380 branches to step S30 “wireless communication mode” (hereinafter abbreviated as S30), and sets the processing mode of the portable terminal to the wireless communication mode.
[0085]
When the mobile terminal 30 is set to the wireless communication mode, the process proceeds to the next S32 “Select connected device (select camera)”, and the display unit 368 of the mobile terminal 30 displays the wireless communication connection destination device shown in FIG. A selection display is made.
[0086]
FIG. 12 is an explanatory diagram of the selection display of the wireless communication connection destination device displayed on the display unit 368 of the mobile terminal 30.
[0087]
As shown in the figure, the display means 368 is present in the vicinity of the portable terminal 30 and the title “Wireless Communication” for notifying the user that the current display is a selection display of the wireless communication connection destination device. The menu display “Portable Phone”, “Digital Camera”, and “Portable Audio Player” of a plurality of communication devices that can be connected by wireless communication are displayed. In addition, when there are a plurality of communication devices capable of wireless communication, the user can specify the icon of the △ button for scrolling up and the icon for the ▽ button (tap with a stylus or a touch pen) to display the present device. It is possible to scroll and display other communication devices that are not.
[0088]
The user selects a desired communication device for performing wireless communication (in the example shown in the figure, the △ button and the ▽ button are tapped to scroll, and “Digital Camera” in the middle framed edge is selected. ) And then tapping the “connection” icon, wireless communication connection and establishment processing is started for the MAC address of the electronic camera 10. Note that the MAC address of the electronic camera 10 is recorded in advance in the nonvolatile memory 382 of the portable terminal 30.
[0089]
When the electronic camera 10 receives the connection request information issued by the portable terminal 30, the information processing means 180 of the electronic camera 10 starts processing for establishing wireless communication with the portable terminal 30 based on the received connection request information. . When a communication connection is established between the electronic camera 10 and the portable terminal 30, the information processing unit 180 outputs connection completion information to the portable terminal 30.
[0090]
The mobile terminal 30 that has received the connection completion information transmits to the electronic camera 10 an image list request instruction for requesting a list of images.
[0091]
When the electronic camera 10 obtains the image list request instruction from the portable terminal 30, the processing performed by the information processing unit 180 proceeds to S12 “image list reproduction”, and a plurality of recordings recorded in a plurality of directories on the recording medium 177 are performed. Attached information in the image file is read out, and an image list shown in FIG. 15 is created and transmitted to the portable terminal 30.
[0092]
FIG. 15 shows an example of creating an image list of a plurality of image files recorded on the recording medium 177.
[0093]
The information of “S” and “M” described in the first column of the image list shown in the figure is that the main image recorded in each image file is a still image (Still) or a moving image. This is information indicating whether or not (Movie). Information such as “001-0001” described in the next column represents a frame number for specifying an image file.
[0094]
In the next column, “JPG” and “AVI” are file type information indicating whether the recorded image file is a still image file or a moving image file. Note that the compression format of images that can be handled is not limited to the compression format having the extensions “JPG” and “AVI”, and may be any other compression format. In the next column, the number of horizontal and vertical pixels of the image recorded in the image file is recorded, and information such as the frame rate of the moving image, the file size of the image file, the shooting date and time, and so on are recorded. Are listed.
[0095]
When the mobile terminal 30 acquires the image list created by the electronic camera 10, the information processing unit 380 sends a still image thumbnail transmission request instruction for each image described in the image list via the transmission / reception unit 357. Send to. This still image thumbnail request is transmitted in order from the first image file described in the image list, for example. At this time, a still image thumbnail request including the number of pixels that can be processed by the mobile terminal 30, gradation, and other information may be transmitted to the electronic camera 10.
[0096]
The information processing means 180 of the electronic camera 10 that has received the still image thumbnail request reads the still image thumbnail of the image file corresponding to the acquired still image thumbnail request in S14 “Still image thumbnail extraction (001-0001)”. Thus, processing for transmitting to the portable terminal 30 via the transmission / reception means 157 is performed. The still image thumbnail data to be transmitted to the mobile terminal 30 is divided into a plurality of packets and then transmitted to the mobile terminal 30. At this time, the information processing means 180 of the electronic camera 10 causes the total transfer time T ( Second) is measured using the timer 190A. Then, the effective transfer rate R (KB / S) = S / T of the wireless communication interface is calculated from the measured total transfer time T (seconds) and the size S (KB) of the transmitted still image thumbnail image data. You may do it.
[0097]
Thereafter, in response to the still image thumbnail transmission request output from the mobile terminal 30, the electronic camera 10 repeatedly performs the processing of S16 "Still Image Thumbnail Extraction (001-0002)" and the like.
[0098]
In the above description, still image thumbnails are collectively transmitted from the electronic camera 10 to the mobile terminal 30 in advance after communication connection. However, the display means of the mobile terminal 30 is not transmitted collectively. Only the still image thumbnails displayed at 368 may be sequentially transmitted according to the scrolling of the display.
[0099]
When the mobile terminal 30 completes the acquisition of the thumbnails of all the images listed in the image list, the processing performed by the information processing means 380 of the mobile terminal 30 proceeds to S34 “Still Image Display Thumbnail Image Select”.
[0100]
In S34, the information processing unit 380 displays a list of still image thumbnails shown in FIG. 13 on the display unit 368 and notifies the user.
[0101]
FIG. 13 is a diagram showing a list display example of still image thumbnails displayed on the display unit 368 of the mobile terminal 30.
[0102]
As shown in the figure, the display means 368 displays a plurality of still image thumbnails as a list display of a plurality of still images and moving images recorded on the recording medium 177 of the electronic camera 10. It should be noted that the moving image and the still image have the same number of pixels or the same image size as the still image thumbnail of the image file in which the moving image is recorded and the still image thumbnail of the image file in which the still image is recorded. It is possible to display uniformly without providing separate thumbnail display fields.
[0103]
On the right side of the thumbnail, a frame number for specifying the image file of the thumbnail and “still” and “movie” for identifying whether the image recorded in the image file is a moving image or a still image. Information such as the display, the shooting date of the image, and the file size of the image file are displayed.
[0104]
The display means 368 does not have a desired image or a Δ button or a ▽ button icon for inputting an instruction to display another still image thumbnail by scrolling the displayed still image thumbnail up or down. In this case, a “disconnect” icon for inputting an instruction to disconnect communication with the electronic camera 10 is displayed.
[0105]
The display unit 368 also has a “GET” button for selecting a still image thumbnail at the middle framed edge and acquiring the main image of the still image thumbnail from the electronic camera 10, and an edge. A “confirmation” icon is displayed for instructing acquisition of the moving image thumbnail recorded in the moving image file from the electronic camera 10 by selecting the still image thumbnail of the moving image in the middle row. Yes.
[0106]
When a still image thumbnail of a still image file is selected, there is no moving image thumbnail in the target still image file, so “gray out display” indicating that the “confirm” icon is invalid is displayed. Thus, the user is notified of this (not shown in FIG. 13).
[0107]
A case will be described in which a user taps the up and down icons while browsing a list display of still image thumbnails, scrolls the still image thumbnail, and selects a still image thumbnail in the middle framed border.
[0108]
For example, when the user selects a still image thumbnail (for example, 001-0004) and taps a “confirm” icon, the processing performed by the information processing unit 380 of the mobile terminal 30 is S36 “selected image (001-00004). ) Check ”. In S <b> 36, the information processing unit 380 transmits a moving image thumbnail request including the frame number of the moving image and the number of pixels that can be processed by the mobile terminal 30, the gradation, the frame rate, and other information to the electronic camera 10.
[0109]
When the electronic camera 10 receives the thumbnail information of the moving image, the processing performed by the information processing unit 180 proceeds to S20 “determine moving image thumbnail specification (frame rate)”, and the specification of the moving image thumbnail to be created as described below. The process of determining is performed.
[0110]
FIG. 11 is a flowchart of an algorithm for determining the specifications of the moving image thumbnail in the electronic camera 10.
[0111]
When the electronic camera 10 and the mobile terminal 30 establish wireless communication, the processing performed by the information processing means 180 proceeds from S100 “Start” to S102 “Acquire effective transfer rate (R)”.
[0112]
In step S102, the information processing unit 180 performs processing for acquiring the effective transfer rate (R) of the still image thumbnail actually transmitted in step S14 and subsequent steps.
[0113]
In the next step S104 “Acquire the number of moving image thumbnail pixels requested by the PDA”, the information processing means 180 determines the number of moving image thumbnail pixels of the moving image that the user desires to view by operating the mobile terminal 30 (PDA). The process acquired from the portable terminal 30 is performed.
[0114]
The number of pixels used when generating a moving image thumbnail is the specification (number of pixels and frame rate) that can be reproduced on the mobile terminal 30 together with the frame number of the moving image in the request information of the moving image thumbnail transmitted from the mobile terminal 30. If specified, the information processing means 180 of the electronic camera 10 performs a process of generating a moving image thumbnail based on the specification of the number of pixels received from the mobile terminal 30.
[0115]
In the next step S106 “Calculate one frame data size of moving image thumbnail with respect to the number of moving image thumbnail pixels (FS)”, the information processing means 180 determines the amount of data (data) per frame based on the number of pixels designated from the mobile terminal 30. Size) is performed. For example, here, the data amount per frame may be calculated by performing a process of actually compressing the first frame of the moving picture main image recorded in the image file to the specified number of pixels.
[0116]
In the next S108 “Calculate the maximum frame rate FRmax = R / FS”, the information processing means 180 actually calculates the data amount per frame of the moving image calculated in S106 and the effective transfer rate of the communication interface calculated in S102. The frame rate (FRmax) of the moving image thumbnail that can be transmitted to is calculated.
[0117]
In the next S110 “Acquire frame rate requested by PDA (FR1, FR2,..., FRn)”, processing for obtaining the frame rate requested by portable terminal 30 is performed.
[0118]
In the next S112 “the maximum value not exceeding FRmax among FR1, FR2,..., FRn is used as the frame rate FR of the moving image thumbnail”, the information processing means 180 exceeds the frame rate presented by the mobile terminal 30. The maximum frame rate that does not exceed the frame rate of the moving image thumbnail that can actually be transmitted is selected, and the process of adopting the maximum frame rate is performed, and the process proceeds to S114 “End”. As described above, the information processing means 180 of the electronic camera 10 can determine the maximum moving image thumbnail frame rate that looks as natural as possible within the allowable range.
[0119]
In S22 “moving image thumbnail generation” shown in FIG. 10, processing for actually generating a moving image thumbnail from the number of pixels and the frame rate determined in S20 is performed. Specifically, first, frames are thinned out in the time direction, and then each thinned frame is resized (reduced) to recompress the image.
[0120]
The moving image thumbnail created in this way is isochronously transferred to the mobile terminal 30 in real time according to the actual frame rate, for example. In S38, the information processing unit 380 displays each frame of the received moving image thumbnail in the still image thumbnail area of the display unit 368 (see FIG. 13).
[0121]
Note that the moving image thumbnails displayed on the display unit 368 may be collectively generated on the electronic camera 10 side and transmitted to the portable terminal 30 as one image file. The mobile terminal 30 that has received the image file reads out the moving image thumbnail from the image file, performs decompression processing, and performs processing to reproduce and display the moving image thumbnail on the display unit 368 at a predetermined frame rate.
[0122]
Here, when the user confirms the contents of the displayed moving image thumbnail and confirms that the image is a target image desired to be browsed, “GET” displayed on the display unit 368 is displayed. Select an icon. Then, the processing executed by the information processing means 380 proceeds to S40 “GET Selected Image (001-0004)”.
[0123]
In S <b> 40, in response to the selection of the “GET” icon, the information processing means 380 transmits a moving image main image transmission request (image file request) with frame numbers 001-0004 to the electronic camera 10.
[0124]
When the electronic camera 10 receives the image file request, the process performed by the information processing unit 180 proceeds to the process of S24 “Read image file (001-0004)”, and 001-0004 recorded in the recording medium 177. A process for reading the image file of the frame number is performed. Then, the information processing unit 180 performs a process of transmitting the read image file 001-0004 to the portable terminal 30 via the transmission / reception unit 157. At this time, the electronic camera 10 may convert the resolution and the frame rate of the moving image main image in accordance with the display capability of the display unit 368 of the mobile terminal 30 and transmit the converted video.
[0125]
When the portable terminal 30 receives the image file 001-0004, the processing performed by the information processing unit 380 proceeds to S42 “display acquired image”, reads out the moving image main image included in the received image file, and decompresses it. After performing image processing such as processing, processing for display on the display means 368 is performed.
[0126]
FIG. 14 shows a display example in which a moving image is displayed on the display means 368.
[0127]
The moving image main image data received from the electronic camera 10 is displayed in high pixels on the display means 368 as shown in FIG. In this manner, the user can operate the portable terminal 30 to select and browse a desired image recorded in the electronic camera 10. When the user stops viewing the moving image displayed, for example, the user taps (selects) the “disconnect” icon displayed on the display unit 368 and inputs an instruction to disconnect the communication connection. To do.
[0128]
If the user wishes to continue to acquire and display another image, the user taps the “return” icon. Then, the display on the display unit 368 returns to the still image thumbnail list display shown in FIG. 13, and it is possible to select another desired image again and repeat the same operation.
[0129]
Note that if the image file recorded in the electronic camera 10 is recorded with audio along with the image, the audio data that has been compressed with a small amount of data together with the moving image thumbnail (audio data with a reduced sampling frequency, etc.) ) May be attached.
[0130]
【The invention's effect】
As described above, according to the image processing apparatus of the present invention, the moving image thumbnail generating means for reducing the number of pixels, the gradation, and the number of frames of the moving image main image, and the number of pixels or the level of the representative image of the moving image main image. Since it has a still image thumbnail generating means for performing processing to reduce the tone and an image recording means for recording the moving picture main image, the moving picture thumbnail and the still picture thumbnail in association with each other, it is suitable for processing on the mobile terminal side. It is possible to generate a moving image thumbnail and a still image thumbnail of a size.
[0131]
According to another aspect of the invention, a receiving unit that receives moving image thumbnail information from a mobile terminal or the like, and a process of reducing the number of pixels, gradation, and number of frames based on the moving image thumbnail information received by the receiving unit. Moving image thumbnail generating means, and image recording means for associating and recording the moving picture main image and the moving picture thumbnail, or transmitting means for associating the moving picture main image and the moving picture thumbnail and transmitting them to other communication devices. It is possible to acquire the playback environment of the communication device on the acquisition side and transmit a moving image suitable for the playback environment of the communication device on the moving image acquisition side.
[0132]
According to another aspect of the invention, moving image thumbnail information related to the number of pixels, gradation, and frame rate of a moving image thumbnail and still image thumbnail information related to the number of pixels of a still image thumbnail and gradation are received from another communication device. Receiving means, moving picture thumbnail generating means for performing processing for reducing the number of pixels, gradation, and number of frames of the moving picture on the moving picture main image based on the received moving picture thumbnail information, and the received still picture thumbnail Still image thumbnail generation means for performing processing for reducing the number of pixels or gradation on the representative image of the moving image main image based on the information, and image recording that records the moving image main image, the moving image thumbnail, and the still image thumbnail in association with each other Means, or transmission means for associating a moving picture main image, a moving picture thumbnail, and a still picture thumbnail with each other and transmitting them to another communication device. Since, to obtain a moving image and the reproduction environment of the communication apparatus acquiring side of the still image, it is possible to transmit a moving image conforming to the moving image and the reproduction environment of the communication apparatus acquiring side of the still image.
[0133]
According to another aspect of the invention, the transmission means for transmitting the moving image thumbnail information related to the number of pixels or the gradation or the frame rate of the moving image thumbnail to the image processing apparatus or the like, and the receiving means for receiving the moving image thumbnail from the image processing apparatus or the like. And a display means for displaying the moving image thumbnail received by the receiving means as a moving image, it is possible to quickly receive and display a moving image based on the moving image thumbnail information from the image processing apparatus or the like.
[0134]
According to the present invention, a user can check a moving image recorded in a recording unit or the like of an electronic camera by browsing a still image thumbnail or a moving image thumbnail on the mobile terminal side. In addition, since the acquisition of video thumbnails with a large amount of image data is performed according to the need for browsing, it is possible to reduce unnecessary waiting time for the user and to perform comfortable image browsing operations. It becomes possible.
[0135]
Further, according to the present invention, the moving image thumbnail for confirming the moving image is adjusted to an optimum size depending on the image processing capability of the mobile terminal on the image acquisition side, the effective communication speed of the communication interface, and the like. The time can be minimized, and the moving image thumbnail can be surely displayed without unsightly frame dropping.
[0136]
Further, according to the present invention, in the moving image pull model, the user on the image acquisition side can confirm the content of the moving image quickly and reliably.
[Brief description of the drawings]
FIG. 1 is an external perspective view of an electronic camera which is an embodiment of an image processing apparatus according to the present invention.
FIG. 2 is a block diagram of a signal processing system of the image processing apparatus according to the present invention.
FIG. 3 is a diagram showing display examples of various setting modes provided on the mode switching dial.
FIG. 4 is an explanatory diagram showing a tree display of image files and their directories.
FIG. 5 is a diagram showing the structure of an image file in which still images are recorded
FIG. 6 is a diagram showing the structure of an image file in which moving images are recorded
FIG. 7 is an external view of a mobile terminal according to the present invention.
FIG. 8 is a block diagram of a signal processing system of a mobile terminal according to the present invention.
FIG. 9 is a diagram illustrating a flow of communication processing performed between the electronic camera and the mobile terminal.
FIG. 10 is a diagram illustrating a flow of communication processing performed between the electronic camera and the mobile terminal.
FIG. 11 is a flowchart of an algorithm for determining the specification of a moving image thumbnail in the electronic camera.
FIG. 12 is an explanatory diagram of selection display of a wireless communication connection destination device displayed on the display unit of the mobile terminal.
FIG. 13 is a list display example of still image thumbnails displayed on the display means of the mobile terminal.
FIG. 14 is a diagram illustrating an example in which a moving image is displayed on the display unit;
FIG. 15 shows an example of creating an image list of a plurality of image files recorded on a recording medium.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Electronic camera, 30 ... Portable terminal, 100 ... Lens barrel, 106 ... Release button, 107 ... Mode switch dial, 113 ... Cross button, 150 ... Imaging means, 153 ... Image processing means, 154 ... A / D converter 156 ... antenna 157 ... transmission / reception means 162 ... communication connector 167 ... compression / decompression means 168 ... display means 169 ... OSD 171 ... frame memory 172 ... D / A converter 174 ... input means 175 ... I / O, 177 ... recording medium, 178 ... recording medium mounting section, 179 ... recording medium interface, 180 ... information processing means, 182 ... nonvolatile memory, 183 ... ROM, 184 ... RAM, 190 ... calendar clock, 190A ... timer 193: Light emission means, 194 ... Light emission control means, 199 ... Bus, 356 ... Antenna, 357 ... Transmitter 368 ... Display means, 369 ... Touch panel, 370 ... Touch panel control means, 371 ... Frame memory, 372 ... Display control means, 374 ... Input means, 375 ... I / O, 377 ... Recording medium, 378 ... Recording medium mounting section, 379 ... Recording medium interface, 380 ... Information processing means, 381 ... Memory, 382 ... Non-volatile memory, 390 ... Calendar clock, 390A ... Timer, 399 ... Bus

Claims (6)

A moving image acquisition method for communicating between a first communication device and a second communication device, and acquiring a moving image main image from the first communication device in response to a moving image acquisition request from the second communication device In
The first communication device creates a still image thumbnail and a moving image thumbnail with a common number of pixels for the moving image main image,
The second communication device requests the still image thumbnail for the moving image main image from the first communication device, acquires the still image thumbnail from the first communication device, and displays the still image thumbnail. ,
The second communication device selects one still image thumbnail based on the displayed still image thumbnail, and requests a moving image thumbnail corresponding to the selected still image thumbnail from the first communication device. , Obtaining a video thumbnail from the first communication device, displaying the video thumbnail as a video,
The second communication device requests a desired moving image main image from the first communication device based on the displayed still image thumbnail or moving image thumbnail, and receives the moving image main image from the first communication device. A moving image acquisition method characterized by acquiring.   When the first communication device creates a moving image thumbnail for the moving image main image, the first communication device acquires the specification of the moving image thumbnail that can be displayed by the second communication device from the second communication device, and the moving image satisfies the specification. The method according to claim 1, wherein a thumbnail is created.   The moving image acquisition method according to claim 1, wherein the second communication device displays the moving image thumbnail after the moving image thumbnail is acquired.   The moving image acquisition method according to claim 1 or 2, wherein the second communication device displays the moving image thumbnail as a stream image while acquiring the moving image thumbnail in real time.   The first communication device calculates an effective transfer rate with the second communication device from the data amount of the still image thumbnail and its transfer time, and adjusts the size of the moving image thumbnail according to the effective transfer rate. The moving image acquisition method according to claim 1, wherein   The first communication device calculates an execution transfer rate with the second communication device from the data amount of the still image thumbnail and its transfer time, and the data amount per unit time of the moving image thumbnail is the execution transfer. 5. The moving image acquisition method according to claim 4, wherein the size of the moving image thumbnail is adjusted according to the execution transfer rate so that the maximum value does not exceed the rate.

Images