JP3542563B2 - Image playback method - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image reproducing method for reproducing a wide image.
[0002]
[Prior art]
In addition to the conventional monitor having an aspect ratio of 3: 4, a horizontally long monitor having an aspect ratio of 9:16 has been commercialized, and there is a demand for obtaining a wide image even with a video camera or an electronic still camera.
[0003]
For such a request, various methods have been proposed. For example, an optical method of attaching a wide converter lens to the front of the taking lens to enlarge the angle of view, or an image editing method for multiple still images taken by overlapping the camera by panning the camera There are a method of manually synthesizing the image on the apparatus, a method of cutting or masking the upper and lower portions of the shooting screen to visually create a wide screen, and the like.
[0004]
[Problems to be solved by the invention]
When displaying a wide image in which the angle of view is actually enlarged, when trying to display the entire wide image, the image becomes small and details cannot be confirmed, while only a part of the wide image is displayed. In this case, it is not possible to grasp which part of the wide image is being displayed.
[0007]
The present inventionWhen displaying a wide image, the entire wide image can be grasped and the details of the wide image can be confirmed while grasping which part of the entire wide image is being displayed.The purpose is to present an image reproduction method.
[0008]
[Means for Solving the Problems]
The image playback method according to the present inventionwideImage to multiple partial imagesdataDivided intoRecordedFrom recording mediaThe wide imageAn image reproducing method for reproducingA step of reading the partial image data from the recording medium, a first mode for sequentially reducing and combining the read partial image data to display the entire wide image, and a step of reading the partial image data A second mode for displaying data on the display means and displaying information for recognizing the position of the displayed partial image data in the wide image on the same screen, and selectively displaying the image; Display process andToYesIt is characterized by doing.
[0009]
【Example】
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0010]
FIG. 1 is a block diagram showing the overall schematic configuration of a video camera according to an embodiment of the present invention. Reference numeral 10 denotes a photographing lens including a zooming lens and a focusing lens, 12 denotes an image pickup device that converts an optical image of the photographing lens 10 into an electric signal, and 14 denotes a sample hold (S / S) that samples and holds an analog output of the image pickup device 12. H) circuit, 16 is an A / D converter for converting an analog output of the S / H circuit 14 into a digital signal, and 18 is gamma conversion, color balance adjustment, luminance / color difference signal conversion to the output of the A / D converter 16. This is a process circuit that performs well-known camera signal processing such as conversion.
[0011]
Reference numerals 20A, 20B, and 20C denote image memories each having a storage capacity of one field, and reference numeral 22 denotes a memory control circuit that controls writing and reading of the image memories 20A, 20B, and 20C. In this embodiment, a widened still image is automatically formed using the three image memories 20A, 20B, and 20C. Details will be described later.
[0012]
Reference numeral 24 denotes a recording medium for recording a captured image and an image synthesized for an ultra-wide image. For example, it comprises a magneto-optical disk device, an optical disk device, a hard disk device, a large-capacity solid-state memory device such as a flash memory, and the like.
[0013]
Reference numeral 26 denotes a motion detection circuit for detecting a motion vector from two consecutive fields of luminance signals from the memory control circuit 22, and a filter 28 for band-limiting the two luminance signals from the memory control circuit 22, and two outputs of the filter 28. And a vector operation circuit 32 for calculating a motion vector from the result of the matching operation of the matching circuit 30.
[0014]
Numeral 34 denotes an output of the process circuit 18, that is, an electronic view finder (EVF) for displaying a captured image as an image, and numeral 36 denotes a timing control circuit for controlling the overall operation timing.
[0015]
FIG. 2 shows a template of a matching operation in the matching circuit 30 and a vector direction pattern. In FIG. 2, the template shown in the center is a template for a matching operation for detecting a motion vector, and is composed of 48 (6 in length and 8 in width). If one template is 64 × 64 pixels, matching is performed for an area of 512 × 384 pixels.
[0016]
Also, in FIG. 2, the arrow indicates the direction of the motion vector. FIG. 2 illustrates motion vectors in eight directions. In each direction, the cross-hatched portion is a region for detecting a joint, and this region is hereinafter referred to as a vector direction pattern. For example, when the motion vector is determined to be rightward, the right end portion is a vector direction pattern, and when the motion vector is determined to be lower right, the right end and lower end portions are vector direction patterns.
[0017]
Hereinafter, the operation will be described, but it is assumed that the camera shown in FIG. 1 is panned in a U shape as shown in FIG. 4, 5, and 6 show an operation flowchart as a whole. FIG. 7 shows a state transition of the memories 20A, 20B, and 20C. In FIG. 7, a blank indicates that valid image data is not stored, a downward-sloping diagonal line indicates that valid image data is stored, and a cross-diagonal line indicates a result of the motion vector calculation. The symbol position of the obtained pattern in the vector direction on the memory is symbolically shown. S1 to S14 on the left side of FIG. 7 correspond to the step numbers in FIGS. 4, 5, and 6.
[0018]
The image sensor 12 converts an optical image from the photographing lens 10 into an electric signal. The output of the image sensor 12 is sampled and held and A / D-converted by an S / H circuit 14 and an A / D converter 16 and input to a process circuit 18. The process circuit 18 processes the output of the A / D converter 16 as a camera signal, and outputs the output to the memory control circuit 22 as a luminance signal and a color difference signal.
[0019]
The memory control circuit 22 freezes the image data of the first field output from the process circuit 18 to the memory 20A and outputs the image data to the recording medium 24 (S1, S2). The recording medium 24 records the image data from the memory control circuit 22.
[0020]
The memory control circuit 22 freezes the image data of the second field output from the process circuit 18 to the memory 20B (S3). At this time, at the same time, the memory control circuit 22 reads out the image data (the image data frozen in the memory 20B) from the process circuit 18 and the image data frozen in the memory 20A to the motion detection circuit 26, and displays the screen between the fields. The whole motion vector is calculated (S4). Here, since panning is performed in a U-shape as shown in FIG. 3, the cross-hatched portion at the right end of the screen 1-1 (memory 20A) in FIG. 3 is a pattern detection area (S5).
[0021]
The operation of S1 to S5 will be described with reference to FIG. 7. In S1, only the memory 20A has image data, and the memories 20B and 20C are empty. In S3, the next field is frozen in the memory 20B. In S4, the images stored in the memories 20A and 20B are compared, and as a result, it is found that the images are moving rightward. Thus, in S5, the same pattern (crossed oblique lines) is detected at the right end of the image frozen in the memory 20A and slightly to the left of the center of the image on the memory 20B.
[0022]
Next, on the memory 20B, the data of the area (crossed oblique line) of the same pattern as the right end pattern of the memory 20A and the data of the area (rightward oblique line) to the right (pan direction) therefrom are transferred to the left end of the memory 20C (S6). ).
[0023]
The third field is frozen in the memory 20A (S7), and the same processing is performed between the memories 20B and 20A. That is, for the image frozen in the memory 20B, a motion vector of the image frozen in the memory 20A is calculated (S8), and a pattern serving as a joint of the images frozen in the memories 20A and 20B is detected (S9). ), Transfer the seam (crossed oblique lines) and the new image portion from the memory 20A to the memory 20C at a position where the existing image is connected (S10).
[0024]
That is, as shown in FIG. 7, in S9, on the memory 20A, the data of the area (crossed oblique line) of the same pattern as the right end image pattern of the memory 20B and the data of the area on the right side of the same are copied to the existing The data is transferred to a blank part (S10). If the blank portion of the memory 20C is narrow, data enough to fill the blank portion of the memory 20C is transferred from the image memory 20A to the memory 20C. At this time, the location on the memory 20A that has been transferred to the memory 20C is stored.
[0025]
It is checked whether or not the memory 20C is full (S11). If the memory 20C is full, it is swept out and recorded on the recording medium 24 (S12). If the splicing switch (not shown) is off, the photographing is ended (S13) and turned on. In the case of (1), the data remaining in the memory 20A is transferred to the memory 20C (S14). Up to this point, images for two screens have been recorded on the recording medium 24.
[0026]
When the memory 20C is not full (S11), or after transferring the data of the memory 20A to the memory 20C (S14), the next field is frozen in the memory 20B (S15), and the operation is performed by a matching operation between the memories 20A and 20B. A vector is detected (S16). Similarly to the above, a pattern in the vector direction on the memory 20A is detected from the memories 20A and 20B (S17), and predetermined data is transferred from the memory 20B to the memory 20C (S18).
[0027]
It is checked again whether the memory 20C is full (S19). If it is full, the memory 20C is swept out and recorded on the recording medium 24 (S20). If the splicing switch (not shown) is off, the photographing is ended (S21). , The data remaining in the memory 20A is transferred to the memory 20C (S22). If the memory 20C is not full (S19), the process returns to S7 and repeats the above operation.
[0028]
FIG. 8 shows a timing chart of the above operation. The same step numbers are assigned to the timings at which the steps in FIGS. 4, 5 and 6 are executed. The V timing signal is a timing signal synchronized with the vertical synchronization signal of the field signal.
[0029]
In the first field, a photoelectric conversion signal is read from the image sensor 12, frozen in the memory 20A (S1), and simultaneously recorded on the recording medium 24 (S2).
[0030]
Also in the next field, reading from the image sensor 12 is performed, and this time, the image is frozen in the memory 20B (S3). At the same time, image data (especially luminance data) stored in the memory 20A is read (S4), and a motion vector is calculated by a matching operation (S4). Before the next reading from the image sensor 12, the image data in the detected movement direction is frozen in the memory 20C (S6).
[0031]
The signal read from the image sensor 12 in the third field is frozen in the memory 20A (S7). At the same time, image data (especially luminance data) stored in the memory 20B is read (S8), and a motion vector is calculated by a matching operation (S8). Before the next reading from the image sensor 12, the image data in the detected movement direction is frozen in the memory 20C (S10). When the memory 20C is full, the image data in the memory 20C is recorded on the recording medium 24 (S12), and the remaining data in the memory 20A is transferred to the memory 20C (S14).
[0032]
In the fourth field, the signal read from the image sensor 12 is frozen in the memory 20B (S15). At the same time, the image data stored in the memory 20A is read (S16), and a motion vector is calculated by a matching operation (S16). Before the next readout from the image sensor 12, the image data in the detected movement direction is frozen in the memory 20C (S18).
[0033]
In the fifth field, the process returns to S7 and repeats the previous operation.
[0034]
In this manner, by repeating the operation of freezing the new image in the movement direction to the memory 20C and recording the image on the recording medium 24 when the memory 20C becomes full, the recording medium 24 can accurately accumulate the image in the pan direction. Can be recorded as a series of still images. Although the rightward pan has been mainly described, the same applies to the leftward and vertical directions.
[0035]
As described above, it is assumed that 12 images have been recorded on the recording medium 24 by the right, bottom, and left pans as shown in FIG. At this time, twelve image position addresses 1-1 to 1-6 and 2-1 to 2-6 are completed. This image position address is recorded at the head of each image data after the splicing and shooting is completed.
[0036]
Next, the operation when the panning operation is slow will be described. FIG. 9 shows transition states of the memories 20A, 20B, and 20C, and FIG. 10 shows a timing chart thereof.
[0037]
First, the first field image is frozen in the memory 20A (S1), and the second field image is frozen in the memory 20B (S3). Motion is detected from both images, and a new image portion in the motion direction is transferred from the memory 20B to the memory 20C based on the detection result (S6). The next field image is frozen in the memory 20A (S7), motion is detected between the image in the memory 20B, and a new image portion in the motion direction is transferred from the memory 20A to the memory 20C based on the detection result (S10). .
[0038]
Since the panning is slow, there is still room in the memory 20C, and the next field is frozen in the memory 20B (S15), motion is detected between the image in the memory 20A and a new image in the motion direction based on the detection result. The part is transferred from the memory 20B to the memory 20C (S18).
[0039]
Also at this stage, since there is room in the memory 20C, S7 and subsequent steps are repeated. When the memory 20C is full, the data is recorded on the recording medium 24.
[0040]
In the panning in the horizontal direction, the camera should be rotated in the horizontal direction in the horizontal plane. However, in the case of hand-held shooting, the camera pans obliquely with respect to the horizontal plane. Therefore, an operation in the case of panning in an oblique direction will be described next.
[0041]
FIG. 11 shows a motion vector and a recorded image when panning in the upper right direction. FIG. 11A shows a motion vector indicating the direction and amount of movement of each field detected by the motion detection circuit 26, and FIG. 11B shows an image of each field by diagonal right panning. In FIG. 11B, the cross-hatched portion is a pattern in the motion direction corresponding to the detected motion vector.
[0042]
FIG. 11C is an image obtained by connecting the images of the respective fields, assuming that the image is panned right next to the field at the start of shooting. A hatched portion is an image portion that cannot be obtained by panning diagonally right upward, and a blank portion is an image portion that is obtained by the image sensor 12.
[0043]
FIG. 11D shows an image synthesized on the memory 20C and recorded on the recording medium 24. The range of the arrow pointing left and right is a range recorded as one screen. The cross hatched portion is a joint with the next screen, and the hatched portion is a portion where image data is missing. In this way, four screen images are sequentially recorded on the recording medium 24.
[0044]
At this time, when the user operates to end the splicing mode, the electronic view finder 34 displays as shown in FIG. This allows the user to check the image just recorded. Also in this case, if the hatched portion is a missing portion of the image data and the user does not like it, the data recorded on the recording medium 24 may be invalidated or deleted later, and the panning may be performed again. It is preferable that motion vectors (FIG. 11A) to be sequentially detected are displayed on the electronic view finder 34 so that panning can be performed accurately, for example, in the same plane.
[0045]
FIG. 11E shows an image synthesized on the memory 20 </ b> C by another connection method and recorded on the recording medium 24. Here, at the time of synthesis on the memory 20C, seams are extracted along the pattern in the direction of movement so as to minimize data loss, and an image of each screen is formed according to the seams thus extracted. . At this time, when recording the image position address after recording the image data, it is necessary to also record the motion vector data. Thereby, at the time of reproduction, each image can be connected naturally even in the oblique direction.
[0046]
In the above embodiment, when a missing portion of the image data occurs, the panning must be restarted from the beginning, but it is convenient if the missing data can be filled or supplemented by supplementary panning. That is, as shown in FIG. 13, the splicing information is displayed on the electronic viewfinder 34, and the image in the area indicated by reference numeral 40 is picked up by the image sensor 12 so as to include the data missing part of the existing recorded image. Shoot. Then, the joint of the data missing portion is searched for by the matching operation, and the missing portion is supplemented.
[0047]
When the seam is found and the filling of the holes is completed in this way, the through screen is displayed intermittently. In response to this, the user pans the camera in a direction in which the next data missing portion can be refilled. In this way, data missing portions are sequentially supplemented.
[0048]
In the above embodiment, when the images captured by panning are connected, a state in which the images are partially missing is displayed on the electronic view finder, so that the synthesis result can be predicted, and the creation of an ultra-wide image is easy. become. Further, since the connection state is displayed, the learning effect of the user can be expected.
[0049]
If it is expected that smooth connection is difficult due to the influence of the imaging lens, the above-described connection processing may be performed before correcting the influence of such aberration. When a moving subject arrives at the joint, the data may be replaced in the memory and fixed as data in either field.
[0050]
Next, a description will be given of a reproducing apparatus that reproduces an image recorded on the recording medium 24 by combining in this manner. FIG. 14 shows a schematic block diagram of one embodiment. Reference numeral 50 denotes a recording medium on which a plurality of super-wide still images are recorded, such as the recording medium 24; 52, a data control circuit that controls reading of the recording medium 50 and writing and reading of the image memory 54; A process circuit for processing image data from the data control circuit 52 for monitor output; 58, a display control circuit for controlling the monitor 60; 62, a system control circuit for controlling the whole; and 64, a predetermined instruction to the system control circuit 62 This is an operating device for inputting.
[0051]
FIG. 15 shows an operation panel surface of the operation device 64. Reference numeral 66 denotes a MODE switch for instructing switching of a display mode, 68, 70, 72, and 74 denote arrow keys for instructing scrolling and screen switching, and 76 denotes a Disp switch for instructing switching of display settings.
[0052]
The playback operation of the playback device shown in FIG. 14 will be described with reference to the flowcharts shown in FIGS.
[0053]
First, the data control circuit 52 reads the image position address data recorded at the head of the recorded image data from the recording medium 50 (S31). As a result, it is possible to know how many still images constitute one wide image. To display the entire wide image, it is necessary to thin out appropriate pixels, and the system control circuit 62 sets a filter for the thinning in the data control circuit 52.
[0054]
Next, the data control circuit 52 sequentially reads out the image data from the recording medium 50 and stores the image data thinned out by the above-described filter in the image memory 54 (S32). The data control circuit 52 reads out the image data stored in the image memory 54 and applies it to the monitor 60 via the process circuit 56 and the display control circuit 58 (S33). That is, the process circuit 56 performs predetermined signal processing on the entire image stored in the image memory 54, and the display control circuit 58 converts the output of the process circuit 56 into a signal suitable for displaying an image on the monitor 60. , To the monitor 60. This is the whole display mode, and the display state is as shown in FIG. FIG. 19 illustrates a wide image including a total of eight still images.
[0055]
The display mode can be changed by operating the MODE switch 66 (S35 to S39). That is, when the MODE switch 66 is turned on, the above-described whole display mode, the both display mode in which a part of the super wide image is displayed in a large size and the position in the super wide image is displayed, and only the partial image is displayed. The partial display mode cycles. The dual display mode includes a first dual display mode Disp # 1 for scrolling and displaying a reproduced image within a screen of a large partial image, and a second dual display mode Disp # 2 for switching and displaying a reproduced image. There is. The display mode of the first both display mode Disp # 1 is shown in FIG. 20, and the display mode of the second both display mode Disp # 2 is shown in FIG. The position of the partial display is indicated by a color or a unique pattern (hatched in FIGS. 20 and 21) in the entire display.
[0056]
When the MODE switch 66 is first turned on (S34), the display mode is set to the both-display mode Disp # 1 shown in FIG. 20 (S35 and S36). In the mode Disp # 1 of the both display modes, the reproduced image is scroll-displayed according to the arrow keys 68, 70, 72, 74 (S43, 45). That is, the system control circuit 62 instructs the data control circuit 52 to issue a movement command in accordance with the operation time and / or the pressing strength of the arrow keys 68 to 74, and the data control circuit 52 responds to the movement command by using the image memory. The image data stored in the image memory 54 is moved in the designated direction on the image memory 54, and necessary data of the insufficient image data is read from the recording medium 50 and stored in the image memory 54.
[0057]
When the Disp switch 74 is operated in the both display mode (S46), the mode is switched from the mode Disp # 1 to the mode Disp # 2 or from the mode Disp # 2 to the mode Disp # 1 (S47).
[0058]
In the mode Disp # 2, the reproduced images are switched and displayed according to the arrow keys 68, 70, 72, 74 (S43, 44). That is, when the arrow key 74 is turned on while the image 1-2 is being reproduced and displayed, the data control circuit 52 reads the data of the image 1-3 from the recording medium 50 and stores it in the image memory 54, and It is displayed at a predetermined location.
[0059]
When the MODE switch 66 is operated in the both display mode (S37), if the mode is Disp # 1 in the both display mode, the entire display mode is set (S38). If the mode is Disp # 2, the partial display mode is set (S38, 39).
[0060]
In the partial display mode, only the partial image of the super wide image is displayed. FIG. 22 shows a display mode on the monitor 60. In the partial display mode, as shown in FIG. 18, the reproduced image is scroll-displayed according to the arrow keys 68, 70, 72, 74 (S48, 49). As shown in FIG. 22, an arrow indicating a scrollable direction is superimposed on the reproduced image. For example, when the image 1-1 is displayed, the arrows displayed on the monitor screen are only rightward and downward.
[0061]
When the MODE switch 66 is operated in the partial display mode (S50), the process exits the partial display mode and returns to the mode Disp # 2 of the both display modes.
[0062]
In the both display mode, as shown in FIGS. 20 and 21, an image indicating the position in the whole is displayed outside the display area of the partial image, but may be superimposed and displayed as a so-called picture-in-picture. . Further, in the above embodiment, the mode is shifted from the mode disp # 2 of the both display mode to the partial display mode. However, the mode is similar to the mode Disp # 1 in that the image is scroll-displayed. The mode may shift to the partial display mode. Further, it goes without saying that a window display system used in a workstation or a personal computer may be utilized so that the size of the display area of the partial image can be enlarged or reduced.
[0063]
It is obvious that the image information may be recorded after being reversibly or irreversibly compressed on the recording medium 24.
[0064]
【The invention's effect】
As can be easily understood from the above description, according to the present invention,The entire wide image can be grasped, and the details of the wide image can be confirmed while grasping which part of the whole wide image is being displayed, which is very convenient for the user.
[Brief description of the drawings]
FIG. 1 is a schematic block diagram of an embodiment of the present invention.
FIG. 2 shows a matching template and a vector direction pattern in the present embodiment.
FIG. 3 is an example of a pan direction.
FIG. 4 is a part of the operation flowchart of FIG. 1;
FIG. 5 is a part of the operation flowchart of FIG. 1;
FIG. 6 is a part of the operation flowchart of FIG. 1;
FIG. 7 is a transition state diagram of memories 20A, 20B, and 20C.
FIG. 8 is a timing chart corresponding to FIG. 7;
FIG. 9 is a transition state diagram of memories 20A, 20B, and 20C for a slow pan.
FIG. 10 is a timing chart for FIG. 9;
FIG. 11 is an explanatory diagram of the operation when panning rightward and obliquely;
FIG. 12 is a display of the electronic view finder 34;
FIG. 13 is a display of the electronic view finder 34 when filling in missing data.
FIG. 14 is a schematic block diagram of an embodiment of a reproducing apparatus.
15 is a diagram showing an operation panel surface of the operation device 64 of FIG.
FIG. 16 is an operation flowchart of FIG.
FIG. 17 is a detailed flowchart of S41 in FIG. 14;
FIG. 18 is a detailed flowchart of S40 in FIG.
FIG. 19 shows a display mode of the monitor screen 60 in the entire display mode.
FIG. 20 shows a display mode of a monitor screen 60 in a mode Disp # 1 of both display modes.
FIG. 21 shows a display mode of a monitor screen 60 in a mode Disp # 2 of both display modes.
FIG. 22 shows a display mode of the monitor screen 60 in the partial display mode.
[Explanation of symbols]
10: Shooting lens
12: Image sensor
14: Sample and hold circuit
16: A / D converter
18: Process circuit
20A, 20B, 20C: Image memory
22: Memory control circuit
24: Recording medium
26: Motion detection circuit
28: Filter
30: Matching circuit
32: Vector operation circuit
34: Electronic view finder
36: Timing control circuit
50: Recording medium
52: Data control circuit
54: Image memory
56: Process circuit
58: Display control circuit
60: Monitor
62: System control circuit
64: Operating device
66: MODE switch
68, 70, 72, 74: Arrow keys
76: Disp switch

Claims (1)

An image reproducing method for reproducing a wide image from a recording medium in which one wide image is divided into a plurality of partial image data and recorded,
Reading the partial image data from the recording medium;
A first mode in which the read-out partial image data is sequentially reduced and combined to display the entire wide image, and the read-out partial image data is displayed on display means, and is displayed on the same screen. A display step of selectively displaying an image by using a second mode for displaying information for recognizing a position of the displayed partial image data in the wide image.
Image reproducing method, characterized by chromatic and.

Images