JP5142811B2 - Imaging apparatus and imaging method in imaging apparatus - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method in the imaging apparatus.

  Conventionally, an imaging device such as a digital camera is known which has a panoramic shooting mode for generating a panoramic image by synthesizing a plurality of captured images, and the subject is photographed for each part while panning the imaging device. By synthesizing the photographed image data, a panoramic image in which the photographed images for each part are connected can be obtained. In digital cameras, the output from the image sensor is displayed on the display unit at a predetermined frame rate and used as an electronic viewfinder. In the panorama shooting mode, the motion between live view images is detected and integrated for each frame. Thus, the amount of movement of the image is obtained, and the joint between the captured images is detected. According to this, it is possible to perform the photographing process at an appropriate timing that does not cause a sense of incongruity at the joint. For example, Patent Literature 1 discloses a technique for obtaining a motion (motion vector) between images accompanying panning and controlling a width when stitched images are joined based on the motion vector.

Japanese Patent No. 3421859

  By the way, since there is no output from the image sensor during the photographing process (exposure, transfer, etc.), the display of the live view image is temporarily stopped, and a blackout state in which nothing is temporarily displayed on the display unit is entered. Here, if the main body of the device is moved during blackout, the movement of the image during this time cannot be detected, so that the joint between the captured images cannot be detected correctly, and the accuracy of calculating the image movement amount decreases. There was a problem that. As a result, there is a case where a gap occurs in the joint at the time of synthesis, and the generated panoramic image is uncomfortable.

  The present invention has been made in view of the above, and an imaging apparatus capable of appropriately calculating the amount of image movement during blackout that occurs during shooting processing and improving the calculation accuracy of the amount of image movement, and the imaging An object is to provide an imaging method in an apparatus.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging unit that captures a subject image and generates image data, and performs a plurality of imaging processes to generate a plurality of captured images. A display unit that continuously displays the image data captured by the imaging unit as a live view image; and a live view period in which a motion amount of the image is detected from a plurality of image data of the live view image. a motion detection unit, and the imaging capturing longitudinal motion detector for detecting the motion amount of the image data or we the image of the image data data and immediately after the completion on the obtained live view image of the live view image obtained immediately before the start of the process , While calculating the live view image movement amount during display of the live view image based on the detection result by the live view period motion detection unit, An image shift amount calculation unit that calculates a shooting image moving amount during the photographing process based on the detection result by the serial shooting longitudinal motion detector, characterized in that it comprises a.

The imaging apparatus according to the present invention is the imaging apparatus according to the above aspect, wherein the imaging time measuring unit that measures the imaging time from the start to the end of the imaging process and the imaging time measured by the imaging time measuring unit are reduced. A reduction rate setting unit for setting a rate, and the pre- and post-shooting motion detection unit includes image data of a live view image obtained immediately before the start of the shooting process and image data of a live view image obtained immediately after the end of the shooting process. Is reduced at a reduction rate set by the reduction rate setting unit.

The imaging apparatus according to the present invention may further include a reliability determination unit that determines the reliability of the detection result by the pre- and post-shooting motion detection unit, and the image movement amount calculation unit may include the reliability determination unit. The amount of movement of the image at the time of shooting is calculated based on the detection result determined to be reliable by.

  Further, in the imaging apparatus according to the present invention, in the above invention, when the reliability determination unit determines that the detection result of the back-and-forth shooting motion detection unit is not reliable, the reduction rate setting unit sets the reduction rate. The setting and motion detection by the pre- and post-shooting motion detection unit are repeated until the reliability determination unit determines that there is reliability, and the reduction rate setting unit performs the post-reduction processing every time the reduction rate is set. The reduction ratio is set so as to reduce the image size.

In the image pickup apparatus according to the present invention, in the above aspect, the image movement amount calculation unit repeats the reduction rate setting by the reduction rate setting unit and the motion detection by the pre- and post-shooting motion detection unit a predetermined number of times. As a result, when the reliability determination unit does not determine that there is reliability, the motion amount of the image detected by the live view period motion detection unit before the shooting process and / or the live view period after the shooting process. The image movement amount during the photographing process is estimated based on the movement amount of the image detected by the motion detection unit.

  In the image pickup apparatus according to the present invention, in the above invention, when the reliability determination unit determines that the detection result of the back-and-forth photographing motion detection unit is reliable, the reduction rate setting unit sets the reduction rate. The setting and motion detection by the pre- and post-shooting motion detection unit are repeated until the reliability determination unit determines that there is no reliability, and the reduction rate setting unit performs the post-reduction processing every time the reduction rate is set. The reduction rate is set so that the image size is increased, and the image movement amount calculation unit repeats the setting of the reduction rate by the reduction rate setting unit and the motion detection by the pre- and post-shooting motion detection unit. When the reliability determination unit determines that there is no reliability, the image movement amount during shooting is calculated based on a previous detection result by the movement detection unit before and after shooting.

  In the image pickup apparatus according to the present invention, in the above invention, the back-and-forth shooting motion detection unit detects a motion amount of an image by performing pattern matching based on a predetermined search range set in advance, and When the sex determination unit determines that the detection result of the back-and-forth shooting motion detection unit is reliable, the reduction is performed based on the previous motion detection result for the predetermined search range by the shooting back-and-forth motion detection unit. An execution determining unit that determines whether or not the reduction rate setting by the rate setting unit and the motion detection by the pre- and post-shooting motion detection unit are performed again, and the reduction rate setting unit is determined to be performed by the execution determination unit. In this case, the reduction ratio is set again based on the previous motion detection result.

  An imaging apparatus according to the present invention is an imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and that performs a plurality of imaging processes to generate a plurality of captured images. A display unit that continuously displays the image data picked up as a live view image, a live view period motion detection unit that detects a motion amount of the image from a plurality of image data between the live view images, and start of the shooting process Obtained immediately before the start of the photographing process, a photographing time measuring unit that measures a photographing time from the end to the end, a search range setting unit that sets a search range according to the photographing time measured by the photographing time measuring unit Pattern matching is performed between the image data and the image data obtained immediately after the completion based on the search range set by the search range setting unit. Calculating a moving amount during live view during the display of the live view image based on a detection result by the motion detection unit before and after shooting for detecting a motion amount of the image from the data and a live view period motion detecting unit, An image movement amount calculation unit that calculates an image movement amount during shooting during the shooting process based on a detection result by the shooting back and forth movement detection unit.

  The imaging apparatus according to the present invention is the imaging apparatus according to the above invention, wherein a shooting start instruction unit that instructs the start of the shooting process based on the image movement amount calculated by the image movement amount calculation unit, and a shooting start instruction unit After the first shooting process instructed to start by, the movement mark that moves according to the image movement amount during live view and the movement target mark that indicates the movement target position of the movement mark are displayed on the live view image. A display control unit that performs control, wherein the display control unit sets an initial display position of the movement mark based on the image movement amount during shooting, and the shooting start instruction unit includes When moving to the movement target position, the start of the next photographing process is instructed.

  An imaging method in an imaging apparatus according to the present invention is an imaging method in an imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of shooting processes to generate a plurality of captured images. A display step of continuously displaying the image data captured by the imaging unit as a live view image, a live view period motion detection step of detecting a motion amount of the image from a plurality of image data of the live view image, Image data obtained immediately before the start of the photographing process and image data obtained immediately after the end of the image data are reduced at a predetermined reduction ratio, and the amount of movement of the image is detected from each image data after the reduction process. During live view during display of the live view image based on the detection result in the detection step and the motion detection step in the live view period Calculates the image movement amount, characterized in that it comprises a, an image movement amount calculating step of calculating a shooting image moving amount during the shooting process on the basis of the detection result in the imaging before and after motion detection step.

An imaging method in an imaging apparatus according to the present invention is an imaging method in an imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of shooting processes to generate a plurality of captured images. A display step of continuously displaying the image data captured by the imaging unit as a live view image, a live view period motion detection step of detecting a motion amount of the image from a plurality of image data of the live view image, a photographing longitudinal motion detection step of detecting the captured image data and the motion amount of the image data or we image obtained immediately after the end of the live view image of the live view image obtained immediately before the start of treatment, the live view period motion Based on the detection result in the detection step, the moving amount of the live view image during the display of the live view image is calculated. Together, characterized in that it comprises a, an image movement amount calculating step of calculating a shooting image moving amount during the shooting process on the basis of the detection result in the imaging before and after motion detection step.

  According to the present invention, during the display of live view images, the amount of image movement can be calculated by detecting the movement between live view images. Also, the live view image immediately before the start of the shooting process and the live view image immediately after the end of the shooting process are reduced at a reduction rate according to the shooting time, and the movement between the images after the reduction process is detected, thereby performing the shooting process. The amount of image movement at can be calculated. According to this, it is possible to appropriately calculate the image movement amount during shooting during the shooting process in which the display of the live view image is temporarily stopped and blackout occurs, and the image movement amount can be calculated with high accuracy. Therefore, even if the apparatus main body is moved during the photographing process, it is possible to correctly detect the joint between the photographed images.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the imaging apparatus of the present invention is applied to a digital camera will be described as an example.

(Embodiment)
FIG. 1 is a rear view of the digital camera 1. As shown in FIG. 1, the digital camera 1 includes a shutter button (release button) 3 for instructing photographing timing provided on the upper surface of the camera body 2, and a power button 4 provided on the back surface of the camera body 2. And menu button 5, cross button 6 having up / down / left / right direction buttons (up button, down button, left button and right button), OK button 7 for confirming operation details, and display unit for displaying various screens 22 etc. Although not shown, a finder, an imaging lens, and the like are disposed on the front surface of the camera body 2. When the user presses the power button 4 to turn on the power, the digital camera 1 is in a state where it can shoot (shooting mode). In this shooting mode, a subject image formed on the image sensor 12 (see FIG. 2) through the imaging lens is captured every frame (for example, 1/30 second) and displayed on the display unit 22 in real time as a live view image. It has come to be. For example, electronic photographing is performed at the timing of pressing the shutter button 3 or the like.

  First, the configuration of the digital camera 1 will be described. FIG. 2 is a schematic block diagram illustrating a configuration example of the digital camera 1. As shown in FIG. 2, the digital camera 1 includes an imaging optical system 11, an imaging device 12 as an imaging unit, an AFE (Analog Front End) 13, a frame memory 14, a motion detection unit 15, an image processing unit 16, and a recording medium I. / F17, recording medium holding unit 18, recording medium 19, video encoder 20, display driver 21, display unit 22, video signal output terminal 23, operation unit 24, RAM 25, ROM 26, controller 27, and the like.

  The imaging optical system 11 includes an imaging lens, a diaphragm, a shutter, and the like, and forms an incident subject image on the imaging element 12. The image pickup device 12 is a solid-state image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), for example, and receives a light beam from a subject via the image pickup optical system 11 and photoelectrically converts it into frame units. Image data (analog electrical signal) is obtained. The AFE 13 performs analog signal processing such as CDS (Correlated Double Sampling) and AGC (Automatic Gain Control) on the image data obtained by the image sensor 12, and then performs A / D conversion processing to convert it into a digital electrical signal. To do. The image data digitized by the AFE 13 is output to the frame memory 14 and the motion detection unit 15 and is temporarily stored in the RAM 25.

  The frame memory 14 is used as a working memory by the motion detection unit 15. The frame memory 14 has an area for storing image data for the past two frames, and stores image data of a live view image captured last time and image data of a live view image captured this time. That is, when a live view image is displayed (hereinafter referred to as “live view”), the image data of the live view image of the immediately preceding frame (immediately preceding frame image) captured immediately before and the live view image of the current frame (currently displayed). Frame image) is stored. In addition, when the shooting process is started, the display of the live view image is temporarily stopped. During this shooting process (hereinafter referred to as “at the time of shooting”), the frame immediately before shooting that was captured at the end before the start of the shooting process. The image data of the live view image (frame image immediately before shooting) is stored, and after the shooting process, when the display of the live view image is restored, the live view image of the current frame is the live view image (frame immediately after shooting). Image).

  The motion detection unit 15 is for detecting a motion between images based on the image data from the AFE 13. FIG. 3 is an explanatory diagram for explaining an outline of motion detection performed by the motion detection unit 15. The motion detection unit 15 performs a reduction process on the frame image immediately after shooting, which is a live view image that is first captured after the shooting process, and the frame image immediately before shooting that is already stored in the frame memory 14, and the frame immediately before shooting after the reduction process. The image is compared with the frame image immediately after shooting after the reduction process. Then, it calculates a motion vector representing the amount of change in the same subject position shown in each image as a detection result before and after shooting, thereby functioning as a motion detection unit before and after shooting. In live view, the motion detection unit 15 compares the live view image (current frame image) of the current frame, which is input from the AFE 13 as needed, with the immediately preceding frame image already stored in the frame memory 14. Then, by calculating a motion vector representing a change amount of the same subject position shown in each image as a detection result between previous frames, it functions as a live view period motion detection unit.

  FIG. 4 is a schematic diagram showing a functional configuration of the motion detection unit 15. As illustrated in FIG. 4, the motion detection unit 15 includes a reduction rate setting unit 151 that sets a reduction rate according to the shooting time measured by the shooting time measurement unit 273 of the controller 27, a frame image immediately before shooting, and a frame immediately after shooting. A reduction processing unit 153 that reduces the image at a reduction rate set by the reduction rate setting unit 151, and a reliability evaluation unit 155 as a reliability determination unit that evaluates the reliability of the detection result between the immediately preceding frames and the detection result before and after the imaging. And the previous frame detection results are integrated to calculate the image movement amount during live view (image movement amount during live view), and the image movement amount during shooting (image movement during shooting) based on the detection results before and after shooting. An image movement amount calculation unit 157 that calculates (amount). Although not shown, the motion detection unit 15 includes a RAM for temporarily storing a detection result between previous frames, a detection result before and after shooting, a reliability evaluation result of each detection result, and the like. .

  As shown in FIG. 2, the image processing unit 16 reads image data once stored in the RAM 25, performs various image processing on the image data, and is suitable for recording, display, panorama synthesis, and the like. To convert the image data into For example, when recording image data of a captured image or a panoramic image, or when displaying recorded image data, image data compression processing or decompression processing based on a JPEG (Joint Photographic Experts Group) method or the like is performed. . In addition, a process of generating a single panoramic image by combining image data of a plurality of continuous captured images shot during the panoramic shooting mode is performed. The image data image-processed by the image processing unit 16 is output to the recording medium I / F 17 and recorded on the recording medium 19 or is output to the video encoder 20 and displayed on the display unit 22.

  The video encoder 20 sends the image data converted for display to the display driver 21. For example, in the shooting mode, an image captured from the image sensor 12 for each frame and subjected to image processing by the image processing unit 16 is switched and displayed on the display unit 22 in units of frames to display a live view image. On the other hand, in the reproduction mode, a captured image or a panoramic image read from the recording medium 19 and subjected to image processing by the image processing unit 16 is displayed on the display unit 22. Further, the video encoder 20 outputs image data for display to an external device connected to the video signal output terminal 23 as necessary. The display unit 22 displays various setting information of the digital camera 1 in addition to the captured image and live view image, and is realized by a display device such as an LCD (Liquid Crystal Display) or an EL display (Electroluminescence Display). Is done.

  The recording medium I / F 17 performs writing of image data converted for recording, reading of recorded image data, and the like with respect to the recording medium 19 that is detachably held by the recording medium holding unit 18. . The recording medium 19 is a memory card such as an xD-picture card (registered trademark) or a compact flash (registered trademark) card.

  The operation unit 24 accepts various operations by the user, such as a shooting timing instruction, a mode setting operation such as various shooting modes including a panoramic shooting mode and a playback mode, and a shooting condition setting operation, and notifies the controller 27 of an operation signal. This is realized by a button switch to which various functions are assigned. The operation unit 24 includes the shutter button 3, the power button 4, the menu button 5, the cross button 6, and the OK button 7 in FIG. 1.

  The ROM 26 stores in advance various camera programs for operating the digital camera 1 and realizing various functions of the digital camera 1, data used during the execution of the camera program, and the like. The RAM 25 is used as a working memory for the image processing unit 16 and the controller 27. For example, image data or the like from the AFE 13 is temporarily stored, for work when generating live view image data to be displayed on the display unit 22, for work when generating a panoramic image, a captured image, or a panoramic image. Is used for work when recording the image on the recording medium 19.

  The controller 27 reads out and executes a camera program from the ROM 26 in accordance with an operation signal from the operation unit 24 and performs operation control and memory control of each unit constituting the digital camera 1 to control the entire operation of the digital camera 1. To control. In addition, processing such as AF (automatic focus), AE (automatic exposure), and AWB (automatic white balance) is performed. The controller 27 counts (measures) the shooting start instruction unit 271 that instructs the start timing of the shooting process in the panoramic shooting mode, and the shooting time from the start to the end of the shooting process started by the instruction of the shooting start instruction unit 271. And a photographing time measuring unit 273 that outputs to the motion detecting unit 15.

  Next, a panoramic shooting mode that is one of the shooting modes of the digital camera 1 configured as described above will be described. FIG. 5 is a diagram for explaining the outline of the panoramic shooting mode. In the panorama shooting mode, a plurality of shooting processes are performed to generate image data of a plurality of images (hereinafter, a shot image shot for a panorama image is referred to as a “partial image”). For example, the user presses the shutter button 3 to generate the first partial image. Thereafter, the camera body 2 is panned in a predetermined panoramic shooting direction, and when the second partial image shooting position is reached, the second partial image is shot. Thereafter, the photographing of an arbitrary number of partial images is repeated to photograph the subject for each portion. The panning operation refers to an operation of moving the camera body 2 while rotating it along the panoramic shooting direction. FIG. 5 shows a case where the user pans the camera body 2 in the horizontal direction from the left side to the right side after the shutter button 3 is pressed and, for example, three partial images I1, I3, and I5 are photographed. Here, the photographing process of each of the partial images I1, I3, and I5 is performed at the timing when the shutter button 3 is pressed for the first partial image I1. The second and subsequent partial images I3 and I5 are automatically performed during the panning operation of the camera body 2 by the user. Then, the partial images I1, I3, and I5 generated for each part are connected and combined so that the positional relationships between the adjacent partial images I1, I3, and I5 are matched, and one panoramic image I7 is combined. Is generated. Specifically, an overlapping portion reflected in the rightmost region A1 of the partial image I1 and the leftmost region A3 of the partial image I3 is searched, and the partial image I1 and the partial image I3 are searched based on the searched overlapping portion. Is synthesized. Similarly, the partial image I3 and the partial image I5 are synthesized based on the overlapping portion shown in the area A5 on the right side of the partial image I3 and the area A7 on the left side of the partial image I5.

  As described above, in order to generate a panoramic image, the predetermined range of each partial image needs to overlap. On the other hand, if the overlapping portion deviates from the predetermined range, it takes time to search for a synthesis location. The synthesis processing time increases. Further, if the overlapping portion is significantly deviated from the predetermined range, the synthesized portion cannot be detected, and as a result of the synthesis, a panoramic image having a sense of incongruity is formed at the joint. For this reason, it is necessary to perform the imaging processing of the second and subsequent partial images at a timing such that the overlapping portion falls within an appropriate range. In the present embodiment, after the first image capturing process, a pointer as a movement mark and a target as a movement target mark indicating the movement target position of the pointer are displayed on the live view image, and the pointer is moved over the target. Encourage panning operation to move to.

  6 and 7 are diagrams for explaining the operation of the digital camera 1 when performing panoramic shooting, and show a transition example of the display screen displayed on the display unit 22 in the panoramic shooting mode. In the panoramic shooting mode, first, display of a live view image is started (FIG. 6A). And it will be in a standby state until the shutter button 3 is pressed down. The user presses the shutter button 3 while viewing the live view image, and instructs the photographing timing of the first partial image. When the photographing timing is instructed, the photographing process is started and a first partial image is generated. Here, when the shooting process is started, the display of the live view image is temporarily stopped and the process proceeds to exposure. That is, during exposure, transfer, and the like, there is no output from the image sensor 12, so a live view image cannot be captured, and the display on the display unit 22 is in a blackout state where nothing is displayed (FIG. 6B). When the exposure is completed, the display of the live view image is restored through image data transfer processing and image processing. In addition, a pointer PM and a target TM are displayed on the live view image that has been restored after the first partial image is captured (FIG. 6C), and a target for capturing the second partial image. Encourage panning operation to TM. Here, FIG. 6C shows an operation example in which the pointer PM is arranged on the right side and the target TM is arranged on the left side, and the panoramic shooting direction is “right”. The panorama shooting direction is determined, for example, when the user selects either “right” or “left”. When the panorama shooting direction is “left”, the pointer is placed on the left side of the screen and the target is placed on the right side of the screen as the initial display position. When the user performs a panning operation in the horizontal direction with the camera body 2 facing right, the pointer PM moves along with the live view image along with the live view image according to the image movement amount during live view calculated as a result of the motion detection processing performed by the motion detection unit 15. It moves to the left side where it is arranged (FIG. 6 (d)).

  As shown in FIG. 7, when the pointer PM moves into the frame of the target TM (FIG. 7A), the photographing process is automatically started. In this shooting process, as in the shooting process of the first partial image, the display of the live view image is temporarily stopped and the display on the display unit 22 is in a blackout state in which nothing is displayed (FIG. 7B). )). Then, the exposure is finished, and the display of the live view image is restored after the image data transfer process and the image process. In addition, the pointer PM and the target TM are displayed on the live view image that has been restored after the second partial image has been shot, in the same manner as after the first shot, and the third partial image is shot. However, after the second and subsequent shooting processes, the joints of the partial images are detected from the amount of image movement during shooting calculated as a result of the movement detection process performed by the movement detection unit 15. The next shooting position is determined based on the joint, and the initial display position of the pointer PM is adjusted. For example, if the camera body 2 is hardly moved during the photographing process, the pointer PM is arranged with the predetermined position at the right end of the live view image as the initial display position (FIG. 7C). When the camera body 2 is moved rightward during the partial image shooting process, the pointer PM is arranged with the target TM side as the initial display position from the above-mentioned predetermined position (FIG. 7D). The extent to which the target TM is approached is determined based on the amount of image movement during shooting.

  Here, the principle of the motion detection process and the pre- and post-shooting motion detection process performed by the motion detector 15 will be described. As described above, the motion detection unit 15 detects a motion between motion detection target images. That is, during live view, a motion detection process is performed to detect a motion between images (between the previous frames) in which the previous frame and the current frame are motion detection targets. On the other hand, during shooting processing that cannot detect the motion of the live view image in a blackout state, motion detection processing is performed before and after shooting, and the frame image immediately before shooting that has been reduced and the frame image immediately after shooting that has been reduced are set as motion detection targets. The movement between each image (before and after photographing) is detected.

  8A and 8B are diagrams for explaining the motion detection between the immediately preceding frames. FIG. 8A illustrates an example of the immediately preceding frame image, and FIG. 8B illustrates the current frame image. An example is shown. First, as illustrated in FIG. 8A, the motion detection unit 15 sets a plurality of representative points B in the immediately preceding frame detection area Eb in the immediately preceding frame image. The position where the representative point B is set, its size, number, etc. are set in advance, for example. In addition, a search range for each representative point set in the immediately preceding frame image is set in the current frame image. The size of the search range to be set is set in advance. Then, pattern matching is performed between each representative point of the immediately preceding frame and the corresponding search range, and a position having the highest correlation with the representative point is searched from each search range in the current frame image. For example, when focusing on the representative point B1 set in the immediately preceding frame image in FIG. 8A, the most correlated point of the representative point B1 from the search range Es in the current frame image corresponding to the position of the representative point B1. A high position P1 is searched. Based on the motion vector of each representative point obtained by this pattern matching, a representative motion vector indicating the motion of the entire screen is obtained. In the motion detection process, the image movement amount during live view is calculated by integrating the detection results between the immediately preceding frames detected at any time during live view.

  On the other hand, in the motion detection process before and after shooting, motion detection is performed using the pattern matching method described with reference to FIGS. 8A and 8B. Since the frame image immediately after shooting is set as a motion detection target, when the movement between the images is large, such as when the camera body 2 is moved greatly during the shooting process, the representative point set in the frame image immediately before shooting is set as a representative point. There may occur a situation in which the corresponding motion vector deviates from the search range in the frame image immediately after shooting. FIG. 9A is a diagram illustrating an example of a frame image immediately before shooting, and FIG. 9B is a diagram illustrating an example of a frame image immediately after shooting. For example, focusing on the representative point B11 set in the frame image immediately before shooting in FIG. 9A, the motion vector P11 corresponding to the representative point B11 in the frame image immediately before shooting is the frame image immediately after shooting in FIG. It has deviated from the middle search range Es. In this case, since the matching about the representative point B11 fails, a correct motion detection result cannot be obtained.

  For this reason, in motion detection before and after shooting, motion detection is performed after the frame image immediately before shooting and the frame image immediately after shooting are reduced at a predetermined reduction rate. 10A is a diagram illustrating an example of a frame image immediately before photographing after the reduction processing, and FIG. 10B is a diagram illustrating an example of a frame image immediately after photographing after the reduction processing. In this case, as shown in FIG. 10A, the motion detection unit 15 sets the number of representative points corresponding to the reduction ratio and the search range to positions corresponding to the reduction ratio in the frame image immediately before shooting after the reduction processing. Set. The position, number, etc. for setting the search range are set in advance for each reduction ratio. Then, pattern matching is performed between the representative point immediately before shooting after the reduction process and the search range in the frame image immediately after shooting after the reduction process. For example, one representative point Bc1 set in the frame image immediately before photographing after the reduction processing in FIG. 10-1 is illustrated, and focusing on this representative point Bc1, the photographing after the reduction processing shown in FIG. Immediately after that, the motion vector Pc1 is detected from the corresponding search range Es in the frame image. If the frame image immediately before shooting and the frame image immediately after shooting are reduced as described above, the motion vector detection probability can be improved without changing the size of the search range, and the processing load and processing time increase. do not do. More specifically, the longer the shooting time, the more likely the image movement before and after shooting is high, so by setting the reduction ratio according to the shooting time, the motion vector corresponding to the representative point is within the search range. To fit.

Next, the operation of the digital camera 1 will be described. FIG. 11 is a state transition diagram of the digital camera 1. As shown in FIG. 11, the digital camera 1, when the power supply button 4 is pressed is ON (a1), a transition from the camera power OFF state C ₁ to the still image shooting standby state C _3, and shooting mode Become. In the shooting mode, the image data of the subject image formed on the image sensor 12 is stored in the RAM 25, a series of processes for switching the image data in units of frames and displaying them on the display unit 22 are repeated, and the live view image is displayed. Display continuously. Further, when the power supply button 4 is pressed it is OFF (a3), the digital camera 1 makes a transition to the camera power OFF state C _1.

When the panoramic mode is selected (a5), the digital camera 1 makes a transition from the still image shooting standby state C ₃ to photographing state C _5. Here, when the first partial image is shot, the shooting process is started when the shutter button 3 is pressed. When completing the photographing (a7), the digital camera 1 makes a transition from the photographing state C ₅ to photographing direction detection state C _7. Here, the panning operation of the camera body 2 is detected by detecting the motion of the live view image, and the panorama shooting direction is determined. When the panoramic photographing direction is determined (a9), the digital camera 1 makes a transition from the photographing direction detection state C ₇ to live view moving amount detecting state C _9. Here, first, an initial display position of the pointer is set based on the panoramic shooting direction, and the pointer and the target are displayed on the live view image. Then, the movement amount during live view is calculated by integrating the motion detection between the immediately preceding frames as needed, and the pointer is moved. Pointer has completed the movement of the pointer overlaps the target (a11), the digital camera 1, the transition to start the shooting process from live view during the movement amount detecting state C ₉ to photographing state C _5. If you have completed the second and subsequent shot (a13), the digital camera 1 makes a transition from the photographing state C ₅ to shooting moving amount detecting state C _11. Here, the movement of the image at the time of photographing is calculated by performing motion detection before and after the photographing. Then, an initial display position of the pointer is set based on the obtained image movement amount during shooting and the panoramic shooting direction, and the pointer and the target are displayed on the live view image. Upon completion of detection (a15), the digital camera 1 transitions from the shooting moving amount detecting state C ₁₁ to live view moving amount detecting state C _9, moves to the imaging of the next partial image. When a predetermined number of partial images have been shot in this way, the generated partial images are combined to generate a panoramic image, and panoramic shooting is completed (a17). At this time, the digital camera 1 transitions from the shooting state C ₅ to the still image shooting standby state C ₃ .

  FIG. 12 is a flowchart showing the operation of the digital camera 1 in the shooting mode. In the shooting mode, first, the controller 27 determines the type of shooting mode. When a shooting mode other than the panoramic shooting mode such as the normal shooting mode is set (step b1: No), the process proceeds to a shooting operation according to the shooting mode (step b3), and the process is ended after the end.

  On the other hand, if the panoramic shooting mode is set (step b1: Yes), first, the first partial image is shot. That is, the shooting start instruction unit 271 instructs the start of shooting processing with the timing when the shutter button 3 is pressed as the shooting timing. As a result, the photographing process is started, and the operation of each part is controlled to photograph the still image in the photographing range at this time as the first partial image (step b5). When this shooting process is started, the display of the live view image is temporarily stopped. The display of the live view image is restored after the image data transfer processing and image processing are completed after exposure.

  Subsequently, the controller 27 performs control to display a shooting direction guidance OSD (On-Screen Display) that prompts an instruction of the panoramic shooting direction on the live view image, and moves to the panoramic shooting direction of the camera body 2 by the shooting direction guidance OSD. The panning operation is urged, and for example, an instruction of either “left” or “right” panorama shooting direction is accepted (step b7). And the process of step b9-step b13 is repeatedly performed for every flame | frame, and the panning operation of the camera main body 2 is detected. That is, the motion detection unit 15 detects a motion between the current frame image and the previous frame image, and integrates the detection results between the previous frames to calculate an integrated movement amount (step b9). Then, the controller 27 performs a shooting direction determination process based on the integrated movement amount (step b11). At this time, if the user rotates the camera body 2 toward the left side in the horizontal direction according to the display of the shooting direction guidance OSD, the panoramic shooting direction is determined as the left direction. Similarly, when the user rotates the camera body 2 toward the right side in the horizontal direction, the panorama shooting direction is determined as the right direction. When the camera body 2 is not moved and the panoramic shooting direction is not determined (step b13: No), the process returns to step b9.

  If the panoramic shooting direction is determined (step b13: Yes), the controller 27 then performs control to display the pointer and target on the live view image as a display control unit (step b15). Specifically, the controller 27 sets an initial display position of the pointer based on the integrated movement amount calculated in step b9, and arranges the pointer and the target on the live view image. At this time, the positional relationship between the pointer and the target is determined according to the panoramic shooting direction.

  Thereafter, the processing from step b17 to step b21 is repeated for each frame until the pointer moves to the target position. That is, first, the motion detection unit 15 performs a motion detection process to calculate a live view image movement amount (step b17). Subsequently, the controller 27 moves the display position of the pointer based on the live view image movement amount calculated by the motion detection process, and updates the display (step b19). Then, the controller 27 determines whether or not the display position of the pointer is within the target frame. If it is not within the target frame (step b21: No), the controller 27 returns to step b17.

  If the pointer has moved into the target frame (step b23: Yes), the current frame image is set to the frame image immediately before shooting, and the image data is stored in the frame memory 14 (step b23). Subsequently, the imaging start instruction unit 271 instructs to start the imaging process. At this time, the imaging time measuring unit 273 starts measuring the imaging time (step b25). Then, the photographing process is started by an instruction from the photographing start instruction unit 271 and the still image in the photographing range at this time is photographed as a partial image (step b27). When the shooting process is completed, the shooting time measuring unit 273 ends the measurement of the shooting time (step b29).

  Subsequently, the controller 27 determines whether or not a predetermined number of images have been taken. For example, if a panoramic image is generated by shooting three partial images, it is determined whether or not three partial images have been shot. The number of partial images may be fixed, or may be configured so that the user can set as appropriate. If the predetermined number of shots has not been completed (step b31: No), the motion detector 15 performs a pre- and post-shooting motion detection process to calculate an image movement amount during shooting (step b33). Subsequently, the controller 27 sets an initial display position of the pointer as a display control unit based on the moving amount of the image at the time of shooting calculated in the movement detection process before and after shooting, and displays the pointer and the target on the live view image. Control is performed (step b35). At this time, the positional relationship between the pointer and the target is determined according to the panoramic shooting direction. Thereafter, the process returns to step b17.

  When a predetermined number of images have been shot (step b31: Yes), the image processing unit 16 joins and combines the shot number of partial images according to the panorama shooting direction to generate a panoramic image (step b). b39). And the controller 27 performs control which displays the produced | generated panoramic image on the display part 22 (step b41), and complete | finishes a process. At this time, the generated panoramic image is recorded in the recording medium 19.

  Next, the shooting direction determination process at step b11, the motion detection process at step b17, and the motion detection process before and after shooting at step b33 will be described.

  FIG. 13 is a flowchart illustrating a detailed processing procedure of the shooting direction determination process. In this shooting direction determination process, the panning operation of the camera body 2 is detected by comparing the absolute value of the horizontal component of the integrated movement amount calculated in step b9 in FIG. 12 with a predetermined direction threshold value set in advance. If the absolute value of the integrated movement amount (horizontal component) is equal to or smaller than the direction threshold value (step c1: No), the controller 27 determines that the shooting direction has not been determined (step c3). Then, the process returns to step b11 in FIG. On the other hand, when the absolute value of the integrated movement amount (horizontal component) is larger than the direction threshold (step c1: Yes), the controller 27 determines that the integrated movement amount (horizontal component) is larger than 0 (step c5: Yes). The panorama shooting direction is determined as “right” (step c7). Then, the process returns to step b11 in FIG. On the other hand, when the integrated movement amount (horizontal component) is less than 0 (step c5: No), the controller 27 determines the panoramic shooting direction as “left” (step c9). Then, the process returns to step b11 in FIG.

  FIG. 14 is a flowchart showing a detailed processing procedure of the motion detection process. In this motion detection process, the motion detection unit 15 performs motion detection between the current frame image and the previous frame image (between the previous frames) (step d1), and the detection result between the previous frames is stored in the RAM of the motion detection unit 15. Store temporarily (step d3). Subsequently, the reliability evaluation unit 155 evaluates the reliability of the detection result between the immediately preceding frames (step d5). If the reliability evaluation unit 155 determines that the previous inter-frame detection result is reliable as a result of the evaluation here (step d7: Yes), the image movement amount calculation unit 157 subsequently moves the image during live view. The total value of the previous value of the amount and the current inter-frame detection result is calculated as the live view image movement amount (step d9). Then, the process returns to step b17 in FIG.

  Here, the reliability evaluation performed by the reliability evaluation unit 155 will be described. The reliability evaluation unit 155 calculates the value of the evaluation parameter based on the detection result between the immediately preceding frames and the detection result before and after photographing, and determines whether or not there is reliability. For example, when the number of matching failures with respect to a plurality of representative points arranged is large, or when the direction of the obtained motion vector is not uniform, it is considered that the reliability of the detection result is low. Therefore, the number of successful matching with respect to the number of representative points is calculated as the first evaluation parameter. Further, the motion vectors that have been successfully matched are grouped in each direction to select the maximum value of the number of elements, and the maximum value of the number of elements with respect to the number of successful matching is calculated as a second evaluation parameter. And the reliability evaluation part 155 determines the presence or absence of the reliability of each detection result by whether the obtained evaluation parameter exceeds the predetermined threshold value preset.

  On the other hand, as a result of the evaluation in step d5, when the reliability evaluation unit 155 determines that the detection result between the previous frames is not reliable (step d7: No), the image movement amount calculation unit 157 performs the movement between the previous frames. An amount estimation process is performed to estimate the amount of motion between the current frame image and the previous frame image (step d11). For example, the detection result between immediately preceding frames for the most recent predetermined number of frames is held in the RAM of the motion detection unit 15. Then, based on the predetermined number of detection results between the previous frames, for example, an average value of these is obtained to obtain an estimated motion amount between the current frame image and the previous frame image. Then, the image movement amount calculation unit 157 calculates the total value of the previous value of the image movement amount during live view and the estimated motion amount as the image movement amount during live view (step d13). Then, the process returns to step b17 in FIG.

  FIG. 15 is a flowchart illustrating a detailed processing procedure of the pre-photographing motion detection process. Prior to the start of shooting back-and-forth motion detection processing, the repetition numbers i and j used during the back-and-forth shooting motion detection processing are initialized to “0”. Then, as shown in FIG. 15, in the motion detection process before and after shooting, first, the reduction rate setting unit 151 of the motion detection unit 15 responds to the shooting time measured by the shooting time measuring unit 273 in steps b25 and b29 of FIG. To set the reduction ratio (step e1). For example, the reduction ratio at the same magnification is set to 1, and a value of 1 or less is set so that the image size after the reduction process becomes smaller as the photographing time becomes longer. Subsequently, the reduction processing unit 153 performs reduction processing at the reduction rate set in step e1, and resizes the frame image immediately before shooting and the frame image immediately after shooting to a small size (step e2).

  Subsequently, the motion detection unit 15 sets the number of representative points according to the reduction ratio set in step e1 in the frame image immediately before shooting after the reduction process (step e3), and the reduction corresponding to the representative point. Motion detection before and after shooting is performed for the search range in the frame image immediately after shooting after processing (step e5). Then, the reliability evaluation unit 155 evaluates the reliability of the detection result before and after photographing (step e6). As a result of the evaluation, when the reliability evaluation unit 155 determines that the detection result before and after photographing is reliable (step e7: Yes), the image movement amount during photographing is calculated based on the detection result before and after photographing. Then, it is updated and stored in the RAM of the motion detector 15 (step e9). Then, the motion detection unit 15 determines the reduction ratio at the time of the reduction process, and if the reduction ratio is not less than 1 (step e11: No), the process returns to step b33 in FIG.

  On the other hand, when the reduction ratio is less than 1 (step e11: Yes), the repetition number i is incremented (step e13). If the incremented number i after the increment is larger than the threshold value Vsi (step e15: Yes), the process returns to step b33 in FIG. When the number of repetitions i is equal to or smaller than a predetermined threshold value Vsi (step e15: No), the image size after the reduction process is incremented every time the number of repetitions i is incremented, based on the previous value of the reduction ratio. Then, the reduction ratio is reset within a range of 1 or less so as to be larger, and the reduction process is performed to resize the frame image immediately before shooting and the frame image immediately after shooting larger than the previous time (step e17). Subsequently, the motion detection unit 15 sets a representative point in the frame image immediately before shooting after the reduction process according to the reduction ratio reset at step e17 (step e19), and immediately after shooting after the reduction process corresponding to the representative point. Motion detection before and after shooting is performed for the search range in the frame image (step e21). Then, the reliability evaluation unit 155 evaluates the reliability of the detection result before and after photographing (step e23). As a result of the evaluation, when the reliability evaluation unit 155 determines that the detection result before and after photographing is reliable (step e25: Yes), the image movement amount during photographing is calculated based on the detection result before and after photographing. Then, it is updated and stored in the RAM of the motion detector 15 (step e27). Then, the process returns to step e11. On the other hand, if it is determined that there is no reliability as a result of the evaluation in step e23 (step e25: No), the process returns to step b33 in FIG. The previous value stored in the RAM is determined. If the reduction ratio value is lowered, the size of the search range with respect to the image size increases, so that the success probability of motion detection is improved. However, on the other hand, there is a problem that the resolution is lowered and the motion vector detection accuracy is lowered, but when the reduction ratio is excessively increased by repeating the processing of step e13 to step e27 in this way. Even if it is, motion detection before and after shooting can be performed by searching for an appropriate reduction ratio that minimizes a decrease in the detection accuracy of the motion vector and reliability is obtained in the detection result before and after shooting. It is possible to achieve a balance between the improvement of the success probability and the reduction of the motion vector detection accuracy.

  As a result of the evaluation in step e6, when it is determined that the detection result before and after photographing is not reliable (step e7: No), the repetition number j is incremented (step e33). If the incremented number j after the increment is larger than the threshold value Vsj (step e35: Yes), the process proceeds to step e49. When the number of repetitions j is less than or equal to a predetermined threshold value Vsj set in advance (step e35: No), the image size after the reduction process is incremented every time the number of repetitions j is incremented with reference to the previous value of the reduction ratio. The reduction ratio is reset within a range that is greater than or equal to a preset minimum reduction ratio and less than or equal to 1 so as to be smaller, and the reduction process is performed to resize the frame image immediately before shooting and the frame image immediately after shooting smaller than the previous time. (Step e37). Subsequently, the motion detection unit 15 sets a representative point in the frame image immediately before shooting after the reduction process according to the reduction ratio reset at step e37 (step e39), and immediately after shooting after the reduction process corresponding to the representative point. Motion detection before and after shooting is performed for the search range in the frame image (step e41). Then, the reliability evaluation unit 155 evaluates the reliability of the detection result before and after photographing (step e43). As a result of the evaluation, when the reliability evaluation unit 155 determines that the detection result before and after photographing is reliable (step e45: Yes), the image movement amount during photographing is calculated based on the detection result before and after photographing. Then, it is updated and stored in the RAM of the motion detector 15 (step e47). Thereafter, the process returns to step b33 in FIG. On the other hand, if it is determined that there is no reliability as a result of the evaluation in step e43 (step e45: No), the process returns to step e33. By repeating the processing from step e33 to step e47 in this manner, the reduction ratio value can be increased in stages, so that even if the set reduction ratio is too low, the detection result before and after photographing is obtained. In addition, it is possible to perform motion detection before and after shooting by searching for an appropriate reduction ratio that minimizes a decrease in motion vector detection accuracy and improves motion detection success probability. Can be balanced with a decrease in detection accuracy.

  If reliability is not obtained in the detection result before and after photographing even if the processing of step e33 to step e47 is repeatedly performed, the image movement amount calculation unit 157 performs a pre- and post-shooting motion amount estimation process to obtain a frame image immediately before photographing. The amount of motion with the frame image immediately after shooting is estimated (step e49). For example, the detection result between the immediately preceding frames for a predetermined number of frames before the photographing process is held in the RAM of the motion detection unit 15. Then, for example, from the predetermined number of immediately preceding inter-frame detection results, the estimated motion amount before and after shooting is calculated according to the shooting time. Note that the estimated motion amount before and after shooting may be calculated from the detection result between the immediately preceding frames for a predetermined number of frames performed after the shooting process, or the immediately preceding frame detection result detected before the shooting process and the shooting process. It is also possible to calculate the estimated motion amount before and after shooting using both of the immediately preceding inter-frame detection results detected after. Then, the image movement amount calculation unit 157 uses the calculated estimated movement amount as the image movement amount during shooting, and updates and stores it in the RAM of the movement detection unit 15 (step e51). Thereafter, the process returns to step b33 in FIG.

  As described above, according to the present embodiment, during live view, the image movement amount during live view can be calculated by detecting a motion between live view images. In addition, the live view image immediately before the start of the shooting process and the live view image immediately after the end of the shooting process are reduced at a reduction rate according to the shooting time, and the movement between the images after the reduction process is detected to detect the image at the time of shooting. The amount of movement can be calculated. According to this, even when the camera body 2 is moved during the photographing process in which the display of the live view image is temporarily stopped and blackout occurs, the image movement amount at the time of photographing can be appropriately calculated. There is an effect that the amount of image movement can be calculated with high accuracy. Therefore, it is possible to correctly detect the joint between the captured images, and it is possible to generate a natural panoramic image without any sense of incongruity by combining the partial images.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the invention.

  For example, in the above-described embodiment, the frame image immediately before shooting and the frame image immediately after shooting are reduced at a reduction rate according to the shooting time, and the movement between the images after shooting is detected by detecting the movement between the images after the reduction processing. Although the case of calculating is described, it is not limited to this. FIG. 16 is a schematic diagram illustrating a functional configuration of the motion detection unit 15a in the present modification. FIG. 17 is a flowchart illustrating a detailed processing procedure of the motion detection process before and after photographing in the present modification. In the following description, the same reference numerals are assigned to the same configurations as those in the above-described embodiment.

  The motion detection unit 15a illustrated in FIG. 16 includes a search range setting unit 154 that sets the size of the search range according to the shooting time measured by the shooting time measurement unit 273 of the controller 27, the detection result between immediately preceding frames, and the detection before and after shooting. The reliability evaluation unit 155 that evaluates the reliability of the result and the image movement amount during live view are calculated based on the detection result between the previous frames, and the image movement amount during shooting is calculated based on the detection result before and after shooting. An image movement amount calculation unit 157.

  In the photographing back-and-forth motion detection process of this modification, as shown in FIG. 17, first, the search range setting unit 154 sets the size of the search range according to the photographing time measured by the photographing time measuring unit 273 ( Step f1). For example, the predetermined maximum size is set as the upper limit so that the size of the search range increases as the shooting time increases. Subsequently, the motion detection unit 15a sets a representative point in the frame image immediately before shooting (step f3). Then, the search range in the corresponding immediately following frame image is set to the size set in step f1, and the motion detection before and after shooting is performed for the search range in the immediately following shooting frame image corresponding to each representative point (step f5). Subsequently, the reliability evaluation unit 155 evaluates the reliability of the detection result before and after photographing (step f7). As a result of the evaluation, when the reliability evaluation unit 155 determines that the detection result before and after photographing is reliable (step f9: Yes), the image movement amount during photographing is calculated based on the detection result before and after photographing. Then, it is updated and stored in the RAM of the motion detector 15a (step f11).

  If it is determined as a result of the evaluation in step f7 that the detection result before and after photographing is not reliable (step f9: No), the size of the current search range is determined. If it is not the maximum size (step f13: No), the search range setting unit 154 resets the search range size with the maximum size as an upper limit so as to be larger than the previous value (step f15), and returns to step f3. On the other hand, in the case of the maximum size (step f13: Yes), the image movement amount calculation unit 157 performs the pre- and post-shooting motion amount estimation processing to estimate the amount of motion between the frame image immediately before shooting and the frame image immediately after shooting. (Step f17). Then, the image movement amount calculation unit 157 uses the calculated estimated movement amount as the image movement amount during shooting, and updates and stores it in the RAM of the movement detection unit 15a (step f19).

  According to this modification, the size of the search range for pattern matching can be set according to the shooting time, and motion detection before and after shooting can be performed. According to this, the success probability of motion detection can be improved without reducing the motion vector detection accuracy. When the shooting time is short, the size of the search range is reduced, so that a reduction in processing efficiency can be suppressed.

  Further, in the above-described embodiment, when it is determined that the detection result before and after photographing is reliable, the reduction ratio is reset so that the size after the reduction processing increases stepwise, and the movement before and after photographing is changed. Although it was decided to search for an appropriate reduction ratio that minimizes a decrease in the detection accuracy of the motion vector by repeated detection, the present invention is not limited to this. FIG. 18A and FIG. 18B are diagrams illustrating a method for setting a reduction ratio in the present modification. For example, as shown in FIG. 18A, when the motion vector Pc11 is detected at a position closer to the center of the search range Es, the motion before and after photographing is detected again by increasing the value of the reduction ratio. On the other hand, as shown in FIG. 18-2, when the motion vector Pc13 is detected near the end of the search range Es, it is expected that the motion vector will not fit within the search range Es if the value of the reduction ratio is increased. Therefore, the process of increasing the reduction ratio and detecting the motion before and after the photographing again is not performed. Specifically, the motion detection unit 15 performs shooting by resetting the reduction rate based on the position of the motion vector in the search range searched when the motion detection before and after the previous shooting was performed as the execution determination unit. It is determined whether to perform forward / backward motion detection again. At this time, it is possible to determine the position of the motion vector for all the search ranges, reset the reduction ratio, and determine whether to perform motion detection before and after shooting again. By selecting and determining the position of the motion vector for the search range, it may be determined whether to reset the reduction ratio and perform motion detection before and after photographing again.

  In the above-described embodiment, the case where the reduction rate is set according to the shooting time has been described. However, the movement amount before and after shooting is estimated from the detection result between the immediately preceding frames detected before the start of the shooting process, and the estimation is performed. The reduction ratio may be set according to the result.

  In the above-described embodiment, the motion detection before and after the shooting is performed after the shooting process of the second and subsequent partial images that are shot while performing the panning operation. In this case, the motion detection before and after photographing may be performed in the same manner as after the photographing processing for the second and subsequent images.

  In the embodiment described above, the case where either “right” or “left” is set as the panorama shooting direction has been described, but directions other than the left and right can also be set as the panorama shooting direction. For example, when performing panoramic shooting with “upper” or “lower” as the panoramic shooting direction, the absolute value of the vertical component of the integrated movement amount is set in advance in the shooting direction determination process shown in FIG. It is configured to compare with the direction threshold value, and the positive / negative of the vertical component of the integrated movement amount is determined, and if it is a positive value, the panoramic shooting direction is determined as “up” and if it is a negative value, “down”.

It is a rear view of a digital camera. It is a schematic block diagram which shows the structural example of a digital camera. It is explanatory drawing explaining the outline | summary of the motion detection which a motion detection part performs. It is a schematic diagram which shows the function structure of a motion detection part. It is explanatory drawing explaining the outline | summary of a panoramic photography mode. It is a figure which shows the example of a transition of the display screen displayed on a display part in panorama imaging | photography mode. It is a figure which shows the example of a transition of the display screen displayed on a display part in panorama imaging | photography mode. It is a figure which shows an example of a frame image just before imaging | photography. It is a figure which shows an example of the present frame image. It is a figure which shows an example of a frame image just before imaging | photography. It is a figure which shows an example of the frame image immediately after imaging | photography. It is a figure which shows an example of the frame image just before imaging | photography after a reduction process. It is a figure which shows an example of the frame image immediately after imaging | photography after a reduction process. It is a state transition diagram of a digital camera. It is a flowchart which shows operation | movement of the digital camera in imaging | photography mode. It is a flowchart which shows the detailed process sequence of an imaging | photography direction determination process. It is a flowchart which shows the detailed process sequence of a motion detection process. It is a flowchart which shows the detailed process sequence of an imaging | photography back-and-front movement detection process. It is a schematic diagram which shows the function structure of the motion detection part in a modification. It is a flowchart which shows the detailed process sequence of the imaging | photography back-and-forth motion detection process in this modification. It is a figure explaining the setting method of the reduction rate in a modification. It is a figure explaining the setting method of the reduction rate in a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Camera body 11 Imaging optical system 12 Imaging element 13 AFE
14 frame memory 15 motion detection unit 151 reduction rate setting unit 153 reduction processing unit 155 reliability evaluation unit 157 image movement amount calculation unit 16 image processing unit 17 recording medium I / F
DESCRIPTION OF SYMBOLS 18 Recording medium holding part 19 Recording medium 20 Video encoder 21 Display driver 22 Display part 23 Video signal output terminal 24 Operation part 3 Shutter button 4 Power button 25 RAM
26 ROM
27 Controller 271 Shooting start instruction unit 273 Shooting time measuring unit

Claims (11)

An imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of shooting processes to generate a plurality of captured images,
A display unit that continuously displays the image data captured by the imaging unit as a live view image;
A live view period motion detector for detecting a motion amount of the image from a plurality of image data of the live view image;
A photographing longitudinal motion detecting unit for detecting a motion amount of the image data or al images of the live view image obtained in the image data and immediately after the completion of the live-view image obtained immediately before the start of the photographing processing,
While calculating the live view image movement amount during display of the live view image based on the detection result by the live view period motion detection unit, and during the shooting process based on the detection result by the motion detection unit before and after shooting An image movement amount calculation unit for calculating an image movement amount during shooting in
An imaging apparatus comprising: A shooting time measuring unit that measures the shooting time from the start to the end of the shooting process;
A reduction rate setting unit for setting a reduction rate according to the shooting time measured by the shooting time measuring unit;
With
The pre- and post-shooting motion detector is configured to reduce the image data of the live view image obtained immediately before the start of the shooting process and the image data of the live view image obtained immediately after the end with the reduction rate set by the reduction rate setting unit. The imaging apparatus according to claim 1, wherein reduction processing is performed. A reliability determination unit that determines the reliability of a detection result by the imaging back-and-forth motion detection unit;
The imaging apparatus according to claim 2, wherein the image movement amount calculation unit calculates the image movement amount during shooting based on a detection result determined to be reliable by the reliability determination unit. . If the reliability determination unit determines that the detection result of the back-and-forth shooting motion detection unit is not reliable, the setting of the reduction rate by the reduction rate setting unit and the motion detection by the back-and-forth shooting motion detection unit are performed as described above. Repeat until the reliability determination unit determines that there is reliability,
The imaging apparatus according to claim 3, wherein the reduction ratio setting unit sets the reduction ratio so that the image size after the reduction processing is reduced every time the reduction ratio is set. The image movement amount calculation unit has not been determined to be reliable by the reliability determination unit as a result of repeating the reduction rate setting by the reduction rate setting unit and the motion detection by the pre- and post-shooting motion detection unit a predetermined number of times. In this case, based on the amount of motion of the image detected by the live view period motion detector before the shooting process and / or the amount of motion of the image detected by the live view period motion detector after the shooting process, The imaging apparatus according to claim 4, wherein an image movement amount during the photographing process is estimated. When the detection result of the back-and-forth shooting motion detection unit is determined to be reliable by the reliability determination unit, the setting of the reduction rate by the reduction rate setting unit and the motion detection by the back-and-forth shooting motion detection unit are performed as described above. Repeat until the reliability determination unit determines that there is no reliability,
The reduction ratio setting unit sets the reduction ratio so that the image size after the reduction process becomes larger every time the reduction ratio is set,
The image movement amount calculation unit, as a result of repeating the setting of the reduction rate by the reduction rate setting unit and the motion detection by the pre- and post-shooting motion detection unit, when the reliability determination unit determines that there is no reliability, The imaging apparatus according to claim 3, wherein the image movement amount at the time of shooting is calculated based on a previous detection result by the movement detection unit before and after shooting. The photographing back-and-forth motion detection unit detects the amount of motion of the image by performing pattern matching based on a predetermined search range set in advance,
When the detection result of the back-and-forth shooting motion detection unit is determined to be reliable by the reliability determination unit, based on the motion detection result that the shooting back-and-forth motion detection unit previously performed for the predetermined search range, An execution determining unit that determines whether or not to perform the setting of the reduction rate by the reduction rate setting unit and the motion detection by the pre- and post-shooting motion detection unit;
The said reduction rate setting part sets the said reduction rate again based on the said motion detection result performed last time, when it determines with performing by the said execution determination part, The reduction rate is characterized by the above-mentioned. Imaging device. An imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of shooting processes to generate a plurality of captured images,
A display unit that continuously displays the image data captured by the imaging unit as a live view image;
A live view period motion detection unit that detects a motion amount of an image from a plurality of image data between the live view images;
A shooting time measuring unit that measures the shooting time from the start to the end of the shooting process;
A search range setting unit for setting a search range according to the shooting time measured by the shooting time measuring unit;
Pattern matching is performed on the basis of the search range set by the search range setting unit between the image data obtained immediately before the start of the photographing process and the image data obtained immediately after the end, and each image A pre- and post-shooting motion detector that detects the amount of motion of the image from the data;
While calculating the live view image movement amount during display of the live view image based on the detection result by the live view period motion detection unit, and during the shooting process based on the detection result by the motion detection unit before and after shooting An image movement amount calculation unit for calculating an image movement amount during shooting in
An imaging apparatus comprising: A shooting start instruction unit for instructing start of the shooting process based on the image movement amount calculated by the image movement amount calculation unit;
After the first shooting process instructed to start by the shooting start instruction unit, a movement mark that moves on the live view image according to the image movement amount during live view, and a movement target mark that indicates a movement target position of the movement mark A display control unit that performs control to display
With
The display control unit sets an initial display position of the movement mark based on the image movement amount at the time of shooting,
The imaging apparatus according to claim 1, wherein the imaging start instruction unit instructs the start of a next imaging process when the movement mark moves to the movement target position. An imaging method in an imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of imaging processes to generate a plurality of captured images,
A display step of continuously displaying the image data captured by the imaging unit as a live view image;
A live view period motion detection step of detecting a motion amount of the image from a plurality of image data of the live view image;
A photographing longitudinal motion detection step of detecting the captured image data and the motion amount of the image data or we image obtained immediately after the end of the live view image of the live view image obtained immediately before the start of the process,
Based on the detection result in the live view period motion detection step, the amount of image movement during live view during the display of the live view image is calculated, and the shooting based on the detection result in the motion detection step before and after shooting. An image movement amount calculation step for calculating an image movement amount during shooting during processing;
An imaging method for an imaging apparatus, comprising: An imaging method in an imaging apparatus that includes an imaging unit that captures a subject image and generates image data, and performs a plurality of imaging processes to generate a plurality of captured images,
A display step of continuously displaying the image data captured by the imaging unit as a live view image;
A live view period motion detection step of detecting a motion amount of the image from a plurality of image data between the live view images;
A shooting time measuring step for measuring a shooting time from the start to the end of the shooting process;
A search range setting step for setting a search range according to the shooting time measured in the shooting time measurement step;
Pattern matching is performed based on the search range set in the search range setting step between the image data obtained immediately before the start of the photographing process and the image data obtained immediately after the end, A pre-shooting motion detection step for detecting the amount of motion of the image from the data;
Based on the detection result in the live view period motion detection step, the amount of image movement during live view during the display of the live view image is calculated, and the shooting based on the detection result in the motion detection step before and after shooting. An image movement amount calculation step for calculating an image movement amount during shooting during processing;
An imaging method for an imaging apparatus, comprising:

Images