JP6946055B2 - Image processing equipment and image processing methods, programs - Google Patents

Description

The present invention relates to image processing for imparting a rewriting effect by a virtual light source to a plurality of frame images constituting a moving image.

An image in which a virtual light source (hereinafter referred to as a virtual light source) that did not exist at the time of shooting is set in the shot image to give the subject in the image the effect of being irradiated with light from the virtual light source. The processing is known. Such image processing is called rewriting processing, and it is possible to brighten dark areas such as shadows generated by ambient light at the time of shooting.

The following Patent Document 1 discloses a method of creating face area information in an image and performing special processing such as rewriting for each area according to the result of the face area information. More specifically, the rewriting process in the face area is performed according to the face area information such as the presence / absence of the face and the position of the face and the scene discrimination result by the scene discrimination means.

JP-A-2007-148537

When the image is rewritten to correct the shadow, it is necessary to appropriately determine the parameters used for the rewriting process such as the position, intensity, and angle of the virtual light source.

In this regard, Patent Document 1 determines the target area and rewriting intensity to be rewritten according to the face area information such as the presence / absence of a face and the position of the face, and the scene discrimination results such as evening scene, night view, and blue sky by the scene discrimination means. ing.

However, Patent Document 1 is intended for still images, and does not consider how to set a virtual light source for moving images. For example, if a moving subject such as walking or running is rewritten with the optimum virtual light source parameters for each frame image, the parameters are switched on a frame-by-frame basis, making it difficult to see the moving image after the rewriting process. There is.

Therefore, the present invention provides image processing that appropriately imparts a rewriting effect by a virtual light source to a moving image.

One embodiment of the present invention is an image processing device that performs image processing for imparting a rewriting effect by a virtual light source to a plurality of frame images constituting a moving image, and is shooting information relating to fluctuations in a shooting image angle during movie shooting. The acquisition means for acquiring the image, the processing means for performing image processing for imparting a rewriting effect which is the effect of irradiating the plurality of frame images with virtual light from the virtual light source, and the parameters used for the image processing by the processing means are set. When the setting means for setting and the fluctuation of the shooting image angle are caused by the zoom operation of the shooting device for shooting the moving image, the mode in which the position of the virtual light source is not changed and the position of the virtual light source are set with respect to the subject. The setting means has a selection means that makes it possible to select a mode that is relatively far away, and the setting means selects the frame image during a period in which the shooting angle of the image fluctuates among the plurality of frame images. It is characterized in that the parameter is set based on the mode selected by the means.

According to the present invention, it is possible to perform an appropriate rewriting process on a moving image.

It is a block diagram which shows the structure of the digital camera in this invention. It is a block diagram which shows the structure of the image processing part in this invention. It is a block diagram which shows the structure of the rewriting processing part in this invention. It is a figure explaining the reflection by the irradiation of the virtual light source in this invention. It is a figure explaining the image before and after the rewriting process in this invention. It is a flowchart which determines a virtual light source parameter in this invention. It is a table which shows the correspondence between the virtual light source parameter and the camera information in this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Although the present invention will be described with reference to an embodiment in which the present invention is applied to a digital camera capable of photographing a moving image, the present invention is not limited to the digital camera and can be applied to other image processing devices.

(First Embodiment)
Embodiments of the present invention will be described with reference to FIGS. 1 to 7.

FIG. 1 is a block diagram showing a configuration example of a digital camera according to an embodiment of the present invention.

In FIG. 1, 100 is an entire digital camera, 101 is a lens group including a zoom lens and a focus lens, 102 is a shutter having an aperture function, and 103 is an image pickup composed of a CCD or CMOS element that converts an optical image into an electric signal. Part 104 is an A / D converter that converts an analog signal into a digital signal, and 105 is white balance processing, γ processing, contour enhancement, and color correction processing for the image data output from the A / D converter 104. It is an image processing unit that performs various image processing such as.

106 is an image memory, 107 is a memory control unit that controls the image memory 106, 108 is a D / A converter that converts an input digital signal into an analog signal, 109 is a display unit such as an LCD, and 110 is a compression codec for image data. -The codec part to be decrypted.

111 is an interface I / F with the recording medium 112, 112 is a recording medium such as a memory card or a hard disk, 113 is a face detection unit that detects the face of the subject from the captured image, and 114 rewrites the captured image. The rewriting processing unit, 50, is a system control unit that controls the entire system of the digital camera 100.

Further, 121 is a non-volatile memory such as EEPROM for storing programs and parameters, and 122 is a system memory for developing constants and variables for operation of the system control unit 50, programs read from the non-volatile memory 121, and the like. Is. Reference numeral 123 denotes a distance measuring sensor that measures the distance to the subject and outputs the distance information corresponding to each pixel of the photographing pixel as a two-dimensional distance map image.

Next, the basic operation at the time of shooting of the digital camera 100 configured as described above will be described. The image pickup unit 103 photoelectrically converts the light incident through the lens 101 and the shutter 102, and outputs the light as an input image signal to the A / D converter 104. The A / D converter 104 converts the analog image signal output from the image pickup unit 103 into a digital image signal and outputs the analog image signal to the image processing unit 105.

The image processing unit 105 performs color conversion processing such as white balance, γ processing, contour enhancement processing, and the like on the image data from the A / D converter 104 or the image data from the memory control unit 107. Further, the image processing unit 105 performs a predetermined evaluation value calculation process (not shown) using the face detection result acquired by the face detection unit 113 and the captured image data, and the system is based on the obtained evaluation value result. The control unit 50 performs exposure control and distance measurement control. As a result, TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, AWB (auto white balance) processing, and the like are performed.

The image data output from the image processing unit 105 is written to the image memory 106 via the memory control unit 107. The image memory 106 stores the image data output from the imaging unit 103 and the image data to be displayed on the display unit 109.

Further, the D / A converter 108 converts the image display data stored in the image memory 106 into an analog signal and supplies it to the display unit 109. The display unit 109 displays on a display such as an LCD according to the analog signal from the D / A converter 108.

The codec unit 110 compresses and encodes the image data recorded in the image memory 106 based on a standard such as MPEG. The system control unit 50 associates the encoded image data and stores it in the recording medium 112 via the recording interface 111.

In addition to the above basic operations, the system control unit 50 realizes each process of the present embodiment described later by executing the program recorded in the above-mentioned non-volatile memory 121. The program referred to here is a program for executing various flowcharts described later in this embodiment. At this time, the constants and variables for the operation of the system control unit 50, the program read from the non-volatile memory 121, and the like are expanded in the system memory 122.

Next, the detailed configuration of the image processing unit 105 will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the image processing unit 105.

In FIG. 2, 200 is a simultaneous processing unit, 201 is a WB amplification unit, 202 is a brightness / color signal generation unit, 203 is a contour enhancement processing unit, 204 is a brightness gamma processing unit, 205 is a color conversion processing unit, and 206 is a color. The γ processing unit, 207 is a color difference signal generation unit, and 208 is a shadow information acquisition unit.

Next, the processing in the image processing unit 105 will be described. The image signal input from the A / D conversion unit 104 of FIG. 1 is input to the image processing unit 105.

The image signal input to the image processing unit 105 is input to the simultaneous processing unit 200. The simultaneous processing unit 200 performs simultaneous processing on the input Bayer RGB image data to generate color signals R, G, and B.

The WB amplification unit 201 applies a gain to the RGB color signal based on the white balance gain value calculated by the system control unit 50, and adjusts the white balance. The RGB signal output by the WB amplification unit 201 is input to the luminance / color signal generation unit 202. The luminance / color signal generation unit 202 generates a luminance signal Y from the RGB signal, outputs the generated luminance signal Y to the contour enhancement processing unit 203, and outputs the color signal RGB to the color conversion processing unit 205.

The contour enhancement processing unit 203 performs contour enhancement processing on the luminance signal and outputs it to the luminance gamma processing unit 204. The luminance gamma processing unit 204 performs gamma correction on the luminance signal Y and outputs the luminance signal Y to the image memory 106.

The color conversion processing unit 205 converts the RGB signal into a desired color balance (color space) by matrix calculation or the like. The color gamma processing unit 206 performs gamma correction on the RGB color signal. The color difference signal generation unit 207 generates a color difference signal RY signal and a BY signal from the RGB signal.

The image signals (luminance signal Y, color difference signal (RY signal, BY signal)) output to the image memory 106 are compressed and encoded by the codec unit 110 and recorded on the recording medium 112.

Further, the RGB signal output by the color conversion processing unit 205 is also input to the shadow information acquisition unit 208. The shadow information acquisition unit 208 acquires information (shadow information) for analyzing the state of the shadow generated on the subject by the environmental light source at the time of shooting. For example, the average brightness information of the subject, the brightness histogram information of the face area, and the like may be acquired as shadow information.

Next, the detailed configuration and operation of the rewriting processing unit 114 will be described with reference to FIG.

For example, when the rewriting process is selected by the user operation, the image signal output from the image processing unit 105 is input to the rewriting processing unit 114, and the rewriting process is performed by the virtual light source.

FIG. 3 is a block diagram showing the configuration of the rewriting processing unit 114.

In FIG. 3, 301 is an RGB signal conversion unit that converts an input image signal (luminance signal Y, color difference signal (RY signal, BY signal)) into an RGB signal, and 302 is a degamma processing unit that performs degamma processing. Is.

Further, 303 is a distance calculation unit that acquires distance information between the image pickup device and the subject output from the distance measurement sensor 123, 304 is a normal calculation unit that calculates the normal of the subject, and 305 is a virtual light source reflected on the subject. The virtual light source reflection component calculation unit for calculating the components, and 306 are virtual light source addition processing units for adding the rewriting effect by the virtual light source.

Further, 307 is a gamma processing unit that applies gamma characteristics to an RGB signal, and 308 is a luminance / color difference signal conversion unit that converts an RGB signal into a luminance signal (Y) / color difference signal (BY signal, RY signal). ..

The operation of the rewriting processing unit 114 having the above-described configuration will be described.

The rewriting processing unit 114 reads out the luminance / color difference signals (Y, BY, RY) recorded in the image memory 106 and uses them as inputs. The RGB signal conversion unit 301 converts the input luminance / color difference signals (Y, BY, RY) into RGB signals and outputs them to the degamma processing unit 302.

The degamma processing unit 302 calculates the gamma characteristic multiplied by the gamma processing unit of the image processing unit 105 and converts it into linear data. The degamma processing unit 302 outputs the RGB signals (Rt, Gt, Bt) after linear conversion to the virtual light source reflection component calculation unit 305 and the virtual light source addition processing unit 306.

On the other hand, the distance calculation unit 303 generates a distance map using the subject distance information (subject distance value) acquired from the distance measurement sensor 123. Here, the subject distance information is two-dimensional distance information (distance value) obtained in pixel units of a captured image. The distance map is, for example, an image in which the subject distance value is shown in pixel units using gradation or the like.

The normal calculation unit 304 calculates the normal map from the distance map generated by the distance calculation unit 303. Although it is possible to use a known technique for generating a normal map from a distance map, a specific processing example in the present embodiment will be described with reference to FIG.

FIG. 4 is a diagram showing the relationship between the camera shooting coordinates and the subject. For example, for a subject 401 as shown in FIG. 4, the gradient information can be calculated from the difference ΔD of the distance (depth) D with respect to the horizontal difference ΔH of the captured image, and the normal N can be calculated from the gradient information. It is possible. By performing the above processing on each of the captured pixels, the normal information N corresponding to each pixel of the captured image can be calculated. The normal calculation unit 304 outputs the normal information corresponding to each pixel of the captured image as a normal map to the virtual light source reflection component calculation unit 305.

The virtual light source reflection component calculation unit 305 calculates a component in which the light emitted from the set virtual light source is reflected by the subject based on the distance K between the light source and the subject, the normal information N, and the virtual light source parameter L. Specifically, the reflection component of the coordinate position corresponding to the captured image is calculated so as to be inversely proportional to the square of the distance K between the light source and the subject and proportional to the inner product of the subject normal vector N and the light source direction vector L.

This will be described with reference to FIG. In FIG. 4, 401 indicates the subject and 402 indicates the position of the set virtual light source. The reflection component at the horizontal pixel position H1 (the vertical pixel position is omitted for simplification of explanation) of the captured image taken by the camera 100 is proportional to the inner product of the normal N1 at the camera coordinates H1 and the direction vector L1 of the virtual light source. , The value is inversely proportional to the distance K1 between the virtual light source and the subject position.

Expressing this relationship with a mathematical formula, the subject reflection components (Ra, Ga, Ba) by the virtual light source are as shown in the following mathematical formula (1).
Ra = α × (−L ・ N) / K ² × Rt
Ga = α × (−L ・ N) / K ² × Gt (1)
Ba = α × (−L ・ N) / K ² × Bt

Here, α is the intensity of the light emitted from the virtual light source, and is the gain value of the rewriting correction amount. L is the three-dimensional direction vector of the virtual light source, N is the three-dimensional normal vector of the subject, and K is the distance between the virtual light source and the subject. Further, Rt, Gt, and Bt are photographed RGB data output from the degamma processing unit 302.

The reflection components (Ra, Ga, Ba) by the virtual light source calculated by the above formula are output to the virtual light source addition processing unit 306. The virtual light source addition processing unit 306 performs processing according to the following mathematical formula (2) so as to add virtual light source components (Ra, Ga, Ba).
Rout = Rt + Ra
Gout = Gt + Ga (2)
Bout = Bt + Ba

The image signals (Rout, Gout, Bout) output from the virtual light source addition processing unit 306 are input to the gamma processing unit 307. The gamma processing unit 307 performs gamma correction on the RGB input signal. The luminance / color difference signal conversion unit 308 generates a luminance signal Y, a luminance signal RY, and a BY signal from the RGB signal.

The above is the operation of the rewriting processing unit 114. An example of the rewriting process by the rewriting process unit 114 is shown in FIG. FIG. 5A is an example of a captured image before the rewriting process, and FIG. 5B is an example of a captured image after the rewriting process. By irradiating the person subject in the center, which was dark in FIG. 5 (a), with the light of the virtual light source and performing the rewriting process, the brightness is corrected as shown in FIG. 5 (b).

The system control unit 50 stores the luminance signal and the color difference signal output by the rewriting correction unit 114 in the image memory 106 under the control of the memory control unit 107, and then compresses and encodes them in the codec unit 110. Further, the image signal is recorded on the recording medium 112 via the I / F 111.

FIG. 6 is a flowchart showing the operation of the system control unit 50. Hereinafter, an operation of determining the parameters of the virtual light source of the rewriting processing unit 114 based on the camera information regarding the fluctuation of the shooting angle of view at the time of moving image shooting acquired by the system control unit 50 will be described according to the flowchart of FIG.

Here, the camera information is shooting information related to fluctuations in the shooting angle of view at the time of shooting, and is, for example, the presence / absence of a zoom operation of the lens of the camera, the presence / absence of a panning operation, and the like.

FIG. 6 shows a flowchart of the system control unit 50 that determines the parameters of the virtual light source of the rewriting processing unit 114.

In step S601, it is determined whether the rewriting process is selected by the operation from the user via the operation unit 120. That is, it is determined whether or not the processing by the rewriting processing unit 114 is performed. When performing the rewriting process, the process proceeds to step 602.

In step S602, a moving image of the subject is taken.

In step S603, camera information such as the distance from the camera body to the subject, the presence / absence of the zoom operation of the lens, and the presence / absence of the panning operation is acquired from the set values of the lens group 101, the shutter 102, and the imaging unit 103 for each frame. In the following description, the image acquired in one frame is referred to as a frame image, and the moving image is composed of a plurality of frame images.

In step S604, the parameters of the virtual light source are determined based on the camera information acquired in step S603 and the parameter setting table of the virtual light source. Specifically, the determination is made based on the camera information in each frame as shown in FIG. 7A and the virtual light source setting table at the time of zoom operation shown in FIG. 7B.

In FIG. 7A, 701 indicates a frame number (frame No.) and 702 indicates camera information. In FIG. 7B, 703 indicates camera information, 704 indicates a mode, and 705 indicates a virtual light source setting. Here, the mode is a code indicating a setting method of a virtual light source preset for each camera information (zoom operation, panning operation). The setting method of the virtual light source is uniquely determined according to the type and mode of camera information. The mode may be selectable from a plurality of modes using the operation unit 120.

For example, in the period when the frame number is 0 to 3, the information that the zoom-in operation is in progress is acquired as the camera information according to FIG. 7A. Next, according to FIG. 7B, if the zoom-in operation is in progress and mode 0 is set, the position of the virtual light source set in the frame before the period is not changed during this period. Control.

If mode 1 is set, it is set again to a position relatively far from the subject from the position of the virtual light source set in the frame before the period, and the position set during that period. Control so that it does not change from. In either mode, the position of the virtual light source is fixed during the zoom operation. This is because when the angle of view of the camera zooms in on the subject and the subject is photographed, the reflected light changes significantly even if the position of the virtual light source is slightly shifted.

More specifically, assuming that the face position of the main subject acquired by the face detection unit 113 is (x ₁ , y _{1 ) when the frame number is 1, the subject reflection component (Ra (x 1} , 1,) by the virtual light source is used. y ₁ ), Ga (x ₁ , y ₁ ), Ba (x ₁ , y ₁ )) are expressed by the following equation (3).
Ra (x ₁ , y ₁ )
= Α × (−L (x ₁ , y ₁ ) ・ N (x ₁ , y ₁ )) / K (x ₁ , y ₁ ) ² × Rt
Ga (x ₁ , y ₁ )
= Α × (−L (x ₁ , y ₁ ) ・ N (x ₁ , y ₁ )) / K (x ₁ , y ₁ ) ² × Gt (3)
Ba (x ₁ , y ₁ )
= Α × (−L (x ₁ , y ₁ ) ・ N (x ₁ , y ₁ )) / K (x ₁ , y ₁ ) ² × Bt

L (x ₁ , y ₁ ) is the 3D direction vector of the virtual light source, N (x ₁ , y ₁ ) is the _{3D normal vector of the subject at the position (x 1} , y ₁ ), and K (x). ₁ , y ₁ ) indicates the distance between the virtual light source and the _{subject at the position (x 1} , y _1). The fact that the light from the virtual light source hits the subject means that (Ra (x ₁ , y ₁ ), Ga (x ₁ , y ₁ ), Ba (x ₁ , y ₁ )) have positive values.

In FIG. 7A, the period of frame numbers 0 to 3 is in the zoom-in operation, and in FIG. 7B, the zoom-in operation is in progress. When the mode is 0, the position of the virtual light source is not changed, that is, the frame number is 0. In the period of ~ 3, the distance K (x ₀ , y ₀ ) to the subject is controlled so as not to be changed as the default. Also, when the mode is 1, the position of the virtual light source is set farther than the default position and controlled so that it does not change during that period, that is, the distance K (x ₀ , y ₀ ) to the subject is set to the default value. Control to set to a value greater than.

Through the above processing, the control unit 50 determines the parameter of the virtual light source of the rewriting processing unit 114 based on the camera information such as the presence / absence of the zoom operation.

In the virtual light source setting table of FIG. 7B, the case where there are two modes 0 and 1 has been described, but the position of the virtual light source, the light emission angle (irradiation angle), the light emission intensity, and the light source are described based on the camera information. It is not limited to this as long as it controls parameters such as the type (color temperature) of. It suffices as long as it sets at least one parameter. It should be noted that the type of camera information and the setting of the virtual light source may be set in more detail to prepare three or more modes.

Further, the virtual light source may be set according to the panning operation separately from the zoom operation. As a method of detecting the panning motion, for example, the target image is divided into arbitrary regions, the temporal change of the motion vector in each region is detected, and the panning motion is performed when there are a predetermined number or more of motion vectors in the same direction. Judge that there is. Alternatively, the motion information output from the sensor mounted on the camera may be used.

An example of setting the virtual light source during the panning operation is shown in FIG. 7 (c).

In FIG. 7C, 706 indicates camera information, 707 indicates mode, and 708 indicates virtual light source setting. When the panning operation is in progress and the mode is 0, the position of the virtual light source is controlled so as not to be changed, and when the panning operation is in progress and the mode is 1, the position of the virtual light source is controlled to be linked with the panning speed, and the panning operation is in progress. Moreover, when the mode is 2, it is an example of controlling the angle of the virtual light source so as to be linked with panning without changing the position of the virtual light source.

As a method of linking the panning operation with the change of the position and angle of the virtual light source, for example, the target image is divided into arbitrary regions, the panning speed is calculated by the temporal change of the motion vector in each region, and the calculated panning speed is calculated. Change the position of the virtual light source at the same speed as. Alternatively, there is a method of changing the position of the virtual light source according to the calculated panning speed multiplied by an arbitrary coefficient.

Further, in the present embodiment, the case where the subject area in the image is corrected to be bright by the rewriting process has been described, but conversely, the rewriting process may be performed so as to be dark. In that case, the gain value α of the virtual light source may be set to minus.

Further, the method of calculating the position of the virtual light source and the distance D of the pixel to be processed is not limited to the above-mentioned calculation method, and other calculation methods may be used. For example, the camera position and the subject position may be acquired as three-dimensional positions and calculated by the distance in three dimensions.

Further, when a virtual light source is added, the calculation is performed by an equation that is inversely proportional to the square of the distance, but the addition amount of the virtual light source is not limited to the one calculated by this method. For example, it may be an equation that is inversely proportional to the distance D or an equation in which the irradiation range changes in a Gaussian distribution.

Further, in the present embodiment, an example in which camera information is acquired for each frame and the parameter setting of the virtual light source is processed for each frame has been described, but the present invention is not limited to this. The camera information may be acquired and controlled not in units of one frame but in units of several frames.

Further, in the present embodiment, an example in which a digital camera as an imaging device determines parameters of a virtual light source based on subject information and performs rewriting processing has been described, but the present invention is not limited to this. For example, in a personal computer or the like, even if the personal computer has a configuration in which the parameters of the virtual light source are determined by capturing the image data taken by the digital camera and analyzing the subject information of the captured image data, the personal computer performs the rewriting process. good.

In addition, in a personal computer or the like, the image data before the rewriting process taken by a digital camera having a rewriting process function, the subject information acquired by the digital camera, and the parameter information of the virtual light source are taken in, and the user in the rewriting process by the digital camera. Only in the scene where the rewriting is not performed as intended, the parameters of the virtual light source may be edited again and the rewriting process may be performed again.

50 System control unit 100 Digital camera 105 Image processing unit 114 Rewriting processing unit 120 Operation unit

Claims (8)

An image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images that make up a moving image.
An acquisition method for acquiring shooting information regarding fluctuations in the shooting angle of view during movie shooting, and
A processing means for performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source,
Setting means for setting parameters used for image processing by the processing means, and
When the fluctuation of the shooting angle of view is caused by the zoom operation of the shooting device that shoots the moving image, the mode in which the position of the virtual light source is not changed and the position of the virtual light source are set relatively far from the subject. It has a selection means that makes it possible to select a mode.
The setting means sets the parameters of the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images based on the mode selected by the selection means. Processing equipment. An image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images that make up a moving image.
An acquisition method for acquiring shooting information regarding fluctuations in the shooting angle of view during movie shooting, and
A processing means for performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source,
Setting means for setting parameters used for image processing by the processing means, and
When the fluctuation of the shooting angle of view is caused by the panning operation of the shooting device that shoots the moving image, a mode in which the position of the virtual light source is not changed and a mode in which the position of the virtual light source is changed in conjunction with the panning operation. Has a selection means that makes it possible to select and
The setting means sets the parameters of the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images based on the mode selected by the selection means. Processing equipment. An image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images that make up a moving image.
An acquisition method for acquiring shooting information regarding fluctuations in the shooting angle of view during movie shooting, and
A processing means for performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source,
Setting means for setting parameters used for image processing by the processing means, and
When the fluctuation of the shooting angle is caused by the panning operation of the shooting device that shoots the moving image, the mode in which the position of the virtual light source is changed in conjunction with the panning operation and the position of the virtual light source are not changed. It also has a selection means that makes it possible to select a mode that changes the light emission angle in conjunction with the panning operation.
The setting means sets the parameters of the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images based on the mode selected by the selection means. Processing equipment. The image processing apparatus according to any one of claims 1 to 3, wherein the parameter set by the setting means further includes at least one of the emission intensity and the type of the virtual light source. It is a control method of an image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images constituting a moving image.
The acquisition process for acquiring shooting information related to fluctuations in the shooting angle of view during movie shooting, and
A processing step of performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source, and
A setting process for setting parameters used for image processing in the processing process, and a setting process for setting parameters.
When the fluctuation of the shooting angle of view is caused by the zoom operation of the shooting device that shoots the moving image, the mode in which the position of the virtual light source is not changed and the position of the virtual light source are set relatively far from the subject. It has a selection process that makes it possible to select a mode.
The setting step is an image characterized in that the parameters are set based on the mode selected in the selection step for the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images. How to control the processing device. It is a control method of an image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images constituting a moving image.
The acquisition process for acquiring shooting information related to fluctuations in the shooting angle of view during movie shooting, and
A processing step of performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source, and
A setting process for setting parameters used for image processing in the processing process, and a setting process for setting parameters.
When the fluctuation of the shooting angle of view is caused by the panning operation of the shooting device that shoots the moving image, a mode in which the position of the virtual light source is not changed and a mode in which the position of the virtual light source is changed in conjunction with the panning operation. Has a selection process that makes it possible to select
The setting step is an image characterized in that the parameters are set based on the mode selected in the selection step for the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images. How to control the processing device. It is a control method of an image processing device that performs image processing that imparts a rewriting effect by a virtual light source to a plurality of frame images constituting a moving image.
The acquisition process for acquiring shooting information related to fluctuations in the shooting angle of view during movie shooting, and
A processing step of performing image processing for imparting a rewriting effect, which is an effect of irradiating the plurality of frame images with virtual light from a virtual light source, and
A setting process for setting parameters used for image processing in the processing process, and a setting process for setting parameters.
When the fluctuation of the shooting angle is caused by the panning operation of the shooting device that shoots the moving image, the mode in which the position of the virtual light source is changed in conjunction with the panning operation and the position of the virtual light source are not changed. It has a selection process that makes it possible to select a mode that changes the light emission angle in conjunction with the panning operation.
The setting step is an image characterized in that the parameters are set based on the mode selected in the selection step for the frame image during the period in which the shooting angle of view fluctuates among the plurality of frame images. How to control the processing device. A computer-executable program that causes a computer to function as each means of the image processing apparatus according to any one of claims 1 to 4.

Images