JP3344600B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for expressing and converting video information obtained by continuously photographing an object into a still image.

[0002]

2. Description of the Related Art Selection of a representative screen from a video according to the content of the video is frequently performed when the video is expressed as a still image.

[0003] As the representative screen, the first frame or the last frame of a shot, which is the minimum unit of a video, is used.

[0004] By looking at these representative screens, it is intended to realize a free video grasp without any time constraint due to reproduction from the head of the video.

[0005] However, the video contains motion information that does not exist in the still image.

A problem is how to express this motion information, that is, a temporal change, in a still image (space).

If it becomes possible to change the expression of the time information into the space, it becomes possible to intuitively grasp the above-described video grasping time and the movement in the photographed space.

The method of expressing the time information in space is considered to be the creation of a representative screen rather than the selection of a representative screen.

That is, it is created by sufficiently reflecting the characteristics of the video on the still image.

Conventionally, as a method of expressing a video as a still image for the purpose of tracking the movement of an object or the like, there is a video expression method using flash photography.

This expression method is a method of irradiating an object or the like with strobe light at fixed time intervals, multiplexing the reflected light from the object on an imaging surface, and expressing a temporal displacement of the object or the like as a locus in a still image (space). It is.

By expressing a time displacement as a trajectory in a still image (space), it is easy to intuitively grasp the movement of an object or the like.

[0013] In digital image processing of this flash photography, the temporal displacement is expressed in a still image (space) by averaging the images in the time direction.

[0014]

However, there is a need for the image rendering device to be an image captured in a camera stationary state in terms of algorithm.

If the camera is moving, the motion represented by the above algorithm is averaged for all the motions of the camera and the object, and it is impossible to intuitively track the motion of the object and the like.

Generally, when photographing a moving object, the camera is shaken or moved in accordance with the movement of the object.

The method of photographing the movement with the camera pulled (wide angle) is problematic in terms of the limitation of the photographing place and the resolution.

Further, in the above-mentioned conventional method, since the time average is simply taken, there is a problem that the motion is buried temporally.

A motion photographed earlier in time does not appear in a still image expressed by time averaging.

This indicates that the movement of an object or the like cannot be clearly (clearly) represented by the above method.

Conventionally reported methods of expressing the temporal displacement of an object or the like in a video as a trajectory in a still image (space) have the above-described problems.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and an object of the present invention is to provide an image processing apparatus that includes an object in a video even when a motion such as a camera operation is included in the video. It is an object of the present invention to provide a technology which enables an image processing apparatus capable of converting a temporal displacement such as a locus clearly into a still image (space) as a locus.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0024]

To SUMMARY OF THE INVENTION To achieve the object, means (1) of the present invention, an image processing apparatus der for converting image information obtained by continuously photographing a subject to still picture
Tsu, as the size and position of the photographic stationary object in the resulting image of the object by continuous shooting is equal between each frame, the video converting means for similarity transformation of the image, the image converting means Of the frame of the image
For each pixel, a local
Features reflecting movement and time lapse from the first frame of the video
A weight coefficient calculation means for calculating a weighting factor by using the over-the previous
For each pixel of the frame of the image that has been subjected to the similarity conversion,
And a weighted integration means for integrating a product of the element and the weight coefficient calculated by the weight coefficient calculation means in the time direction to generate a still image .

The means (2) according to the present invention is characterized in that, in the means (1), the object is obtained by continuously photographing the object.
The added to that shooting state detecting means calculates a shooting state of the image
It is characterized by having been done .

The means (3) of the present invention is the same as the means (2), except that a spatiotemporal projection image creating means is added.
The spatiotemporal projection image creating means continuously shoots the object.
Rearrange the images obtained by shading on the time axis of video frames
A storage means to create a space image when performed example, frame image
Cutting the spatiotemporal image at the plane containing the image normal,
Reordering means for creating a cross-sectional image;
The image is filtered to detect edges and lines, and
And filter processing means for creating a Tsu di-image column, the space-time
The normal direction of the spatio-temporal edge image sequence
And a feature extraction unit for creating a spatio-temporal projection image , wherein the photographing state detecting unit includes
Detecting shooting conditions using feature statistical analysis means for images
Characterized in that it.

[0027] The means of (4) of the present invention is the means of (3), wherein the feature statistic analysis means comprises
Spatio-temporal projection represented in a two-dimensional image having axes and spatial axes
Compares the spatial distribution of any two times of the shadow image and the spatial coordinates
Means for associating, and the associated spatial coordinates
Statistical processing to calculate the video shooting state parameters
Means .

[0028]

[0029]

[0030]

[0031]

According to the above-mentioned means, when converting video information obtained by continuously photographing a target object into a still image, the size and position of the photographed still target object in the video become equal in each frame. In this manner, the video is converted in a similar manner, a weighting factor is calculated from the converted video using the characteristic of temporal displacement of the video, and the converted video is integrated in the time direction using the weighting factor, thereby converting the video into a still image. Therefore, even when movement such as camera operation is included in an image, such as a generally shot image, the temporal displacement of an object or the like in the image is clearly defined as a locus in a still image (clear). Can be expressed as follows.

[0032]

Embodiments of the present invention will be described below in detail with reference to the drawings.

In all the drawings for explaining the embodiments, parts having the same functions are denoted by the same reference numerals, and their repeated description will be omitted.

FIG. 1 is a functional block diagram of a video converter according to one embodiment of the present invention.

The hardware configuration of the video converter of this embodiment is composed of a computer, an image input device, a memory, an output device, etc., but there is no difference between the hardware configuration of the conventional image processing system. Therefore, the detailed description is omitted.

In FIG. 1, 101 is an input unit, 102 is a spatio-temporal projection image creation unit, 103 is a photographing state detection unit, 10
4 is a determination unit, 105 is a similarity conversion unit, 106 is a weight coefficient calculation unit, 107 is a weighted integration unit, and 108 is an output unit.

In this embodiment, the input video from the camera is converted into a digital signal and input from the input unit 101.

Here, the input image includes luminance, R,
G and B values.

The input image input from the input unit 101 is
A spatio-temporal projection image creation unit 102 creates a spatio-temporal projection image.

Next, the photographing state detection unit 103 detects the photographing state using the spatiotemporal projection image created by the spatiotemporal projection image creating unit 102.

Next, from the photographing state parameters detected by the photographing state detecting unit 103, the determining unit 104 determines whether or not a camera operation has been performed during photographing.

When it is determined that the camera operation has been performed during the photographing, the similarity conversion unit 105 performs similarity conversion on the input video using the shooting state parameters detected by the shooting state detection unit 103.

The meaning of the similarity conversion is to cancel the operation performed during photographing.

Thus, for example, when a camera operation is performed at the time of shooting, an image without camera operation is obtained by this conversion.

A weighting factor is calculated by a weighting factor calculating unit 106 for an input image for which no camera operation has been performed at the time of shooting and an input image to which similarity has been converted by the similarity converting unit 105.

Next, a weighted integration performed by the weighted integration unit 107 is used to perform a weighted integration process on the video in the time direction using the weight calculated by the weighting factor calculation unit 106 to generate a still image. Output from the output unit 108.

Hereinafter, each component will be described with reference to the drawings.

Elements that determine the state at the time of shooting include camera operation, camera on / off, camera shake, and the like.

Hereinafter, a movement caused by a change in the photographing state is called a global movement.

FIG. 2 is a diagram showing a camera operation which is one element of the image capturing state.

As shown in FIG. 2, camera operation is performed in the basic
It consists of operations and their combination operations.

The basic operations include a fix which is fixed to the camera, a pan 501 which is an operation for shaking the camera left and right, a zoom 502 which is an operation for enlarging and reducing a subject by changing the angle of view, and a camera shake up and down. Tilt 503 as an operation, track 5 as an operation to move the camera left and right
04, boom 50 for moving the camera up and down
5. There is a dolly 506 for moving the camera back and forth.

That is, a fix is an operation involving a change in the optical axis direction fixed to the camera projection center, a pan and a tilt are operations involving a change in the angle of view, and a track, boom and dolly are operations involving a change in the position of the camera projection center.

Tracks, booms, and dollies are operations in which the three-dimensional arrangement information of the subject is included in the motion of the image in accordance with the change in the position of the camera projection center.

The motion of an image photographed by a track, a boom, and a dolly indicates a fast motion when the subject is relatively close to the camera, and a slow motion when the subject is far from the camera.

First, the camera parameter estimation will be described in detail.

FIG. 3 is a functional block diagram of the spatiotemporal projection image creation unit 102 in this embodiment.

In FIG. 3, reference numeral 201 denotes a storage unit, 202 denotes a rearrangement unit, 203 denotes a filter processing unit, and 204 denotes a feature extraction unit.

FIG. 4 is a flowchart showing a processing procedure performed in each functional block in FIG.

FIG. 5 is a diagram for explaining the processing procedure of FIG.

First, in FIG. 3, several frames of an input image input from the input unit are stored in the storage unit 201.

The processing of the storage unit 201 is performed in a manner similar to the processing of 301 in FIG.
And rearrangement of video frames on the time axis.

FIG. 5 schematically illustrates this processing on the drawing.
And 401 is called a normal spatio-temporal image.

Next, the video stored in the storage unit 201 is
The rearranging unit 202 rearranges the video information arrangement in preparation for the next filtering process.

The rearrangement process in the rearrangement unit 202 corresponds to the process of 302 in FIG. 4, and is a process of cutting a spatiotemporal image on a plane including a normal line of a frame (screen).

FIG. 5 schematically illustrates this processing on the drawing.
This is the processing of 402 shown in FIG.

FIG. 6 is a diagram for explaining the normal line of the frame in the present embodiment.

In FIG. 6, reference numeral 551 denotes a frame.
The direction 552 perpendicular to the X and Y coordinate axes of this frame is defined as the normal direction of the screen.

A plane including the cut time axis is generally called a spatiotemporal sectional image.

As an example of the spatio-temporal cross-sectional image, a cut plane (epipolar plane image) obtained by cutting the spatio-temporal image on a plane used in the field of computer vision and including the direction of movement of the camera and the normal of the screen. (Epipolar Plane Im
age)).

The three-dimensional position of the subject is estimated from the spatiotemporal sectional image.

This is because on the epipolar plane image, the locus of the feature point of the object becomes a straight line, and the inclination of the straight line becomes the magnitude of the movement of the feature point of the object [RCBoll
es, H. Baker, and DHMarimont, “Epipolar-plane im
age analysis: An approachto determing structure fr
om motion "IJCV, 1, 1, pp7-55, june 1989.].

A spatio-temporal image obtained by cutting the spatio-temporal image so as to include the x and t coordinate axes is particularly called an xt spatio-temporal image.
Similarly, a spatiotemporal cut image including the y and t coordinate axes is referred to as a yt spatiotemporal image.

An arbitrary xt spatiotemporal image is cut out from an arbitrary y value, and the plurality of xt spatiotemporal images are extracted as x-time images.
It is called a t-space image sequence. The same applies to the yt spatiotemporal image sequence.

Next, the cut surface of the spatiotemporal image cut by the rearrangement unit 202 is subjected to a filter (first-order differentiation, second-order differentiation, etc.) process in the filter processing unit 203. This processing aims at detecting an edge or a line.

The processing of the rearranging unit 202 is performed in the order of 3 in FIG.
This processing corresponds to the processing in step 403, and the processing in the form of a diagram is equivalent to the processing in step 403 shown in FIG.

The intensity of an edge or a line is calculated from the filter processing unit 203.

The flow pattern along the time axis seen on the cut surface in the cut image is caused by the movement in the image.
The direction of this flow corresponds to the magnitude of the movement.

In the edge detection described above, an edge or a straight line indicating the direction of the flow is detected, and only the motion information is emphasized from the cut image. The cut image sequence whose edge has been detected is referred to as a spatiotemporal edge image sequence.

Next, in the feature extraction unit 204, the spatiotemporal edge image sequence is added in the normal direction of the edge image.

The process of the feature extracting unit 204 is performed by
04.

FIG. 5 schematically illustrates this processing on the drawing.
Is a process of performing an addition process in the dashed line direction 405.

The processing of the feature extracting section 204 aims at further enhancing the movement emphasized in the preceding filter processing section 203.

That is, if the movement of the feature point of the object is an object caused by a global movement, it is emphasized by performing the addition processing, and is significantly reflected in the result of the addition processing.

On the other hand, when the motion of the feature point of the object is a local motion, the addition process makes the feature point weaker and less likely to be reflected in the result of the addition process.

[0086] Also, this addition process is a strong processing to noise difference and the difference processing, which means that it is possible to extract the motion information from any image containing a large amount of noise.

The image obtained as a result of this addition is called a spatiotemporal projection image.

An xt spatiotemporal image is obtained from the xt spatiotemporal image sequence, and a yt spatiotemporal image is obtained from the yt spatiotemporal image.

The meaning of the flow pattern along the time axis of the xt spatio-temporal projection image expresses the horizontal movement of the image, and the pattern of the yt spatio-temporal projection image expresses the vertical movement. are doing.

Thus, the global motion is visualized as a flow on the spatiotemporal projection (integration) image, and is effective as an intuitive visualization of the global motion.

Next, using the spatio-temporal projection image calculated by the spatio-temporal projection image creating unit 102, the photographing state detecting unit 1
At 03, the photographing state is detected.

In this embodiment, Houg is used as a method for calculating the camera operation parameters using a spatiotemporal projection image.
The h conversion method is used.

Here, the spatiotemporal projection image creation unit 102
The xt spatio-temporal projection image calculated in is calculated as F (x,
t).

The xt spatiotemporal projection image is a function of space x and time t.

The spatial distribution of the xt spatiotemporal projection image at time t0 is represented by F (x, t0), and the spatial distribution of the xt spatiotemporal projection image at time t1 is similarly represented by F (x, t1). .

The global motion parameters calculated below are represented by a, b, and c, where a is a zoom parameter in camera operation, b is a pan parameter, and c is a tilt parameter.

If a global motion exists in the created xt spatiotemporal projection image, F (x, t
0) and F (x, t1) have the following relationship.

[0098]

F (x ', t1) = F (ax + b, t0) Similarly, the yt spatiotemporal projection image includes:

[0099]

## EQU2 ## There is a relationship of F (y ', t1) = F (ay + c, t0).

Here, x 'and y' indicate space coordinates at time t0, and x and y indicate space coordinates at time t1.

First, the above-mentioned x 'is associated with x, y' and y.

FIG. 7 is a diagram for explaining a camera parameter calculation method, and is a diagram for explaining the correspondence between x 'and x, y' and y.

In FIG. 7, reference numeral 601 denotes an xt spatiotemporal projection image, 602 denotes a spatial distribution F (x, T) of time T,
Reference numeral 603 denotes a spatial distribution F (x, T-1) at time T-1.

The coordinate values 604 are associated with each other as shown in FIG. 6, and the corresponding coordinates 605 are calculated.

The associated 604 and 605 indicate arbitrary coordinate values, and the relationship between these coordinates is represented by a straight line 6.
06.

The inclination of this straight line represents the zoom parameter a, and the intercept represents the pan parameter b.

Here, the unit of the zoom parameter a is the change ratio, and the unit of the pan parameter b is the number of changed pixels per frame (pixel / frame).

Next, the maximum value 607 of the projection space is extracted by projecting (voting) onto the parameter space using the coordinate values associated with each other to calculate a and b of the straight line 606 using the following relational expression. Is calculated.

This conversion is generally called a Hough conversion [for the Hough conversion, PV
C. Hough, “Method and Means for Recognizing Complex
Patterns ", US Patent No. 306954, 1962].

If the arbitrary coordinates associated with each other are represented by x 'and x,

[0111]

B = x'cos (a) + xsin (a) The Hough transform is generally established as a method for estimating a straight line constituted by a plurality of points.

One point in the image space represents one curve in the Hough space (projection space), and the coordinate value of the intersection 607 of the plurality of projected curves represents the slope and intercept of the straight line to be extracted. I have.

In this embodiment, a plurality of points constituting a straight line are voted in the projective space, and the slope and intercept of the straight line to be extracted are calculated with the coordinate values indicating the maximum number of votes.

The parameters are calculated by voting in the projection space from a plurality of sets of coordinate values associated with each other.

Similarly, the tilt parameter (c) can be calculated from the yt spatiotemporal projection image using the following equation.

[0116]

C = y'cos (a) + ysin (a) Here, the unit of the tilt parameter c is the number of changed pixels per frame (pixel / frame).

Further, since the feature seen in the spatiotemporal projection (integration) image for the operation including the three-dimensional information is microscopically (partially) equivalent to the operation not including the three-dimensional operation, Is partially (blockwise) applied.

Detection of global movement factors other than camera operation, camera on / off, and camera shake can be detected from discontinuous features seen in a spatiotemporal projection (integration) image. And a relatively short period (10 to 20 Hz) for camera shake.
This is considered to be a pan / tilt operation, and after the camera operation is detected, it can be detected by processing such as Fourier transform.

If the determining unit 104 determines that the camera operation has been performed at the time of shooting based on the shooting state parameters detected by the shooting state detecting unit 103, the similarity conversion unit 10
At 5, the similarity conversion processing of the input video is performed.

Here, if an image is represented by a function of f (x, y, t) as a function of time t and space x, y, the image obtained by performing the parameterized camera operation has the following relationship. There is.

[0121]

F (X, Y, t + 1) = f (ax + b, ay + c, t) The above expression indicates that the image becomes larger or smaller, or moves left and right or up and down by operating the camera. I have.

The similarity conversion unit 105 performs similarity conversion on the input video so that the size and position of the object to be photographed and stationary become equal in each frame (for example, t and t-1).

In the above formula, the luminance of (X, Y) at time (t + 1) (or RGB) can be expressed by (ax) at time t.
The operation of the camera performs similarity conversion of the spatial coordinates so as to be equal to the luminance of + b, ay + c).

To make the image at time (t + 1) the same size as the image at time t, (x = X / ab, y = Y /
The similarity conversion of a-c) may be performed on the space coordinates at time (t + 1).

By the conversion, the global movement (movement by camera operation) included in the movement in the video such as camera operation is canceled.

In this process, the input image becomes a two-dimensional expression image by the photographing system of the photographed space.

FIG. 8 is a diagram for explaining that the input image becomes a two-dimensional expression image of the photographing space by the similarity conversion in the present embodiment.

FIG. 8 is a view showing an example in which a moving car is photographed by shaking the camera. When an image 701 in FIG. 8 is observed from a direction 702, a panoramic image (703) of the photographing space is obtained. .

As described above, by making the input image a two-dimensional expression image of the photographing space, it is easy to detect only the local movement of the object or the like in the following process, and the weight reflecting the movement of the object or the like is easily obtained. Enables calculation of coefficients.

If the determination unit 104 determines that the camera operation has not been performed at the time of shooting, the input image is processed by the weight coefficient calculation unit 106 without conversion.

Next, a description will be given of a method in which the weight coefficient calculating section 106 calculates a weight coefficient reflecting local motion using the similarity-converted video.

FIG. 9 is a diagram for explaining a method of calculating a weight coefficient in this embodiment.

In FIG. 9, reference numeral 801 denotes a spatiotemporal image after conversion, 802 denotes a frame at time t, and 803 denotes a frame at time t + n.

As shown in FIG. 9, 801 represents XYT
It can be represented by a function F (X, Y, T) of X, Y, T in a coordinate system.

The weighting factor to be calculated is 801 pixels (X,
Y, T) are calculated and expressed as w (X, Y, T).

Reference numeral 804 denotes an arbitrary pixel f (x0, y0) in the frame 802 at time t, and F (X0, Y0,
t).

806 is a frame 80 at time t + n
3 is an arbitrary pixel (x1, y1) and F (X1, Y
1, t + n).

However, there is a relationship of X1 = X0, Y1 = Y0. Reference numeral 805 denotes a straight line connecting 804 and 806, and the values (luminance, R, G, B values, R,
A time function L (X0, Y0, T) represented by a color difference signal (y, cb, cr value, etc.) based on the G and B values.

If there is no local motion of an object or the like in the spatiotemporal image 801,

[0140]

L (X0, Y0, t + 1) = L (X0, Y0, t + 2)... = L (X0, Y0, t + n).

The change in the value of L (X0, Y0, T) is due to local motion, and the local motion is reflected by defining the weight coefficient w (X, Y, T) using this function. The coefficient can be calculated.

The weight coefficient can be generally expressed as follows using the function L (X0, Y0, T).

[0143]

W = AB where A represents a group of changes in the function L (X0, Y0, T) and is, for example, as follows.

[0144]

(Equation 8)

B is a group of functions using constants and time as variables, and is, for example, as follows.

[0146]

(Equation 9)

A is a term reflecting local motion, and B is a term reflecting time. With B, the density, color, and the like of the luminance of the still image obtained as a result of the integration process described below change, and the time can be represented by these changes.

For example, B can make the past sepia tone.

Next, in the weighting factor calculating unit 106, the weighted integrating unit 107 performs integration processing on the converted image in the time direction using the weighting factor calculated previously.

For convenience, the weighting factor is defined as follows so that the integration result can be represented as an image on a CRT or hard copy.

Here, it is assumed that the time width for performing the integration process is n.

[0152]

(Equation 10)

W 'represents a normalized form.

The converted image is expressed as F (X, Y, Ti, i = t to t
+ N) and the weighting factor is w ′ (X, Y, Ti, i = t to t +
n), the still image G (X, Y) output from the output unit 108 according to this embodiment is

[0155]

[Equation 11]

It can be expressed as follows.

As described above, the present invention has been specifically described based on the embodiments. However, it is needless to say that the present invention is not limited to the above-described embodiments, and can be variously modified without departing from the gist thereof.

[0158]

As described above, according to the present invention, when converting video information obtained by continuously photographing a target object into a still image, the size of the photographed still target object in the video is reduced. The image is similar-transformed so that the position is equal between frames, a weighting factor is calculated from the converted image using the characteristic of temporal displacement of the image, and the converted image is integrated in the time direction using the weighting factor. The video is converted to a still image, so that even if the video includes motion such as camera operation, as in a generally shot video, the temporal displacement of an object or the like in the video can be statically captured. It is possible to clearly (clearly) express a locus on an image.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a video converter according to an embodiment of the present invention.

FIG. 2 is a diagram showing a camera operation which is one element of a video shooting state.

FIG. 3 illustrates a spatiotemporal projection image creation unit 10 according to the present embodiment.
2 is a functional block diagram of FIG.

FIG. 4 is a flowchart showing a processing procedure performed in each functional block in FIG. 3;

FIG. 5 is a diagram for explaining the processing procedure of FIG. 4;

FIG. 6 is a diagram for describing a normal line of a frame in the present embodiment.

FIG. 7 is a diagram for describing a camera parameter calculation method according to the present embodiment.

FIG. 8 is a diagram for explaining that an input image becomes a two-dimensional expression image of a shooting space by similarity conversion in the present embodiment.

FIG. 9 is a diagram for explaining a method of calculating a weight coefficient in the present embodiment.

[Explanation of symbols]

101: input unit, 102: spatiotemporal projection image creation unit, 10
3. Photographing state detecting unit, 104 determining unit, 105 similarity converting unit, 106 weighting factor calculating unit, 107 weighted integrator, 108 output unit, 201 storing unit, 202 rearranging unit, 203 Filter processing unit, 204: feature extraction unit, 551: frame, 552: normal direction, 601: spatiotemporal projection image.

Continuation of front page (56) References JP-A-5-232912 (JP, A) JP-A-6-38091 (JP, A) JP-A-6-98206 (JP, A) JP-A-6-253197 (JP) , A) One person outside Akito Akutsu, study of camera operation information extraction method using spatiotemporal images, Technical Report of the Television Society, January 28, 1993, Vol. 17, No. 8, pp. 7-12 Suzuki, S., et al., Extraction of Flexible Objects by Spatio-Temporal Contour Integration Method, Transactions of Information Processing Society of Japan, October 15, 1993, Vol. 34, No. 10, pp. 2184-2192 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T ^7/ 00-7/60 G06T 1/00 H04N 7/18 H04N 5/91 H04N 9/79

Claims (4)

(57) [Claims] 1. An image processing apparatus for converting video information obtained by continuously photographing an object into a still image, wherein the size of a photographed still object in a video obtained by continuously photographing the object is as the to position is equal between each frame, the video converting means for similarity transformation of the image, a frame of video that is similarity transformation by the image transformation means
Local pixel based on the pixel value change over time
And the time from the first frame of the video
Weighting factor calculating means for calculating a weighting factor using the progress, and for each pixel of the similarity-converted video frame,
An image processing apparatus comprising: a weighted integration unit that integrates a product of a pixel and a weight coefficient calculated by the weight coefficient calculation unit in a time direction to generate a still image . The image processing apparatus according to the claim 1, characterized in that the object continuous shooting you calculate the shooting state of the resulting image by shooting state detecting means is added Image processing device. 3. The image processing apparatus according to claim 2, further comprising a spatio-temporal projection image creating unit, wherein the spatio-temporal projection image creating unit continuously shoots the object.
The video obtained from the video frame to the time axis of the video frame
A storage means for producing a spatiotemporal image performed, cutting the time space image in a plane including the normal line of the frame image
Reordering means for creating a spatio-temporal cross- sectional image, filtering the spatio-temporal cross-sectional image,
Filter processing means for detecting and generating a spatio-temporal edge image sequence; and
The image processing apparatus further includes a feature extracting unit that performs an addition process in a normal direction to create a spatio-temporal projection image, and the photographing state detecting unit performs processing on the spatio-temporal projection image.
An image processing apparatus characterized in that a photographing state is detected using a feature statistical analysis unit . 4. The image processing apparatus according to claim 3, wherein the feature statistical analysis means has a time axis and a space axis.
Any two times of the spatio-temporal projection image represented in the two-dimensional image
Line Cormorant means associates the comparison spatial coordinates spatial distribution between
If, performs statistical processing on the the associated spatial coordinates, the image shooting
An image processing apparatus comprising: means for calculating a shadow state parameter .