JP6611256B2 - Video filtering method, video filtering device, and video filtering program - Google Patents

Description

  The present invention relates to a video filtering method, a video filtering device, and a video filtering program.

  With recent advances in semiconductor technology, the video acquisition speed of video imaging equipment has greatly improved. Currently, the use of high frame rate video acquired by a high-speed camera is classified into high image quality during video playback and high accuracy of video analysis. The former aims to approach the upper limit of the frame rate that can be detected by the visual system (can be displayed on the display), and aims to express smooth movement by increasing the frame rate. For this reason, real-time playback on the display is assumed.

  The latter aims to improve the accuracy of video analysis by using a high frame rate video that exceeds the visual detection limit. Analysis of high-speed moving objects (sport images, FA / inspection, automobiles, etc.) by slow reproduction is a typical application example. This is because the upper limit of the frame rate of the video input / output system is asymmetric. Currently, a high-speed camera capable of acquiring a high frame rate video exceeding 10,000 [frame / sec (fps)] is available as an imaging system.

  On the other hand, the upper limit of the current display is 120-240 [frame / sec]. For this reason, the video image | photographed with such a high-speed camera is used for a slow reproduction use.

  As a prior art, there is known a technique for converting a moving image shot at 30 frames per second or 60 fields per second into a moving image of 48 frames per second that does not feel unnatural motion. In this method, a part of a frame of a progressive image of 30 frames per second is continuously generated twice, thereby generating an image of 48 frames per second that does not feel unnatural motion. Alternatively, an interlaced image of 60 fields per second is converted into a progressive image of 60 frames per second, and a part of the frame is extracted, thereby generating an image of 48 frames per second that does not feel unnatural motion (for example, , See Patent Document 1).

JP 2004-201165 A

  By the way, there is a possibility that a video signal for real-time reproduction having high affinity for video coding can be generated by using a high frame rate video exceeding the visual detection limit. Such a high frame rate video includes a group of frames sampled at a high density in the time direction, and information on the time axis direction of the imaging target is acquired with a high time resolution. Thus, if a video group (for example, 30 Hz) for real-time reproduction is generated using a frame group (for example, 1000 Hz) sampled at high density time, the generation of the video can be controlled with high temporal resolution.

  However, the conventional method relating to the pre-processing of video encoding for the purpose of reducing the amount of generated code has been based on the premise that video is acquired at a playback frame rate. For this reason, sampling a frame with a temporal resolution higher than the playback frame rate is out of scope.

  In simple frame thinning processing, image quality degradation due to temporal aliasing becomes a problem. In order to avoid such a problem, band limiting filtering in the time axis direction is necessary. On the other hand, in the case of an encoder using motion compensated interframe prediction, reduction of temporal aliasing is not directly linked to reduction of prediction error. In addition, the frame sampled with high density was not fully utilized, and the degree of freedom as a time filter was limited.

  In the case of a conventional video with a frame rate of 30 fps or 60 fps, it is difficult to approximate the characteristics of the filter with high accuracy because sufficient samples (that is, frames) for filtering cannot be secured. For example, when a 30 fps video signal is generated by filtering a 60 fps video signal, the number of frames to be filtered is limited to 2 frames under conditions that do not allow duplication of the filtering target frames. On the other hand, in the case of high frame rate video, the degree of freedom in filter design is expanded. For example, when a 62.5 fps video signal is generated by filtering a 1000 fps video signal, 16 frames can be the filtering target even under conditions that do not allow duplication of the filtering target frame.

  That is, when a high frame rate video is input and a low frame rate video is obtained by filtering, the degree of freedom in filtering design is increased. Therefore, there is a possibility that the coding efficiency can be improved by using this high degree of freedom, and thus there is room for optimization with respect to the time direction filter from the viewpoint of coding efficiency.

  The present invention has been made in view of such circumstances, and provides a video filtering method, a video filtering device, and a video filtering program that can reduce the amount of computation while maintaining the suitability of temporal filtering. For the purpose.

  According to one embodiment of the present invention, video filtering performed by a video filtering device that performs temporal filtering on an input video signal based on video coding using motion compensated prediction to generate a low frame rate video signal A method for generating a reference frame that minimizes an evaluation measure of coding efficiency for all selected frames when selecting a filter coefficient from a set of filter coefficients, and the evaluation A minimum value calculating step for calculating a minimum value of the scale, and when the filter coefficient is selected again, the minimum value of the evaluation scale is used for calculation of a cumulative value of the evaluation scale in the frame selection; A filter coefficient selecting step for selecting a filter coefficient that minimizes the sum of the evaluation scales, and the filter to be selected Coefficients dynamically updates, using the updated the filter coefficients, a video filtering method and a repeating step of repeating the process of selecting the filter coefficients to converge condition is satisfied.

  One aspect of the present invention is the video filtering method, wherein the reference frame is used when the difference in accumulated cost of the reference frame is equal to or less than a predetermined threshold before and after the update of the filter coefficient. The accumulated cost for the frame is obtained by referring to the accumulated cost when the filter coefficient before update is used.

  One aspect of the present invention is a video filtering device that performs filtering in a time direction on an input video signal based on video coding using motion compensated prediction to generate a low frame rate video signal, When selecting a filter coefficient from the set of filter coefficients, a frame generation unit that generates a reference frame that minimizes an evaluation measure of coding efficiency for all selected frames, and a minimum value of the evaluation measure When calculating the minimum value to be calculated and selecting the filter coefficient again, the minimum value of the evaluation scale is used for calculation of the cumulative value of the evaluation scale in frame selection, so that the sum of the evaluation scales for all frames is obtained. A filter coefficient selection unit that selects a filter coefficient that minimizes the filter coefficient, and dynamically updates the filter coefficient to be selected, and updates the filter Using the coefficients, a video filtering device having a repeating unit repeating the process of selecting the filter coefficients to converge condition is satisfied.

  One aspect of the present invention is a video filtering program for causing a computer to execute the video filtering method.

  According to the present invention, when performing filtering to generate a low frame rate video signal using a temporal filter in video encoding processing, it is possible to reduce the amount of computation while maintaining the suitability of temporal filtering. The effect of becoming is obtained.

It is a block diagram which shows the structure of the video filtering apparatus by embodiment of this invention. It is a flowchart which shows operation | movement of the video filtering apparatus shown in FIG. It is a flowchart which shows the detailed operation | movement of step S3 shown in FIG. It is a flowchart which shows the detailed operation | movement of step S3 shown in FIG.

  Hereinafter, a video filtering apparatus according to an embodiment of the present invention will be described with reference to the drawings. First, a time filter setting process for generating a video suitable for encoding using the video on the premise that the video can be acquired with a high temporal resolution will be described. In the following, each frame is represented as a one-dimensional signal for the sake of simplicity.

ここで、ｉはダウサンプリング後のフレームを指定するインデックスであり、非負の整数値をとる。時間フィルタの入力信号のフレーム間隔をδｔとして、各フレームは、ｔ＝ｊδｔ（ｊ＝０，１，・・・）においてサンプリングされる。 The i-th frame generated by the (2Δ + 1) -tap time filter is expressed by the following equation.

Here, i is an index for designating a frame after Dow sampling, and takes a non-negative integer value. Each frame is sampled at t = jδt (j = 0, 1,...) Where δt is the frame interval of the input signal of the time filter.

f (x, t) (x = 0,..., X−1) is a pixel value at the position x in the t-th frame. wi [j] is a filter coefficient for the reference frame, and satisfies the relationship of the following equation.

  Further, wi is a vector wi = (wi [−Δ],..., Wi [Δ]) having filter coefficients as elements, and is called a coefficient vector. M is a parameter for determining the frame rate of the video generated by the temporal filter. In the case of Equation (1), the frame rate of the video output by the temporal filter is 1 / Mδt.

In the present embodiment, 2Δ + 1 ≦ M is assumed. As a special form of equation (1), a filter having a filter coefficient of a constant value w [i] = 1 / (2Δ + 1) is called an average filter. The i-th frame output by the average filter is as follows.

  As candidate vectors to be selected as coefficient vectors (hereinafter referred to as coefficient candidate vectors), N types of coefficient candidate vectors γn = (γn [−Δ],..., Γn [Δ]), (n = 0,. ., N-1). From the N types of coefficient candidate vectors, an optimum coefficient candidate vector for generating an encoded frame is selected. Hereinafter, a set of coefficient candidate vectors is referred to as a dictionary, and in order to simplify the notation, a dictionary composed of N types of coefficient candidate vectors is represented as ΓN = (γ0,..., ΓN−1).

[Filter coefficient optimization criteria]
As an optimization criterion in the temporal filter design, the generated code amount for the frame generated by the temporal filter is used. The generated code amount is assumed to be obtained by a lossless encoder with motion compensation prediction (MC prediction). Consider a case where a frame consisting of X pixels is divided into K and motion compensation interframe prediction is performed for each divided section. Frame {circumflex over (f)} (x, iMδt, wi) ({circumflex over (文字)}] is a section B [k] of size X / K (k = 0, 1,..., K−1) ) And a frame ^ f (x, (i−1) Mδt, wi−1) as a reference frame, for each section B [k] (k = 0, 1,..., K−1) Thus, when motion compensation (displacement amount di = (di [0],..., Di [K−1]) is performed, a motion compensation inter-frame prediction error within the frame (hereinafter simply abbreviated as prediction error) is It can be expressed as follows.

The generated code amount of the encoder that encodes the motion compensation inter-frame prediction error is expressed as follows.

  Here, Re (ei (0, wi, wi-1),..., Ei (X-1, wi, wi-11)) is a code amount for a motion compensation interframe prediction error, and Rd (di [0]). ,..., Di [K-1]) are code amounts for the estimated displacements di [0],..., Di [K-1], and Rh is a code amount of header information generated by the encoder. .

  As described above, since the generated code amount of the lossless encoder is used, the motion compensation inter-frame prediction error depends only on the encoding target frame and the reference frame. Therefore, the generated code amount Ψ () in Expression (5) is determined by the filter coefficient vector wi for the i-th frame and the filter coefficient vector wi−1 for the i−1th frame if the displacement amount and the header information are determined.

[Optimization of filter coefficients]
In order to generate a frame that minimizes the generated code amount, it is necessary to solve the minimization problem using the generated code amount of Equation (5) as a cost function and obtain J / M filter coefficient vectors satisfying the following equation: .

When N types of filter coefficient vectors are used as candidate vectors, there are N ^{J / M} possible combinations of filter coefficient vectors, and the optimal filter coefficient vector selection requires an exponential order calculation amount. For this reason, it is not realistic from the viewpoint of computational complexity to search for the optimal combination (w ^* ₀ ,..., W ^* _{J / M-1} ) brute force.

Focusing on the fact that Ψ [w _i , w _i-1 ] depends only on w _i and w _i-1 , equation (6) can be formulated as an optimization problem in a simple Markov process. The optimization problem is based on dynamic programming, and an optimal solution can be obtained with a calculation amount of a polynomial order. The following shows a solution method using dynamic programming. First, S _i (w _i ) of the following equation is defined for w _i (i = 1,..., J / M−1).

S _i (w _i ) is a sum of costs when an optimum filter coefficient vector is used in a path to a state where a frame is generated by w _i in the i-th stage. Here, when w _i is fixed, paying attention to the fact that Ψ [w _i , w _i−1 ] depends only on w _i−1 , S _i (w _i ) can be expressed as a recurrence formula as follows: I can express.

Note that S _i-1 (w _i-1 ) has been calculated using the same recurrence formula, and is stored in the register as a referenceable value when S _i (w _i ) is calculated. In this case, for the calculation of S _i (w _i ), the dictionary Γ that minimizes Ψ [w _i , w _i−1 ] + S _i−1 (w _i−1 ) is calculated based on the recurrence relation of Equation (8). It is sufficient to select _N candidate vectors. Assuming that n _i is a candidate vector index for w _i, for each n _i , the index of the candidate vector that gives the minimum value of equation (8) is stored as ^ n _i−1 (n _i ). Make it referable.

By using the recurrence formula of formula (8) recursively, the minimization problem of formula (6) can be expressed as the following formula.

Thus, as long as the method of using the recurrence formula of the formula (8), the optimal solution of equation _{^{(6) (w * 0,}} ···, w * J / M-1) ^is, N 2 J / M It can be reduced to a problem of searching for an optimal solution from the street, and can be calculated with a calculation amount of a polynomial order. Σ ^{J / M−1} _{i = 1 After} obtaining the minimum value of Ψ [w _i , w _i−1 ], the optimal solution (w ^* ₀ ,..., W ^* _{J / M−1} ) is Obtained by the track process. The w _{J / M-1} that minimizes the expression (10) is set as w ^* _{J / M-1} as the following expression.

Let n _{J / M-1} be the index of the candidate vector representing w ^* _{J / M-1} . When the candidate vector index of the J / M-1 frame is n _{J / M-1} , the optimal candidate vector index for the J / M-2 frame is ^ n _{J / M-2} (n _{J / M -1} ). Therefore, the filter coefficient vector of the J / M-2th frame is identified as w ^* _{J / M-2} = γ ^ n _{J / M-2} (n _{J / M-1} ). Below, the same reference processing _{^{w * J / M-3 =}} γ ^ n J / M-3 (n J / M-2), ···, w * 0 = γ ^ n 0 (n 1). Repeat as.

[Update filter coefficient dictionary]
The filter coefficient dictionary is updated by the following iterative process until the convergence condition is satisfied, and candidate vectors in the dictionary are configured.

The optimum filter coefficient solution is obtained using the given dictionary by the algorithm of [Filter coefficient optimization]. A histogram h [n] (n = 0,..., N-1) of candidate vectors in the dictionary for the obtained optimal solution W ^* _i (i = 0,..., J / M-1). ) Is calculated. Here, h [n] (n = 0,..., N−1) stores the frequency at which the candidate vector γn in the dictionary is selected as the optimal solution.

  Based on the frequency of the candidate vectors in the dictionary, the dictionary is updated according to the following policy. If a candidate vector with a low frequency selected as the optimal solution exists in the dictionary, it is deleted from the dictionary. Specifically, candidate vectors that satisfy h [n] ≦ θl are deleted from the dictionary. Here, θl is a threshold value given from the outside.

  When a candidate vector having a high frequency selected as the optimal solution exists in the dictionary, a filter coefficient vector obtained by correcting the candidate vector is added to the dictionary. Specifically, in a candidate vector that satisfies h [n] ≧ θu, θu is a threshold given from the outside. The correction of the candidate vector is as follows.

γｎ［＾ｊ］をαγｎ［＾ｊ］へ修正する。ここで、α＜１のパラメータとして設定される。さらに、＾ｊ＝−Δ＋１，・・・，Δ−２の場合、γｎ［＾ｊ＋１］をγｎ［＾ｊ＋１］＋α／２・γｎ［＾ｊ］に修正し、γｎ［＾ｊ−１］をγｎ［＾ｊ−１］＋α・２・γｎ［＾ｊ］に修正する。＾ｊ＝Δ−１の場合、γｎ［＾ｊ−１］＋αγｎ［＾ｊ］に修正する。＾ｊ＝−Δ の場合、γｎ［＾ｊ＋１］＋αγｎ［＾ｊ］に修正する。 [Candidate vector correction method]
The maximum value in the element γn [j] (j = −Δ,..., Δ−1) of the candidate vector γn is identified.

γn [^ j] is corrected to αγn [^ j]. Here, it is set as a parameter of α <1. Further, in the case of ^ j = −Δ + 1,..., Δ−2, γn [^ j + 1] is corrected to γn [^ j + 1] + α / 2 · γn [^ j], and γn [^ j−1] is changed. The correction is made to γn [^ j−1] + α · 2 · γn [^ j]. If ^ j = Δ−1, the correction is made to γn [^ j−1] + αγn [^ j]. If ^ j = −Δ, the correction is made to γn [^ j + 1] + αγn [^ j].

  In addition, although the said method was shown as an example as a correction method of a candidate vector, even if it is methods other than the above, this invention can be utilized.

  The candidate vectors in the dictionary are updated by deleting, modifying, and adding to the candidate vectors. Using the updated dictionary, the optimum filter coefficient and the generated code amount when the filter coefficient is used are obtained based on the algorithm of [Filter coefficient optimization]. Specifically, an algorithm of “optimization of filter coefficient after dictionary update” described later is used.

  Comparing before and after updating the dictionary, the update is terminated when the difference in the generated code amount when the optimum filter coefficient is used becomes sufficiently small (below a given threshold). If the number of candidate vectors in the dictionary exceeds a predetermined upper limit, the addition of candidate vectors is stopped and the update process is terminated.

[Optimization of filter coefficients after dictionary update]
Consider a case where one new candidate vector is added to the dictionary as a result of the above-described modification of the candidate vector, and the dictionary is composed of N + 1 candidate vectors. At this time, it is possible to refer to the accumulated cost S _i (w _i ) (i = 0,..., J / M−1) when the dictionary before update is used. In order to distinguish the accumulated cost before and after the update, the accumulated cost before the update is expressed as S ^(k−1) _i (w _i ) (i = 0,..., J / M−1), The accumulated cost is expressed as S ^(k) _i (w _i ) (i = 0,..., J / M−1).

The recurrence formula of S ^(k) _i (w _i ) when w _i is fixed to a specific candidate vector can be expressed as the following formula, similarly to formula (8).

When w _i is fixed to the candidate vector added to the dictionary, the calculation is the same as [Optimization of filter coefficient]. On the other hand, when w _i is fixed to the candidate vector held in the dictionary before update, an approach that omits redundant operations is taken as described below.

^{_{S (k) i-1 (}} w i-1) reads the update flag indicating indicating whether or not updated for _{^{S (k-1) i-}} 1 (w i-1). This update flag is stored together when S ^(k) _i-1 (w _i-1 ) is calculated. First, S ^(k−1) _i (w _i ) is set as an initial value of S ^(k) _i (w _i ). When S ^(k) _i-1 (w _i-1 ) has been updated from S ^(k-1) _i-1 (w _i-1 ), Ψ [w _i , w _i- in equation (12) ₁ ] + S ^(k) _i−1 (w _i−1 ) is calculated.

Further, the calculated value S ^(k) _i-1 (w _i-1 ) is compared with the provisional minimum value at that time, and if the calculated value is smaller than the provisional minimum value, the value obtained by calculating the provisional minimum value Update with. On the other hand, when S ^(k) _i-1 (w _i-1 ) is the same as S ^(k-1) _i-1 (w _i-1 ), the above-mentioned Ψ [w _i , w _i-1 ]. The calculation of + S ^(k) _i-1 (w _i-1 ) and the comparison process with the provisional minimum value are omitted. The other processing is the same calculation as [Optimization of filter coefficient].

  FIG. 1 is a block diagram showing the configuration of the video filtering apparatus of the embodiment. In this figure, reference numeral 1 denotes an original image storage unit for storing an original image. Reference numeral 2 denotes a dictionary storage unit that stores a dictionary. Reference numeral 3 denotes an encoded image generation unit that generates an encoded image. Reference numeral 4 denotes a generated image storage unit that stores the generated image. Reference numeral 5 denotes a reference image selection unit that selects a reference image. Reference numeral 6 denotes an encoding processing unit that encodes an image. Reference numeral 7 denotes a subjective distortion amount calculation unit that calculates the subjective distortion amount.

  Reference numeral 8 denotes an optimal cumulative cost calculation unit that calculates the optimal cumulative cost. Reference numeral 9 denotes an accumulated cost storage unit that stores the calculated accumulated cost. Reference numeral 10 denotes a filter coefficient index storage unit that stores a filter coefficient index. Reference numeral 11 denotes a final level determination unit that determines the final level. Reference numeral 12 denotes a filter coefficient update unit that updates the filter coefficient. Reference numeral 13 denotes a filter coefficient update determination unit that determines whether or not to update a filter coefficient. Reference numeral 14 denotes a convergence determination unit that determines convergence. Reference numeral 15 denotes a filter processing unit that performs filter processing.

  Next, the operation of the video filtering apparatus shown in FIG. 1 will be described. FIG. 2 is a flowchart showing the operation of the video filtering apparatus shown in FIG. First, the video filtering apparatus reads video data to be filtered from the outside and a parameter M for determining the frame rate after filtering (step S1).

  Next, the filter coefficient updating unit 12 reads a dictionary of filter coefficients used for filtering (step S2). Subsequently, the video filtering apparatus reads as input the video data to be filtered, the parameter M for determining the frame rate after filtering, and the filter coefficient dictionary, and calculates the optimum filter coefficient for each encoded frame (step S3). ). Specific processing in step S3 will be described later.

  Next, the filter coefficient update unit 12 reads as input the video data to be filtered, the parameter M for determining the frame rate after filtering, and the filter coefficient dictionary, and updates the candidate vectors in the dictionary (step S4). As a specific update process, for example, the above-described algorithm for updating the filter coefficient dictionary is used.

  Next, the video filtering device determines whether or not there is an updated filter coefficient (step S5). As a result of the process of step S4, when there is no updated candidate vector in the dictionary, the process proceeds to step S7, and otherwise, the process proceeds to step S6.

となる場合、ステップＳ７の処理に移り、それ以外の場合、ステップＳ８の処理に移る。 Next, the filter coefficient update determination unit 13 determines whether or not the reduction rate of the generated code amount using the dictionary before and after the update is less than a threshold (step S6).

If so, the process proceeds to step S7. Otherwise, the process proceeds to step S8.

  Next, the filter processing unit 15 reads as input the video data to be filtered, the parameter M for determining the frame rate after filtering, and a dictionary of filter coefficients, and is designated by the parameter M using the filter coefficients in the dictionary. The video data is filtered in the time direction so that the frame rate becomes the same, and the filtered video data is output (step S7).

  On the other hand, when the reduction rate of the generated code amount is not less than the threshold, the dictionary storage unit 2 reads the candidate vector including the updated filter coefficient, adds the candidate vector to the dictionary, and outputs the updated dictionary (step S8). Return to and repeat the process.

  Next, the detailed operation of step S3 shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart showing the detailed operation of step S3 shown in FIG. The process shown in FIG. 3 is a process that is performed before the dictionary is updated.

  First, the video filtering device reads video data to be filtered, a parameter M for determining a frame rate after filtering, and a dictionary of filter coefficients (step S11). Subsequently, the video filtering apparatus repeats the processing from step S12 to step S28 for the index i = 0,..., J / M−1 indicating the encoding target frame (step S12).

Next, the video filtering apparatus repeats the processing from step S14 to step S27 for the filter coefficient w _i εΓ _N of the encoding target frame (step S13). The video filtering device reads input video data, performs a temporal filtering process using the filter coefficient w _i , and generates an encoding target frame (step S14). The video filtering apparatus repeats the processing from step S18 to step S26 for the filter coefficient w _i-1 εΓ _N of the reference frame (step S15).

Next, the accumulated cost S _i (bsw _i−1 ) in the _i− 1th stage when the filter coefficient w _i−1 is used is read (step S18). Subsequently, the reference image selection unit 5 reads the frame to be encoded at the i−1 stage generated using the filter coefficient w _i−1 as the reference frame at the i stage (step S <b> 19).

Next, the encoding processing unit 6 reads the encoding target frame generated in step S14 and the reference frame read in step S19 as inputs, performs the encoding process, and generates the generated code amount Ψ [w _i , w _i−1. ] Is calculated (step S20).

Next, the optimum accumulated cost calculation unit 8 sets the filter coefficient of the reference frame read in step S20 as the number w _i−1 and the accumulated cost at the i-th stage when the filter coefficient of the encoding target frame is set as the number w _i. Is calculated as ψ [w _i , w _i-1 ] + S _i-1 (w _i-1 ) (step S21).

Next, the video filtering device compares the calculated accumulated cost with the provisional value of S _i (w _i ). If the former is smaller than the latter, the video filtering device moves to the process of step S23. (Step S22).

Next, the optimal accumulated cost calculation unit 8 updates the provisional value of S _i (w _i ) with the calculated accumulated cost. The processes of Step S22 and Step S23 are processes for realizing Expression (8) (Step S23).

Next, the filter coefficient index storage unit 10, the filter coefficient when the w _i, and stores the index of the candidate vectors of filter coefficients of the reference frame to provide a temporary minimum value of the accumulated cost of the i stage (step S24) .

Next, the optimal accumulated cost calculation unit 8 calculates w ^* _{J / M-1} shown in Expression (10) as a filter coefficient that minimizes the accumulated cost in the J / M-1 stage (step S29).

  Next, the video filtering apparatus repeats the process of step S31 for the index i = J / M−2,..., 1 indicating the encoding target frame (step S30).

Next, when the index of the candidate vector representing the w ^* _i and n _i, the index of the optimal candidate vector for the i-1 frame when the index of the candidate vectors of the i-th frame and the n _i is ^ n _{i- 1} (n _i ). Therefore, when the filter coefficient is a candidate vector representing w _i , the optimal accumulated cost calculation unit 8 calculates the filter coefficient of the reference frame (filter coefficient of the i−1th frame) that gives the minimum accumulated cost of the i-th stage. _{^{w * i-1 = γ ^}} n i-1 (n i) is identified as (step S31).

  Next, a modified example of the detailed operation of step S3 shown in FIG. 2 will be described with reference to FIG. FIG. 4 is a flowchart showing the detailed operation of step S3 shown in FIG. The process shown in FIG. 4 is a process performed when an updated dictionary is used. 4, the same operations as those shown in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted. The operation shown in FIG. 4 is different from the operation shown in FIG. 3 in that steps S16, S17, and S25 are newly added. Here, only the added processing will be described. Other processes are as described above.

Optimal cumulative cost calculation unit ⁸ updates showing the indicating whether _{S (k) i-1 (} w i-1) is updated for _{^{S (k-1) i-}} 1 (w i-1) A flag is read (step S16). Note that S ^(k−1) _i (w _i ) is set as the initial value of S ^(k) _i (w _i ).

  When the cumulative cost update flag is ON, the video filtering apparatus proceeds to the process of step S18, and otherwise, proceeds to the process of step S26 (step S17).

The optimum accumulated cost calculation unit 8 sets the update flag to ON when S ^(k) _i-1 (wi _-1 ) is updated with respect to S ^(k-1) _i-1 (wi _-1 ). Set (step S25).

  As described above, in a video encoding process in which a low frame rate video signal obtained by a temporal filter is input to a high frame rate video signal sampled at a high density, all frames of the low frame rate video signal are input. In consideration of the encoding efficiency, filtering can be performed using a time filter that determines a low frame rate video signal.

  You may make it implement | achieve all or one part of the video filtering apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  In video encoding processing, the present invention can be applied to applications in which it is indispensable to perform filtering using a time filter that determines a low frame rate video signal.

  DESCRIPTION OF SYMBOLS 1 ... Original image memory | storage part, 2 ... Dictionary memory | storage part, 3 ... Encoded image generation part, 4 ... Generated image memory | storage part, 5 ... Reference image selection part, 6 ... Code | symbol Processing unit, 7 ... subjective distortion amount calculation unit, 8 ... optimum cumulative cost calculation unit, 9 ... cumulative cost storage unit, 10 ... filter coefficient index storage unit, 11 ... final level determination , 12 ... Filter coefficient update unit, 13 ... Filter coefficient update determination unit, 14 ... Convergence determination unit, 15 ... Filter processing unit

Claims (3)

A video filtering method performed by a video filtering device that performs time direction filtering on an input video signal based on video coding using motion compensated prediction and generates a low frame rate video signal,
When selecting a filter coefficient from the set of filter coefficients, the filter coefficient is selected for every selected frame so as to minimize the generated code amount of the reference frame constituting the low frame rate video signal, which is an evaluation measure. Generating a frame of reference using a filter coefficient selected from the set of :
A minimum value calculating step for calculating a minimum value of the evaluation scale;
When selecting again the filter coefficients, the minimum value of the evaluation scale found in the previous, by using the calculation of the cumulative value of the evaluation scale in selecting again filter coefficients in the reference frame, the all A filter coefficient selection step of selecting, for each of the selected frames , a filter coefficient that minimizes the sum of the evaluation measures of the reference frames of
The filter coefficient to be selected is dynamically updated, the updated filter coefficient is used, and the updated filter coefficient based on the generated code amount of the reference frame using the set of filter coefficients before the update is used. And a repetition step of repeating the process of selecting the filter coefficient until a reduction rate of the generated code amount of the reference frame using a set becomes less than a threshold . A video filtering device that performs temporal filtering on an input video signal based on video coding using motion compensated prediction to generate a low frame rate video signal,
When selecting a filter coefficient from the set of filter coefficients, the filter coefficient is selected for every selected frame so as to minimize the generated code amount of the reference frame constituting the low frame rate video signal, which is an evaluation measure. A frame generation unit that generates the reference frame using a filter coefficient selected from the set of :
A minimum value calculating step for calculating a minimum value of the evaluation scale;
When selecting again the filter coefficients, the minimum value of the evaluation scale found in the previous, by using the calculation of the cumulative value of the evaluation scale in selecting again filter coefficients in the reference frame, the all A filter coefficient selection unit that selects, for each of the selected frames , a filter coefficient that minimizes the sum of the evaluation scales of the reference frames of
The filter coefficient to be selected is dynamically updated, the updated filter coefficient is used, and the updated filter coefficient based on the generated code amount of the reference frame using the set of filter coefficients before the update is used. A video filtering apparatus comprising: a repetition unit that repeats the process of selecting the filter coefficient until a reduction rate of the generated code amount of the reference frame using a set is less than a threshold value . A video filtering program for causing a computer to execute the video filtering method according to claim 1 .

Images