JP3793333B2 - Time variable image speed conversion device - Google Patents

Description

を満足するｉをｉ_opt とし、フレームＣj は入力フレームＯ_(iopt)-1とＯ_ioptの間に位置するものと考え、
この時、Ｏ_(iopt)-1からの内挿位置（Ｐｊ）を次式（２）により演算して与える
【数４】

ことを特徴とする。
【０００８】
ここで、前記可変速指示手段は、映像の記録再生または編集を行なう機能を有する装置における映像の速度を可変にするための可変速制御部であることを特徴とすることができる。
【００１３】
本発明は、上記構成により、（例えばＶＴＲ等の）可変速制御部、もしくは編集機の可変速制御部などの外部から指示する可変速倍率（伸張/ 短縮コード）に合わせて、（テレビジョン映像のように）時間方向に離散的な映像に対し、適宜、映像を内挿する時間位置を決定し、決定した時間位置に映像を内挿するので、原映像に比べて速度のゆったりした映像、または速度の速い映像を、自然性を保ったまま高品質に、提示することができる。
【００１４】
また、本発明は、上記構成により、映像の提示速度を可変速する場合、一様に変化させるのではなく、時々刻々変化する可変長のブロック単位で、映像の提示速度を時々刻々変化させるので、一様に映像の提示速度を変化させるのではなく、きめ細かに時々刻々可変速することが可能である。
【００１５】
【発明の実施の形態】
以下、図面に示す本発明の実施の形態に基づき本発明を詳細に説明する。
【００１６】
なお、本発明は複数の機器からなるシステムにおいて達成されてもよく、１つの機器からなる装置に達成されてもよい。また、システムあるいは装置にプログラムを供給することにより、本発明を達成される場合にも適用されることは言うまでもない。例えば、専用の装置だけでなく、パーソナルコンピュータ（パソコン）等を使用しても実現することができる。また、本発明に係る画速変換アルゴリズムを実現する制御手順をプログラム形態で記録する記録媒体は、ＦＤ（フロッピーディスク）以外にも、ＣＤ−ＲＯＭ、ＩＣメモリカード等であってもよい。更に、本プログラムをＲＯＭに記録しておき、これをメモリマップの一部となすように構成し、直接ＣＰＵで実行することも可能である。
【００１７】
図１は本発明の一実施形態の時可変型画速変換システムの構成を機能ブロックで示す。ここで、画速変換はハイビジョン、ＮＴＳＣ、ＰＡＬ等の各種カラーテレビジョン方式に適応可能であるが、図１では、代表例としてハイビジョン（高品位テレビジョン、高精細度テレビジョン）についての場合を示す。
【００１８】
まず、入力するアナログの映像（映像信号）を例えばＲＧＢ４：４：４のフォーマットのＡ／Ｄ変換部２においてＡ／Ｄ変換する。Ａ／Ｄ変換部２でデジタル化された映像信号は動きベクトル検出系と内挿系とに送出される。
【００１９】
例えば動きベクトル検出系は前処理部３、動き検出部４、およびベクトル検出割り付け部５を有する。Ａ／Ｄ変換部２でデジタル化された映像信号に対して、動きべクトル検出・割付のための前処理を前処理部３で行う。一例としては、前処理部３において映像信号をインターレースの１１２５／６０／２：１からノンインターレースの６５２／６０／１：１に変換する。前処理部３で前処理を施された映像信号は動き検出部４とベクトル検出割り付け部５とに送られる。
【００２０】
動き検出部４は、例えば勾配法に基づく初期偏位べクトル（候補べクトル８種、ブロックサイズ：８画素、８ライン）を用いた反復勾配法（最大反復回数：２回、ブロックサイズ：８画素、８ライン）により、映像のフィールド間での動きべクトルを検出する。入力画像と時間的タイミングの異なるフイールドを新たに内挿するため、入力映像信号から動き検出部４で検出した動きべクトルを、ベクトル検出割り付け部５により新たに内挿するフイールド上に割り当てる。
【００２１】
（例えばＶＴＲ等の）可変速制御部、もしくは編集機の可変速制御部などの外部の可変速指示部１から指示された可変速倍率を示す伸張／短縮コードをフィールド内挿部６へ転送する。この可変速倍率は時々刻々任意に設定可能である。
【００２２】
上記内挿系はフィールド内挿部６からなる。フィールド内挿部６は、Ａ／Ｄ変換部２から送られてくる入力映像に対して、可変速指示部１から指示された可変速倍率を示す伸張／短縮コードをもとに、適宜、可変時間長内で、任意の時間位置のフィールド（任意のフィールド数）を内挿することで、時々刻々、映像の速度を可変にする。即ち、フィールド内挿部６は、伸張／短縮コードが示す可変速倍率が例えば１／６０秒（１フィールド分）を越えた場合、映像の動きベクトルをもとに伸張量に相当する任意の時間位置のフィールドの内挿を行なう。そのため、フィールド内挿部６では伸張／短縮コードを１／６０秒（ｌフィールド分）で量子化し、原映像に対応した映像のフィールド数と、伸張／短縮コード（可変速倍率）に対応した映像のフィールド数とを算出する。
【００２３】
フィールド内挿部６で内挿処理を受けた映像信号はＤ／Ａ変換部７へ出力され、Ｄ／Ａ変換部７でアナログ化した映像信号は時々刻々変化する画速変換映像として出力される。
【００２４】
図２は図１のフィールド内挿部６による、ある可変ブロック長内における、フィールド内挿位置の決め方を示す。
【００２５】
２／６０秒（２フィールド分）〜１０／６０秒（１０フィールド分）程度のある分析区間内（可変ブロック長内）において（各分析区間の時間長は夫々異なる）、原画のフィールド数：Ｍ、画速変換により作成するフィールド数：Ｎとすると、図２の（ａ）にＭ＜＝Ｎの場合、図２の（ｂ）にＭ＞Ｎの場合を示している。この両者の内挿位置の決定方法は同一であり、以下に示す手順で決定する。
【００２６】
１）原画のフィールドを、Ｏ1 、Ｏ2 、…、Ｏi 、Ｏi+1 、…、Ｏm とする。
【００２７】
２）画速変換により作成するフィールドを、Ｃ1 、Ｃ2 、…、Ｃj 、Ｃj+1 、…、Ｃn とする。
【００２８】
３）各フレームＣj に対して、次式（１）
【００２９】
【数１】

【００３０】
を満足するｉをｉ_opt とし、フレームＣj は入力フレームＯ_(iopt)-1とＯ_ioptの間に位置するものと考える。
【００３１】
この時、Ｏ_(iopt)-1からの内挿位置（Ｐｊ）を次式（２）により演算して与える。
【００３２】
【数２】

【００３３】
４）以上の内挿処理を、各可変ブロック毎（各可変ブロックの時間長はそれぞれ異なる）に行い、画速変換する。
【００３４】
任意の可変ブロック長単位で、時々刻々速度の変化する画速変換の例を図３に示す。図３は原画の映像を６フィールドから７フィールドへ変換出力した後、３フィールドから４フィールドへ変換して出力する場合の例を示している。図３の破線矢印で示すように、可変ブロックごとに、決定した内挿位置（時間位置）に、その内挿位置の原画（入力画像）のフィールドの映像と、隣接する原画のフィールドから合成した映像とを内挿し、画速変換している。
【００３５】
【発明の効果】
以上説明したように、本発明によれば、映像を伴ったメディア（例えば、テレビジョン、ＶＴＲやＤＶＤのような映像音声の記録メディア、コンピュータ上での動画音響再生等）や装置、および医療機器（例えば、Ｘ線ＣＴやＮＭＲ−ＣＴなど）等での動画再生等において、（例えばＶＴＲ等の）可変速制御部、もしくは編集機の可変速制御部などの外部から時々刻々任意に設定される可変速倍率をもとに、時々刻々変化する可変長のブロック単位で、（テレビジョン映像のように）時間方向に離散的な映像に対し、映像の伸張量に相当する数のフィールドを内挿する時間位置を決定し、決定した時間位置に映像を内挿するので、原映像に比べて速度のゆったりした映像、または速度の速い映像を、自然性を保ったまま高品質に、提示することができる。
【００３６】
また、本発明によれば、映像の提示速度を可変速する場合、一様に変化させるのではなく、時々刻々変化する可変長のブロック単位で、映像の提示速度を時々刻々変化させるので、一様に映像の提示速度を変化させるのではなく、きめ細かに時々刻々可変速することができる。
【図面の簡単な説明】
【図１】本発明の一実施形態の時可変型画速変換システムの構成を示すブロック図である。
【図２】本発明の一実施形態における映像のフィールドの内挿位置を示すタイミングチャートである。
【図３】本発明の一実施形態における可変ブロック長単位で、時々刻々速度の変化する画速変換の例を示すタイミングチャートである。
【符号の説明】
１ 可変速指定部
２ Ａ／Ｄ変換部
３ 前処理部
４ 動き検出部
５ ベクトル検出割り付け部
６ フィールド内挿部
７ Ｄ／Ａ変換部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to media (for example, television, video / audio recording media such as VTR and DVD, moving image sound reproduction on a computer) and apparatus, and medical equipment (for example, X-ray CT and NMR). -When playing a video with a variable speed every time, such as when playing a video, etc., the video speed is variable with high quality while maintaining the naturalness of the video by interpolating the video with high quality. when it is possible to relate the variable image speed conversion apparatus.
[0002]
[Prior art]
As described above, the variable image speed conversion technology that makes the video speed variable with high quality while maintaining the naturalness of the video is high-quality video while maintaining the naturalness of the video every moment. It has not been developed yet because it was difficult to make it convertible.
[0003]
On the other hand, it is possible to change the video speed to a variable speed by a variable speed playback function of a VTR (video tape recorder).
[0004]
[Problems to be solved by the invention]
However, in such VTR variable speed playback, etc., the same field is used in units of 1/60 second or 1/50 second (for example, HDTV, NTCS system: 1/60 second, PAL system: 1/50 second). Since it is repeatedly presented more than once or a continuous field is omitted, high-quality image speed conversion cannot be performed due to unnatural movement or image jitter in the vertical direction.
[0005]
Also, conventional variable playback such as VTR does not perform operations such as field interpolation, and due to the limitations of the drive system, it is impossible to change the video speed with high accuracy from moment to moment. It is.
[0006]
The present invention has been made in view of the above points, and its object is to follow a variable speed magnification instructed from a variable speed control unit (for example, a VTR) or a variable speed control unit of an editing machine. By appropriately determining the interpolation position of the video in units of variable length blocks that change and interpolating the video at the determined position, the speed of the video can be changed from moment to moment, thereby presenting the video uniformly. An object of the present invention is to provide a time-variable image speed conversion device capable of finely varying the speed of an image while maintaining the naturalness of the image with high quality, without changing the speed.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides a time-variable image speed that performs discrete image speed conversion in the time direction based on a variable speed magnification that is arbitrarily set every moment by an external variable speed instruction means. In the conversion device, motion detection means for detecting a motion vector of the video between the fields of the input video, and the input video in units of variable length blocks that change from time to time based on the variable speed magnification. , a predetermined formula time position for interpolating the field number below which correspond to images Shin Choryou (1), (2) and the interpolation position determining means for determining using said variable length block units Then, when the video is interpolated at the time position determined by the interpolation position determination means, the time position determined by the interpolation position determination means according to the motion vector of the video detected by the motion detection means. Field and this Field interpolation means for interpolating video synthesized from adjacent fields, wherein the interpolation position determination means is a variable length block unit, wherein the number of fields of the input video is M, and The number of fields of the output video created by the speed change magnification is N, and the fields of the input video are O1, O2,..., Oi, Oi + 1,. The video field is defined as C1, C2,..., Cj, Cj + 1,..., Cn.
[Equation 3]

I is assumed to be i _opt and the frame Cj is _assumed to be located between the input frames O _{(iopt) -1} and O _iopt ,
At this time, the interpolation position (Pj) from O _{(iopt) -1 is} calculated and given by the following equation (2).

It is characterized by that.
[0008]
Here, the variable speed instructing means may be a variable speed control unit for making the video speed variable in an apparatus having a function of recording / reproducing or editing a video .
[0013]
According to the above configuration, the present invention (television video) is adapted to a variable speed magnification (expansion / shortening code) designated from the outside such as a variable speed control unit (for example, a VTR) or a variable speed control unit of an editing machine. Since the time position to interpolate the video is appropriately determined for the discrete video in the time direction and the video is interpolated at the determined time position, the video has a slower speed than the original video, Alternatively, a high-speed video can be presented with high quality while maintaining naturalness.
[0014]
In addition, according to the present invention, when the video presentation speed is variable, the video presentation speed is changed from time to time in variable-length block units that change from time to time. Instead of changing the video presentation speed uniformly, it is possible to change the speed at fine intervals from time to time.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail based on embodiments of the present invention shown in the drawings.
[0016]
In addition, this invention may be achieved in the system which consists of a some apparatus, and may be achieved in the apparatus which consists of one apparatus. Further, it goes without saying that the present invention can also be applied when the present invention is achieved by supplying a program to a system or apparatus. For example, not only a dedicated device but also a personal computer (personal computer) can be used. In addition to the FD (floppy disk), the recording medium for recording the control procedure for realizing the image speed conversion algorithm according to the present invention may be a CD-ROM, an IC memory card, or the like. Furthermore, it is also possible to record this program in the ROM, configure it as a part of the memory map, and execute it directly by the CPU.
[0017]
FIG. 1 is a functional block diagram showing the configuration of an hourly variable image speed conversion system according to an embodiment of the present invention. Here, the image speed conversion can be applied to various color television systems such as Hi-Vision, NTSC, PAL, etc., but FIG. 1 shows the case of Hi-Vision (high-definition television, high-definition television) as a representative example. Show.
[0018]
First, an analog video (video signal) to be input is A / D converted in the A / D conversion unit 2 in the format of RGB 4: 4: 4, for example. The video signal digitized by the A / D converter 2 is sent to a motion vector detection system and an interpolation system.
[0019]
For example, the motion vector detection system includes a preprocessing unit 3, a motion detection unit 4, and a vector detection allocation unit 5. Preprocessing for motion vector detection / allocation is performed by the preprocessing unit 3 on the video signal digitized by the A / D conversion unit 2. As an example, the pre-processing unit 3 converts the video signal from interlaced 1125/60/2: 1 to non-interlaced 652/60/1: 1. The video signal preprocessed by the preprocessing unit 3 is sent to the motion detection unit 4 and the vector detection allocation unit 5.
[0020]
The motion detection unit 4 uses, for example, an iterative gradient method (maximum number of iterations: 2 times, block size: 8) using an initial displacement vector (eight candidate vectors, block size: 8 pixels, 8 lines) based on the gradient method. A motion vector between fields of a video is detected by pixels (8 lines). In order to newly interpolate a field having a temporal timing different from that of the input image, the motion vector detected by the motion detection unit 4 from the input video signal is allocated on the field to be newly interpolated by the vector detection allocation unit 5.
[0021]
A decompression / shortening code indicating a variable speed magnification indicated by an external variable speed instruction unit 1 such as a variable speed control unit (for example, a VTR) or a variable speed control unit of an editing machine is transferred to the field interpolation unit 6. . This variable speed magnification can be arbitrarily set every moment.
[0022]
The interpolation system includes a field interpolation unit 6. The field interpolation unit 6 can change the input video sent from the A / D conversion unit 2 as appropriate based on the expansion / contraction code indicating the variable speed magnification instructed from the variable speed instruction unit 1. By interpolating a field (arbitrary number of fields) at an arbitrary time position within the time length, the video speed is changed every moment. That is, when the variable speed magnification indicated by the expansion / contraction code exceeds 1/60 seconds (for one field), the field interpolation unit 6 performs an arbitrary time corresponding to the expansion amount based on the motion vector of the video. Interpolate the position field. Therefore, the field interpolation unit 6 quantizes the expansion / contraction code in 1/60 seconds (for 1 field), and the number of video fields corresponding to the original image and the video corresponding to the expansion / contraction code (variable speed magnification). And calculate the number of fields.
[0023]
The video signal subjected to the interpolation processing by the field interpolation unit 6 is output to the D / A conversion unit 7, and the video signal analogized by the D / A conversion unit 7 is output as an image speed conversion video that changes every moment. .
[0024]
FIG. 2 shows how to determine the field interpolation position within a certain variable block length by the field interpolation unit 6 of FIG.
[0025]
In an analysis section (within variable block length) of about 2/60 seconds (2 fields) to 10/60 seconds (10 fields) (the time length of each analysis section is different), the number of fields of the original picture: M Assuming that the number of fields created by the image speed conversion is N, FIG. 2A shows a case where M <= N, and FIG. 2B shows a case where M > N. The method for determining both the interpolation positions is the same, and is determined by the following procedure.
[0026]
1) The original picture fields are O1, O2,..., Oi, Oi + 1,.
[0027]
2) The fields created by the image speed conversion are C1, C2,..., Cj, Cj + 1,.
[0028]
3) For each frame Cj, the following equation (1)
[0029]
[Expression 1]

[0030]
And i _opt to i satisfying the frame Cj is considered to be located between the input frame O _{(iopt) -1} and O _iopt.
[0031]
At this time, the interpolation position (Pj) from O _{(iopt) -1 is} calculated and given by the following equation (2).
[0032]
[Expression 2]

[0033]
4) The above interpolation processing is performed for each variable block (the time length of each variable block is different), and the image speed is converted.
[0034]
FIG. 3 shows an example of image speed conversion in which the speed changes every moment in an arbitrary variable block length unit. FIG. 3 shows an example in which the original image is converted and output from 6 fields to 7 fields and then converted from 3 fields to 4 fields. As indicated by the broken line arrows in FIG. 3, for each variable block, the determined interpolation position (time position) is synthesized from the field image of the original image (input image) at the interpolation position and the field of the adjacent original image. Interpolated with video to convert image speed.
[0035]
【The invention's effect】
As described above, according to the present invention, media with video (for example, television, video / audio recording media such as VTR and DVD, moving image sound reproduction on a computer, etc.), apparatus, and medical equipment In moving image reproduction, etc. (for example, X-ray CT, NMR-CT, etc.), etc., it is arbitrarily set from the outside such as a variable speed control unit (for example, VTR) or a variable speed control unit of an editing machine. Based on the variable speed magnification, the number of fields corresponding to the amount of video expansion is interpolated with respect to discrete images in the time direction (such as television images) in variable length blocks that change from moment to moment. Since the video is interpolated at the determined time position, the video with a slower speed than the original video or the video with a higher speed is presented with high quality while maintaining the naturalness. You can.
[0036]
In addition, according to the present invention, when the video presentation speed is variable, the video presentation speed is changed from moment to moment in variable length block units that change from moment to moment, instead of changing uniformly. Instead of changing the video presentation speed in the same way, it can be finely adjusted from moment to moment.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an hourly variable image speed conversion system according to an embodiment of the present invention.
FIG. 2 is a timing chart showing an interpolation position of a video field in an embodiment of the present invention.
FIG. 3 is a timing chart showing an example of image speed conversion in which the speed changes every moment in variable block length units according to an embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Variable speed designation | designated part 2 A / D conversion part 3 Pre-processing part 4 Motion detection part 5 Vector detection allocation part 6 Field interpolation part 7 D / A conversion part

Claims (2)

In a time-variable image speed conversion device that performs image speed conversion of discrete input video in the time direction based on a variable speed magnification arbitrarily set every moment by an external variable speed instruction means,
Motion detection means for detecting a motion vector of the video between the fields of the input video;
With respect to the input image, on the basis of the variable speed ratio, in blocks of variable length which changes from moment to moment, the time position for interpolating the field number which corresponds to the image of Shin Choryou below a predetermined Interpolation position determining means for determining using equations (1) and (2);
The interpolation position is determined according to the motion vector of the video detected by the motion detection means when the video is interpolated at the time position determined by the interpolation position determination means in units of the variable length. A field interpolation means for interpolating the video synthesized from the field at the time position determined by the means and the field adjacent thereto,
The interpolation position determining means is
In a variable-length block unit, the number of fields of the input video is M, and the number of fields of the output video created by the speed variable magnification is N.
The input video fields are O1, O2,..., Oi, Oi + 1,.
The fields of the output video created by the image speed conversion are C1, C2,..., Cj, Cj + 1,.
For each frame Cj, the following equation (1)

I is assumed to be i _opt and the frame Cj is _assumed to be located between the input frames O _{(iopt) -1} and O _iopt ,
At this time, the interpolation position (Pj) from O _{(iopt) -1 is} calculated and given by the following equation (2).

A variable-speed image speed conversion device characterized by that.   2. The time-variable type image according to claim 1, wherein the variable speed instruction means is a variable speed control unit for changing a video speed in an apparatus having a function of recording / reproducing or editing a video. Speed converter.

Images