JP3080485B2 - Orthogonal transform coding method for multi-view stereoscopic video - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system applied to a system for obtaining a multi-view stereoscopic image using outputs from a plurality of cameras, and more particularly, to a method for applying a DC to an output from each camera.
The present invention relates to orthogonal transform coding systems such as T, WAVELET, Hadamard, and KL.

[0002]

2. Description of the Related Art FIG. 9 is a block diagram showing the configuration of a general multi-view stereoscopic video system.
Reference numeral 101 denotes a subject, and 102 denotes a camera for photographing the subject 101. A plurality of cameras 102 are provided, and the cameras 102 are arranged with their positions (imaging directions and mounting angles) slightly shifted vertically and horizontally. I have.

The reason why the arrangement positions of the cameras 102 are slightly shifted is that an output from one camera is used as an input to one eye to form binocular parallax so that stereoscopic vision can be obtained. . Therefore, the camera 10
The use of a large number of 2 makes it possible to use the display 10 in the output system.
Even if the person who views the display 107 changes the position at which the display 107 is viewed, a natural stereoscopic view can be obtained.

[0004] An encoder 103 encodes image data input from each camera 102, and an encoder 104 encodes image data.
A multiplexing device (multiplexing device) 105 for multiplexing the image data encoded in step 3 on the transmission path for transmission, and a demultiplexer 105 for separating the multiplexed image data input via the transmission line ( And a display unit 106 for displaying the image data separated by the demultiplexer 105 on the display 10.
7 is a decoder that performs decoding to display the image on the 7. In addition,
As the display 107, for example, lenticular
A lens (when there is parallax only in the horizontal direction) or a fly's eye lens (when there is parallax in the vertical and horizontal directions) is used.

[0005] With the above configuration, the conventional multi-view stereoscopic video system operates as follows. First, it is stationary,
Alternatively, the moving subject 101 is photographed by a plurality of cameras 102 whose arrangement position and direction are slightly shifted vertically and horizontally. Next, the image data obtained from the plurality of cameras 102 is encoded with high efficiency, and multiplexed (multiplexed) by the multiplexer 104.

[0006] Thereafter, the signal is transmitted via a transmission path or the like, and is separated (demultiplexed) by a demultiplexer 105 on the receiving side, and then decoded by a decoder 106.
The image is displayed on the display 107. This display 10
As an example of the image shown in FIG. 7, FIG. 10 shows the outputs from the five vertical and five horizontal cameras 102 when an octopus is the subject. In this example, it can be seen that there is also a parallax in the vertical direction.

As another example, FIG. 11 shows an example of a system for forming a holographic stereogram. In FIG. 11, up to the point where decoding is performed by the decoder 106, it is the same as the multi-view stereoscopic video system shown in FIG. Reference numeral 108 denotes an image size conversion unit that converts an image size as preprocessing of hologram phase calculation by the phase calculation unit 109.
A hologram phase calculation for outputting to a holographic output system 110 such as a display after an image size change is performed.

With such a configuration, the holographic stereogram described above is the same as the multi-view stereoscopic video system shown in FIG. After the decoding, the image size is converted by the image size conversion unit 108 so that the phase calculation unit 109 calculates the hologram phase. After that, the phase calculation section 109 calculates the phase, and the holographic output system 110 outputs an image.

By the way, usually, in the case of two-dimensional orthogonal transformation,
The output image (one frame) from the camera 102 is divided into blocks of N × N pixels and orthogonal transformation is performed (in the case of one-dimensional orthogonal transformation, the blocks are divided into elongated blocks of N × 1 pixels and orthogonal transformation is performed). Next, a conventional orthogonal transform coding method (orthogonal transform scan method) performed by the encoder 103 of the system shown in FIGS. 9 and 11 will be described with reference to FIG. FIG. 12 is a block diagram showing a detailed configuration of an encoder in the case of a single eye (normal television). However, the orthogonal transform encoding method performed by the encoder 103 of the multiview stereoscopic video system is also used. Since this is basically the same as the case of a single eye, the case of a single eye will be described here.

In FIG. 12, reference numeral 111 denotes an orthogonal transformer for performing orthogonal transformation on the output from the camera 102, 112 denotes a quantizer for quantizing the data after the orthogonal transformation, and 113 denotes 2
In the case of a dimension, the data after quantization is shown in FIGS.
A one-dimensional rearrangement unit 114 for performing one-dimensional conversion by means shown in (1) is a variable length encoder (VLC; Valiable) that performs variable-length coding on the one-dimensionally rearranged data.
word Length Cording).

With such a configuration, first, the camera 10
An image data signal is obtained by capturing an image of a subject by the method 2. At this time, prediction may be performed by matching with a past reproduced image or an original image obtained by locally decoding a past encoding result. One-dimensional orthogonal transformation or two-dimensional orthogonal transformation is performed by the orthogonal transformer 111 on the original signal or the prediction error signal. Types of orthogonal transform include DCT, WAVELET, Hadamard,
KL, etc., but in any orthogonal transform, as a basic property, after orthogonally transforming an image data signal,
It is known that the distribution concentrates on the low frequency region side.
Utilizing such a property, highly efficient encoding is performed as follows. In addition, using the visual characteristics that human vision is more sensitive to coding distortion in the lower frequency region,
After the orthogonal transform by the orthogonal transformer 111, a quantization matrix (weighting) for finely quantizing the low frequency region
May be applied.

After orthogonal transformation or multiplication by a quantization matrix, quantization is performed by a quantizer 112,
In the case of two dimensions, the one-dimensional rearranging unit 113 scans the data to convert it into one dimension, and then performs variable-length encoding by the variable-length encoder 114 (for VLC, for example, MPEG1 which is known as a standardized method) The two-dimensional VC is composed of a combination of a zero run length and a non-zero coefficient size).

[0013] The scanning means by the one-dimensional rearrangement section 113 corresponds to the characteristics of the image data, as shown in FIG.
A zigzag scan as shown in FIG.
A vertical scan as shown in FIG. 13 and a horizontal scan as shown in FIG. In addition, FIG.
(C) shows an example of the scan order in the case of a block of 4 × 4 pixels, and the number written in each block indicates the scan order.

The zigzag scan shown in FIG.
This is a general example employed for standardization and the like, in which scanning is performed obliquely from the low frequency side to the high frequency side. FIG.
The vertical scan shown in (b) is means for vertically scanning from the low frequency side to the high frequency side, and is effective when the input image data signal to the orthogonal transformer 111 has a strong vertical correlation. The horizontal scan shown in FIG. 13C is a means for performing a horizontal scan from the low frequency side to the high frequency side.
This is effective when the input image data signal to 1 has a strong correlation in the horizontal direction.

In the multi-view stereoscopic video system, a configuration for orthogonal transform coding as shown in FIG. 12 is provided for each camera 102 shown in FIG.
02, the same orthogonal transform coding processing as described above is performed.

[0016]

By the way, in the conventional orthogonal transform coding method of a multi-view stereoscopic video, even in the case of a multi-view type, the image data obtained by each camera 102 can be obtained in the same manner as in the case of a monocular view. Orthogonal transform, quantization, and variable length coding are performed for each signal. In the case of multi-view stereoscopic video, each camera 10
As shown in FIG. 10, the output image data from each of the cameras 2 has a feature of being similar (having a strong correlation) in many cases. Since the above-mentioned features are not used by performing coding and variable-length coding, efficient coding cannot be performed, resulting in deterioration of image quality.

The present invention has been made in view of the above-described problems, and utilizes a characteristic correlation strength between cameras in a multi-view three-dimensional image using a plurality of cameras to achieve coding efficiency and visual recognition. It is an object of the present invention to provide an orthogonal transform encoding method for multi-view stereoscopic video with improved characteristics.

[0018]

FIG. 1 is a block diagram showing the principle of the present invention. In FIG. 1, reference numeral 1 denotes an object (not shown).
The camera 1 is provided with a plurality of cameras 1. Each camera 1 is vertically and horizontally displaced in position (imaging direction, mounting angle) little by little, and outputs the output from one camera to one side. As an input to the eye, a stereoscopic image is obtained by forming binocular parallax.

Reference numeral 2 denotes a cell forming means.
With respect to the output from each camera 1, a set of pixels at the same position or a position near the corresponding position is output as one block (this block processing is called cell processing).
The positional relationship of each pixel in the block corresponds to the positional relationship of the camera 1. Reference numeral 3 denotes a pixel number switching unit. The pixel number switching unit 3 uses the correlation of the boundary between outputs from the cameras 1 as an evaluation function, and calculates the number of pixels dependent on the same camera in a block obtained by the cellizing unit 2. Is switched adaptively.

Reference numeral 4 denotes one-dimensional rearrangement means for converting two-dimensional data into one-dimensional data, and reference numeral 5 designates a two-dimensional orthogonal transform or 1 Correlation determining means for adaptively performing rearrangement for one-dimensional orthogonal transformation; when switching to the output side of one-dimensional rearrangement means for performing one-dimensional orthogonal transformation, performing two-dimensional orthogonal transformation; Is a switch that switches to a side that bypasses the one-dimensional rearranging unit 4. The above-described correlation determining unit 5 performs either two-dimensional orthogonal transformation or one-dimensional orthogonal transformation according to the correlation of the data from the celling unit 2. When the switch 6 is switched by performing the one-dimensional orthogonal transformation, the one-dimensional rearranging unit 4 is switched according to the correlation of the data from the celling unit 2.
Is used to switch the two-dimensional data scanning method.

Reference numeral 7 denotes an orthogonal transformer for orthogonally transforming the data passed through the switch 6, 8 a quantizer for quantizing the data after the orthogonal transformation, and 9 a code for the data from the quantizer 8. Encoder.

[0022]

In the multi-view stereoscopic video orthogonal transform coding system of the present invention described above, the outputs from the plurality of cameras 1 are converted into a set of pixels at the same position or a position near the corresponding position by the celling means 2. It is cellized as one block.
At this time, the positional relationship of each pixel in the block corresponds to the arrangement relationship of the camera 1, and the pixel number switching means 3 uses the correlation of the boundary between the outputs from the cameras 1 as an evaluation function in the block. The number of pixels depending on the same camera is adaptively switched.

The correlation determining means 5 determines whether to perform two-dimensional orthogonal transformation or one-dimensional orthogonal transformation in accordance with the correlation of the data from the celling means 2. Indicates that the switch 6 is switched to the side that bypasses the one-dimensional rearranging means 4, while
When performing the dimensional orthogonal transformation, the switch 6 is switched to the output side of the one-dimensional rearranging means 4 and the two-dimensional data is scanned by the one-dimensional rearranging means 4 according to the correlation of the data from the celling means 2. The system is switched. Thereafter, the data that has passed through the switch 6 is orthogonally transformed by the orthogonal transformer 7, quantized by the quantizer 8, and then encoded by the encoder 9.

[0024]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 2 is a block diagram showing an embodiment of the present invention. In FIG. 2, reference numeral 1 denotes a plurality of cameras provided for photographing a subject (not shown). (Shooting direction, mounting angle) are shifted slightly, and an output from one camera is set as an input to one eye, and a binocular parallax is formed to obtain a stereoscopic image.

Reference numeral 2A denotes a cell forming means.
Is for outputting a set of pixels in the vicinity of the same position or a corresponding position with respect to the output from each camera 1 as one block (cellized block) to perform cellization. It comprises a switching element 2a that switches and outputs the output from each camera 1 according to the cell matrix.

At this time, each pixel in the cellized block
Corresponds to the arrangement relationship of the camera 1. An example
For example, 5 cameras x 5 eyes, 25 cameras 1 are provided.
Output from camera 1 is K₁₁~ K₅₅Then,
As shown in FIG.₁₁~ K₅₅At
A pixel at one position (in FIG. 3, it is a pixel unit,
2 ^Two, 3^Two, ..., N^TwoMay be in pixel units)
Each camera 1 is switched by the switching element 2a forming the stage 2A.
Placed at a position corresponding to the placement position of
Create In FIG. 3, each camera output K₁₁~ K₅₅Upper left
Pixel KB in₁₁₁~ KB₅₅₁Brochure
An example of a lock is shown.

Reference numeral 3A denotes a pixel number switching means. The pixel number switching means 3A uses the correlation of the boundary between outputs from the cameras 1 as an evaluation function for the same camera in the block obtained by the celling means 2A. The number of dependent pixels is adaptively switched, and includes a correlation determination unit 3a and a cell matrix selection unit 3b. Here, the correlation judging unit 3a is configured as shown in FIG. 4, and according to the procedure shown in FIG. 5, the correlation between the boundaries from the outputs from the cameras 1 is used as an evaluation function in the block obtained by the celling means 2A. To determine the number of pixels depending on the same camera in the cell matrix selection unit 3b
Is for outputting a cell matrix according to the judgment result from the correlation judgment unit 3a to the cell unit 2A to operate the cell unit 2A.

The correlation judging section 3a shown in FIG. 4 is configured to correspond to the calculation operation in the judging procedure (steps A2 to A8) described with reference to FIG.
Sixteen cameras 1 with four eyes x 4
The outputs from the K ₁₁ ~K _44, as shown in step A1 of FIG. 5, for each camera output K ₁₁ ~K ₄₄ 8
× 8 pixels KB _klij (k = 1 to 4, l = 1 to 4, i =
1-8, j = 1-8). k and l are suffixes indicating that the pixel is from the camera output K _kl , and i and j indicate at which position in the 8 × 8 pixel data the camera output K _{11 to} K ₄₄ is. Subscript.

Also, in this embodiment, the types of cellization are 1 × 1 pixel, 2 × 2 pixels, 4 pixels per camera.
It is assumed that one of the three types of × 4 pixels is selected, and the cell arrangement is the same as the arrangement order of the camera 1. The reason why the pixel of the cell type is set to a power of 2 is that, in the case of the orthogonal transform (DCT), the vertical and horizontal sizes may be different, but it is desirable that each of them is a power of 2. It is. Therefore, in this case, the number of cameras 1 arranged in the vertical and horizontal directions is also set to a power of 4 × 4.

In FIG. 4, reference numerals 11a to 11c denote pixels K of the respective camera outputs K _{11 to} K ₄₄ by counting a predetermined clock.
A counter for sequentially calling B _klij , 12 is a delay circuit for delaying the called pixel KB _klij and outputting the _immediately preceding pixel data, 13 is a called pixel KB _klij and pixel data delayed by the delay circuit 12 A subtractor 14 for calculating a difference from KB _{kl + 1ij} or KB _{k + 1lij} ;
The absolute value of the difference calculated by the following equation is taken and ABS (KB _klij −
KB _{kl + 1ij} ), ABS (KB _klij −KB _{k + 1lij} ), an absolute value calculator, 15 is an adder, 16 is a delay circuit,
The adder 15 and the delay circuit 16 accumulate the absolute value of the difference from the absolute value calculator 14, and the pixel boundary between each camera output for the 1 × 1 pixel / 1 camera cell in step A2 in FIG. Evaluation function H ₁ representing the correlation of
Is calculated. Further, 17 is a divider for calculating a dividing the value of the evaluation function H ₁ is the accumulated result pixel number 8 × 8 × 4 × 3 × 2 Mean value, the average value from the divider 17 is the evaluation value H _It is intended to be used as ₁ .

Reference numeral 18 denotes a delay circuit for further delaying the pixel data from the delay circuit 12 to output pixel data two pixels before the called pixel, and 19 denotes a called pixel and a pixel delayed by the delay circuit 18. A subtractor 20 calculates the difference from the data.
9 is a selector for selectively outputting the output from "9" and "0". The ROM 21 stores in advance data relating to k, l, i, j of the pixel KB _klij located at the boundary of the cell of 2 × 2 pixels / 1 camera. When the difference between the pixels located at the boundary is output from the subtractor 19, the difference result is output from the selector 20, and otherwise, "0" is output. I have. 22 takes the absolute value of the difference from the selector 20 and _calculates ABS (KB _{klij + 1} -K
B _{kl + 1ij} ), ABS (KB _{kli + 1j} −KB _{k + 1lij} ), an absolute value calculator, 23 is an adder, 24 is a delay circuit,
The adder 23 and the delay circuit 24 accumulate the absolute value of the difference from the absolute value calculator 22, and the pixel boundary between each camera output for the 2 × 2 pixels / 1 camera cell in step A2 in FIG. Evaluation function H ₂ representing the correlation of
Is calculated. Further, 25 is a divider for calculating a dividing the value of the evaluation function H ₂ is the cumulative result by the pixel number 8 × 8 × 4 × 1 × 2 Mean value, the average value from the divider 25 is the evaluation value H _It is used as ₂ .

Further, reference numeral 26 denotes a delay circuit for further delaying the pixel data from the delay circuit 18 to output pixel data three pixels before the called pixel, and reference numeral 27 denotes a called pixel and a pixel delayed by the delay circuit 26. A subtractor 28 for calculating the difference from the data is a selector for selectively outputting the output from the subtractor 27 and "0" in accordance with an instruction from the ROM 29. The ROM 29 stores 4 × 4 pixels / 1 camera cell. Data regarding k, l, i, and j of the pixel KB _{kl ij} located at the boundary is stored in advance, and when the difference between the pixels located at the boundary is output from the subtractor 27, the difference result is output from the selector 28. It is output, and in other cases, "0" is output. 30 takes the absolute value of the difference from the selector 28 and _{calculates the} ABS (KB _klij -K
B _{kl + 1ij-3} ) and ABS (KB _klij −KB _{k + 1li-3j} ), 31 is an adder, 32 is a delay circuit, and 32 is a delay circuit. The absolute value of the difference from the value calculator 30 is accumulated, and an evaluation function H ₄ representing the correlation of the pixel boundary between the respective camera outputs for the 4 × 4 pixel / 1 camera cell in step A2 in FIG. 5 is calculated. You. Further, 33 is a divider which calculates an average value by dividing the value of the evaluation function H ₄ is the cumulative result by the pixel number 8 × 2 × 3 × 4 × 2, the average value from the divider 33 is the evaluation value H _It is used as ₄ .

Reference numeral 34 denotes a comparator for comparing the magnitudes of the evaluation values H ₁ , H ₂ , and H ₄ from the dividers 17, 25, and 33, and reference numeral 35 denotes three evaluation values H based on the comparison result by the comparator 34. A selector for selecting and outputting the minimum value among ₁ , H ₂ , and H ₄ , and a cell type (1 × 1, 2 × 2, or 4 × 4) corresponding to the selector 35 evaluation value output is
The cell type having the highest correlation between the pixels at the cell boundary is selected, and the cell-forming matrix corresponding to the cell type is selected by the cell-forming matrix selection section 3b and output to the cell-forming means 2A. I have.

In FIG. 2, reference numeral 4 denotes one-dimensional rearranging means for converting two-dimensional data into one-dimensional data, and reference numeral 5 denotes pixel data in the cell-forming block from the cell-forming means 2A before performing the orthogonal transformation. Correlation determination means for determining the correlation of the two in accordance with the procedure shown in FIG. 7 and adaptively performing rearrangement for two-dimensional orthogonal transformation or one-dimensional orthogonal transformation. When performing two-dimensional orthogonal transformation while switching to the output side of the reordering means 4, the switch is switched to the side that bypasses the one-dimensional reordering means 4, and the correlation determining means 5 determines the correlation of the data from the celling means 2A. Accordingly, it is determined whether to perform the two-dimensional orthogonal transform or the one-dimensional orthogonal transform, and the switch 6 is switched. When the one-dimensional orthogonal transform is performed, the data from the celling means 2A is output. According to the correlation of data, 8 the scan method of the two-dimensional data by the one-dimensional rearranging means 4 (a)
To (c) (the details will be described later).

Reference numeral 7 denotes an orthogonal transformer for performing orthogonal transform on the data passing through the switch 6, 8 denotes a quantizer for quantizing the data after the orthogonal transform, and 9A denotes a quantizer for the data from the quantizer 8. Variable-length encoder (V
LC). With the above-described configuration, the outputs from the plurality of cameras 1 are cellized by the celling unit 2A, with a set of pixels at the same position or a position near the corresponding position as one block. At this time, as shown in FIG. 3, the positional relationship of each pixel in the block corresponds to the arrangement relationship of the camera 1, and the pixel number switching means 3A outputs the output from each camera 1 according to the procedure shown in FIG. The number of pixels in the block that depends on the same camera is adaptively switched using the correlation between boundaries between the pixels as an evaluation function.

For example, as described above, 1 × 4 eyes × 4 eyes
It has six cameras 1 and outputs from each camera 1 are K _{11 to} K ₄₄ , respectively, and each camera output is K _{11 to} K ₄₄ as 8
× 8 pixels KB _klij (k = 1 to 4, l = 1 to 4, i =
1 to 8, j = 1 to 8) (see step A1 in FIG. 5), and one type per camera is used as a type when cellization is performed.
In order to select any one of the three types of × 1 pixel, 2 × 2 pixel, and 4 × 4 pixel, first, each correlation is determined by using the correlation determination unit 3a having the circuit configuration described with reference to FIG. For each cell type, evaluation functions H ₁ , H ₂ , and H ₄ representing the correlation of the pixel boundary between the camera outputs (the cumulative value of the absolute value of the difference between the pixels at the cell boundary) are calculated (FIG. 5). Step A2).

The operation result H₁, H_Two, H_FourEach
Dividers 17, 25, and 33 calculate the number of pixels of each cell type.
Divide to obtain an average value, and calculate the average value as the evaluation value H₁,
H_Two, H _Four(See step A3 in FIG. 5), and the comparator 3
Evaluation value H by 4₁, H_Two, H_FourCompare the size of
5 Steps A4 and A6) Evaluation value H which has become the minimum value
₁Or H_TwoOr H_FourIs selected by the selector 35.
Output (see steps A5, A7, A8 in FIG. 5).
See). Evaluation value H₁, H_Two, H_FourIs, the smaller the value,
Indicates that the correlation is high.

In this way, the cell type (1 × 1, 2 × 2, or 4 × 4) corresponding to the selector 35 evaluation value output is selected as the cell type having the highest correlation between the pixels at the cell boundary. The cell-forming matrix corresponding to the cell type is selected by the cell-forming matrix selecting section 3b and output to the cell-forming means 2A, whereby 1
In the case of a cell of × 1 pixel / 1 camera, a cell block as shown in FIG. 6A is obtained. In the case of a cell of 2 × 2 pixel / 1 camera, the cell is formed as shown in FIG. 6B. When a block is obtained and the cell is 4 × 4 pixels / 1 camera, FIG.
A cellized block as shown in (c) is obtained. Note that the matrix of these cells may be used alone without being selected.

The block of the pixel set thus cellified is generally a flat image in which the correlation of each pixel in the cellized block is high. Therefore, the encoding efficiency when performing orthogonal transformation and the like also increases. In the first place, conventionally, there has been no concept of performing cellization as described above in a multi-view stereoscopic video. After the above-described cell processing, the correlation determining means 5 determines the correlation of the data in the cellized block SB (x, y) from the cell converting means 2A according to the procedure shown in FIG. That is, x × y pixel data SB _{ij of the cellized} block SB (x, y) obtained by the cellifying means 2A
For (i = 1 to x, j = 1 to y) (see step B1 in FIG. 7), the vertical correlation V and the horizontal correlation H are calculated as shown in steps B2 and B3, respectively. The meanings of the arithmetic expressions shown in Steps B2 and B3 are horizontal,
The average value of the cumulative absolute value of the difference between the pixels adjacent in the vertical direction. The larger the value, the lower the correlation.

The thus calculated correlations V and H are compared with preset threshold values THV and THH, respectively (steps B4, B5 and B8). By step B4, V
If> THV is determined and H> THH is determined in step B5, it is determined that the correlation of the cellized block SB is low both in the vertical direction and the horizontal direction, and the switch 6 is output from the one-dimensional rearranging unit 4. Switch to side, one-dimensional sorting means 4
For example, the mosquito coil incense-like scanning method as shown in FIG. 8A is selected, the two-dimensional cellized block SB is rearranged one-dimensionally (step B6), and the orthogonal transformer 7 is used.
Performs a one-dimensional orthogonal transformation.

If V> THV is determined in step B4 and H ≦ THH is determined in step B5,
The cellized block SB determines that the correlation is high in the horizontal direction, and switches the switch 6 to the output side of the one-dimensional reordering unit 4;
For example, a horizontal scanning method as shown in FIG. 8B is selected by the one-dimensional rearranging means 4, and the two-dimensional cellized block SB is rearranged one-dimensionally (step B).
7) One-dimensional orthogonal transformation is performed by the orthogonal transformer 7.

Similarly, if V≤THV is determined in step B4 and H> THH is determined in step B8, it is determined that the cellized block SB has a high correlation in the vertical direction, and the switches 6 are arranged one-dimensionally. Switching to the output side of the reordering means 4 and the one-dimensional reordering means 4 for example, FIG.
Is selected, the two-dimensional cellized block SB is rearranged into one dimension (step B9), and the orthogonal transformer 7 performs one-dimensional orthogonal transformation.

If V ≦ THV is determined in step B4 and H ≦ THH is determined in step B8, it is determined that the cellized block SB has a high correlation both in the vertical direction and the horizontal direction. To one-dimensional sorting means 4
Is switched to the bypass side, and the cellized block SB is set to 1
It is sent to the orthogonal transformer 7 as it is without being rearranged in two dimensions, and two-dimensional orthogonal transformation is performed (step B1).
0).

After the orthogonal transform by the orthogonal transformer 7, the data is quantized by the quantizer 8 and then encoded by the encoder 9A as in the conventional case. As described above, according to the orthogonal transform encoding method of the multi-view stereoscopic video of the present embodiment, the cell from which the set of pixels at the same position or a position near the corresponding position is one block for the output from each camera 1 Since the conversion process is performed while adaptively switching the number of pixels depending on the same camera in the block using the correlation of the boundary between outputs from each camera 1 as an evaluation function, the correlation within the cellized block becomes high. In addition to improving the coding efficiency, the block distortion peculiar to the orthogonal transform can be dispersed and inconspicuous when the cellularized block is transformed into the original multi-view image, and the visual characteristics are greatly improved. improves.

In the present embodiment, after the cell processing and before the orthogonal transform, based on the correlation in the cell block,
If the correlation is high both in the vertical and horizontal directions, the two-dimensional orthogonal transformation is performed without performing one-dimensional rearrangement. On the other hand, when the correlation is low in the vertical and horizontal directions, or one of the vertical and horizontal directions On the other hand, when the correlation is high, the optimal scanning method is selected, one-dimensional rearrangement is performed, and one-dimensional orthogonal transform is performed, so that it is possible to further improve the coding efficiency.

[0046]

As described above in detail, according to the orthogonal transform coding method of the multi-view stereoscopic video of the present invention, a set of pixels at the same position or a position near the corresponding position with respect to the output from each camera is obtained. By performing orthogonal transformation and encoding as one block, the correlation in each block is increased by utilizing the characteristic correlation strength between cameras in a multi-view stereoscopic image using a plurality of cameras. Not only can the coding efficiency be improved, but also the block distortion peculiar to the orthogonal transform can be dispersed and inconspicuous when the cellularized block is transformed into the original multiview image, and the visual This has the effect of significantly improving the characteristics.

[Brief description of the drawings]

FIG. 1 is a principle block diagram of the present invention.

FIG. 2 is a block diagram showing one embodiment of the present invention.

FIG. 3 is a diagram for explaining a cell forming method in the present embodiment.

FIG. 4 is a circuit diagram illustrating a configuration of a determination unit in a pixel number switching unit according to the present embodiment.

FIG. 5 is a flowchart illustrating an operation of a determination unit in a pixel number switching unit according to the present embodiment.

FIGS. 6A to 6C are diagrams each illustrating an example of a cellized block according to the present embodiment.

FIG. 7 is a flowchart for explaining the operation of the correlation determining means of the embodiment.

FIGS. 8A to 8C are diagrams illustrating examples of a one-dimensional expansion order by the one-dimensional rearranging unit according to the present embodiment.

FIG. 9 is a block diagram illustrating a configuration of a general multiview stereoscopic video system.

FIG. 10 is a diagram illustrating an output example of a multi-view (5 × 5) camera.

FIG. 11 is a block diagram showing a configuration of a general holographic stereogram system.

FIG. 12 is a block diagram showing an encoder to which a conventional orthogonal transform coding method (orthogonal transform scan method) is applied.

FIGS. 13A to 13C are diagrams illustrating an example of a scan order by a conventional one-dimensional rearranging unit.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Camera 2, 2A Celling means 2a Switching element 3, 3A Pixel number switching means 3a Correlation judging part 3b Celling matrix selecting part 4 One-dimensional rearrangement means 5 Correlation judging means 6 Switch 7 Orthogonal transformer 8 Quantizer 9 Code 9A Variable length encoder (VLC) 11a to 11c Counter 12 Delay circuit 13 Subtractor 14 Absolute value calculator 15 Adder 16 Delay circuit 17 Divider 18 Delay circuit 19 Subtractor 20 Selector 21 ROM 22 Absolute value calculator Reference Signs List 23 adder 24 delay circuit 25 divider 26 delay circuit 27 subtractor 28 selector 29 ROM 30 absolute value calculator 31 adder 32 delay circuit 33 divider 34 comparator 35 selector

────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Kiichi Matsuda 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Limited (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N 13/00-15 / 00

Claims (5)

(57) [Claims] An apparatus for obtaining a multi-view stereoscopic image using outputs from a plurality of cameras (1), wherein a set of pixels at the same position or a position near a corresponding position is determined for each output from each camera (1). An orthogonal transform coding method for multi-view stereoscopic video, wherein orthogonal coding is performed by performing orthogonal transform as one block. 2. The orthogonal transform coding method for multi-view stereoscopic video according to claim 1, wherein the positional relationship between the pixels in the block corresponds to the positional relationship between the cameras. 3. The orthogonal transform coding method for multi-view stereoscopic video according to claim 1, wherein the number of pixels in the block that depends on the same camera is adaptively switched. 4. The block according to claim 3, wherein the number of pixels dependent on the same camera in the block is adaptively switched using the correlation of the boundary between outputs from each camera as an evaluation function. An orthogonal transform coding method for multi-view stereoscopic video. 5. The orthogonal transform code for a multi-view stereoscopic video according to claim 1, wherein a rearrangement for two-dimensional orthogonal transform or one-dimensional orthogonal transform is adaptively performed before performing the orthogonal transform. System.