CN106067966B - Video 3 dimensional format automatic testing method - Google Patents

Abstract

This invention discloses an automatic video 3D format detection method that leverages the uniqueness of the format for the same video program, performing only necessary calculations/detection for each new program. Simultaneously, based on the characteristics of vertical and horizontal parallax in 3D video, this method rapidly estimates parallax by extracting feature points from image corners and matching feature point sets, thus achieving a preliminary format determination. Further, multi-frame voting is performed in the time domain to determine the format of the current video program, and finally, the format of the current video program is output, controlling the display device to play according to the detected video format. This method significantly reduces the computational complexity of video format detection while maintaining high detection accuracy. Therefore, compared to existing technologies, this automatic video program format detection method is particularly suitable for systems with stringent requirements for power consumption and manufacturing costs, such as mobile devices (tablets, laptops).

Description

Automatic Detecting Method of Video 3D Format

technical field

The invention belongs to the technical field of video format detection, and relates to a video format detection method, in particular to a video three-dimensional format automatic detection method.

Background technique

Along with the film of three-dimensional (3D) video becomes more and more popular on the video website and 3D digital television program broadcasts, the quantity of 3D video will become explosive growth. A large category of these 3D videos is to arrange 3D content in 2D frames so as to be compatible with existing 2D display systems. According to the arrangement format of 3D subframes in 2D frames, such 3D videos are divided into the following formats:

(1) Left and right 3D format (SBS): 3D left and right views are arranged side by side in 2D frames;

(2) Top and bottom 3D format (TB): 3D left and right views are arranged in the upper and lower parts of the 2D frame;

(3) Line-interleaved 3D format (LI): Interleave the lines of the 3D left and right views so that the left and right views are arranged in a 2D frame;

(4) Column interleaved 3D format (CI): Interleave the columns of the 3D left and right views so that the left and right views are arranged in a 2D frame;

(5) Grid-interleaved 3D format (CB): stagger the 3D left and right views in a 2D frame in a grid pattern;

(6) Grid-side-by-side 3D format (CBS): 3D left and right views are arranged side by side in a 2D frame in grid mode;

(7) 2D+depth format (2D+D): 2D subframes and depth information subframes are arranged side by side in a 2D frame.

Among them, the left-right 3D format and the top-bottom 3D format are the two most common 3D formats. Meanwhile, 2D video content is here to stay (and always will). How to automatically detect whether the video program format is 2D or 3D, and if it is a 3D video format, which 3D format it belongs to is indispensable for correct display.

At the same time, according to the existing practice of video program production, the format of each program is determined and unique. Either 2D video, or 3D video in side-to-side format, or 3D video in top-bottom format. There is no such thing as a program that contains multiple video formats.

At the same time, in order to ensure viewing comfort, in addition to requiring a high degree of consistency in the brightness/chromaticity of the left and right views when producing 3D video content, only horizontal parallax is allowed, and the horizontal parallax must be controlled within a certain range. The left and right image pairs can be fused into a single stereoscopic image; otherwise, the human eye and the brain cannot coordinate the relationship between the depth information provided by various physiological stereoscopic factors such as lens adjustment, binocular convergence, and binocular vision difference. The images are fused into a single stereoscopic image, and the result is a great deal of discomfort for the viewer.

In view of this, there is an urgent need to design a new 3D video format detection method in order to overcome the above-mentioned defects in the existing format detection methods.

Contents of the invention

The technical problem to be solved by the present invention is to provide an automatic detection method for a video three-dimensional format, which can improve the detection accuracy and reduce the computational complexity of video format detection.

In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

A method for automatic detection of a video three-dimensional format, characterized in that the detection method comprises the following steps:

Step S1. When a new video program starts, the first counter and the second counter are cleared;

Step S2. Obtain a frame of image, and divide the input image into views; format according to left and right 3D, the first left view is on the left, and the first right view is on the right; according to up and down 3D format, the second left view is on the top, and the second The right view is below;

Step S3. Detect the feature points of each sub-region, and use the corner points in the image as feature points for detection. The corner point detection method adopts mature SUSAN or Harris algorithm; obtain the first left view area, the first right view area, the second Corner sets C11, C12, C21, C22 of the left view area and the second right view area respectively;

Step S4. Approximate the set of corner points of the whole picture with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if so, it is considered to be a valid frame, and step S5 is executed; otherwise, step S2 is executed;

Step S5. Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs;

Step S6. According to the matching of the corner point set C11 and the C12 feature point, the vertical disparity Disparity_1v and Disparity_1h are counted, and according to the C21 and C22 feature point matching, the vertical disparity Disparity_2v and Disparity_2h are counted;

Step S7. If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value plus 1; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value plus 1 ;

Step S8. Repeat steps S2 to S7 until the determination step S7 is executed m times;

Step S9. If the value of the first counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a left-right 3D video format;

Step S10. If the value of the second counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a top-bottom 3D video format;

Step S11. If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format.

A method for automatically detecting a video three-dimensional format, the detection method comprising the steps of:

Step S1. When a new video program starts, the first counter and the second counter are cleared;

Step S2. Obtain a frame of image, and divide the input image into views; format according to left and right 3D, the first left view is on the left, and the first right view is on the right; according to up and down 3D format, the second left view is on the top, and the second The right view is below;

Step S3. Detect the feature points of each sub-area, use the corner points in the image as feature points for detection, and obtain the respective corners of the first left view area, the first right view area, the second left view area, and the second right view area Point set C11, C12, C21, C22;

Step S4. Approximate the set of corner points of the whole picture with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if so, it is considered to be a valid frame, and step S5 is executed; otherwise, step S2 is executed;

Step S5. Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs;

Step S6. According to the matching of the corner point set C11 and the C12 feature point, the vertical disparity Disparity_1v and Disparity_1h are counted, and according to the C21 and C22 feature point matching, the vertical disparity Disparity_2v and Disparity_2h are counted;

Step S7. If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value increases; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value increases;

Step S8. Repeat steps S2 to S7 until the determination step S7 has been executed for a set number of times;

Step S9. If the value of the first counter is greater than or equal to the set value, the detected current video program format is a left-right 3D video format;

Step S10. If the value of the second counter is greater than or equal to the set value, the detected current video program format is a top-bottom 3D video format;

Step S11. If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format.

As a preferred solution of the present invention, in step S3, the corner point detection method adopts mature SUSAN or Harris algorithm.

As a preferred solution of the present invention, in the step S7, if Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value plus 1; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than Horizontal parallax threshold Thh, then the second counter value plus 1;

In the step S8, steps S2-S7 are repeated until the determination step S7 is executed m times;

In the step S9, if the value of the first counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a left-right 3D video format;

In the step S10, if the value of the second counter is greater than or equal to k*m,(k∈(0.5,1]), the detected current video program format is a top-bottom 3D video format.

The beneficial effect of the present invention is that: the method for automatically detecting the video three-dimensional format proposed by the present invention first utilizes the fact that the format of the same video program is unique, and only performs necessary calculation/detection for each new program.

At the same time, according to the characteristics of 3D video vertical disparity and horizontal disparity, the disparity is quickly estimated by extracting the feature points of the image corners and the matching of feature point sets, so as to realize the preliminary judgment of the format; again, multi-frame voting discrimination is performed in the time domain , and finally output the format of the current video program, and control the display device to play according to the detected video format. Using this method greatly reduces the computational complexity of video format detection under the premise of ensuring high detection accuracy. Compared with the prior art, the method for automatically detecting the video program format of the present invention is especially suitable for systems such as mobile devices (tablet computers and notebook computers) that have very strict requirements on power consumption and manufacturing cost.

According to the present invention, the parallax estimation based on the left and right views can be completed based on the feature point matching of the image. Since the left and right views are samples of the same scene at the same time, and the imaging conditions are highly consistent, the feature points detected by the left and right views are also highly consistent. This is the same as Normal 2D video has significant differences. Therefore, the horizontal disparity and vertical disparity can be estimated through the left and right view feature point sets, and the 3D/2D format of the video can be detected.

Description of drawings

FIG. 1 is a flow chart of the method for automatically detecting a three-dimensional format of a video according to the present invention.

Detailed ways

Preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

Embodiment one

Please refer to Fig. 1, the present invention discloses a kind of video three-dimensional format automatic detection method, described detection method comprises the following steps:

[Step S1] A new video program starts, and the first counter and the second counter are cleared;

[Step S2] Obtain a frame of image, divide the input image into views; format according to left and right 3D, the first left view is on the left, and the first right view is on the right; format according to up and down 3D, the second left view is on the top, the second Two right view below;

[Step S3] Detect the feature points of each sub-region, using the corner points in the image as feature points, and the corner point detection method adopts the mature SUSAN or Harris algorithm; get the first left view area, the first right view area, the second left view area Corner sets C11, C12, C21, C22 of the view area and the second right view area respectively;

[Step S4] Approximate the set of corner points of the whole image with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if it is considered to be a valid frame, execute step S5; otherwise, execute step S2;

[Step S5] Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs;

[Step S6] Statistical vertical disparity Disparity_1v and Disparity_1h according to the matching pair of corner point set C11 and C12 feature point, and statistical vertical disparity Disparity_2v and Disparity_2h according to the matching pair of C21 and C22 feature point;

[Step S7] If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value is added by 1; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value is added 1;

【Step S8】Repeat steps S2～S7 until the determination step S7 is executed m times;

[Step S9] If the value of the first counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a left-right 3D video format;

[Step S10] If the value of the second counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a top-bottom 3D video format;

[Step S11] If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format.

Embodiment two

A method for automatically detecting a video three-dimensional format, the detection method comprising the steps of:

Step S1. When a new video program starts, the first counter and the second counter are cleared;

Step S2. Obtain a frame of image, and divide the input image into views; format according to left and right 3D, the first left view (11) is on the left, and the first right view (12) is on the right; format according to up and down 3D, the second left The view (21) is on the top, and the second right view (22) is on the bottom;

Step S3. Detect the feature points of each sub-area, use the corner points in the image as feature points for detection, and obtain the respective corners of the first left view area, the first right view area, the second left view area, and the second right view area Point set C11, C12, C21, C22;

Step S4. Approximate the set of corner points of the whole picture with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if so, it is considered to be a valid frame, and step S5 is executed; otherwise, step S2 is executed;

Step S5. Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs;

Step S6. According to the matching of the corner point set C11 and the C12 feature point, the vertical disparity Disparity_1v and Disparity_1h are counted, and according to the C21 and C22 feature point matching, the vertical disparity Disparity_2v and Disparity_2h are counted;

Step S7. If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value increases; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value increases;

Step S8. Repeat steps S2 to S7 until the determination step S7 has been executed for a set number of times;

Step S9. If the value of the first counter is greater than or equal to the set value, the detected current video program format is a left-right 3D video format;

Step S10. If the value of the second counter is greater than or equal to the set value, the detected current video program format is a top-bottom 3D video format;

Step S11. If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format.

To sum up, the automatic three-dimensional video format detection method proposed by the present invention first utilizes the fact that the format of the same video program is unique, and only performs necessary calculation/detection for each new program.

At the same time, according to the characteristics of 3D video vertical disparity and horizontal disparity, the disparity is quickly estimated by extracting the feature points of the image corners and the matching of feature point sets, so as to realize the preliminary judgment of the format; again, multi-frame voting discrimination is performed in the time domain , and finally output the format of the current video program, and control the display device to play according to the detected video format. Using this method greatly reduces the computational complexity of video format detection under the premise of ensuring high detection accuracy. Compared with the prior art, the method for automatically detecting the video program format of the present invention is especially suitable for systems such as mobile devices (tablet computers and notebook computers) that have very strict requirements on power consumption and manufacturing cost.

The description and application of the invention herein is illustrative and is not intended to limit the scope of the invention to the above-described embodiments. Variations and changes to the embodiments disclosed herein are possible, and substitutions and equivalents for various components of the embodiments are known to those of ordinary skill in the art. It should be clear to those skilled in the art that the present invention can be realized in other forms, structures, arrangements, proportions, and with other components, materials and components without departing from the spirit or essential characteristics of the present invention. Other modifications and changes may be made to the embodiments disclosed herein without departing from the scope and spirit of the invention.

Claims (4)

1. a video three-dimensional format automatic detection method, is characterized in that, described detection method comprises the steps: Step S1. When a new video program starts, the first counter and the second counter are cleared; Step S2. Obtain a frame of image, and divide the input image into views; format according to left and right 3D, the first left view (11) is on the left, and the first right view (12) is on the right; format according to up and down 3D, the second left The view (21) is on the top, and the second right view (22) is on the bottom; Step S3. Detect the feature points of each sub-region, and use the corner points in the image as feature points for detection. The corner point detection method adopts mature SUSAN or Harris algorithm; obtain the first left view area, the first right view area, the second Corner sets C11, C12, C21, C22 of the left view area and the second right view area respectively; Step S4. Approximate the set of corner points of the whole picture with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if so, it is considered to be a valid frame, and step S5 is executed; otherwise, step S2 is executed; Step S5. Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs; Step S6. According to the matching of the corner point set C11 and the C12 feature point, the vertical disparity Disparity_1v and the horizontal disparity Disparity_1h are counted, and according to the C21 and C22 feature point matching, the vertical disparity Disparity_2v and the horizontal disparity Disparity_2h are counted; Step S7. If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value plus 1; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value plus 1 ; Step S8. Repeat steps S2 to S7 until the determination step S7 is executed m times; Step S9. If the value of the first counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a left-right 3D video format; Step S10. If the value of the second counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a top-bottom 3D video format; Step S11. If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format. 2. A video three-dimensional format automatic detection method, is characterized in that, described detection method comprises the steps: Step S1. When a new video program starts, the first counter and the second counter are cleared; Step S2. Obtain a frame of image, and divide the input image into views; format according to left and right 3D, the first left view (11) is on the left, and the first right view (12) is on the right; format according to up and down 3D, the second left The view (21) is on the top, and the second right view (22) is on the bottom; Step S3. Detect the feature points of each sub-area, use the corner points in the image as feature points for detection, and obtain the respective corners of the first left view area, the first right view area, the second left view area, and the second right view area Point set C11, C12, C21, C22; Step S4. Approximate the set of corner points of the whole picture with the set {C11, C12}, and judge whether the number of corner points in the set is greater than the first preset threshold; if so, it is considered to be a valid frame, and step S5 is executed; otherwise, step S2 is executed; Step S5. Perform feature point matching on the corner point sets C11 and C12, and on the corner point sets C21 and C22; the matching principle is based on the principle of minimum vertical parallax of feature point matching pairs; Step S6. According to the matching of the corner point set C11 and the C12 feature point, the vertical disparity Disparity_1v and the horizontal disparity Disparity_1h are counted, and according to the C21 and C22 feature point matching, the vertical disparity Disparity_2v and the horizontal disparity Disparity_2h are counted; Step S7. If Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value increases; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value increases; Step S8. Repeat steps S2 to S7 until the determination step S7 has been executed for a set number of times; Step S9. If the value of the first counter is greater than or equal to the set value, the detected current video program format is a left-right 3D video format; Step S10. If the value of the second counter is greater than or equal to the set value, the detected current video program format is a top-bottom 3D video format; Step S11. If the values of the above two counters do not meet the conditions, the detected current video program is in 2D video format. 3. The video three-dimensional format automatic detection method according to claim 2, characterized in that: In the step S3, the corner detection method adopts mature SUSAN or Harris algorithm. 4. The video three-dimensional format automatic detection method according to claim 2, characterized in that: In the step S7, if Disparity_1v is less than the vertical parallax threshold Thv, and Disparity_1h is less than the horizontal parallax threshold Thh, then the first counter value plus 1; if Disparity_2v is less than the vertical parallax threshold Thv, and Disparity_2h is less than the horizontal parallax threshold Thh, then the second counter value plus 1; In the step S8, steps S2-S7 are repeated until the determination step S7 is executed m times; In the step S9, if the value of the first counter is greater than or equal to k*m, (k∈(0.5,1]), the detected current video program format is a left-right 3D video format; In the step S10, if the value of the second counter is greater than or equal to k*m,(k∈(0.5,1]), the detected current video program format is a top-bottom 3D video format.