JP4183499B2 - Video file processing method and video processing method - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to a video file processing method and a video processing method.
[0002]
[Prior art]
Conventionally, as a stereoscopic technique, a stereoscopic method without glasses using a parallax barrier, a stereoscopic method with glasses using polarized glasses, liquid crystal shutter glasses, and the like are known. In addition, as a stereoscopic image, there is not only a live-action image but also a 3D drawing image in which an object placed in a virtual space is projected onto a plane and drawn using computer graphics. Furthermore, a right-eye image and a left-eye image can be created by performing the drawing process from two viewpoints. In addition, a stereoscopic video receiving apparatus and a stereoscopic video system that generate a stereoscopic video based on depth information extracted from a two-dimensional video signal and the two-dimensional video signal have been proposed (see Patent Document 1). If a video file composed of two-dimensional video and depth information is created, a stereoscopic video can be generated when this file is opened. In addition, a method has been proposed in which two videos are broadcast as one-channel video and can be stereoscopically viewed on the receiver side (see Patent Document 2). If a video file composed of two videos is created, a stereoscopic video can be generated when the file is opened.
[0003]
[Patent Document 1]
JP 2000-78611 A [Patent Document 1]
Japanese Patent Laid-Open No. 10-174064
[Problems to be solved by the invention]
By the way, in such a video file, it is desired that when the file is opened, the video is displayed reflecting the intention of the creator.
[0005]
In view of the above circumstances, an object of the present invention is to provide a video file processing method capable of displaying a video reflecting the creator's intention when the video file is opened. Provided is a video processing method capable of reducing the processing load of drawing processing.
[0006]
[Means for Solving the Problems]
The video file processing method according to the present invention is a video file processing method for generating video based on a video video file, a still video video file, or a computer graphics video file in order to solve the above-mentioned problem. If the control information indicating whether the video to be displayed is a stereoscopic video or a planar video is recorded in the file, and the control information is detected, the control information is based on the control information. And generating a stereoscopic video or a planar video.
[0007]
With the above configuration, the creator of the video file, for example, if the initial video is a planar video and the video file is created with the intention of displaying a stereoscopic video by the viewer's operation, the file is opened. It is only necessary to write control information indicating that the video to be displayed as a planar view video is written in the file, and when the device receiving the file executes the video file processing method, the file is opened. It is possible to display the initial image when it is displayed as a planar view image.
[0008]
The video file processing method of the present invention is a video file processing method for generating a video based on a moving picture video file. Whether the video file processing method is a stereoscopic video or a planar video for each frame or scene change. It is detected whether or not the control information to be recorded is recorded in a file, and when the control information is detected, a stereoscopic video or a planar video is generated based on the control information.
[0009]
With the above configuration, the creator of the video file, for example, when creating a video file with the intention of displaying a video with intense motion as a planar view video, for each frame or scene switching of the video with intense motion. It is only necessary to write control information indicating that the image is to be a planar view image every time, and the device on the side of receiving this file executes the above-mentioned image file processing method. The image can be displayed as a planar view image.
[0010]
The video file processing method of the present invention is a video file processing method for generating a video based on a moving image video file, a still image video file, or a computer graphics video file, which is displayed as a stereoscopic video or a planar video. Whether or not control information indicating the maintenance time is recorded in the file is detected, and when the control information is detected, a stereoscopic video or a planar video is generated based on the control information.
[0011]
With the above configuration, the creator of the video file, for example, creates a video file with the intention of displaying a stereoscopic video image for the first 3 seconds after the file is opened and then displaying a stereoscopic video image. In this case, it is only necessary to write in the file control information indicating that the image is to be a plane view image for 3 seconds after the file is opened, and the device that receives this file executes the above-mentioned image file processing method. Thus, the video can be displayed as a planar video for only the first 3 seconds when the file is opened. It is also possible to repeatedly display a planar video and a stereoscopic video at predetermined time intervals.
[0012]
In the video file processing method described above, when the control information recorded in the file is changed by a user operation, the changed control information may be recorded in the file. . According to this, editing reflecting the user's preference can be performed.
[0013]
The video file processing method according to the present invention is a video file processing method for generating video based on a video video file, a still video video file, or a computer graphics video file, wherein the playback video display of the file is a stereoscopic video. State information indicating whether the state is a state or a state of a planar view video is stored in the file itself or in another management file. Having the status information makes it easy to manage video files (whether the file playback video is displayed as a stereoscopic video or the file playback video is displayed as a planar video). .
[0014]
In the above video file processing method, when reproducing a file, the state information may be read, and a stereoscopic video or a planar video may be generated based on the status information. According to this, when the file is closed and then opened again, the previous state can be understood. For example, if the previous time was displaying the playback video of the file as a stereoscopic video, Even in the video playback of the file this time, the playback video of the file can be displayed as a stereoscopic video.
[0015]
Further, the video processing method of the present invention detects whether or not control information indicating whether the object constituting the video is a stereoscopic video or a planar video is recorded, and the control information is detected. In this case, for an object having control information indicating a stereoscopic video image, a rendering process is performed from two directions with different viewpoint positions to obtain a stereoscopic video image. A drawing process from one direction is executed.
[0016]
The video processing method of the present invention is a video file processing method for generating a video based on a moving image or a still image video file, wherein the depth position of each object constituting the video is detected, and the depth position is determined. For an object on the near side of the predetermined position, drawing processing from two directions with different viewpoint positions is executed to obtain a stereoscopic image, and one object is obtained for the object whose depth position is on the far side of the predetermined position. A drawing process from a direction is executed.
[0017]
With these video processing methods, the processing load is reduced and the drawing speed can be improved as compared with the case where drawing processing from two directions with different viewpoint positions is executed for all objects constituting the video.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
The video file processing method of the present invention will be described below with reference to FIGS.
[0019]
First, a video file processing method according to an embodiment of the present invention will be described with reference to FIG. In this figure, a description will be given of a system including a transmission side device 1 configured as a server on the Internet 2 and a reception side device 3 such as a personal computer or a mobile phone having a network connection environment.
[0020]
On the recording medium (not shown) of the transmission side device 1, a video file of a moving image or a still image (for example, a two-dimensional stereoscopic image including a two-dimensional image and depth information, or a left-eye image and a right-eye image). Existing. If this video file is an MPEG file, an identifier indicating it is given, and if it is a motion JPEG file, an identifier showing it is given. It can also be a video file in an HTML file.
[0021]
For example, the reception-side device 3 is provided with a liquid crystal barrier at a position in front of the screen, so that both a stereoscopic video image and a stereoscopic video image can be displayed. If the stereoscopic image is, for example, a right-eye image and a left-eye image that are alternately arranged in a vertical stripe shape, the vertical stripe-shaped light-shielding region in the liquid crystal barrier is controlled by a control unit (CPU) (not shown). Is formed. In addition, if a stereoscopic video is to be displayed in a partial area on the screen (a window portion for file playback or a partial video portion in an HTML file), the control unit may cause the window or one of the images to be displayed. Based on the display coordinates and size of the partial video portion, the size and position of the vertical stripe-shaped light shielding region are controlled. Further, the receiving side device 3 is equipped with MPEG player software, browser software, and the like, and can open a video file and display a video on the screen. Note that the present invention is not limited to the stereoscopic video in which the right eye video and the left eye video are alternately arranged in a vertical stripe shape, and may be, for example, a stereoscopic video in which the right eye video and the left eye video are diagonally arranged (Japanese Patent No. 3096613). In this case, an oblique barrier is formed in the liquid crystal barrier.
[0022]
The receiving side device 3 executes the processing shown in the flowchart of FIG. 2 when displaying the video on the screen based on the video file. Although this flowchart shows processing for a moving image file, similar processing may be performed for a still image file. First, the control unit of the receiving side device 3 detects whether or not control information exists in the video file (step S1). Here, it is assumed that the control information includes the following first control information, second control information, and third control information. The control information may be stored in the header portion of the video file, and any additional information may be used as long as the control information can be recognized when the receiving side device 3 that has received the video file opens the file. You can do it.
[0023]
First control information: indicates whether a video to be displayed when a file is opened is a stereoscopic video or a planar video.
Second control information: indicates whether the image is a stereoscopic image or a planar image for each frame or scene change.
Third control information: Indicates a display maintenance time for stereoscopic video or planar video.
[0024]
When control information does not exist in the video file (NO in step S1), normal file reproduction processing is executed (step S23). On the other hand, if the control information exists in the video file (YES in step S1) and this control information is the first control information, it is determined whether the initial video is a stereoscopic video based on the first control information. It is determined whether or not the video is to be viewed (step S2). In the case of a stereoscopic video, a stereoscopic video display process is executed (step S3), and in the case of a planar video, a flat video display process is executed (step S6). Then, in the case of a stereoscopic video, it is determined whether or not a planar instruction has been given by the user (step S4), and in the case of a planar video, it is determined whether or not a stereoscopic instruction has been given by the user (step S7). For example, in the case where “stereoscopic” and “planar” buttons are provided as operation buttons of the MPEG player, the user can perform a plane instruction and a stereoscopic instruction by operating the buttons by a mouse operation or the like.
[0025]
When the user gives a plane instruction, a plane display process is performed (step S6), and when the user gives a stereoscopic instruction, a stereoscopic display process is performed (step S3). Usually, the viewer's impression is higher when switching from the state of flat image display to stereoscopic image display. When the plane instruction is not given by the user or when the stereoscopic instruction is not given, it is determined whether or not the reproduction of all the frames has been completed (steps S5 and S8). If all the frames have not been completed, the process proceeds to step S3 or step S6. On the other hand, if the processing is completed, final state information (whether it was flat display or stereoscopic display) is acquired (step S9) and recorded in a file (step S10). As described above, by storing the final state information in the file, it becomes easy to manage the video file (setting when the file is opened next time). If state information at the time of video display is held, it is possible to manage whether a playback video display of a file is displayed as a stereoscopic video or a playback video display of a file as a planar video, and a plurality of images can be managed. Is displayed and it is useful for managing the display state when a plane and a solid display are mixed. This is especially convenient when there are multiple images in the same window. The status information may be managed in a separate file, but if it is stored in individual files, as described above, the last final status can be known when opening the file, and it is originally given to the file. Processing such as giving priority to status information over control information becomes possible.
[0026]
If the control information exists in the video file (YES in step S1) and this control information is the second control information, whether the video is a stereoscopic video or a planar video based on the second control information. Is determined (step S11). In the case of a stereoscopic video, a stereoscopic video display process is executed (step S12), and in the case of a planar video, a flat video display process is executed (step S13). Then, it is determined whether or not all frames have been reproduced (step S14). If all the frames have not been completed, the process proceeds to step S11. As described above, the second control information is information indicating whether a stereoscopic video or a planar video is used for each frame or scene change. For example, if the creator of the video file creates a video file with the intention of displaying a video with a lot of movement as a flat-view video, the video file is a plane-view video for every frame of a video with a high movement or every scene change. It is sufficient to write the second control information indicating that the image is to be written into the file, and the receiving device 3 that has received the file executes the above-described processing, so that the image is displayed as a planar view image at the portion of the image where the motion is intense. be able to. In addition, when adding 2nd control information for every flame | frame, what is necessary is just to add 2nd control information to the header etc. of each flame | frame. In addition, when the second control information is added for each scene change, the second control information may be added to the header of the scene change frame. In creating such a video file, for example, the degree of similarity (similarity) of the video between frames is calculated, and if the degree of approximation is above a predetermined reference value (threshold), it is determined that the scenes are the same, and approximation When the degree is below a predetermined reference value (threshold value), a method of determining that the scene has been switched can be adopted. Further, for example, on a video editing apparatus (not shown), a person detects a scene change while viewing the video and operates a stereoscopic instruction button or a plane instruction button, and the video editing apparatus adds the control information based on this operation. It is good as well.
[0027]
If control information exists in the video file (YES in step S1) and this control information is the third control information, a timer is started to execute processing based on the time indicated by the third control information. (Step S15) and it is determined whether the image is a stereoscopic image or a planar image (Step S16). In the case of a stereoscopic video, a stereoscopic video display process is executed (step S17), and in the case of a planar video, a flat video display process is executed (step S20). Then, it is determined whether or not the time indicated by the third control information has elapsed (steps S18 and S21). If the designated time is reached in the stereoscopic video display state (YES in step S18), the planar video display processing is executed (step S20), and if the designated time is reached in the planar video display state (YES in step S21). Then, a stereoscopic video display process is executed (step S17). If all the frames are not finished in the stereoscopic video display state (NO in step S19), the process proceeds to step S17. If all the frames are not finished in the planar video display state (NO in step S22), the process proceeds to step S20. As described above, the third control information is information indicating a display maintenance time for a stereoscopic video or a planar video. The creator of a video file, for example, when creating a video file with the intention of displaying a stereoscopic video for the first 3 seconds after opening the file and then displaying a stereoscopic video, the file is opened. It is only necessary to write third control information indicating that the image is to be a planar view image for 3 seconds from the beginning of the file, and when the device receiving this file executes the above-mentioned image file processing method, the file is opened. The image can be displayed as a planar view image only for the first 3 seconds. It is also possible to repeatedly display a planar video and a stereoscopic video at predetermined time intervals. For example, if the video of a product repeats a planar video and a stereoscopic video at predetermined time intervals on an online shopping HTML screen, the viewer's eyes are easily caught. Note that when the video position of a product to be stereoscopically viewed is shifted by the scroll operation, the stripe formation region in the liquid crystal barrier is also shifted corresponding to the video position of the product. In addition, the stripe area in the liquid crystal barrier is turned on / off in synchronization with the repetition of the stereoscopic video and the planar video.
[0028]
Further, for example, when a video file is played back on an MPEG player, an arbitrary frame can be displayed by performing a stop operation. When a “stereoscopic” button is operated as an operation button in this display state, control information “solid” can be associated with the frame being displayed. That is, the control information is recorded in the header of the frame. The MPEG player performs stereoscopic video display in the video display after the frame to which the control information “stereoscopic” is attached. By such processing, it is possible to create (edit) a video file in which the user side is set to stereoscopically view a desired portion.
[0029]
FIG. 3 is an explanatory diagram showing a state in which objects A, B, and C are arranged in the virtual coordinate space by computer graphics processing. Each object is a collection of polygon images. When the virtual space is displayed as a video, a drawing process for projecting the object onto a plane corresponding to the state seen from the line-of-sight direction D1 is executed, and a two-dimensional image obtained by the drawing process is displayed on the screen. Become. Further, when the virtual space is stereoscopically viewed, a drawing process for projecting the object onto a plane corresponding to the state viewed from the line-of-sight direction D2 is executed, and a two-dimensional image obtained by the drawing process is displayed on the right side. A stereoscopic video display may be performed using an eye image and a two-dimensional image based on the line-of-sight direction D1 as a left eye image. For example, in a video display device configured as a game device, an object to be arranged in a space to be displayed based on a command from a game operation unit (joystick, etc.) is stored from a storage unit (CD-ROM, ROM cassette, etc.). Reading and drawing processing are executed by the drawing processor, and a video signal is generated by the video signal generator.
[0030]
Control information indicating whether the object is a stereoscopic video or a planar video is associated with the objects A, B, and C. The video display device determines whether the video is a stereoscopic video or a planar video based on the control information added to the object read from the storage unit. Here, it is assumed that control information for stereoscopic video is added to the objects A and B, and control information for planar video is added to the object C. When the video display apparatus stereoscopically views the virtual space, first, the video display apparatus executes a drawing process for projecting the objects A, B, and C onto a two-dimensional plane in correspondence with the state viewed from the line-of-sight direction D1. Then, a drawing process for projecting the objects A and B onto a two-dimensional plane is executed in correspondence with the state viewed from the line-of-sight direction D2. For the object C in the video viewed from the line-of-sight direction D2, an image obtained by simply moving the image of the object C obtained by the process in the line-of-sight direction D1 by several pixels may be used. For example, on the game software creator side, control information to be viewed in plane view is added to objects that do not have a problem even if they are displayed in a plane, such as the background, or objects that will be displayed far away. Will do. Thereby, the drawing processing speed is improved.
[0031]
Even when the control information is not added to the object read from the storage unit, the video display device acquires the Z value information (depth information) of the object, and whether the Z value information is behind the predetermined threshold (Z value). Whether the object is a stereoscopic video or a planar video can be determined depending on whether the object is on the near side. That is, when stereoscopically viewing the virtual space, the video display device first executes a drawing process for projecting the objects A, B, and C onto a two-dimensional plane in correspondence with the state viewed from the line-of-sight direction D1. Then, when projecting the objects A and BC on the two-dimensional plane in correspondence with the state viewed from the line-of-sight direction D2, based on the respective Z value information, the object whose Z value information is on the near side of a predetermined threshold value. Only perform the drawing process. For an object that is not subject to processing in the line-of-sight direction D2 drawing process, an object image obtained by the line-of-sight direction D1 process may be simply moved horizontally by several pixels. In the above processing, status information indicating whether each object is processed as a stereoscopic video or a planar video may be held for each object. An object that flies from the front position of the screen toward the back of the screen is processed as a stereoscopic video until halfway, and is processed as a planar video after that, and the state information changes.
[0032]
In the drawing process, the object C may be completely or partially hidden by another object in the drawing from the line-of-sight direction D1, and the hidden part may be drawn in the drawing from the line-of-sight direction D2. . Therefore, the following processing may be performed.
[0033]
Other drawing processing (part 1)
{Circle around (1)} Only an object that is a plane view image or an image whose Z value information is behind the predetermined threshold (hereinafter referred to as a plane view image or the like) is first drawn. By this drawing, the entire drawing data of the object is obtained (there is no hidden part). The drawing line-of-sight direction may be D1 or D2. Further, the line-of-sight direction may be set in the middle of D1 and D2.
{Circle around (2)} An object other than a planar view image or the like is drawn in the line-of-sight direction D1 and the line-of-sight direction D2 to obtain two images.
{Circle around (3)} The drawing data of the object to be the above-mentioned planar view video is synthesized with each video. This drawing data is arranged behind an object other than a planar view video or the like. In addition, in the two videos, the arrangement positions of the objects that are the planar video and the like are shifted. In other words, the parallax is given by shifting horizontally.
[0034]
Other drawing processing (part 2)
{Circle around (1)} Only an object that becomes a planar view image or the like is first drawn (same as above).
{Circle around (2)} This drawing result is arranged as a flat background image, and an object other than a planar view image is arranged before that.
{Circle around (3)} In this state, drawing processing in the line-of-sight direction D1 and the line-of-sight direction D2 is performed.
[0035]
In any drawing processing, drawing processing from two directions with different viewpoint positions is executed for an object other than a planar video and the like, and drawing processing from one direction is executed for an object that is a planar video and the like. To do.
[0036]
FIG. 4 is an explanatory diagram showing an example of calculating the “shift amount”. A typical distance of an object such as a planar image is L, a range in which the angle of view in the line-of-sight direction D1 is desired (the number of horizontal pixels in the entire image) is W, and a distance between the optical axes in the line-of-sight direction D1 and the line-of-sight direction D2 is P. It represents as. When the number of horizontal pixels of a drawn image of an object such as a planar video is N, the shift ratio is P / W, and the shift amount (number of pixels) is N × P / W.
[0037]
【The invention's effect】
As described above, according to the present invention, it is possible to display a video reflecting the creator's intention when the video file is opened. In addition, the drawing processing speed can be improved.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a system in which a video file processing method according to an embodiment of the present invention is used.
FIG. 2 is a flowchart showing a video file processing method according to the embodiment of the present invention.
FIG. 3 is an explanatory diagram showing objects in a virtual space.
FIG. 4 is an explanatory diagram showing a calculation example of a shift amount.
[Explanation of symbols]
1 Sending device 2 Internet 3 Receiving device

Claims (3)

  A video file processing method for generating a video based on a video file having stereoscopic information for stereoscopic display of the video, wherein control information indicating a display maintaining time for stereoscopic video or planar video is the video file A video file processing method comprising: generating a stereoscopic video or a planar video based on the control information when the control information is detected by detecting whether the video is recorded on the video.   Control information indicating stereoscopic video when control information indicating whether to make stereoscopic video or planar video is recorded for each object constituting the video is detected. Execute drawing processing from two directions with different viewpoint positions for an object with 3D images, and execute drawing processing from one direction for objects with control information indicating a planar video. And a video processing method.   The depth position of each object constituting the video is detected, and rendering processing from two directions with different viewpoint positions is performed for an object whose depth position is in front of the predetermined position in order to obtain a stereoscopic video. A video processing method that executes, and executes drawing processing from one direction for an object whose depth position is behind the predetermined position.

Images