JP3201533B2 - Presentation method of video and its shooting information - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for editing a recorded video, which includes information on position information of a subject which supports the editing and information on a photographing condition of a camera, and a video which expresses the information in an easily understandable manner. The present invention relates to a method of presenting the shooting information.

[0002]

2. Description of the Related Art In a conventional video editing method, a person called a scripter describes information at the time of capturing a video in a script, and an editor searches for video required for editing from the description. The method has been adopted. This method is used in broadcasting stations whose main business is video creation and in editing companies called post production.

[0003]

However, the description of the situation using this scripter requires a great deal of effort, and if the character of the script description does not match well with the information desired by the editor, the content of the description is poor. There are problems such as the case where the information becomes insufficient, and it is necessary to mechanize the problem to obtain information of constant quality.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image capable of displaying the position information of a subject, information on a photographing condition of a camera, and the like in an easily understandable manner, and a method of presenting the photographing information.

[0005]

According to the present invention, the position information of a subject as a target and the photographing conditions of a camera for photographing the subject are collected by various sensors such as the movement of the subject and the photographing conditions of the camera. It extracts information from video and manages and stores them together with video and audio.

[0006] Further, when editing using this, the collected information is developed into a bird's-eye view showing the shooting site displayed on the display device, and a means for providing information on the relationship between the camera and the subject and the shooting situation is provided. Using. In addition, the bird's-eye view and the image captured by the camera are simultaneously displayed on the display device, and the video and audio are displayed in synchronization with the subject in the bird's-eye view and changes in the camera. The camera status can be visually recognized, and the dialogue and voice can be synchronously displayed and output.

[0007]

According to the present invention, the photographing information and the position information of the subject and the camera shown in the bird's eye view can be simultaneously and visually recognized, and the video and the audio can be viewed in synchronization. . For this reason, it becomes possible to visually determine how a plurality of cameras have recorded the video at the same site, and it is easy to determine which camera's video contains the video intended by the editor. In addition to speech, you can listen to reminders of the situation at hand, as well as hear surrounding background sounds, in addition to knowing the dialogue. Will be able to save. Also, by displaying the dialogue, it becomes possible to visually recognize the timing at which a necessary scene starts. Therefore, by moving information at a high speed to the position where the dialogue appears, it becomes possible to search for a purpose and a scene.

[0008]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a configuration block diagram of an apparatus used in one embodiment of the present invention. In FIG. 1, reference numeral 1 denotes an input device such as a keyboard having character keys and function keys, a mouse, and a joystick. Reference numeral 2 denotes a display device such as a CRT or a liquid crystal display that displays a character string input by the input device 1, a video for viewing, a guide message, and other information. Reference numeral 3 denotes an external storage device that stores video, audio, and information added to various types of video, and a plurality of the storages are installed as necessary. Reference numeral 4 denotes an internal storage device for storing programs and messages and temporarily storing various information in the external storage device 3, and 5 controls these devices.
And a control device for processing necessary information.

Next, an image displayed on the display device 2 and information necessary for the display will be described. FIG. 2 is a display example of the display device 2. On the position display screen 6, the position of the subject 7, the position of the camera 8, the direction in which the camera 8 shoots, and the angle of view thereof are indicated by broken lines and solid lines extending in a fan shape extending from the camera 8. Here, the subject 7 and the camera 8 are schematically shown, and their actual shapes are not shown. A broken line indicates a region where the camera 8 is out of focus, and a solid line indicates a region where the camera 8 is in focus, and indicates a range where the expansion is taken. The angle of view can be determined from this spread. The distinction by the broken line or the solid line used here can be changed to a method using different types of lines or different colors of the lines, a method of separating images within the range, or using blurs. Below this display, the title of the video, the location where the video is displayed, and the video time are displayed. A button display area 9 displays buttons for moving the position display screen 6 forward and backward at a standard speed, buttons for fast-forwarding, and buttons for stopping.
Reference numeral 10 denotes an information display area for displaying an azimuth of the position display screen 6, a direction from which light comes, and the like. Reference numerals 11 and 12 denote screens of images captured by the cameras C1 and C2, respectively, and images at the same time as the time display below the position display screen 6 are displayed.
Reference numeral 13 denotes a screen indicating a dialogue, which indicates that the voice has reached the part indicated by a black circle (●) at the present time, and a white circle (○) indicates a speech that is not spoken. To display this information, the following information is required. To display the position display screen 6, the position information of each subject 7, the position of each camera 8,
Information such as a shooting direction, an angle of view, a focal point of a lens, a depth or an aperture of a lens, a shooting date and time, and the like are required. Based on these information, the position of the subject 7 and the camera 8 at each time and the photographing state are displayed on the position display screen 6. Further, as a classification of such information, information on a title and a place is required. The information in the button display area 9 is the position display screen 6
Display information to help understand the display. Here, information on the direction of the screen and the direction from which light comes is shown. On the screens 11 and 12 showing images, time information based on a time code and a frame number in a stored medium exist so that the image captured at the time shown on the position display screen 6 can be displayed, and these are associated with each other. Management information may be required separately. This frame number is displayed on the lower right of screens 11 and 12 as FL. No. As shown in the following figures. Also, here, the screens of the cameras C1 and C2 are shown on the screen 11 and the screen 12, respectively, but there are screens that cannot be displayed here as shown on the screen 14 in FIG. The presence of this screen 14 is determined from the camera numbers C1 to C3 on the position display screen 6 and the presence of a fan-shaped line indicating the range of image capturing. By exchanging the screen 14 with the screen 11 or the screen 12, it is possible to actually watch the video. By using a separately provided display terminal, the content can be viewed. Further, it is also possible to display all images on a separately provided display terminal without displaying images on the screen of the display device 2.

[0010] The screen 13 showing the dialogue needs information on the time when the dialogue was actually spoken. Further, it is necessary to collect information on the time at which the sound was recorded so that the sound synchronized with the time on the position display screen 6 can be reproduced. In this way, by collecting various information,
Each screen can be displayed on the display device 2 of FIG.

Next, a method of collecting such information will be described. As for the position information of each subject 7 and the position information of the camera 8, there is a method of collecting the position information of each subject 7 and the camera 8 from a screen shot by a television camera provided on the ceiling. A device that obtains this location information (Video Tracker:
Etc.) are commercially available. Further, a method of calculating the motion of each subject 7 as a difference between each frame of the video, converting the difference value into a binarized image, obtaining the position of the center of gravity, and using it as position information can be used. is there.

The method of collecting images from the ceiling can be used in a special place such as a studio, but is difficult to use outdoors. In such a case, the video captured from a high position is converted into the same video captured from directly above by image processing, and positional information is obtained in the same manner. There is another method using a measuring means. As an example of this, there is a method in which a radio transmitter is attached to each of the subjects 7 and the camera 8 and the direction from which the radio wave comes is obtained by triangulation. Ma
If the subject 7 or the camera 8 moves, the position on the map
Becomes important. For this information, signals from satellites are used.
Satellite position measurement method to determine position, signal from ground station
Ground station position measurement method to determine the position using
Using the radio waves from the transmitting station installed in other places
Proximity wireless position measurement method to determine position, course, speed sensor
There is an autonomous position measuring method for estimating a position by using a method. This
Among them, the satellite position measurement method is the GPS satellite positioning system
Has been put into practical use by the U.S. Department of Defense,
The advantage is that it can be used anywhere. Also this satellite
Places in tunnels or building shadows or indoors where radio waves cannot be received
The ground station position measurement method, proximity wireless position measurement method,
Compensate for the shortcomings of GSP in combination with the key position measurement method
Have already been developed. Collecting such location information
By using the information of the collecting device,
Even in this case, the position information can be obtained accurately. Also high
By using the meter together, the position on the map is the same,
By obtaining height information, information on the number of floors of the building
Can be obtained, and the accuracy can be improved by specifying the shooting location.
It becomes possible to have. This height information is provided by the GSP.
But this method has an error of about 30m.
Therefore, in a building where each floor has an average height of 3m, an error of 10 floors
It is difficult to use it as it is. on the other hand,
With an altimeter using a barometer, there is an error of about 1m from the ground.
Since the height can be detected, if the pressure on the ground is specified,
The height can be easily identified, and the
The height can be specified automatically.

Next, the operation conditions of the camera 8 include information such as a shooting direction, an angle of view, a focal point of a lens, an aperture of a lens, and a depth of a subject. These are measured by attaching sensors, respectively, and there is a camera 8 equipped with the sensors. These data are stored in the memory simultaneously with the recording of the video with the time code, and are managed by time. Other titles and video location, information such as the direction of the light, it is necessary to enter the data each time the situation changes.
Of these, the image location and the direction of the light are the position measurements described earlier.
Automatic data acquisition by the combination of methods and optical sensors
Nesting is possible. In addition, since speech recognition cannot be easily performed for the start of speech and subsequent progress, it is necessary to rely on humans at present. However, it is relatively easy for a person to insert the time associated with the progress of the speech into the speech input by the word processor at each knot, thereby creating information that is somewhat synchronized with the video.

It is necessary to describe such information for each time, but a simple description increases the amount of information.
For example, management is performed as follows. First, information to be described will be described. The descriptive information includes date, reference direction indicating the relationship between screen information and orientation, scale indicating scale,
Collection time of each data, number of cameras used for shooting, their coordinate position, shooting camera direction, angle of view, focus position, aperture, etc., number of moving subjects, their coordinate position, And the number of still subjects, and their coordinate positions. FIG. 4A shows the order in which these pieces of information are described, the method of describing numerical values, and the number of digits. Here, (A)
* In each of the numbers indicates a numerical value, and the number indicates the number of digits. In addition, commas, slashes, colons, semicolons, and the like are used as delimiters for each item. FIG. 4B is a description example in which numerical values are specifically entered. However, the last part is shown as n because of the omitted description part. Here, the upper part shows the contents and the lower part shows the numerical values. Here, when there are a plurality of cameras 8, moving subjects, and still subjects, the number and the information necessary for them are repeatedly described. FIG. 4 (C)
Is a state in which these data are described as continuous data, and (D) is a data string indicating a state 0.08 seconds after the data in (C). In (D), it is assumed that only the underlined portion is changed and other conditions are not changed.
(E) is the difference between (C) and (D) ((E) = (C) −
(D)). Items having no difference are indicated by 0, and the difference is described by indicating positive / negative by +-. (F) is a state in which the delimiter mark of (E) is removed. In (F), the item having no difference is indicated as 0, but in (G), the corresponding 0 is indicated by *, and the 0 generated as the difference value is indicated by 0 as it is. Here, the number of * is shown in the upper row, and (H) shows the value and the difference value side by side. By using such a description method, items that do not change can be handled collectively. For this reason, the data amount can be significantly reduced. (I) is another description method having the same contents. However, this is basically the same as (G), except that an item having a difference uses the data itself of the item in (D).
(J) is data enumerated and shown in the same manner as (H). In this description method, the amount of data is significantly reduced as compared with the original data, and the changed value is shown as it is, so that it has a feature that it is more resistant to errors. As described above, when the description method shown in FIG. 4 is used, information that does not change can be described collectively, and only the changed information can be described in detail. This makes it possible to store information by adding very little information compared to video information having a large amount of information. Here, the example of FIG. 4 is shown as the description content, but information is managed in a hierarchical (directory) manner, and fixed information such as date, scale, the number of cameras and the like are collectively managed therein, and the number of description items is reduced. Or
Obviously, missing information not shown here can be added.

A screen on the display device 2 shown in FIG. 2 is formed on the basis of the information stored in this manner. By using this method, a multifaceted screen more than the current scripter describes. Information can be collected and the effort of the splitter can be reduced.

By the way, a bird's-eye view can be obtained on the screen on the display device 2 shown in FIG. 2, but it is difficult to display distant view information in this. That is, when the position display screen 6 is displayed with information on the distant view, detailed information near the camera 8 cannot be understood, and when the information is near the camera 8, information on the distant view cannot be understood. Therefore, here, as shown in FIG.
, And the video is displayed on the screen 15 in synchronization with the time on the position display screen 6. This video is rarely used for the actual editing screen, but can be used as an auxiliary means for clarifying the situation at the moment. At this time, the screen 15 is a panorama-like image obtained by cutting out a part of the entire screen, so that the limited screen of the display device 2 can be used effectively. Furthermore, redundant information such as sky and floor can be omitted. Of course, if you need sky or floor information, as long as it is in the recorded video,
Clearly, it is possible to display it. Further, in the description of FIG. 2, it is assumed that all the cameras 8 are recording, but in FIG. 5, only the C1 of the cameras 8 is recording. In such a case, in this method, the line indicating the fan-shaped spread indicating the angle of view is
It only exits from one and does not exit from the other cameras C2 and C3. In this way, the camera that is taking a picture can be visually and clearly identified.

[0017]

As described above, according to the present invention, the position and movement of the camera and the subject, the shooting conditions, and the like are recorded with the lapse of time when the image is shot, and the overall state is determined based on this. Is shown as a bird's-eye view, and the camera conditions, the angle of view, and the shooting conditions corresponding to the in-focus area are visually understood. Furthermore, the images of the cameras taken at the same time are displayed. However, a panoramic image is used as a method for displaying a distant view image which causes a problem when expressed in a bird's-eye view as a video, reducing the area of the display device used by the screen, and not reducing the information. As a result, the method manually described by the conventional scripter can be mechanized, and the problem of biased content description due to individual differences due to the scripter can be solved. In addition, since the information of the camera and the subject is described for each time, the data amount increases.However, since the data amount can be compressed and stored, the load on the memory of the conventional tape or optical disk is reduced. You can now.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of an apparatus used in an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a display image displayed on the display device of FIG. 1;

FIG. 3 is a diagram illustrating an example of a video.

FIG. 4 is a diagram showing an example of collected data.

5 is a diagram showing an example of a display image different from that of FIG. 2 displayed on the display device of FIG. 1;

[Explanation of symbols]

 1 input device 2 display device 3 external storage device 4 internal storage device 5 control device 6 position display screen 7 subject 8 camera 9 button display area 10 information display area 11 screen 12 screen 13 screen 14 screen 15 screen

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Hiroyuki Kimiyama 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Akito Akutsu 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo (56) References JP-A-4-294694 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 5/262 H04N 5/222 H04N 5/76 H04N 5/91

Claims (5)

(57) [Claims] 1. The method according to claim 1, further comprising: collecting position information of a camera and a subject to be photographed in association with a photographing time when photographing an image; displaying the positions of the camera and the subject on a display device in a bird's-eye view according to the photographing time; A method for presenting a video and its shooting information, wherein the video shot at the same time is displayed on the display device or a separately provided display device. 2. A panoramic image is displayed on the display device at the same time when the positional information is collected, as the position information of the camera that has captured the image and the position of the subject are shown on the display device as a bird's-eye view. 2. The method for presenting a video and its photographing information according to claim 1, wherein: 3. Collecting information on shooting conditions of a camera for shooting in association with a shooting time, and displaying the shooting conditions on a display device in a fan-shaped range centered on the position of the camera according to the shooting time. The method for presenting a video and its photographing information according to claim 1, characterized in that: 4. The video according to claim 1, wherein the dialog used at the same time as when the information on the positional relationship indicated by the bird's eye view is collected is displayed on a display device or a separately provided display device. How information is presented. 5. At the time of photographing a video, at least the positional information of the camera and the subject and the information of the photographing conditions of the camera collected in association with the photographing time are collected as enumerated information and obtained in association with the photographing time. For information that has not changed from the previous information, the number of consecutive information that does not change is described, and for changed information, the value after the change or the difference value from the previous information is described. 2. The method according to claim 1, wherein the information is used to represent the positional information of the camera and the subject and the information of the photographing conditions of the camera as a bird's-eye view.