JP3227350B2 - Video playback method and video playback device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for realizing a video reproducing apparatus, and more particularly, to reproducing another video in which the subject is photographed while reproducing the video by using the subject photographed in the video as a key. The present invention relates to a video reproducing method and a video reproducing apparatus for performing reproduction while switching a plurality of videos without interrupting reproduction of the subject by performing switching.

[0002]

2. Description of the Related Art In order to accurately convey the movement of a plurality of objects existing in a target world to a viewer or a user, a plurality of videos of the target world are reproduced while being switched according to a user's request. It is hoped that it can be done.

As a first prior art capable of performing such a reproduction, Japanese Patent Application No. 5-246512, "Video Reproducing Apparatus"
Is known. Here, we set the target world to N
A configuration of an apparatus for switching and displaying N videos taken from different angles according to a user's request is disclosed.

In this apparatus, since the video is switched based on the continuity of the photographing time, it is necessary to prepare N videos synchronized in time in advance. For this reason, there has been a problem that imaging is extremely difficult, or a problem is that the photographable target world is limited. Further, there is also a problem that a necessary video cannot be prepared due to severe restrictions on the above-mentioned shooting, so that a system that can sufficiently satisfy switching based on a user's request cannot be configured.

[0005] As a second prior art aiming to accurately convey the movement of an object existing in the object world to a user, Japanese Patent Application No. 5-251559, "Moving-image reproducing apparatus" is known. In this apparatus, N images of the target world are prepared in advance, and for each individual object in the entire area to be captured, the target object is located in the entire area. By providing position information storage means for detecting and storing position information, the image is switched and displayed so that the object selected by the viewer is displayed at a desired position on the screen, for example, at the center of the screen.
Switching and display of video based on viewers' requests are realized by a simple system that can be used by ordinary viewers.

However, even in this apparatus, it is necessary to determine the position of the object in the entire area to be photographed.
The photographable target world was very limited. That is, in the case of a plane such as soccer or baseball, the position of the object can be represented by two axes of the x-axis and the y-axis, and the positions of the players and the ball can be detected and stored and managed. It is extremely difficult to uniquely determine the position of an object in the entire area from videos of athletes in a ski tournament and images of fish moving in the water from multiple angles. Even if it is possible to create position information that can specify the position of an object by using a complex coordinate space, a great deal of processing is required to identify the object from the video or to determine the video from the object. The amount of calculation was required, and a sufficiently responsive system could not be constructed.

[0007] Even in soccer or baseball, in order to reproduce a video such that a specific player is tracked based on a viewer's request from a video shot of the entire field as a video of sufficient quality, It is necessary to prepare an extremely large amount of high-definition video in advance, and there is a problem that an excessive burden is imposed on a video photographer or a system provider.

[0008]

SUMMARY OF THE INVENTION It is an object of the present invention to simultaneously solve the above-mentioned problems of the first and second prior arts and to provide interactive video switching according to a viewer's request at a high level. It is an object of the present invention to provide a video playback device that enables easy operation, and further reduce the load on a photographer or a photographer so that the system can handle a wider target world.

[0009]

In order to achieve the above object, a video reproducing method according to the present invention comprises the steps of, when a plurality of video streams are stored, one or more subjects A on a certain video stream i; , And another video stream j, the relevance is given through subject information ,
Time between video stream i and said video stream j
When the subject A is designated when a certain video stream i is reproduced and displayed, the given relationship is traced, and the corresponding video stream j
The retrieved from the stored in the video stream, the object to be
The time corresponding to the time of the video stream i when the object A is specified
The video stream j is reproduced and displayed from the corresponding position .

Further, the video reproducing apparatus of the present invention comprises a video storage means for storing N (N ≧ 2) video streams photographing a target world, and one of the stored N video streams. In a video reproducing apparatus provided with video access means for selecting and reading a video stream and video reproducing means for reproducing the read video stream, the position of a subject photographed in each video stream is identified in the video stream. Information management means for managing subject information for a video stream i (1 ≦ i ≦ N) and position information of a subject captured in the video stream i in a video stream j (1 ≦ j ≦ N) To the video stream j
Together when mapped to the position information of the subject captured in the
Time of video stream i and time of video stream j
And a subject information mapping means for associating the video stream i read by the video access means with the video playback means while reproducing the video stream i. Is determined using the subject information managing means, and the determined subject position is input to the subject information mapping means, and the subject information mapping means determines The reproduced video stream is switched to a desired video stream j in which the mapped object is photographed, and the reproduced video stream i is switched to the desired video stream j .
The reproduction is continued from the time of the video stream j corresponding to the time .

[0011]

According to the above means, the position information of the object to be photographed is not detected and managed in the entire area to be photographed, but the position is detected in the video stream in which the object is photographed. Just managing is enough. For this reason,
The processing for detecting the position of the object is greatly reduced, and the restriction on the images that can be photographed is also greatly reduced, so that the load at the time of photographing can be reduced and the world where images can be photographed can be expanded.

[0012]

FIG. 1 is a block diagram of a video reproducing apparatus for explaining an embodiment of the present invention. The video storage unit 10 is a device for storing at least N (N ≧ 1) captured videos. For example, the video is digitally encoded and stored in a magnetic disk device having a desired capacity. Can be easily realized.

The video access unit 11 controls the video storage unit 1 so that one of the N videos stored in the video storage unit 10 can be continuously played back while maintaining the temporal order.
This is a circuit for reading out the video data stored in 0.

The video reproduction / instruction input unit 12 is provided with a video reproduction circuit for presenting to the user as a video using the video data read by the video access unit 11 and a video reproduction circuit for displaying the video presented by the user. , A circuit for inputting a video switching request to the video reproducing apparatus. The user directly specifies a point on the screen with a fingertip, or inputs coordinates in the screen corresponding to the point on the screen using a pointing device such as a mouse.

The subject information management unit 13 includes a video storage unit 10
For each of the N videos stored in the image, the position information of the subject photographed in the video is managed using a coordinate system in the video. As a simple coordinate system, for example, the position information of the subject is managed using a coordinate system consisting of a frame number calculated from the beginning of the video and x and y coordinates having the origin at the upper left in the frame determined by the frame number. I do. When such a management method is used, video identification information for specifying a video, a frame number from the head of the video, and coordinates x and y in the frame are input to the subject information management unit 13. Thus, the subject being photographed can be identified from the position information of the subject managed by the subject information management unit 13. In addition, by inputting information for identifying a subject, for example, a subject identifier, a “reverse” that identifies a video in which the subject is captured can be performed.

The subject information mapping unit 14 is a circuit for mapping the position information of the subject for each video managed by the above-mentioned subject information management unit 13 to the position information of the subject in another video. By the subject information mapping unit 14,
The relative positional relationship between the images is determined via the subject,
The video can be switched and displayed based on a user's instruction. Hereinafter, a specific switching operation will be described in detail with reference to the drawings.

FIG. 2 is a diagram specifically illustrating the relationship between two videos, video A and video B, stored in the video storage unit 10. As shown in FIG. The image A is an image captured so as to follow the movement of the athlete a (skiing), and the imaging is performed around the athlete a. On the other hand, the image B is an image in which the movement of the player b is chased, and the photographing is performed centering on the player b.

While the user is watching the video A mainly shot by the athlete a, the interest of the other athlete b shot in the video A is changed, and the user is shot around the athlete b. To the video playback device. In the example of the figure, the player a moves from left to right with the progress of time, and the player b moves from right to left with the progress of time. For this reason, the player B (along with the player a) is photographed only for a certain time section in the video A mainly shot by the player a. Conversely, also in the video B mainly shot by the player b, the player a is shot (along with the player b) only for a certain time section. FIG. 3 shows this part in detail with time progress on the horizontal axis, and the relationship between the video and the subject will be described in detail with reference to FIG.

The movements of the players in the video are stored and managed in the above-described subject information management unit 13 as the position information of the subject in a closed form in each video. In FIG. 3, the subject of the player b is photographed in the section from the time t ₁ +3 to the time t ₁ +4 of the video A capturing the player a. On the other hand, time t ₂ +1 of video B shooting player b
From time t ₂ +3, the subject of player a is photographed. As described above, the two videos have independent time series, and the position of the subject photographed therein is managed in an independent coordinate system.

Here, from time t ₁ +3 of image A to time t
_When the user instructs the player b during the playback of the section of ₁ + 4, it is possible to switch and display the image to the image B mainly shot by the player b. Also, the time t ₂ +
During playback of the section from 1 to time t ₂ +3, the user
, It is possible to switch and display the video to the video A that is mainly shot by the player a. For this,
A circuit that maps subject information between videos is a subject information mapping unit 14.

Next, the configuration and operation of the subject information mapping section 14 will be described in detail step by step. FIG. 4 is a diagram illustrating a configuration example of the subject information mapping unit 14 and an operation procedure thereof. In the figure, a subject identification unit 42 is a circuit that receives a user switching request via the video reproduction / instruction input unit 12 and identifies the subject designated by the user using the subject information management unit 13 described above. .

The video search section 43 is provided with the subject identification section 42.
The identification information of the subject obtained in the step is input to the subject information management unit 13, a video image of the subject is searched, and the searched video is instructed to be played back to the video playback / instruction input unit 12. Circuit. This video search unit 43
A time correspondence table for managing a temporal correspondence relationship between videos is stored. For example, a time t _{1 of the} video A and a time t ₁ of the video B are stored.
It is described that ₂ is matched. With this time correspondence table, a one-to-one correspondence can be created between frames of the video A and the video B.

The mapping control unit 41 includes the object identification unit 4
2. Control the video search unit 43 to switch to the video that satisfies the instruction issued by the user. Hereinafter, the flow of the operation related to the video switching will be described in detail with reference to FIGS.

(1) A video switching request signal based on a user's instruction is input from the video reproduction / instruction input unit 12 to the subject identification unit 42. The switching request signal can be composed of, for example, four sets of (video identifier, time, x coordinate, y coordinate). Here, the xy coordinates are the coordinates in the screen displaying the video, the x-coordinate in the horizontal direction with the origin at the upper left of the screen,
It is assumed that values from 0 to 100 are assigned to the vertical direction as the y coordinate.

For example, assuming that the user designates the player a on the screen at the time t ₂ +2 in the video B in the process of reproducing the video B in FIG. The switching request signal shown in FIG. (B, t ₂ +2, _10, 10) This corresponds to a request for switching to the video A because the interest of the user watching the video B has shifted to the player a.

(2) The subject identification section 42 passes the switching request signal input from the video reproduction / instruction input section 12 to the subject information management section 13 and the signal (B, t ₂ +2, _10, 10) is A search request for identifying the subject at the designated position is issued to the subject information management unit 13.

The subject information management unit 13 identifies, for each video, position information consisting of a frame number from the beginning of the video, coordinates x and y in the frame, and a subject photographed at the position. It has a correspondence table with the subject identifier, and can search for the subject identifier using the given switching request signal.

(3) The subject identification unit 42 receives the subject identifier as a result of the search request given to the subject information management unit 13. The subject identifier received in the present embodiment is an identifier given to the player a sliding from the upper right to the lower left of the screen.

(4) The subject identifier as the search result is passed to the video search unit 43. Usually, only one subject satisfies the search request.
However, even if a plurality of subject identifiers are received as a search result, the subsequent processing can be continued.

(5) The video search section 43 is provided with the object identification section 42
Using the subject identifier input from, a request is made to the subject information management unit 13 to search for a video image of the subject identified by the subject identifier. As described in (2) above, since the subject information management unit 13 has a correspondence table of the subject identifier and the positional information including the frame number for each video, the subject information management unit 13 receives the subject identifier as an input and Can search for the video that is being shot.

(6) The video search unit 43 receives the result searched by the subject information management unit 13 as a set of video identifiers. The video identifier is composed of information that enables the reproduction of a desired video to be started. For example, it can be composed of information for specifying a video such as a video name or a video number and information indicating a reproduction start position of the specified video (a frame number or the like from the head of the video).

The video identifier received in this embodiment has the video name A and the reproduction start time t ₁ . This is due to the fact that the video A is are taken continuously from the time t ₁ without losing sight of the players a.

If a plurality of subject identifiers are obtained in step (4), steps (5) and (6) are repeatedly executed, and
The final set of video identifiers is obtained by calculating the sum of the set of video identifiers obtained in (6).

(7) The video search unit 43 makes a desired selection for the set of video identifiers obtained in (6), and then sends the selected video to the video playback / instruction input unit 12. Instruct playback.

In this embodiment, the reproduction start time t ₁ +2 is obtained from the time correspondence table of the video search unit 43 using the time information t ₂ +2 of the video B included in the request from the user. The reproduction / instruction input unit 12 is instructed to reproduce the video.

The above-described video reproducing apparatus of the present invention is characterized in that a plurality of videos are switched and reproduced while paying attention to the photographed subject, and a mechanism for managing the position information of the subject in each video is provided. The method is characterized in that the image is switched as a two-stage operation provided with a subject information mapping mechanism for mapping the position information of the subject between the videos. Therefore, if the video is not sufficiently selected in the above step (7) and two or more video images which become reproduction candidates remain as selection results, the video reproduction / instruction input unit 12 is instructed to reproduce them. , Giving the user the choice of viewing multiple presented videos, or selecting a video in which the object of interest to the user is photographed large (large area on the screen) Although any of the above methods can be considered, the video reproducing apparatus according to the present invention can be realized by either method.

In the embodiment, the switching between the two video streams of the video A and the video B has been described in detail. However, the switching between two temporally different areas of one video can be easily realized. it can. Also disclosed is a technique in which a player who is skiing as a subject, that is, a method of switching images by pivoting an object existing as one object in the real world, is used for switching images through a virtual object and switching between objects. Video switching via a multi-stage pivot with a database describing the relationship can be easily realized based on the above description of the present invention.

[0038]

The video reproducing method and the video reproducing apparatus according to the present invention are characterized in that a video appearing in a video is switched as an intermediary, and that the creation and management of the subject information therefor are facilitated. Can be.

Therefore, the video playback method of the present invention is a concept independent of the shooting and storage methods of individual video streams, and the video playback method of the present invention can be combined with various shooting and storage methods. By a simple operation in which a user who is watching a video instructs a subject projected on the video, the video can be switched to another video related to the interested subject and the video reproduction can be continued. There is no restriction on the shooting of other related videos, and videos captured at completely different times and spaces are combined by the video playback method of the present invention, and video playback is performed by switching the video via a subject appearing in the video. The device can also be implemented.

For example, an image of a museum in which a plurality of works of art are exhibited is combined with an image of the production process of each of the works of art taken in the image, and the image of the museum is displayed. When the user designates an art piece of interest during the process of reproducing the art piece, it is possible to easily realize an application such as switching to a video in the process of producing the designated art piece and continuing the reproduction.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a video reproducing apparatus for explaining an embodiment of the present invention.

FIG. 2 is a diagram specifically illustrating a relationship between two videos (video A and video B) stored in a video storage unit.

FIG. 3 is a diagram for explaining a relationship between a video and a subject in detail.

FIG. 4 is a diagram illustrating a configuration example of a subject information mapping unit and an operation procedure thereof.

[Explanation of symbols]

 Reference Signs List 10 video storage unit 11 video access unit 12 video playback / instruction input unit 13 subject information management unit 14 subject information mapping unit 41 mapping control unit 42 subject identification unit 43 video search unit

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-7-105227 (JP, A) JP-A-7-129624 (JP, A) JP-A-7-107437 (JP, A) Video in a network environment Hypermedia mounting method, Koji Hirata, Symposium on Information Processing Society of Japan, Vol. No. 13, December 7, 1994, p. 165-173 (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17/30 H04N 5/91, 5/93

Claims (2)

(57) [Claims] 1. A video playback method for playing back and displaying a video stream selected from a plurality of stored video streams, comprising: one or more subjects A on a video stream i;
Together keep impart relevance through the object information between different video stream j, the video stream i and the previous
The temporal correspondence information with the video stream j is stored.
Lifting, and the when reproducing and displaying the video stream i with the the subject A is specified, follow the grant and relevance of the searches the appropriate video stream j from the stored in the video stream , At the time of the video stream i when the subject A is specified
A video reproducing method comprising: reproducing and displaying a video stream j from a position corresponding to an hour . 2. A video storage means for storing N (N ≧ 2) video streams photographing a target world, and a video access means for selecting and reading any one of the stored N video streams. And a video playback device comprising: a video playback unit that plays back the read video stream; and a subject that manages subject information for identifying a position of a subject photographed in each video stream in the video stream. Information management means, and in any video stream i (1 ≦ i ≦ N) and video stream j (1 ≦ j ≦ N), position information of a subject captured in the video stream i is captured in the video stream j. and while mapping the position information of the subject is, movies
The time of the image stream i and the time of the video stream j are paired.
And a position of an arbitrary subject photographed in the video stream i being reproduced while reproducing the video stream i read by the video access means using the video reproduction means. Is determined using the subject information management means, and the determined position of the subject is input to the subject information mapping means, and the position information of the subject photographed in a video stream other than the video stream i by the subject information mapping means is determined. And a desired video stream j in which the mapped object is photographed.
The video string switches the video stream in said reproduction
A video playback apparatus configured to continue playback from the time of the video stream j corresponding to the time of the video i .