JP4335087B2 - Sound playback device - Google Patents

Description

  INDUSTRIAL APPLICABILITY The present invention is suitable in the field of packaged video / music playback for viewing in consumer / business use using CDs / DVDs, and in the field of environmental video / environmental music distributed for commercial purposes by broadcasters / business operators in public facilities TECHNICAL FIELD OF THE INVENTION

  Conventionally, a technique for displaying a plurality of videos on the same screen has been used. When trying to display multiple videos at the same time, the computer actually processes in order from the first video file, but when the computer's processing power cannot keep up, the frames of the subsequent video file are processed. It may disappear. In particular, when the motion of the video recorded in the later video file is fast, there is a problem that if there is a frame that is not processed, the video has awkward motion.

In the case of displaying a plurality of videos, there is also a technique of giving priority to videos to be displayed preferentially and displaying them on a large screen (for example, see Patent Document 1).
JP 2001-34250 A

  However, the conventional technology has a problem that a multi-display screen cannot be smoothly performed under the limitation of limited hardware resources.

  Therefore, an object of the present invention is to provide a video signal playback apparatus that can smoothly play back a plurality of videos in a limited hardware environment.

  In order to solve the above-described problems, the present invention provides a sound video playback apparatus that plays back a plurality of video files with sound, each having a plurality of video frames and an audio data block associated with each video frame. Extract an inter-frame difference table that records inter-frame difference feature values based on differences between video frames of an attached video file, and an audio data block attached to each corresponding video frame from the plurality of video files with sound. Acoustic data block synthesis means for synthesizing into two synthesized acoustic data blocks; acoustic output means for writing the synthesized acoustic data block to an acoustic output device for acoustic reproduction; and synthetic acoustic data block being reproduced in the acoustic output means Processing control means for acquiring a playback time, and the acquired playback time in each video file with sound Referring to the inter-frame difference table for the corresponding video frame, priority calculation means for calculating the priority for the video frame, and the priority of the video frame of each sound-added video file corresponding to the acquired playback time Whether or not to display the video frame is determined based on the above, and the display control means for writing the video frame determined to be displayed to the memory of the video display device and displaying the screen is provided. .

  According to the present invention, when a plurality of sound-added video files are simultaneously played back, the priority of each video frame is calculated with reference to the inter-frame difference that is the difference between video frames in each file to be played back, and according to this priority Since it is determined whether to display each video frame, a video with a large difference and fast movement is displayed with priority, and smooth playback is possible even in a limited hardware environment. It becomes possible.

(1. Structure of video file with sound)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, the structure of a sound-added video file to be played back by the sound-added video playback apparatus according to the present invention will be described. FIG. 1 schematically shows the structure of a general-purpose sound-added video file. In FIG. 1, V indicates a video frame, A indicates an audio data block, and 0-N indicate frame numbers corresponding to the video frames. The frame number is a number indicating the number of the frame from the beginning. When the number of frames is N + 1 as a whole, numbers 0 to N are assigned. The general-purpose video format is composed of 30 frames per second. For example, if the video data is 3 minutes, it is composed of 5400 frames. In addition, the video frame V is usually compressed, and a still image may be restored from one video frame V by a compression method. If no other video frame V is used, the still image is restored. Sometimes it is not possible. The acoustic data is divided in units of frames, that is, in units of 1/30 seconds, and recorded as an acoustic data block A. For example, when a stereo sound signal is sampled at a sampling frequency of 48 kHz, 3200 samples corresponding to 1/30 seconds are recorded in one sound data block.

  The sound-added video file as shown in FIG. 1 can be created by a known method. That is, a sound-attached video file in which video data obtained by shooting with a video camera or the like is associated with audio data may be used as it is, or only video data that has been shot is used. May be prepared and additionally recorded in association with the previous video data (the latter method is generally used in terms of sound quality). In the present embodiment, it is created by associating and recording separately obtained video data and audio data.

(2. Creation of video file with sound)
Next, creation of a video file with sound in the present embodiment will be described. In the present embodiment, description will be made assuming that this system is used in an apparatus that synthesizes and reproduces audio data recorded as a material in a sound-added video file. In this system, the number of sound-added video files to be synthesized can be changed as appropriate. In the present embodiment, a case where a maximum of five video files can be selected will be described. In this case, the acoustic data is set to five tracks and reproduced. However, if, for example, five pieces of music can be selected for each track, a total of 25 acoustic data are required. Therefore, first, analog sound data obtained by recording or the like is digitized to obtain 25 digital sound data. Digitization of an analog acoustic signal is performed using a conventional general PCM method. Specifically, an analog sound signal is sampled at a predetermined sampling frequency, and an amplitude is converted into digital data using a predetermined number of quantization bits. The acoustic data digitized in this way becomes a set of time series of samples having a value corresponding to the number of quantization bits. For example, when the sampling frequency is 48 kHz and the number of quantization bits is 16 bits, an analog sound signal for 1 second is converted into digital sound data composed of 48000 samples having values of −32768 to 32767.

  In order to synthesize a plurality of pieces of sound data and reproduce them as one piece of reproduced sound data, it is necessary to process the sound data to be synthesized to have the same reproduction time. This is because each sample of other acoustic data (quantized with a predetermined number of bits at each time) is synchronized temporally and musically with the reference material acoustic signal on the basis of one acoustic data. Process to adjust to. Further, in this embodiment, each sound data is created by dividing it into parts such as melody, chord, rhythm, etc. so that the user who reproduces can freely change the composition of music. Each acoustic data is obtained by converting an analog acoustic signal into digital data by a technique such as PCM as described above.

  Here, each acoustic data set as five tracks will be described. FIG. 2 schematically shows a signal waveform of acoustic data of each track. In the example of FIG. 2, a case is shown in which each piece of acoustic data is stereo acoustic data composed of two left and right (LR) channels. In FIG. 2, for simplification of explanation, a portion where the amplitude value of the signal has a level of a certain level or more, that is, a non-silent portion shows a waveform with the same amplitude, and a silent portion is shown without a waveform. . When combining and playing music from multiple tracks and making it possible to select multiple songs for each track, to ensure that the combined music is decent regardless of the combination In addition, each acoustic data needs to be created according to a predetermined rule. Therefore, each track is configured such that, regardless of which piece of music is selected, the temporal positions of the silent section and the silent section are basically the same. That is, for example, each of the five acoustic data prepared for the track 1 has a non-silent part (sound part) and a silent part at the same position in order to avoid poor musical change. The lengths of the non-silence portion and the silence portion are also changed somewhat within a range that does not hinder the music rules.

  When the acoustic data having the waveforms as shown in FIG. 2 are synthesized and reproduced, the sound from the track 1 and the track 5 is first heard, and then the sound from the track 5 disappears and the sound from the track 3 is heard. Then, the sound from track 1 and track 3 disappears and the sound from track 2 and track 4 is heard, then the sound from tracks 2 and 4 disappears and the sound from tracks 1 and 3 Then, the sound from the track 3 disappears, the sound from the track 5 is heard, and finally the sounds from the track 1 and the track 5 disappear.

  On the other hand, the video data is photographed according to the sound content recorded in the sound data, and the photographed data is compression-coded by a predetermined method. For example, when the acoustic data of a musical instrument representing each country is set for each track, an image of the scenery of the country is captured as image data. Subsequently, the separately obtained video data and audio data are integrated into one sound-added video file, which corresponds to an audio data block corresponding to the time corresponding to each frame of the video data. This is done by recording. For example, when video data is sampled at 30 fps (frame / second) and audio data is sampled at 48 kHz stereo, 3200 samples are recorded as one audio data block in association with one video frame. That is, it is recorded in the form as shown in FIG.

  In the present embodiment, data is further processed and recorded for the menu screen. Specifically, the first 10 seconds are extracted from the 25 sound-added video files, and a menu video file and a demo playback sound file are created. The menu video file generates a menu composite frame by synthesizing 25 pieces of video data for 10 seconds extracted from the sound-added video file for each video frame. Thus, a menu video file composed of 300 menu composite frames is obtained. On the other hand, the demo playback audio file is obtained by continuously recording 10-second audio data blocks extracted from the sound-added video file. In addition, when a video is synthesized and played back, a background video file, which is video data for background video, is prepared separately as the background. The background video file is structurally a normal video file of 30 frames per second.

(3. Creation of difference table between frames)
First, as an advance preparation, an inter-frame difference table is created in which feature amounts of differences between video frames in each sound-added video file are recorded. The inter-frame difference feature amount indicates how much the recorded video is moved between two consecutive video frames. In calculating the inter-frame difference feature quantity, first, a difference d (x, y) of the same coordinates (x, y) between consecutive frames is calculated, and the difference for all the pixels (x, y) in the frame is calculated. d (x, y) is obtained. The difference d (x, y) is calculated by the following [Equation 1].

[Formula 1]
d (x, y) = [{R (n, x, y) -R (n-1, x, y)} ² + {G (n, x, y) -G (n-1, x, y) )} ² + {B (n, x, y) -B (n-1, x, y)} ² +] ^1/2

  In the above [Equation 1], n−1 and n indicate frame numbers. The average value of the pixel differences d (x, y) calculated for all pixels is recorded in the inter-frame difference table as the inter-frame difference feature amount. An example of information recorded in the inter-frame difference table is shown in FIG. As shown in FIG. 3, the difference feature amount between all the continuous frames of the video file with all sounds is recorded.

(4. System configuration)
FIG. 4 is a system configuration diagram showing an embodiment of a video signal reproducing apparatus according to the present invention. In FIG. 4, 10 is a sound video file storage means, 11 is an inter-frame difference table, 12 is a demo playback sound file storage means, 13 is a menu video file storage means, 14 is a background video file storage means, and 20 is a file. Selection means, 30 is an acoustic data block synthesizing means, 40 is a processing control means, 50 is a priority calculation means, 60 is a display control means, 70 is an acoustic output means, and 80 is a video display device.

  In FIG. 4, a sound-added video file storage means 10 is a storage device for storing a sound-added video file. The demo playback sound file storage means 12 is a storage device that records a demo playback sound file. The menu video file storage means 13 is a storage device in which a menu video file is recorded. The background video file storage means 14 is a storage device that records a background video file. The sound data block synthesizing unit 30 has a function of extracting and synthesizing the sound data block from the sound-added video file. The processing control means 40 has a function of controlling processing timings for sound reproduction and video reproduction. The priority calculating means 50 has a function of calculating display priority of video frames of a plurality of sound-added video files. The display control means 60 has a function of writing a video frame in the memory of the video display device 80 and displaying it on the screen according to the calculated display priority. The sound output means 70 has a function of outputting the synthesized sound data block as sound according to the instruction timing of the processing control means 40. Specifically, a process of writing a synthesized sound data block attached to a computer to a sound device and reproducing the sound is performed. The system shown in FIG. 3 is actually realized by hardware such as a computer and its peripheral devices, and dedicated software installed in the computer. In particular, the acoustic data block synthesizing unit 30, the process control unit 40, the priority calculation unit 50, and the display control unit 60 are realized by the CPU executing instructions with dedicated software.

(5. Processing operation)
Next, the processing operation of the system shown in FIG. 4 will be described. First, when the system is activated, a menu screen as shown in FIG. 5 is displayed. In FIG. 5, E is a video display area for displaying a video, and the video is divided into 25 sections of a video section E11 to a video section E55. c1 to c5 are cursors indicating that a video section is selected, and one cursor is prepared for each line. That is, one video can be selected per line, and in the initial state, the leftmost video section is selected as shown in FIG. On this menu screen, it seems that 25 images are displayed as shown in FIG. 5, but the system only performs a process of reproducing the menu image file. One video is displayed at a pace of 30 frames per second. However, since 25 different video frames are originally synthesized, the content of the video is different for each video section when viewed from the viewing side. Specifically, this display processing is performed by the display control unit 60 extracting the menu video file from the menu video file storage unit 13 and playing it back.

  On such a menu screen, the user selects a video file with sound to be synthesized and reproduced. Specifically, this is performed by the user clicking on the displayed video section. The user selects one video section for each row, and a total of five video sections are selected. When the user clicks on another video section, the cursor moves on the clicked video, and a demo playback sound file corresponding to the video section is extracted from the demo playback sound file storage means 12 and output as sound. Reproduced and output by means 70. That is, for the user, the sound corresponding to the clicked video section can be heard on the spot. As described above, the demo playback sound file has a length of about 10 seconds and is used by the user to confirm the contents of a performance or the like to be synthesized and played back. In this way, the user selects one video section for each row. The state of the screen after the user has selected is shown in FIG. In the example of FIG. 6, the video segment E12 is selected in the first row, the video segment E23 is selected in the second row, the video segment E35 is selected in the third row, the video segment E41 is selected in the fourth row, and the video segment E53 is selected in the fifth row. It is shown that.

  Here, the structure of such a menu screen will be described. The menu screen is composed of three layers of a material selection button group, a menu video frame, and a cursor overlay window as shown in FIG. Then, a menu video frame is superimposed on the material selection button group, and a cursor overlay window is further superimposed on the menu video frame, whereby display in the video display area shown in FIGS. 5 and 6 is performed. When the user clicks on the video section, the material selection button below it reacts, and the cursor placed on the corresponding line on the cursor layer moves. For example, when the user clicks the video section in the third row and the third column as shown in FIG. 7D on the menu screen, the cursor in the third row starts from the first column as shown in FIG. 7E. It will move to the third row. At this time, the audio file for demo reproduction corresponding to the selected video section is reproduced. When the play button is clicked in a state where five video sections are selected, a sound-added video file corresponding to the selected video section is extracted from the sound-added video selection file storage means 10 and played back.

  Hereinafter, reproduction of a plurality of selected video files with sound will be described. When a reproduction instruction is given, the audio data block synthesizing unit 30 extracts and synthesizes an audio data block corresponding to the first video frame from the selected five video files with sound. Specifically, one synthesized audio data block is created by calculating the sum of values corresponding to the same time for the audio data block attached to the video frame of the video file with sound. At this time, if processing is performed in units of acoustic data corresponding to 1/30 second for one frame, the efficiency is too short and the time for processing to perform video playback described later becomes insufficient. An acoustic data block is extracted every 15 frames (0.5 seconds) and synthesized. And according to the instruction | indication by the process control means 40, the sound output means 70 reproduces | regenerates a synthetic | combination sound data block as a sound.

  When the sound output means 70 starts reproduction of the synthesized sound data block and the buffer in the sound output means 70 for waiting for the synthesized sound data block to be subsequently reproduced is also full, the CPU Since the sound data block cannot be processed and written to the sound output means 70, the process control means 40 uses the waiting time to process the video frame from the sound data processing. Then, the priority level calculation unit 50 and the display control unit 60 process the video frame. For the first video frame (0th frame) of each video file with sound, the display control means 60 performs a process of displaying the video display device 80 unconditionally. At this time, it is also possible to extract a background video frame from the background video file stored in the background video file storage means 14 and display it on the background video frame. Although a moving image can be used as the background video, the video display load increases and deviates from the spirit of the present application. Therefore, in the embodiment, only the first frame is displayed as a still image. The state of the video display screen at this time is shown in FIG. At this time, the display control means 60 performs a process of outputting the selected video frame of each sound-added video file with an offset by a predetermined pixel address, and outputs the images of the five selected sound-added video files. Outputs are made so that the frames do not overlap each other.

  From the next first frame, the priority is calculated, and display according to the priority is performed. Specifically, first, the reproduction time T corresponding to the synthetic sound data block being reproduced is acquired. Then, in order to extract a video frame to be reproduced, a frame number n corresponding to the reproduction time T is calculated. Since the video frame in the video frame with sound is composed of 30 frames per second and the acoustic data is composed of 48000 samples per second, the reproduction time T of the acoustic data and the frame number n generally do not correspond one-to-one. . For example, when the sample of the synthesized sound data block being reproduced by the sound device is the kth sample and the sampling frequency is 48000, n = 30 · k / 48000 is calculated. If the display speed of the video frame is fast and only 1000 samples have elapsed since the time when the previous video frame was played back, the amount of change in the frame number is n = 0, and video frames with the same frame number overlap. On the contrary, when the display speed of the video frame is low and when 10000 samples have elapsed from the time when the previous video frame was played back, the amount of change in the frame number is n = 6 and 5 frames are skipped and 6 The video of the frame is reproduced. Then, using the acquired frame number n, inter-frame difference feature amounts D1 to D5 are obtained for the five sound-added video files selected from the inter-frame difference table. When the inter-frame difference feature amounts D1 to D5 are obtained, the display priorities P1 to P5 are calculated using the following [Equation 2] using these.

[Formula 2]
When Dm / Di> 2.0, Pi = 6
When 1.8 <Dm / Di ≦ 2.0, Pi = 5
When 1.6 <Dm / Di ≦ 1.8, Pi = 4
When 1.4 <Dm / Di ≦ 1.6, Pi = 3
When 1.2 <Dm / Di ≦ 1.4, Pi = 2
When Dm / Di ≦ 1.2, Pi = 1

  In [Formula 2], Dm is the maximum value of D1 to D5, and i takes a value of 1 to 5. That is, according to [Equation 2], the smaller the value obtained by dividing Dm by Di, the smaller the value of Pi and the higher the display priority. In addition, since the values of the display priorities P1 to P5 allow overlapping, for example, the values of P1 to P5 may all be the same.

  When the display priority for each sound-added video file is obtained for a certain frame number, the frame display counter value C is used to determine whether to display each video frame. Specifically, it is calculated by the following [Equation 3].

[Formula 3]
ON when C% Pi = Pi-1
OFF if not C% Pi = Pi-1.

  That is, the remainder obtained by dividing the frame display counter value C by Pi is calculated, and if this coincides with Pi-1, the display is turned ON, otherwise the display is set OFF. Subsequently, a video frame is displayed. Specifically, only the video frame set to ON is output to the video display device 80. At this time, if the video frame is compressed, the video frame is extracted and then decoded and output. In the video frame set to OFF, the data of the previous video frame already displayed is maintained as a still image. When this process ends, the frame display counter value C is incremented by one. The above processing is repeated until the sound output means 70 has finished waiting for processing of the synthesized sound data block and has received a playback stop instruction from the processing control means 40.

  Here, how a video frame is processed by this apparatus will be described using a specific example. FIG. 9 shows the first to sixth frames when the five selected video files with sound are processed from the first track to the fifth track. In FIG. 9, the rectangle indicates the inter-frame difference feature value. For example, “frame 01 difference 200” in the figure indicates that the interframe difference feature quantity between the 0th frame and the first frame is 200. As a display process, first, the 0th frame which is the first frame of all tracks is displayed unconditionally. The priority Pi (i = 1 to 5) is calculated by the above [Equation 2] using the inter-frame difference feature amount Di (i = 1 to 5) of the 0th frame and the first frame. In the example of FIG. 9, since D1 = 200, D2 = 150, D3 = 130, D4 = 115, D5 = 105, Dm = D1 = 200. Therefore, since Dm / D1 = 1.0, P1 = 1 from the above [Equation 2], and Dm / D2≈1.33, so from the above [Equation 2], P2 = 2 and Dm / D3≈ Since it is 1.54, P3 = 3 from the above [Equation 2] and Dm / D4≈1.74, and from [Equation 2], P4 = 4 and Dm / D5≈1.90. From the above [Equation 2], P5 = 5.

  Since the frame display counter value C is set to “0” in the initial state and is incremented by 1 every time the frame number is incremented by 1 thereafter, C = 0 because of the initial state in the processing of the first frame. Then, from the above [Equation 3], only the first track is turned on, and the second to fifth tracks are turned off. Since the frame display counter value C is incremented by 1 after the processing of the first frame, C = 1 in the processing of the second frame. Then, from the above [Equation 3], the first track and the second track are turned on, and the third to fifth tracks are turned off. Similarly, frames to be displayed and frames not to be displayed are determined. Therefore, when the inter-frame difference feature amount shown in FIG. 9 is provided, display / non-display as shown in FIG. 10 is determined. In FIG. 10, a frame in which a rectangular frame is shown is a frame to be displayed. According to the above [Equation 3], when Pi = 1, it is always displayed, when Pi = 2, it is displayed when the counter value C is an odd number, and when Pi = 3 to 6, the remainder obtained by dividing by Pi respectively. In the case of 2-5, it will be displayed. This indicates that the display rates are 30 fps, 15 fps, 10 fps, 7.5 fps, 6 fps, and 5 fps according to Pi = 1 to 6, respectively.

  In the example of FIG. 9, the case where the inter-frame difference feature amount does not change in each track is shown, but usually, the inter-frame difference feature amount also changes in the same track. FIG. 11 shows an example when the inter-frame difference feature value fluctuates. When the interframe difference feature amount as shown in FIG. 11 is included, the display / non-display of each frame is as shown in FIG.

  The processing by the sound data block synthesizing unit 30, the priority calculating unit 50, the display control unit 60, and the sound output unit 70 as described above is performed to reproduce all the sound data blocks of the sound-added video file in accordance with the instruction of the processing control unit 40. It is repeated until is finished. Thereby, on the video display screen shown in FIG. 8, while the acoustic data is continuously displayed as sound, the corresponding video data is reproduced as a moving image. As described above, in the selected video files with sound, video frames with low priority are not subjected to display processing for a predetermined number of frames from the beginning, so there is room for CPU processing capacity. On the contrary, high-priority, fast-moving video can be displayed without missing video frames, even if the CPU processing power is low and all video frames cannot be displayed smoothly on the specs, Overall smooth movement can be expressed.

  In addition, in this system, since processing is performed on a sound-added video file that is recorded by associating an audio data block with a video frame, the user can add a sound-added video in a format commonly used by computers. If even a file is prepared and stored in the sound-added video file storage means 10 of this system, it can be used in this system. In other words, users can add variations to video and music content to be played by simply preparing a video file with sound as a material and replacing multiple video files with sound as needed. It is also possible to provide environmental images and BGM that change over time.

(6. When audio is recorded longer than video)
In the above embodiment, the case where the video frame and the audio data block are recorded in the sound video file for the same time has been described. However, the video frame and the audio data block are recorded for the same time in the sound video file. There may not be. Next, the case where the recording time of the audio data block is longer than the video frame will be described. FIG. 13A is a diagram schematically showing the state of the video frame and the audio data block in the sound-added video file in which the recording time of the audio data block is longer than that of the video frame.

  As shown in FIG. 13A, the audio data block is recorded over the entire video file with sound, whereas the video frame is recorded only up to the reproduction time Te. After Te, it is a blank frame. In such a case, this system performs processing in the same manner as in the above embodiment until time Te, but after time Te, processing is performed from the 0th frame. Specifically, the processing control unit 40 recognizes that the video is recorded only until the reproduction time Te when the audio video file is first read from the audio video file storage unit 10, and the audio When the reproduction time T acquired for the video file exceeds Te, the remainder value obtained by dividing the reproduction time T by Te is passed to the priority calculation means 50 as the reproduction time. At this time, as shown in FIG. 13 (b), the video frame reproduced by time Te is stored in high-speed storage means such as a buffer memory, and the display control means 60 reads out from here and repeatedly reproduces it. Become. In actual calculations, the times T and Te are calculated using the frame number.

  The preferred embodiment of the present invention has been described above. However, the present invention is not limited to the above embodiment, and various modifications can be made. For example, in the above embodiment, when the selected video is displayed, it is displayed together with the background video. However, the background video may not be used. Further, in the above embodiment, when displaying a menu screen for selecting a sound-added video file, a menu video frame file in which a plurality of videos are combined is created and displayed. May be displayed as it is.

  In the above embodiment, the audio data is not compressed, but the compressed data may be recorded in a video file with sound in association with each video frame. In this case, the sound output means 70 is required to have a function of decoding and reproducing in accordance with the compression encoding method. However, since the amount of audio data is smaller than that of video data, there is not much problem in this system that handles a large amount of video frames.

  Further, in the above embodiment, when displaying video frames, they are displayed side by side as shown in FIG. 7, but processing such as masking of each video may be performed. In this case, mask data for identifying whether to overwrite each pixel is attached to the video frame of the video file with sound. Then, when displaying, the display control means 30 performs control not to display on some pixels of the video frame of each selected video file with sound based on the mask data.

It is a figure which shows the structure of the moving image file with a sound made into reproduction object by this invention. It is a figure which shows the signal waveform of each acoustic signal made into a synthetic | combination object. It is a figure which shows an example of the difference table between frames. It is a functional block diagram which shows one Embodiment of the reproduction | regeneration apparatus of a video with a sound concerning this invention. In this system, it is a figure which shows the initial state of the menu screen for selecting the object to reproduce | regenerate. It is a figure which shows the state of the menu screen in the state as which the reproduction | regeneration object was selected by the user. It is a figure which shows the structure of each layer which comprises the video display area of a menu screen. It is a figure which shows the state which displayed multiple selected images | videos. It is a figure which shows the interframe difference feature-value of five video files with a sound used as reproduction | regeneration object. It is a figure which shows the display / non-display of the flame | frame shown in FIG. It is a figure which shows the case where the interframe difference feature-value of a video file with sound changes. It is a figure which shows the display / non-display of the flame | frame shown in FIG. It is a figure which shows the process about the inside of a sound-added video file whose recording time of an audio data block is longer than a video frame.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Sound-added video file storage means 11 ... Inter-frame difference table 12 ... Demo reproduction sound file storage means 13 ... Menu video file storage means 14 ... Background video file storage means 20. ..File selection means 30 ... acoustic data block synthesis means 40 ... processing control means 50 ... priority calculation means 60 ... display control means 70 ... acoustic output means 80 ... video display device C: Frame display counter value c1-c5: Video segment selection cursor d (x, y): Pixel difference Di: Inter-frame difference feature amount E1-E5: Video segment F ...・ Sampling frequency n ・ ・ ・ Frame number Pi ・ ・ ・ Display priority

Claims (11)

A device that has a plurality of video frames and reproduces a plurality of video files with sound in which an audio data block is associated with each video frame,
An inter-frame difference table that records inter-frame difference features based on differences between video frames of a sound-added video file;
An audio data block synthesizing unit that extracts an audio data block attached to each corresponding video frame from the plurality of video files with sound, and synthesizes the audio data block into one synthesized audio data block;
A sound output means for writing the synthesized sound data block to a sound output device and reproducing the sound;
Processing control means for obtaining a reproduction time corresponding to the synthesized sound data block being reproduced in the sound output means;
Priority calculation means for calculating the priority for the video frame with reference to the inter-frame difference table for the video frame corresponding to the acquired reproduction time in each video file with sound;
It counts the display execution number of times of the execution display frame, based on the priority of the video frame of the video file with the sound corresponding to the playback time and the display execution times and acquisition, whether to display the video frame Display control means for writing the video frame determined to be displayed to the memory of the video display device and displaying it on the screen,
An apparatus for reproducing video with sound, comprising: A device that has a plurality of video frames and reproduces a plurality of video files with sound in which an audio data block is associated with each video frame,
An inter-frame difference table that records inter-frame difference features based on differences between video frames of a sound-added video file;
An audio data block synthesizing unit that extracts an audio data block attached to each corresponding video frame from the plurality of video files with sound, and synthesizes the audio data block into one synthesized audio data block;
A sound output means for writing the synthesized sound data block to a sound output device and reproducing the sound;
Processing control means for obtaining a reproduction time corresponding to the synthesized sound data block being reproduced in the sound output means;
Priority calculation means for calculating the priority for the video frame with reference to the inter-frame difference table for the video frame corresponding to the acquired reproduction time in each video file with sound;
Based on the priority of the video frame of each video file with sound corresponding to the acquired playback time, it is determined whether to display the video frame, and the video frame determined to be displayed is stored in the memory of the video display device Display control means for writing to and displaying on the screen ,
When no video frame exists in the sound-added video file after a predetermined playback time Te and only an audio data block is recorded, the display control means is configured such that the acquired playback time T is greater than the playback time Te. the reproducing apparatus of the sound with the image, characterized in der Rukoto those that substitute a reproduction time T to T% Te. In claim 2 ,
The video frame up to the reproduction time Te in the sound-added video file is stored in high-speed storage means that can be accessed at high speed, and the display control means reads out the video frame from the high-speed storage means. A sound video playback device. A device that has a plurality of video frames and reproduces a plurality of video files with sound in which an audio data block is associated with each video frame,
Sound-added video file storage means for storing a plurality of the sound-added video files;
File selection means for selecting a plurality of sound-added video files to be reproduced from the sound-added video file storage means;
An inter-frame difference table that records inter-frame difference features based on differences between video frames of a sound-added video file;
An audio data block synthesizing unit that extracts an audio data block attached to each corresponding video frame from the plurality of selected video files with sound, and synthesizes the audio data block into one synthesized audio data block;
A sound output means for writing the synthesized sound data block to a sound output device and reproducing the sound;
Processing control means for obtaining a reproduction time corresponding to the synthesized sound data block being reproduced in the sound output means;
Priority calculation means for calculating the priority for the video frame with reference to the inter-frame difference table for the video frame corresponding to the acquired reproduction time in each video file with sound;
Based on the priority of the video frame of each video file with sound corresponding to the acquired playback time, it is determined whether to display the video frame, and the video frame determined to be displayed is stored in the memory of the video display device Display control means for writing and displaying on the screen,
An apparatus for reproducing video with sound, comprising: In claim 4 ,
The file selection means displays a menu composite frame created by synthesizing video frames extracted from a predetermined sound-added video file among the sound-added video files stored in the sound-added video file storage means. A sound-equipped video reproduction apparatus, wherein a sound-added video file to be reproduced is selected by writing to a memory of a device, displaying on a screen, and selecting a video section on the screen. In claim 5 ,
It has a function of preparing a demo playback audio file in which only audio data is extracted from a sound-added video file in advance, and playing back the demo playback audio file corresponding to the video section selected on the screen. To play video with sound. In any one of Claims 1-6 ,
The acoustic data block synthesizing unit and the acoustic output unit continue processing until waiting for writing to the acoustic output device, and the processing control unit, when waiting for writing, the priority calculating unit, An apparatus for reproducing sound-added video, wherein each process in the display control means is executed, and the processing by the sound data block synthesizing means and the sound output means is performed again. In any one of Claims 1-7 ,
The sound data block synthesizing unit calculates one summation sound by calculating a sum of values corresponding to the sound data blocks for the sound data blocks attached to the video frames of the plurality of sound-added video files. An apparatus for reproducing video with sound, characterized in that it creates a data block. In any one of Claims 1-8 ,
The video frames of the audio video file stored in the audio video file storage means are recorded in a compressed and encoded state by a predetermined method, and the display control means An apparatus for reproducing sound-added video, wherein a video frame is extracted from a video file and then decoded, and the decoded video frame is written in a memory of the video display device. In any one of Claims 1-9 ,
The sound data block recorded in the sound-added video file stored in the sound-added video file storage means is recorded in a state compressed and encoded by a predetermined method, and the sound data block combining means The sound-added video is characterized in that one audio data block attached to the video frames of the selected plurality of sound-added video files is extracted and then decoded to create one composite audio data block. Playback device. A program for causing a computer to function as the sound-added video reproduction apparatus according to any one of claims 1 to 10 .

Images