JP7136458B2 - Encoding device, playback device, creation device, playback system, encoding method, playback method, creation method, and program - Google Patents

Description

The present invention relates to an encoding device, a playback device, a creation device, a playback system, an encoding method, a playback method, a creation method, and a program.

A system for distributing or storing moving images by computer is used.

In such systems, MPEG-1, MPEG-2, MPEG-4, H.264, etc. are used in order to reduce the amount of moving image data. H.264 and other image coding techniques are used. In these image coding techniques, in order to maintain image quality and reduce the amount of data, intra-frame coding and inter-frame coding are used to generate a coded frame.

Patent document 1 describes an encoding method using intra-frame encoding and inter-frame encoding, in which intra-encoded frames are encoded using positions on a storage device where intra-encoded frames are recorded. Techniques for performing the search are disclosed.

Japanese Patent No. 2773594

When moving pictures are distributed using intra-frame coding and inter-frame coding, in a device that receives and processes moving pictures, whether each frame constituting the moving picture is an intra-coded frame or a frame If it is unclear whether the frame is inter-encoded, there is a problem that appropriate processing is not performed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide an encoding apparatus and the like that enable appropriate processing of moving images.

In order to solve the above problems, an encoding device according to an aspect of the present invention includes an acquisition unit that sequentially acquires a plurality of frames that form a moving image; a coding unit that intra-codes the first frame of and inter-codes a second frame other than the first frame among the plurality of frames acquired by the acquisition unit; and a communication interface that communicates with the processing device. and a control unit that transmits interval information indicating the predetermined interval to the processing device, and sequentially transmits the plurality of frames encoded by the encoding unit to the processing device.

According to this, the encoding device provides the processing device with information on the spacing of the intra-coded frames together with the plurality of frames. A processing device that receives the interval information uses the received interval information to identify which frame among a plurality of frames received from the encoding device is an intra-frame encoded frame, and Processing of the moving image using the encoded frames (specifically, reproduction processing, file creation processing, etc.) can be executed. In this way, the coding device enables proper video processing in the processing device.

The control unit acquires quality information indicating the quality of communication performed by the communication interface with the processing device, and the encoding unit converts the first frame according to the quality indicated by the acquired quality information. A frame interval for intra-frame encoding may be changed.

According to this, the encoding device transmits a plurality of frames in which intra-frame encoded frames are inserted at different frame intervals according to the quality of communication between the encoding device and the processing device. In general, the smaller the frame interval of intra-frame coded frames, the higher the resistance to loss. Also, in a communication channel with a larger bandwidth, it is permissible to increase the amount of communication data by increasing the number of intra-frame coded frames. In addition, loss may be allowed to some extent depending on the type of moving image. Therefore, by changing the frame interval for intra-frame encoding according to the communication quality, it is possible to suppress the influence of possible loss while appropriately maintaining the amount of communication data. Therefore, the encoding device allows appropriate video processing in the processing device while maintaining an appropriate amount of communication data in consideration of communication quality.

Further, the control unit may change the frame interval to a larger value as the quality indicated by the acquired quality information is higher.

According to this, the encoding device transmits a plurality of frames in which intra-frame encoded frames are inserted at larger frame intervals as the quality of communication between the encoding device and the processing device is higher. When communication quality is relatively low, frame loss can occur with a relatively high probability. Therefore, the encoding device allows appropriate video processing in the processing device while maintaining an appropriate amount of communication data in consideration of communication quality.

Further, a playback device according to an aspect of the present invention includes a display unit, and a plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame encoded, and receiving a plurality of frames in which a second frame other than the first frame among the plurality of frames is inter-frame encoded, from an encoding device, decoding the received plurality of frames, and displaying the moving image and a decoding unit that receives interval information indicating an interval of the first frame among the plurality of frames from the encoding device, and reproduces an image based on the received interval information together with the moving image that is being played. and a control unit for displaying on the display unit.

According to this, the playback device displays a moving image based on a plurality of frames received from the encoding device and an image based on intervals between intra-frame encoded frames in the moving image. A user viewing such a display can know not only the moving image but also the inter-frame coded frame interval. can be avoided. Therefore, the playback device as the processing device can appropriately process the moving image.

In addition, the control unit causes the display unit to display a bar indicating a playback time position in the moving image together with the moving image being played, and displays the time position of the first frame included in the moving image in the bar. A figure indicating the time position of the first frame in the bar specified using the interval information may be displayed on the display unit.

According to this, the playback device displays the position of the intra-encoded frame in the video on a bar (so-called seek bar) indicating the playback time position in the video, based on the interval between the intra-encoded frames. The user can know the position of the intra-coded frame on the seek bar. Here, when playback is started from a frame that is not an intraframe-encoded frame (that is, an interframe-encoded frame), the frames from the latest intraframe-encoded frame before that frame to the current frame must be decoded. Therefore, it takes time for the decoding process. On the other hand, when playback is started from an intraframe-encoded frame, it is sufficient to decode the frame, and the time required for the decoding process is relatively short. Therefore, when the user prompts an operation to start reproduction from the intra-frame-encoded frame, the reproduction device as the processing device can appropriately process the moving image.

Further, when detecting a loss of a second frame among the plurality of frames, the control unit uses the interval information to determine the first frame after the second frame in which the loss was detected. may be specified, and decoding by the decoding unit may be prohibited from detection of the loss to arrival of the timing.

According to this, when a loss of an inter-coded frame is detected, the reproduction device prohibits decoding processing of the inter-coded frame that cannot be decoded due to the loss. Therefore, the playback device as a processing device can appropriately process moving images by reducing the amount of processing involved in frame decoding processing.

Further, the creating apparatus according to an aspect of the present invention further includes: (a) a plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame encoded; a plurality of frames in which a second frame other than the first frame out of the frames is inter-encoded, and interval information indicating an interval between intra-encoded frames out of the plurality of frames are encoded. (b) using the received interval information to identify an interframe-encoded frame among the received plurality of frames; a creation unit that creates a moving image file using one or more frames excluding at least part of the frames specified by the unit.

According to this, since the creation device creates a moving image file using a part of the plurality of frames received from the encoding device, the size of the moving image file to be created is reduced compared to the case where the above part is used. can be reduced. Also, when creating the above file, if a video file is created without including intra-frame encoded frames, it is impossible to decode the inter-encoded frames encoded with reference to the above intra-frame encoded. become. Therefore, by creating a moving image file using all the intra-frame coded frames included in the plurality of frames, it is possible to avoid the occurrence of frames that cannot be decoded as described above. Therefore, the creation device as a processing device can appropriately process moving images.

Further, the creation unit creates the moving image file having a frame rate equal to or lower than a predetermined upper limit by using one or more frames excluding at least part of the frames specified by the specifying unit among the plurality of frames. may be created.

According to this, the creation device determines the frames to be added to the moving image file so that the frame rate of the created moving image file does not exceed the upper limit. This makes it possible to reduce the size of the moving image file to be created. Therefore, the creation device as a processing device can appropriately process moving images.

Further, a reproduction system according to an aspect of the present invention includes the above-described encoding device, and the above-described playback device that receives the interval information and the plurality of frames from the encoding device.

According to this, the same effect as that of the reproducing apparatus described above can be obtained.

Further, the encoding method according to an aspect of the present invention includes: an acquisition step of sequentially acquiring a plurality of frames constituting a moving image; an encoding step of inter-frame encoding a second frame other than the first frame among the plurality of frames acquired in the acquiring step; and transmitting interval information indicating the predetermined interval to a processing device. and a control step of transmitting the plurality of frames encoded in the encoding step to the processing unit.

According to this, the same effect as that of the encoding device described above can be obtained.

Further, in the reproduction method according to an aspect of the present invention, a plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame encoded, Decoding for receiving a plurality of frames, of which the second frame other than the first frame is interframe-encoded, from an encoding device, decoding the received plurality of frames, and reproducing the moving image on a display unit a step of receiving interval information indicating an interval of the first frame among the plurality of frames from the encoding device, and displaying an image based on the received interval information on the display unit together with the moving image being reproduced; and a control step.

According to this, the same effect as that of the reproducing apparatus described above can be obtained.

A program according to an aspect of the present invention is a program for causing a computer to execute the above reproducing method.

According to this, the same effect as that of the reproducing apparatus described above can be obtained.

Further, in the creation method according to an aspect of the present invention, a moving image is composed of a plurality of frames, wherein first frames at predetermined intervals among the plurality of frames are intra-frame encoded, and a plurality of frames in which a second frame other than the first frame is inter-encoded, and interval information indicating an interval between intra-encoded frames among the plurality of frames, from an encoding device; a receiving step of receiving; a identifying step of identifying an interframe-coded frame among the plurality of received frames using the received interval information; and a identifying step among the plurality of frames identified in the identifying step and a creating step of creating a moving image file using one or more frames excluding at least part of the frames.

According to this, there exists an effect similar to the said production apparatus.

In order to solve the above problems, a program according to one aspect of the present invention is a program for causing a computer to execute the above processing method.

According to this, the same effect as that of the communication terminal described above can be obtained.

The present invention can be realized not only as an apparatus, but also as a method in which processing means constituting the apparatus are used as steps, as a program for causing a computer to execute the steps, or as a computer-readable program in which the program is recorded. It can also be implemented as a recording medium such as a CD-ROM, or as information, data, or signals indicating the program. These programs, information, data and signals may then be distributed over a communication network such as the Internet.

The invention allows the encoding device to process the moving images appropriately.

FIG. 1 is an explanatory diagram schematically showing the overall system configuration and functional blocks of an encoding device according to an embodiment. FIG. 2 is an explanatory diagram showing the configuration of the playback device according to the embodiment. FIG. 3 is an explanatory diagram illustrating an example of an image displayed on a display unit according to the embodiment; FIG. 4 is a flowchart showing processing of the encoding device according to the embodiment. FIG. 5 is a flow chart showing processing of the display device according to the embodiment. FIG. 6 is a flow diagram showing a process in which the control unit according to the embodiment controls whether or not the decoding unit can execute the decoding process. FIG. 7 is a sequence diagram illustrating processing in which the control unit according to the embodiment controls decoding processing by the decoding unit. FIG. 8 is an explanatory diagram showing the configuration of a creation device according to a modification of the embodiment. FIG. 9 is an explanatory diagram of processing performed on a frame by a specifying unit and a creating unit according to the modification of the embodiment; FIG. 10 is a flowchart showing processing of the creation device according to the modification of the embodiment.

Hereinafter, embodiments will be specifically described with reference to the drawings.

All of the embodiments described below represent preferred specific examples of the present invention. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in independent claims representing the highest level concept of the present invention will be described as arbitrary constituent elements constituting more preferred embodiments. In addition, the same code|symbol may be attached|subjected to the same component and description may be abbreviate|omitted.

(Embodiment)
In the present embodiment, an encoding device and the like that enable appropriate processing of moving images will be described.

FIG. 1 is an explanatory diagram schematically showing the overall configuration of a reproduction system 1 and functional blocks of an encoding device 10 according to this embodiment.

As shown in FIG. 1, the playback system 1 includes a camera 5, an encoding device 10, and a playback device 20. In FIG. The encoding device 10 and the reproducing device 20 are connected via a network N. FIG.

The camera 5 is an imaging device that captures an image of the surroundings and generates a frame that is image data. The camera 5 provides the encoding device 10 with frames generated by shooting. The speed at which the camera 5 captures images and generates frames is, for example, 30 fps (frames per second). Although this case will be described as an example, it is not limited to this.

The encoding device 10 is an encoding device that acquires frames generated by the camera 5 , encodes the acquired frames, and transmits the encoded frames to the playback device 20 via the network N. The encoding device 10 encodes frames using intra-frame encoding and inter-frame encoding in order to reduce communication traffic over the network N. FIG. The encoding device 10 also transmits interval information indicating the interval between intra-frame encoded frames to the playback device 20 . The processing of the encoding device 10 will be described in more detail later.

The playback device 20 is a playback device that receives encoded frames from the encoding device 10, decodes the received frames, and plays back moving images. The playback device 20 plays back the video by decoding the received frames. Further, the reproducing device 20 receives the interval information transmitted by the encoding device 10 and uses it for reproducing frames. Note that the playback device 20 is an example of a processing device that processes moving images.

The encoding device 10 will be described in more detail.

As shown in FIG. 1 , encoding device 10 includes acquisition section 11 , encoding section 12 , communication IF (interface) 13 , and control section 14 . The functional units provided in the encoding device 10 can be realized by a processor executing a predetermined program.

The acquisition unit 11 is a processing unit that sequentially acquires a plurality of frames that form a moving image. The acquisition unit 11 is connected to the camera 5 and sequentially acquires a plurality of frames from the camera 5 through the connection. The connection between the acquisition unit 11 and the camera 5 may be wired (that is, cable) or wireless.

The encoding unit 12 is a processing unit that encodes the frame acquired by the acquisition unit 11 . The encoding unit 12 performs intra-frame encoding on frames (corresponding to the first frame) at predetermined intervals among the plurality of frames acquired by the acquisition unit 11, and encodes frames other than the first frame among the plurality of frames acquired by the acquisition unit 11. (corresponding to the second frame) is interframe-encoded. The predetermined interval is, for example, 30 frames, and this value is also called an interval value. Intra-frame coding is coding that uses only information within the frame. Inter-frame coding refers to coding that refers to a frame prior to the current frame and uses information that includes the difference between the frame being referenced and the frame to be coded. A frame intra-encoded by the encoding unit 12 is also referred to as an intra-encoded frame (corresponding to an I frame or key frame in standards such as MPEG-1), and an inter-encoded frame is called an inter-encoded frame. (P frame or B frame in standards such as MPEG-1).

The communication IF 13 is a communication interface device that communicates with the playback device 20, which is an example of a processing device. The communication IF 13 is communicably connected to the playback device 20 via the network N according to a predetermined communication standard. The communication standard may be a wireless communication standard such as IEEE802.11a, b, g, or n, or a wired communication standard such as IEEE802.3. Also, the network N may include a local area network (LAN) in a company or factory, a mobile phone carrier network, the Internet, and the like.

The control unit 14 is a processing unit that controls transmission of information or data to the playback device 20 . The control unit 14 transmits the interval information to the reproduction device 20 when a moving image acquisition request (simply referred to as a request) is received from the reproduction device 20 via the communication IF 13 . Here, the interval information is an interval value that is information indicating the interval between frames intra-encoded by the encoding unit 12 . Further, when receiving the request, the control unit 14 sequentially transmits the plurality of frames encoded by the encoding unit 12 to the playback device 20 .

Here, the control unit 14 acquires quality information indicating the quality of communication performed by the communication IF 13, and changes the frame interval at which the encoding unit 12 intraframe-encodes frames according to the quality indicated by the acquired quality information. You can also Here, the quality information indicating the quality of communication may be, for example, information indicating that the lower the loss rate in communication, the higher the quality, or information indicating that the smaller the delay in communication, the higher the quality. . Note that when the quality of communication performed by the communication IF 13 changes with time, the acquired quality information also changes. can also be changed on the fly. Note that when the encoding unit 12 changes the frame interval for intra-coding frames as described above, interval information (that is, an interval value) indicating the frame interval after the change is transmitted to the reproduction device 20. good too.

Furthermore, the control unit 14 may change the frame interval to a longer value as the quality indicated by the acquired quality information is higher. If the quality is relatively high, the interval between intra-coded frames can be increased, that is, the number of intra-coded frames can be reduced, but the quality of the reproduced video can be maintained high, so the frame interval is increased. This is in order to reduce the amount of communication by doing so.

FIG. 2 is an explanatory diagram showing the configuration of the playback device 20 according to this embodiment.

As shown in FIG. 2 , the playback device 20 includes a communication IF 21 , a decoding section 22 , a display section 23 and a control section 24 . The above-described functional units included in the playback device 20 can be realized by the processor executing a predetermined program.

Communication IF 21 is a communication interface device that communicates with encoding device 10 . The communication IF 21 is communicably connected to the encoding device 10 via the network N according to a predetermined communication standard. The communication standard is the same as that of the communication IF 13 .

The decoding unit 22 is a processing unit that decodes frames received from the encoding device 10 via the communication IF 21 and causes the display unit 23 to reproduce moving images. The decoding unit 22 performs intra-frame coding on a plurality of frames constituting a moving image (corresponding to the first frame) at predetermined intervals among the plurality of frames, and decodes the first frame of the plurality of frames. A plurality of frames other than the frame (corresponding to the second frame) are interframe-encoded are received from the encoding device 10, the received plurality of frames are decoded, and the display unit 23 reproduces the moving image. Also, the position where the decoding unit 22 starts reproduction is controlled by the control unit 24 .

In the frame decoding by the decoding unit 22, the intra-frame encoded frame can be decoded as a single frame. On the other hand, an inter-coded frame can be decoded by tracing back to the latest intra-coded frame before the current frame and sequentially decoding the inter-coded frames from the intra-coded frame to the current frame. be. Therefore, playback can be started in a relatively short time when playback is started from an intra-coded frame, while the time required to start playback is compared when playback is started from an inter-coded frame. long. The larger the time difference between the inter-coded frame to be reproduced and the latest intra-coded frame preceding it, the longer the time required to start reproduction.

Note that the decoding unit 22 may prohibit frame decoding for a certain period of time under the control of the control unit 24 when the loss of the inter-coded frame is detected. Specifically, when detecting a loss of an inter-coded frame among a plurality of frames, the control unit 24 uses the interval information to determine the first inter-coded frame after the inter-coded frame in which the loss is detected. Specifies the timing of receiving an intra-coded frame. Then, the control unit 24 prohibits decoding by the decoding unit 22 from the detection of the loss to the arrival of the timing. By doing so, the reproducing device 20 reproduces one or more inter-encoded frames after the inter-encoded frame in which the loss is detected and one or more inter-encoded frames before the specified intra-encoded frame. Decoding of inter-coded frames by the decoding unit 22 can be prohibited.

The display unit 23 is a display device that displays an image. The display of images by the display unit 23 is controlled by the control unit 24 . The display unit 23 is, for example, a liquid crystal display panel or an organic EL (Electroluminescence) panel.

The control unit 24 is a processing unit that controls decoding by the decoding unit 22 and display by the display unit 23 . The control unit 24 receives interval information indicating the interval of the first frame among the plurality of frames from the encoding device 10, and causes the display unit 23 to display an image based on the received interval information together with the moving image being reproduced. The control unit 24 causes the display unit 23 to display a bar indicating a playback position in the moving image, that is, a so-called seek bar, together with the moving image being reproduced. The seek bar indicates the playback position of the moving image. Note that the seek bar will be described in detail with reference to FIG. The playback position is also called a playback time position.

Also, the control unit 24 receives an operation on the seek bar for changing the playback position of the moving image from the user via a user interface (specifically, a touch panel, a mouse, or the like). When receiving the operation, the control unit 24 starts reproducing the moving image from the reproduction position corresponding to the position on the seek bar that received the operation.

The control unit 24 specifies the position (also referred to as time position) of the intra-frame encoded frame included in the moving image in the seek bar using the interval information, and causes the display unit 23 to display a figure indicating the specified position. When the user who has visually recognized the displayed graphic tries to start playing the video from a desired position in the video by operating the seek bar, the video can be played in a relatively short time if the video is started from the position of the graphic. is displayed.

Note that the control unit 24 may transmit a moving image request to the encoding device 10 in advance. In this case, the decoding unit 22 and the control unit 24 respectively receive a plurality of frames and interval information that the encoding device 10 transmits in response to receiving the request.

FIG. 3 is an explanatory diagram showing an example of an image displayed on the display unit 23 according to this embodiment.

The image displayed on display unit 23 shown in FIG. 3 includes content 30, seek bar 35, and figure 36. FIG.

The content 30 is video content configured by a plurality of frames received from the encoding device 10 .

The seek bar 35 is a bar that indicates the playback position within the moving image. The left end of the seek bar 35 is associated with the initial position of the moving image, and the right end of the seek bar 35 is associated with the final position of the moving image. Each position between the left end and the right end of the seek bar 35 is associated with a reproduction position having a later reproduction time in the moving image toward the right. The seek bar 35 generally has a linear shape as shown in FIG. 3, but the shape is not limited to this, and may be curved. Further, the playback position in the moving image indicates a position where the playback time becomes later as it progresses to the right, but it is not limited to this.

A figure 36 is a figure indicating the position of the intra-frame encoded frame on the seek bar 35 . Specifically, a graphic 36A of the graphic 36 indicates the first position of the moving image. This is because an intra-frame coded frame is always positioned at the beginning of a moving image.

Also, a figure 36B shows a position where the interval of the intra-frame encoded frames indicated by the interval information is advanced from the initial position of the moving image. A figure 36C shows a position further advanced by the distance from the position indicated by the figure 36B. The same applies thereafter. Although the figure 36 is illustrated as having a rectangular shape, the shape may be a triangle, a circle, an arrow, or any other shape.

FIG. 4 is a flowchart showing processing of the encoding device 10 according to this embodiment.

As shown in FIG. 4 , in step S<b>101 , the acquisition unit 11 determines whether or not a moving image request has been received from the playback device 20 . If the request has been received (Yes in step S101), the process proceeds to step S102; otherwise (No in step S101), step S101 is executed again. That is, the acquisition unit 11 waits in step S101 until receiving a request.

In step S102, the control unit 14 transmits the interval information to the reproducing device 20 in response to the request received in step S101.

In step S103, the encoding unit 12 initializes the frame count to zero, for example. A frame count is a serial number associated with each frame in order of encoding processing.

In step S<b>104 , the encoding unit 12 acquires the frame acquired by the acquisition unit 11 from the camera 5 .

In step S105, the encoding unit 12 determines whether the remainder (modulo) of the frame count by the interval value is zero. If the remainder is zero (Yes in step S105), proceed to step S106; otherwise (No in step S105), proceed to step S107.

In step S106, the encoding unit 12 intraframe-encodes the frame obtained in step S104.

In step S107, the encoding unit 12 interframe-encodes the frame acquired in step S104.

In step S<b>108 , the encoding unit 12 transmits the frame intra-encoded in step S<b>106 or inter-encoded in step S<b>107 to the playback device 20 .

In step S109, the encoding unit 12 increments the frame count by 1, and proceeds to step S104.

As a result, the encoding device 10 performs intra-frame encoding on a plurality of frames constituting a moving image at predetermined intervals, and inter-frame encoding on other frames, and sequentially transmits the encoded frames to the processing device.

FIG. 5 is a flowchart showing processing of the reproducing device 20 according to this embodiment.

As shown in FIG. 5 , in step S<b>201 , the control unit 24 transmits a moving image request to the encoding device 10 via the communication IF 21 .

In step S<b>202 , the control unit 24 determines whether interval information has been received from the encoding device 10 . If it is determined that the interval information has been received (Yes in step S202), the process proceeds to step S203. Otherwise (No in step S202), step S202 is executed again. That is, the control unit 24 waits in step S202 until the interval information is received.

In step S203, based on the interval information received in step S202, the control unit 24 determines at which position the intra-frame-encoded frame is reproduced in the entire moving image, that is, the intra-encoding position in the entire moving image. Identify the playback position of the frame and display the identified position on the seek bar.

In step S<b>204 , the decoding unit 22 determines whether or not a frame has been received from the encoding device 10 . If the frame has been received, proceed to step S205; otherwise, perform step S204 again. That is, the decoding unit 22 waits in step S204 until the frame is received. Note that if no frame is received even after waiting for the predetermined time, it may be determined that a timeout has occurred with respect to frame reception, and step S204 may be executed again. When a timeout occurs, predetermined error processing such as outputting an error message indicating the timeout may be executed.

In step S<b>205 , the decoding unit 22 decodes the frame received in step S<b>205 and reproduces the moving image on the display unit 23 . After finishing step S205, step S204 is executed.

As a result, the playback device 20 displays the motion based on the frames received from the encoding device 10 and also displays the position of the intra-frame encoded frame in the moving image.

Next, processing for prohibiting frame decoding when the decoding unit 22 detects loss of an inter-frame encoded frame will be described in detail.

FIG. 6 is a flow chart showing the process of controlling whether or not the decoding section 22 can execute the decoding process by the control section 24 of the present embodiment.

As shown in FIG. 6, in step S221, the control unit 24 determines whether or not a loss of an inter-coded frame has been detected. If a loss is detected, the process proceeds to step S222; otherwise, step S221 is executed again. That is, the control unit 24 waits in step S221 until the loss of the inter-frame encoded frame is detected.

It should be noted that the detection of the loss of the inter-coded frame in step S221 means that the next frame to be received is the inter-coded frame, and the time-out has occurred in the reception of the frame (step S204 in FIG. 5). Sometimes it can be done.

In step S<b>222 , the control unit 24 stops decoding processing by the decoding unit 22 . Even if the communication IF 21 receives a frame, the decoding unit 22 does not perform the decoding process until the decoding process is started (step S225, which will be described later) after the decoding process is stopped.

In step S223, the control unit 24 specifies the intra-frame coded frame to be received next and its decoding timing. Specifically, the control unit 24 advances the timing corresponding to the frame interval of the intra-coded frame from the decoding timing of the intra-coded frame preceding the inter-coded frame in which the loss is detected. , is specified as the decoding timing of the next intraframe-encoded frame.

In step S224, the control unit 24 determines whether or not the decoding timing of the intra-frame coded frame identified in step S223 has arrived. If it is determined that the decoding timing of the intra-frame encoded frame has arrived, the process proceeds to step S225, otherwise step S224 is executed again. That is, the control unit 24 waits in step S224 until the decoding timing of the intra-frame encoded frame arrives.

In step S<b>225 , the control unit 24 starts (that is, restarts) the decoding process by the decoding unit 22 . As a result, the decoding unit 22 decodes the intra-frame coded frame identified in step S223, and sequentially decodes subsequent inter-frame coded frames. After that, the process of step S221 is executed.

In this manner, the control unit 24 controls to stop or start the decoding process by the decoding unit 22 .

FIG. 7 is a sequence diagram showing the process of controlling the decoding process by the decoding section 22 by the control section 24 of the present embodiment.

As shown in FIG. 7, playback device 20 transmits a request to encoding device 10, and encoding device 10 transmits interval information and frame #1 in response to the request. The reproduction device 20 performs decoding processing on the transmitted frame #1 and reproduces the frame #1.

After that, the encoding device 10 sequentially transmits a plurality of frames #2, #3, . . . , #10. The reproduction device 20 sequentially receives the plurality of frames, decodes them, and reproduces them.

Assume that the frame #11 transmitted by the encoding device 10 is lost in the communication channel.

The playback device 20 waits for the frame #11 for a predetermined time-out period after receiving the frame #11, and detects that the frame #11 is lost because the frame #11 cannot be received during this time. In other words, a loss is detected for frame #11. Triggered by the detection of such a loss, the control unit 24 of the playback device 20 stops the frame decoding by the decoding unit 22 (steps S221 and S222). It should be noted that the predetermined timeout period may adopt a value determined by the communication standard or the moving image transfer standard, or may be a value determined by other arbitrary conditions.

Next, the control unit 24 identifies the next intra-frame encoded frame as frame #31 (step S223). By advancing 30 frames, which is the frame interval of the intra-frame coded frames, from the first intra-frame coded frame (frame #1), the frame #31 can be specified. Further, the control unit 24 specifies the point of time after one second from the reception timing of the frame #1 as the reception timing of the frame #31.

After that, the encoding device 10 sequentially transmits frames #12, #13, . . . , #30. Since the playback device 20 has stopped the decoding process, even if these frames are received, they are not decoded, and as a result, these frames are not played back.

When the reception timing of frame #31 arrives, the control unit 24 starts (that is, restarts) the decoding process by the decoding unit 22 . After that, when the reproducing device 20 receives the frame #31, the decoding process is performed on the frame #31, and the frame #31 is reproduced.

After that, the encoding device 10 sequentially transmits a plurality of frames #32, #33, . The reproduction device 20 sequentially receives the plurality of frames, decodes them, and reproduces them.

By doing so, the processing amount of the playback device 20 can be reduced by not performing, that is, prohibiting, the decoding processing of the inter-encoded frames that become undecodable due to the loss of the inter-encoded frames. Therefore, it is possible to appropriately process moving images.

(Modification of Embodiment)
In this modified example, a configuration in which a creating device 40 is used instead of the reproducing device 20 in the reproducing system 1 shown in the above embodiment will be described. Note that the creation device 40 is an example of a processing device.

The overall configuration of the playback system 1 according to this modification corresponds to the creation device 40 instead of the playback device 20 in FIG. 1 (not shown).

FIG. 8 is an explanatory diagram showing the configuration of the creation device 40 according to this modification.

As shown in FIG. 8 , the creating device 40 includes a communication IF 41 , a specifying unit 42 and a creating unit 43 . The functional units provided in the creation device 40 can be implemented by a processor executing a predetermined program.

Communication IF 41 is a communication interface device that communicates with encoding device 10 . The communication IF 41 is communicably connected to the encoding device 10 via the network N according to a predetermined communication standard. The communication standard is the same as that of the communication IF 13 .

The specifying unit 42 specifies whether the plurality of frames received from the encoding device 10 are intra-frame encoded frames or inter-frame encoded frames.

Specifically, the specifying unit 42 determines that the first frames of the plurality of frames constituting the moving image are intra-frame encoded at a predetermined interval, and the first frame of the plurality of frames is the first frame. A plurality of frames in which a second frame other than one frame is interframe-encoded, and interval information indicating an interval between intraframe-encoded frames among the plurality of frames are received from the encoding device 10 . Then, using the received interval information, the specifying unit 42 specifies an interframe-encoded frame among the plurality of received frames.

The creating unit 43 creates a moving image file using one or more frames excluding at least some of the frames specified by the specifying unit 42 among the plurality of frames. When creating a moving image file, the creation unit 43 sequentially adds a plurality of frames received from the encoding device 10 to the moving image file. At this time, the creation unit 43 creates the moving image file having a frame rate equal to or lower than a predetermined upper limit by using one or more frames excluding at least part of the frames specified by the specifying unit 42 among the plurality of frames. do. The predetermined upper limit is, for example, 15 fps or 20 fps. When the predetermined upper limit is 15 fps, the creation unit 43 uses one or more frames among a plurality of frames constituting a video file with a frame rate of 30 fps transmitted by the encoding device 10 to set the frame rate to 15 fps. Create the above video file.

Note that although the above upper limit value is transmitted from the encoding device 10 to the creation device 40 as an example, it may be preset in the creation device 40 .

FIG. 9 is an explanatory diagram showing the processing performed on the frame by the specifying unit 42 and the creating unit 43 according to this modification. Here, an example will be described in which the received interval information (that is, the interval value) is 30 frames and the upper limit of the frame rate of the created moving image file is 20 fps.

(a) of FIG. 9 shows a frame received by the identifying unit 42 from the encoding device 10 . Here, the frame received first after the specifying unit 42 transmits the request to the encoding device 10 is numbered #1, and the frames received thereafter are numbered #2, #3, . It is numbered consecutively.

(b) of FIG. 9 shows the result of the frame identification processing by the identification unit 42 . The identifying unit 42 identifies the first received frame #1 after transmitting the request to the encoding device 10 as an intra-frame encoded frame. The identifying unit 42 also identifies frame #31, which is located 30 frames ahead of frame #1 as indicated by the interval information, as an intra-frame coded frame. After that, the specifying unit 42 also specifies the frame located 30 frames ahead of the frame specified as the intra-frame-encoded frame as the intra-frame-encoded frame.

In addition, the identification unit 42 identifies frames that have not been identified as intra-coded frames, such as frames #2, #3, . and specify.

Next, the specifying unit 42 specifies, among the inter-frame-encoded frames, frames that are determined to exceed a predetermined frame rate if added to the moving image file created by the creating unit 43, as frames not to be added to the moving image file. In addition, the identifying unit 42 identifies all the frames received by the identifying unit 42 as frames to be added to the moving image file, excluding frames not to be added to the moving image file. In FIG. 9 , reference M indicates “frames to be added to the moving image file”, and reference N indicates “frames not to be added to the moving image file”.

(c) of FIG. 9 shows frames included in the moving image file created by the creating unit 43 . The frames added to the moving image file created by the creating unit 43 are the frames specified as “frames to be added to the moving image file” among the frames received by the specifying unit 42 . In other words, among the frames received by the specifying unit 42, the frames specified as “frames not added to the moving image file” (specifically, frames #3, #6, . . . , #30 in FIG. 9) are , is not included in the moving image file created by the creation unit 43 .

By doing so, when the frame rate of the received frame exceeds the upper limit of the frame rate of the moving image file to be created (that is, 20 fps), the creating unit 43 converts the intra-frame encoded frames into the moving image file. A video file can be created that has a frame rate that is equal to or less than the upper limit while still being included in the upper limit.

FIG. 10 is a flowchart showing processing of the creating device 40 according to this modification. The processing shown in FIG. 10 is a flowchart showing processing for lowering the frame rate of a moving image file using interval information and the upper limit value of the frame rate.

As shown in FIG. 10 , in step S<b>301 , the identifying unit 42 transmits a request to the encoding device 10 via the communication IF 41 .

In step S<b>302 , the specifying unit 42 determines whether or not the interval information and the upper limit value have been received from the encoding device 10 . If it is determined that the interval information and the upper limit value have been received (Yes in step S302), the process proceeds to step S303. Otherwise (No in step S302), step S302 is executed again. That is, the specifying unit 42 waits in step S302 until receiving the interval information and the upper limit value.

In step S303, the specifying unit 42 calculates the thinning frame rate. The thinning frame rate is the number of frames thinned per unit time from a plurality of frames forming a moving image received from the encoding device 10, and is calculated by the following (Equation 1).

Thinning frame rate = video frame rate - upper limit (formula 1)

In step S304, the specifying unit 42 calculates a thinning interval value. The thinning interval value is a numerical value indicating the interval of frames to be thinned out of a plurality of frames forming the moving image received from the encoding device 10, and is calculated by the following (Equation 2).

Thinning interval value = Video frame rate ÷ Thinning frame rate (Formula 2)

In step S<b>305 , the specifying unit 42 determines whether or not a frame has been received from the encoding device 10 . If it is determined that a frame has been received (Yes in step S305), the process proceeds to step S306, otherwise (No in step S305), step S305 is executed again. In other words, the specifying unit 42 waits in step S305 until the frame is received.

In step S306, the identifying unit 42 increments the frame count by 1, and proceeds to step S307.

In step S307, the identifying unit 42 determines whether or not the remainder (modulo) of the frame count by the thinning interval value is zero (in other words, whether or not the frame count is an integral multiple of the thinning interval value). do. If the remainder is zero (Yes in step S307), proceed to step S308; otherwise (No in step S307), proceed to step S309.

In step S308, the creation unit 43 does not add the inter-frame-encoded frame received in step S305 to the moving image file, in other words, prohibits adding the frame to the moving image file. After completing step S308, the process proceeds to step S305.

In step S309, the creating unit 43 adds the inter-frame encoded frames received in step S305 to the moving image file. After completing step S309, the process proceeds to step S305.

Thereby, the creation device 40 creates a moving image file including all intra-frame coded frames and at least part of inter-frame coded frames. If an intra-coded frame is lost, it becomes impossible to decode an inter-coded frame by referring to the intra-coded frame. Since the moving image file created by the creation device 40 includes all inter-encoded frames, the creation device 40 avoids the possibility that the inter-encoded frames cannot be decoded as described above. , you can create a video file that is less than a predetermined upper limit.

As described above, the encoding device according to the present embodiment provides the processing device with interval information between intra-frame encoded frames together with a plurality of frames. A processing device that receives the interval information uses the received interval information to identify which frame among a plurality of frames received from the encoding device is an intra-frame encoded frame, and Processing of the moving image using the encoded frames (specifically, reproduction processing, file creation processing, etc.) can be executed. In this way, the coding device enables proper video processing in the processing device.

Also, the encoding device transmits a plurality of frames in which intra-frame encoded frames are inserted at different frame intervals according to the quality of communication between the encoding device and the processing device. In general, the smaller the frame interval of intra-frame coded frames, the higher the resistance to loss. Also, in a communication channel with a larger bandwidth, it is permissible to increase the amount of communication data by increasing the number of intra-frame coded frames. In addition, loss may be allowed to some extent depending on the type of moving image. Therefore, by changing the frame interval for intra-frame encoding according to the communication quality, it is possible to suppress the influence of possible loss while appropriately maintaining the amount of communication data. Therefore, the encoding device allows appropriate video processing in the processing device while maintaining an appropriate amount of communication data in consideration of communication quality.

Also, the encoding device transmits a plurality of frames in which intra-frame encoded frames are inserted at larger frame intervals as the quality of communication between the encoding device and the processing device is higher. When communication quality is relatively low, frame loss can occur with a relatively high probability. Therefore, the encoding device allows appropriate video processing in the processing device while maintaining an appropriate amount of communication data in consideration of communication quality.

Also, the playback device according to the present embodiment displays a moving image based on a plurality of frames received from the encoding device, and images based on intervals between intra-frame encoded frames in the moving image. A user viewing such a display can know not only the moving image but also the inter-frame coded frame interval. can be avoided. Therefore, the playback device as the processing device can appropriately process the moving image.

Also, the playback device displays the position of the intra-encoded frame in the moving image on a bar (so-called seek bar) indicating the playback time position in the moving image based on the interval between the intra-encoded frames. The user can know the position of the intra-coded frame on the seek bar. Here, when playback is started from a frame that is not an intraframe-encoded frame (that is, an interframe-encoded frame), the frames from the latest intraframe-encoded frame before that frame to the current frame must be decoded. Therefore, it takes time for the decoding process. On the other hand, when playback is started from an intraframe-encoded frame, it is sufficient to decode the frame, and the time required for the decoding process is relatively short. Therefore, when the user prompts an operation to start reproduction from the intra-frame-encoded frame, the reproduction device as the processing device can appropriately process the moving image.

Further, when a loss of an inter-encoded frame is detected, the reproducing apparatus prohibits decoding processing of inter-encoded frames that cannot be decoded due to the loss. Therefore, the playback device as a processing device can appropriately process moving images by reducing the amount of processing involved in frame decoding processing.

In addition, since the creating apparatus according to the present embodiment creates a moving image file using a part of the plurality of frames received from the encoding apparatus, the moving image to be created is less than the case of using the above part. File size can be reduced. Also, when creating the above file, if a video file is created without including intra-frame encoded frames, it is impossible to decode the inter-encoded frames encoded with reference to the above intra-frame encoded. become. Therefore, by creating a moving image file using all the intra-frame coded frames included in the plurality of frames, it is possible to avoid the occurrence of frames that cannot be decoded as described above. Therefore, the creation device as a processing device can appropriately process moving images.

Also, the creation device determines frames to be added to the moving image file so that the frame rate of the created moving image file does not exceed the upper limit. This makes it possible to reduce the size of the moving image file to be created. Therefore, the creation device as a processing device can appropriately process moving images.

The present invention can be realized not only as an apparatus, but also as a method in which processing means constituting the apparatus are used as steps, as a program for causing a computer to execute the steps, or as a computer-readable program in which the program is recorded. It can also be implemented as a recording medium such as a CD-ROM, or as information, data, or signals indicating the program. These programs, information, data and signals may then be distributed over a communication network such as the Internet.

Although the communication terminal and the like of the present invention have been described above based on the embodiments, the present invention is not limited to these embodiments. As long as it does not deviate from the spirit of the present invention, the present embodiment includes various modifications that a person skilled in the art can think of, and the form constructed by combining the components of different embodiments is also included in the scope of the present invention. .

INDUSTRIAL APPLICABILITY The present invention can be used in encoding devices, playback devices, or creation devices that enable suitable moving image processing.

1 playback system 5 camera 10 encoding device 11 acquisition unit 12 encoding unit 13, 21, 41 communication IF
14, 24 control unit 20 playback device 22 decryption unit 23 display unit 30 content 35 seek bar 36, 36A, 36B, 36C graphics 40 creation device 42 identification unit 43 creation unit N network

Claims (13)

an acquisition unit that sequentially acquires a plurality of frames that make up a moving image;
First frames at predetermined intervals among the plurality of frames acquired by the acquisition unit are intra-frame encoded, and second frames other than the first frame among the plurality of frames acquired by the acquisition unit are inter-coded. an encoding unit that encodes
a communication interface in communication with the processing unit;
a control unit that transmits interval information indicating the predetermined interval to the processing device and sequentially transmits the plurality of frames encoded by the encoding unit to the processing device ;
The control unit changes a frame interval at which the encoding unit intra-encodes the first frame,
The encoding unit intra-encodes the first frame using the changed frame interval as the predetermined interval.
Encoding device. The control unit acquires quality information indicating the quality of communication performed by the communication interface with the processing device, and according to the quality indicated by the acquired quality information, the encoding unit converts the first frame into a frame. The encoding device according to claim 1, wherein the encoding frame interval is changed. The encoding device according to claim 2, wherein the control unit changes the frame interval to a larger value as the quality indicated by the acquired quality information is higher. a display unit;
A plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame coded, and second frames other than the first frame among the plurality of frames are frames. a decoding unit that receives a plurality of inter-encoded frames from an encoding device, decodes the received plurality of frames, and causes the display unit to reproduce the moving image;
a control unit that receives interval information indicating an interval of the first frame among the plurality of frames from the encoding device, and displays an image based on the received interval information on the display unit together with the moving image that is being reproduced; a playback device. The control unit
displaying a bar indicating a playback time position in the moving image together with the moving image being played on the display unit;
Identifying the time position of the first frame included in the moving image in the bar using the interval information;
5. The playback device according to claim 4, wherein the specified graphic indicating the time position of the first frame in the bar is displayed on the display unit. The control unit
When the loss of the second frame is detected among the plurality of frames, the interval information is used to identify the timing of receiving the first frame after the second frame in which the loss was detected. ,
6. The reproducing apparatus according to claim 4, wherein decoding by the decoding unit is prohibited after the loss is detected and until the timing arrives. (a) Among a plurality of frames constituting a moving image, first frames at predetermined intervals among the plurality of frames are intra-frame encoded, and second frames other than the first frame among the plurality of frames are receiving, from an encoding device, a plurality of inter-coded frames and interval information indicating an interval between intra-encoded frames among the plurality of frames;
(b) an identifying unit that identifies an inter-encoded frame among the plurality of received frames using the received interval information;
using all of the first frames received from the encoding device and one or more frames excluding at least part of the frames specified by the specifying unit, among the second frames received from the encoding device and a creation unit for creating a moving image file. The creation unit creates the moving image file having a frame rate equal to or lower than a predetermined upper limit by using one or more frames excluding at least part of the frames specified by the specifying unit, among the plurality of frames. The production device according to claim 7. an encoding device according to any one of claims 1 to 3;
and a reproducing device according to any one of claims 4 to 6, which receives the interval information and the plurality of frames from the encoding device. an acquisition step of sequentially acquiring a plurality of frames constituting a moving image;
Intra-frame coding is performed on first frames at predetermined intervals among the plurality of frames obtained in the obtaining step, and inter-frame coding is performed on second frames other than the first frame among the plurality of frames obtained in the obtaining step. an encoding step to
a control step of transmitting interval information indicating the predetermined interval to a processing device and transmitting the plurality of frames encoded in the encoding step to the processing device ;
in the control step, changing a frame interval for intra-coding the first frame in the encoding step;
In the encoding step, intra-frame encoding is performed on the first frame using the changed frame interval as the predetermined interval.
Encoding method. A plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame coded, and second frames other than the first frame among the plurality of frames are frames. a decoding step of receiving a plurality of inter-encoded frames from an encoding device, decoding the received plurality of frames, and causing a display unit to reproduce the moving image;
a control step of receiving interval information indicating an interval of the first frame among the plurality of frames from the encoding device, and displaying an image based on the received interval information on the display unit together with the moving image being reproduced; Including how to play. A program for causing a computer to execute the reproduction method according to claim 11. A plurality of frames constituting a moving image, wherein first frames at predetermined intervals among the plurality of frames are intra-frame coded, and second frames other than the first frame among the plurality of frames are frames. a receiving step of receiving, from an encoding device, a plurality of inter-encoded frames and interval information indicating an interval between intra-encoded frames among the plurality of frames;
an identifying step of identifying an inter-coded frame among the plurality of received frames using the received interval information;
using one or more frames among all of the first frames received from the encoding device and the second frames received from the encoding device , excluding at least part of the frames identified in the identifying step and a creation step of creating a video file.

Images