JP6598376B2 - Encoding apparatus and encoding method - Google Patents

Description

  The present invention relates to an encoding device and an encoding method.

  When moving image content is transferred through a network, a compression technique that reduces the amount of communication in the network is used. In the compression technique, a transmission device compresses and encodes a frame constituting the content, transmits the frame to a network, and the reception device receives and decodes the compressed and encoded data to restore the content.

  There are differential encoding schemes that improve the compression rate in compression techniques. In the differential encoding method, intra-frame encoding is performed using a part of the frame included in the content as a key frame. Further, the remaining part excluding the part of the frame included in the content is set as a difference frame, and the difference from the key frame is encoded, that is, inter-frame encoded.

  Japanese Patent Laid-Open No. 2004-228688 suppresses a rapid increase in the coding processing amount of a key frame in a compression coding technique, and also shows a difference that appears during reproduction between an intra-frame coded frame and an inter-frame coded frame. Disclosed is a technique for making the size smaller.

JP 2002-171527 A

  The technology disclosed in Patent Document 1 is expected to have an effect of suppressing a rapid increase in the coding processing amount of a key frame by introducing the concept of a key GOB (block area) in addition to a key frame. The data size of the converted data is indifferent. For example, when the motion in the video is intense, the data size of the inter-frame encoded frame may exceed the data size of the intra-frame encoded frame.

  For this reason, there is a problem that the data size generated by the encoding becomes relatively large, and the amount of communication on the network is reduced, or the content played back by the playback device is disturbed due to a communication loss that may occur in the network. is there.

  The present invention has been made to solve the above problem, and an object thereof is to provide an encoding device or the like that reduces the size of encoded data.

  In order to solve the above problem, an encoding apparatus according to an aspect of the present invention includes an encoding unit that performs intra-frame encoding or inter-frame encoding on a frame included in a series of frames, and (a) the series of frames. When the encoding unit intra-frame-encodes the first frame of the frames, the second frame next to the first frame in the series of frames is controlled to be inter-frame encoded, and (b ) When the encoding unit performs inter-frame encoding on the third frame of the series of frames, a first predetermined number of consecutive frames that end with the third frame among frames that have already been inter-frame encoded. A control unit that controls the fourth frame following the third frame to be intra-coded when the data size of each of the frames is equal to or greater than a first threshold; Provided.

  According to this, the encoding device reduces the size of the encoded data by inter-frame encoding the frame after intra-frame encoding. In addition, when the data size of the inter-frame encoded frame is relatively large continuously, the encoding device reduces the data size of the frame by intra-frame encoding the subsequent frames. The data size of an inter-frame encoded frame may be larger than the data size of an intra-frame encoded frame when the motion of the entire image in the frame is relatively large. At this time, if a new frame is encoded in the frame, the data size of the frame becomes smaller than that in the case of inter-frame encoding. With these controls, the encoding device can reduce the size of the encoded data.

  In addition, in (c) and (b), when the data size of each of the frames that are interframe-coded is equal to or larger than the first threshold, the control unit is configured to perform the fourth frame of the series of frames. Control may be performed so that a second predetermined number of consecutive frames starting from the frame are intra-coded.

  According to this, when the data size of the inter-frame encoded frame is continuously relatively large, the encoding apparatus performs intra-frame encoding on a predetermined number of subsequent frames. In a series of frames of content, when the entire image in the frame moves continuously (for example, when there is scrolling or when there is continuous camera shake at the time of shooting), an inter-frame encoded frame Therefore, the effect of differential encoding by inter-frame encoding becomes relatively small. In that case, the encoding apparatus can make the data size of those frames smaller than the case of inter-frame encoding by performing a predetermined number of intra-frame encodings after the new frame.

  In addition, in the step (c), the control unit sequentially encodes the second predetermined number of frames in the order of reproduction and sequentially encodes a fifth frame of the second predetermined number of frames. , The data size of the fifth frame that is intra-encoded is changed by a third predetermined value or more from the data size of the sixth frame that is intra-encoded immediately before the fifth frame in the series of frames. In such a case, control may be performed so that the seventh frame next to the fifth frame is inter-coded.

  According to this, in the encoding device, when the intra-frame encoding is continuously performed by continuously moving the entire image in the frame, the data size of the encoded frame changes relatively large. In some cases, the intra-frame encoding is stopped and the frame is inter-frame encoded. One reason for the relatively large change in data size as described above may be that the scene in the frame has been switched. In such a case, there is a possibility that the continuous movement of the entire image in the frame may be lost after that, so that the size of the encoded data can be reduced by stopping the intra-frame encoding and performing the inter-frame encoding. Can be reduced.

  In addition, in the step (a), when the encoding unit encodes the first frame in the frame, the control unit converts the frames after the second frame in the series of frames in a reproduction order. Control is performed so that interframe coding is performed sequentially, and when the eighth frame of the second and subsequent frames is interframe-coded, the data size of the eighth frame that is interframe-coded is the second threshold value. When it is above, control may be performed so that the ninth frame after the eighth frame in the series of frames is intra-coded.

  According to this, the encoding apparatus performs intra-frame encoding of a new frame when the data size of the inter-frame encoded frame is relatively large. When the difference between the inter-frame encoded frame and the key frame is relatively large, the difference between the subsequent frames and the new key frame is calculated by intra-frame encoding using the new frame as the new key frame. This is because it can be made relatively small.

  The encoding apparatus further includes: a transmission unit that transmits encoded data generated by the encoding unit performing intra-frame encoding or inter-frame encoding; and a frame included in the encoded data is transmitted between frames. When the frame is an encoded frame and the data size of the frame is greater than or equal to a third threshold value, a transmission control unit that prohibits the transmission unit from transmitting the frame may be provided.

  According to this, the encoding device can suppress the amount of communication in the network by prohibiting transmission of a frame having a relatively large data size among frames that are inter-frame encoded. If the frame transmission is prohibited in this manner, the frame cannot be reproduced by the decoding device. However, it is considered that the reproduction of the content is not greatly affected as long as one frame is not reproduced.

  An encoding apparatus according to an aspect of the present invention includes an encoding unit that performs intraframe encoding or interframe encoding on a frame included in a series of frames, and (d) a first of the series of frames. When the encoding unit encodes the frame within the frame, control is performed so that the second frame next to the first frame among the series of frames is inter-coded, and (e) the sequence of frames When the encoding unit performs inter-frame encoding on the third frame, the data size of each of a predetermined number of consecutive inter-frame encoded frames including the third frame is greater than or equal to the first threshold. A control unit that controls the third frame to be intra-coded.

  According to this, the encoding apparatus reduces the size of encoded data by performing inter-frame encoding after intra-frame encoding. Further, the encoding device reduces the data size of the frame by performing intra-frame encoding on the encoded frame again when the data size of the inter-frame encoded frame is relatively large continuously. With these controls, the encoding device can reduce the size of the encoded data.

  An encoding method according to an aspect of the present invention includes an encoding step of intra-frame encoding or inter-frame encoding of frames included in a series of frames, and (a) a first of the series of frames. When the frame is intra-frame encoded in the encoding step, control is performed so that the second frame next to the first frame in the series of frames is inter-coded, and (b) When the third frame is inter-frame encoded in the encoding step, the data size of each of the first predetermined number of consecutive frames ending with the third frame among the frames already inter-frame encoded Is greater than or equal to the first threshold value, a control step for controlling the fourth frame following the third frame to be intra-coded in the series of frames. Including the door.

  Thereby, there exists an effect similar to the said encoding apparatus.

  An encoding method according to an aspect of the present invention includes an encoding step of intra-frame encoding or inter-frame encoding of frames included in a series of frames, and (d) a first of the series of frames. When the frame is intra-frame encoded in the encoding step, control is performed so that the second frame next to the first frame in the series of frames is inter-coded, and (e) the series of frames When the third frame is inter-frame encoded in the encoding step, the data size of each of a predetermined number of consecutive inter-frame encoded frames including the third frame is greater than or equal to the first threshold. A control step for controlling the third frame to be intra-coded.

  Thereby, there exists an effect similar to the said encoding apparatus.

  The encoding apparatus according to the present invention can reduce the size of encoded data.

FIG. 1 is a configuration diagram showing a configuration of a content reproduction system according to an embodiment. FIG. 2 is a block diagram illustrating a hardware configuration of the encoding apparatus according to the embodiment. FIG. 3 is a block diagram showing a configuration of the encoding apparatus according to the embodiment. FIG. 4 is a flowchart showing processing of the encoding apparatus according to the embodiment. FIG. 5 is a block diagram showing a configuration of the decoding apparatus according to the embodiment. FIG. 6 is a flowchart showing processing of the decoding apparatus according to the embodiment. FIG. 7 is a block diagram showing in more detail the configuration of the encoding apparatus according to the embodiment. FIG. 8 is an explanatory diagram of an example of scrolling in the content detected by the control unit according to the embodiment. FIG. 9 is an explanatory diagram illustrating an example of scene switching in content detected by the control unit according to the embodiment. FIG. 10 is a flowchart showing processing of the encoding apparatus according to the embodiment. FIG. 11 is a flowchart showing an encoding process of the encoding device according to the embodiment. FIG. 12A is a first flowchart illustrating an encoding control process of the encoding device according to the embodiment. FIG. 12B is a second flowchart illustrating the encoding control process of the encoding device according to the embodiment. FIG. 13 is a flowchart showing a scroll detection mode according to the embodiment. FIG. 14 is a first explanatory diagram for explaining the effect of scroll detection by the encoding apparatus according to the embodiment. FIG. 15 is a second explanatory diagram for explaining the effect of scroll detection by the encoding apparatus according to the embodiment. FIG. 16 is a third explanatory diagram for explaining the effect of scroll detection by the encoding apparatus according to the embodiment. FIG. 17 is a fourth explanatory diagram illustrating the effect of scroll detection by the encoding device according to the embodiment. FIG. 18 is a first explanatory diagram illustrating the effect of scene switching detection by the encoding device according to the embodiment. FIG. 19 is a second explanatory diagram illustrating the effect of scene switching detection by the encoding device according to the embodiment. FIG. 20 is a third explanatory diagram illustrating the effect of scene switching detection by the encoding device according to the embodiment. FIG. 21 is a first explanatory diagram illustrating an effect of early determination of scroll detection by the encoding device according to the embodiment. FIG. 22 is a second explanatory diagram illustrating the effect of the early determination of scroll detection by the encoding device according to the embodiment. FIG. 23 is a block diagram illustrating a configuration of an encoding device according to a modification of the embodiment. FIG. 24 is a flowchart showing processing of the encoding device according to the modification of the embodiment. FIG. 25 is a flowchart showing processing of the encoding device according to the modification of the embodiment.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  Each of the embodiments described below shows a preferred specific example of the present invention. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present invention are described as optional constituent elements that constitute a more preferable embodiment. In addition, the same code | symbol is attached | subjected to the same component and description may be abbreviate | omitted.

(Embodiment)
In the present embodiment, an encoding apparatus and the like that reduce the size of encoded data will be described.

  FIG. 1 is a configuration diagram showing a configuration of a content reproduction system S according to the present embodiment.

  As shown in FIG. 1, the content reproduction system S includes a transmission device 1 and a reproduction device 2. The transmission device 1 and the playback device 2 are connected by a network 3 so that they can communicate with each other.

  The content reproduction system S is a content reproduction system in which moving image content (hereinafter also simply referred to as content) held by the transmission device 1 is reproduced by one or more reproduction devices 2.

  The transmission device 1 is a transmission device that has content and transmits the content to the playback device 2 through the network 3. The transmission device 1 includes an encoding device 10 that compresses and encodes content (hereinafter also simply referred to as encoding). The encoding device 10 generates encoded data in which the amount of content data is reduced by encoding, and transmits the generated encoded data to the network 3.

  The playback device 2 is a playback device that receives content from the transmission device 1 via the network 3 and plays back the received content. The playback device 2 receives the encoded data generated by the encoding device 10 based on the moving image content held by the transmission device 1 through the network 3. The playback device 2 includes a decoding device 20 that decodes encoded data. The decoding device 20 restores the content by decoding the encoded data received from the network 3, and the playback device 2 plays back the restored content.

  The network 3 is a network that connects the transmission device 1 and the playback device 2 so that they can communicate with each other. Specifically, the network 3 is a wireless LAN (Local Area Network) conforming to the IEEE 802.11a, b, g, n standard or the like, or a wired LAN conforming to the IEEE 802.3 standard or the like.

  The content reproduction system S may include two or more reproduction devices 2. In that case, it is necessary that each of the two or more reproducing apparatuses 2 is connected to the network 3 and can receive the encoded data transmitted by the transmitting apparatus 1. When there are two or more playback devices 2, the transmission device 1 may perform unicast transmission to each of the two or more playback devices 2, or broadcast transmission so that the two or more playback devices 2 can receive. May be.

  FIG. 2 is a block diagram showing a hardware configuration of encoding apparatus 10 according to the present embodiment. Note that the decoding device 20 also has the same hardware configuration as the encoding device 10.

  As shown in FIG. 2, the encoding device 10 includes a CPU 11, a main memory 12, and a storage 13.

  The CPU 11 is a processor that executes a control program stored in the main memory 12 or the storage 13. The processing executed by the encoding device 10 can be realized by the CPU 11 executing a control program.

  The main memory 12 is a volatile storage area used as a work area used when the CPU 11 executes a control program.

  The storage 13 is a non-volatile storage device that stores data of frames constituting content. The storage 13 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and may be built in the encoding device 10 or a USB (Universal Serial Bus) standard. May be externally attached.

  Hereinafter, configurations of the encoding device and the decoding device according to the present embodiment will be described in detail.

  FIG. 3 is a block diagram showing a configuration of encoding apparatus 10 according to the present embodiment.

  As illustrated in FIG. 3, the encoding device 10 includes a frame holding unit 100, an acquisition unit 101, a blocking filter 102, a conversion unit 103, a difference unit 104, a quantization unit 105, and an encoding unit 106. And an inverse quantization unit 107 and a reference holding unit 108.

  The frame holding unit 100 is a storage unit that holds data of frames constituting the content to be encoded. The frame holding unit 100 is realized by, for example, the main memory 12 or the like.

  The acquisition unit 101 is a processing unit that acquires an encoding target frame. The acquisition unit 101 acquires the frame to be encoded by reading out the encoding target frame from the frames included in the content stored in the frame holding unit 100. For example, frames that are associated with current time information managed by the encoding device 10 among a plurality of frames constituting the content are sequentially assigned to the encoding target frame.

  The blocking filter 102 is a processing unit that performs a blocking filter process on a block boundary that is a processing unit included in the encoding target frame acquired by the acquisition unit 101. The blocking filter process is a process to be performed before the conversion process in order to suppress discontinuity that may occur at a block boundary that is a processing unit of the conversion process (described later).

  The conversion unit 103 is a processing unit that performs conversion processing for each block on the frame after the blocking filter 102 performs blocking filter processing. An example of the conversion process is a frequency conversion process. In the frequency conversion, a frame to be encoded is converted into a coefficient (frequency coefficient) in a case where the frame to be encoded is expressed as an addition of a plurality of base trigonometric functions. As the conversion process, an order conversion (for example, Hadamard conversion) process or the like may be employed.

  The difference unit 104 calculates the difference between the frame to be encoded (frequency coefficient) after the conversion unit 103 performs the conversion process and the reference frame held by the reference holding unit 108, thereby calculating the difference frame. Is a processing unit for generating The difference unit 104 reads the reference frame from the reference holding unit 108 before calculating the difference. The difference unit 104 calculates the difference when the encoding target frame is not a key frame, and does not calculate the difference when it is a key frame. Whether the encoding target frame is a key frame or not a key frame may be determined by the time when the difference unit 104 acquires the encoding target frame from the conversion unit 103, or for some control. The difference unit 104 may determine based on this. This control includes, for example, control by the control unit 111 in a modification described later.

  The timing at which the difference unit 104 reads the reference frame is after the timing at which the acquisition unit 101 reads the encoding target frame from the frame holding unit 100. This is because the processing of the blocking filter 102 and the conversion unit 103 is executed after the acquisition unit 101 reads out the encoding target frame from the frame holding unit 100, and then the difference unit 104 reads out the reference frame. In other words, it can be said that the blocking filter 102 and the conversion unit 103 function as a delay unit that delays the reading of the reference frame by the difference unit 104 with respect to the reading of the encoding target frame by the acquisition unit 101.

  The quantization unit 105 quantizes the encoding target frame after the conversion unit 103 performs the conversion process or the difference frame generated by the difference unit 104 (that is, converts the frame into a quantization coefficient). Part.

  The encoding unit 106 is a processing unit that generates encoded data by encoding the frame quantized by the quantization unit 105. The encoded data generated by the encoding unit 106 corresponds to content data transmitted by the encoding device 10.

  The inverse quantization unit 107 is a processing unit that inversely quantizes the frame after quantization by the quantization unit 105. The inverse quantization unit 107 performs inverse quantization when the frame after quantization by the quantization unit 105 is a key frame. The inverse quantization unit 107 stores the inversely quantized frame in the reference holding unit 108 as a reference frame.

  The reference holding unit 108 is a storage unit that holds the key frame after the inverse quantization unit 107 performs inverse quantization as a reference frame. The key frame held by the reference holding unit 108 is referred to by the difference unit 104 and used for the process of calculating the difference. The reference holding unit 108 is realized by the main memory 12, for example.

  Processing of the encoding device 10 configured as described above will be described.

  FIG. 4 is a flowchart showing processing of the encoding apparatus 10 according to the present embodiment.

  In step S <b> 101, the acquisition unit 101 acquires the encoding target frame by reading it from the frame holding unit 100.

  In step S <b> 102, the blocking filter 102 performs a blocking filter process on the block boundary in the encoding target frame acquired by the acquisition unit 101.

  In step S <b> 103, the conversion unit 103 performs conversion processing for each block on the frame after the blocking filter processing by the blocking filter 102.

  In step S104, the encoding apparatus 10 branches the process depending on whether the encoding target frame is a key frame. If the frame to be encoded is a key frame (Yes in step S104), the process proceeds to step S105. If not (No in step S104), the process proceeds to step S111.

  In step S105, the quantization unit 105 performs the quantization process on the encoding target frame after the conversion unit 103 performs the conversion process.

  In step S <b> 106, the inverse quantization unit 107 performs an inverse quantization process on the key frame quantized by the quantization unit 105.

  In step S107, the inverse quantization unit 107 stores the frame subjected to the inverse quantization in step S106 in the reference holding unit 108.

  In step S111, the difference unit 104 generates, as a difference frame, a difference between the encoding target frame after the conversion unit 103 performs the conversion process and the reference frame read from the reference holding unit 108.

  In step S112, the quantization unit 105 quantizes the difference frame after the difference unit 104 performs the conversion process.

  In step S108, the encoding unit 106 encodes the key frame quantized by the quantization unit 105 to generate encoded data. The generated encoded data is transmitted to the reproduction device 2 by the transmission device 1.

  FIG. 5 is a block diagram showing a configuration of decoding apparatus 20 according to the present embodiment.

  As illustrated in FIG. 5, the decoding device 20 includes a decoding unit 201, an addition unit 202, an inverse quantization unit 203, a reference holding unit 204, an inverse transformation unit 205, a deblocking filter 206, and a generation Unit 207.

  The decoding unit 201 is a processing unit that receives and decodes encoded data that is content data transmitted by the encoding device 10 as a decoding target frame. A frame generated by decoding by the decoding unit 201 is a frame to be encoded (quantization coefficient) that is original data that has been encoded by the encoding unit 106.

  The addition unit 202 is a processing unit that generates an addition frame (quantization coefficient) by adding the decoding target frame decoded by the decoding unit 201 and the reference frame held by the reference holding unit 204. . The addition unit 202 reads the reference frame from the reference holding unit 204 before performing the above addition process. The addition frame generated by the addition unit 202 is a frequency coefficient generated by the conversion unit 103 based on the encoding target frame.

  The inverse quantization unit 203 is a processing unit that calculates a decoding target frame (frequency coefficient) by inversely quantizing the addition frame generated by the addition unit 202 through the addition process. The processing of the inverse quantization unit 203 is the same as that of the inverse quantization unit 107 of the encoding device 10.

  The reference holding unit 204 is a storage unit that holds a key frame of the decoding target frames generated by the inverse quantization unit 203 as a reference frame. The key frame held by the reference holding unit 204 is referred to by the adding unit 202 and used for the addition process. The reference holding unit 204 is realized by the main memory 12 or the like, for example.

  The inverse transform unit 205 is a processing unit that performs an inverse transform process on a decoding target frame (frequency coefficient) generated by the inverse quantization unit 203 through inverse quantization. In the inverse conversion process, conversion opposite to the conversion by the conversion unit 103 of the encoding device 10 is performed. Specifically, the inverse conversion process is an inverse frequency conversion process when the conversion unit 103 performs frequency conversion, and an inverse order conversion process when the conversion unit 103 performs order conversion.

  The deblocking filter 206 is a processing unit that performs a deblocking filter process on the block boundary in the decoding target frame after the inverse conversion unit 205 performs the inverse conversion process.

  The generation unit 207 is a processing unit that acquires an addition frame that has been subjected to deblocking filter processing by the deblocking filter 206, generates a frame by adding attribute information or the like to the acquired addition frame, and passes the frame to subsequent processing. is there.

  Processing of the decoding device 20 configured as described above will be described.

  FIG. 6 is a flowchart showing processing of the decoding device 20 according to the present embodiment.

  In step S201, the decoding unit 201 receives and decodes encoded data to obtain a decoding target frame.

  In step S202, the decryption apparatus 20 branches the process depending on whether or not the decryption target frame is a key frame. If the frame to be decrypted is a key frame (Yes in step S202), the process proceeds to step S203. If not (No in step S202), the process proceeds to step S211.

  In step S203, the inverse quantization unit 203 inversely quantizes the key frame that is the decoding target frame.

  In step S204, the inverse quantization unit 203 stores the key frame inversely quantized in step S203 in the reference holding unit 204.

  In step S <b> 205, the inverse transform unit 205 performs an inverse transform process on the addition frame that is inversely quantized by the inverse quantization unit 203.

  In step S206, the deblocking filter 206 performs a deblocking filter process on the block boundary in the addition frame after the inverse transform unit 205 performs the inverse transform process.

  In step S207, the generation unit 207 generates a frame and passes it to a subsequent process.

  In step S211, the addition unit 202 adds the frame generated by the decoding unit 201 by decoding and the reference frame read from the reference holding unit 204 to generate an addition frame.

  In step S212, the inverse quantization unit 203 performs inverse quantization on the addition frame generated by the addition unit 202 through the addition process.

  Hereinafter, a technique for reducing the size of encoded data in the encoding apparatus according to the present embodiment will be described in more detail. More specifically, the encoding apparatus according to the present embodiment performs an intraframe encoding or an interframe encoding on a new frame according to the encoding processing result, in other words, which frame By appropriately determining whether to be a key frame, the size of the encoded data is reduced.

  FIG. 7 is a block diagram showing in more detail the configuration of the coding apparatus according to the present embodiment. FIG. 8 is an explanatory diagram of an example of scrolling in content detected by the control unit 111 according to the present embodiment. FIG. 9 is an explanatory diagram illustrating an example of scene switching in content detected by the control unit 111 according to the present embodiment.

  As illustrated in FIG. 7, the encoding device 10A includes a frame holding unit 100, an acquisition unit 101, a blocking filter 102, a conversion unit 103, a difference unit 104A, a quantization unit 105, and an encoding unit 106. An inverse quantization unit 107, a reference holding unit 108, a control unit 111, and a switching unit 112.

  Here, the frame holding unit 100, the acquiring unit 101, the blocking filter 102, the transforming unit 103, the quantizing unit 105, the encoding unit 106, the inverse quantization unit 107, and the reference holding unit 108 are described. Has the same function as the component of the same name of the encoding device 10, and detailed description thereof will be omitted.

  Similar to the difference unit 104 in the encoding device 10, the difference unit 104 </ b> A is a difference between the encoding target frame after the conversion unit 103 performs the conversion process and the reference frame held by the reference holding unit 108. Is a processing unit that generates a difference frame. At the time when the difference unit 104A acquires a frame from the conversion unit 103, it is not determined whether this frame is a key frame or a difference frame, and the state of the switching unit 112 is controlled by the control unit 111 in the subsequent stage. Thus, it is determined whether this frame becomes a key frame or a difference frame. When the frame input to the difference unit 104A is inter-frame encoded (becomes a difference frame), the frame is provided to the quantization unit 105 through the subsequent switching unit 112. On the other hand, when the frame input to the difference unit 104A is intra-frame encoded (becomes a key frame), the frame input to the difference unit 104A is provided to the quantization unit 105, and the difference unit 104A generates The difference frame is not used for subsequent processing.

  The control unit 111 is a processing unit that performs determination based on the encoded data generated by the encoding unit 106 by encoding, and controls encoding of a new encoding target frame according to the determination result. Specifically, in the case where the encoding unit 106 intra-frame-encodes the first frame of the series of frames included in the content, the control unit 111 performs the next step after the first frame of the series of frames. Controls that two frames are inter-coded. In addition, when the encoding unit 106 performs inter-frame encoding on the third frame of the series of frames, the control unit 111 continues to end with the third frame among the frames that have already been inter-frame encoded. When the data size of each of the first predetermined number of frames is greater than or equal to the first threshold value, control is performed so that the fourth frame following the third frame in the series of frames is intra-coded. Here, when the data size of each inter-frame encoded frame is equal to or larger than the first threshold, the control unit 111 selects a second predetermined number of consecutive frames starting from the fourth frame in the series of frames. Control may be performed so that intra-frame coding is performed. In addition, when the encoding unit 106 encodes the first frame in the frame, the control unit 111 performs control so that the frames after the second frame in the series of frames are sequentially encoded in the order of reproduction. When the data size of the eighth frame that has been interframe-coded is greater than or equal to the second threshold at the time when the eighth frame of the second and subsequent frames is interframe-encoded, Control may be performed so that the ninth frame following the eight frames is intra-frame encoded.

  The control unit 111 acquires encoded data (or a copy of the encoded data) generated by the encoding unit 106 by encoding, and indicates whether or not the frame included in the encoded data is a key frame. Information and the data size of the encoded data are obtained. Then, a determination is made based on the information and the data size, and whether a new encoding target frame to be encoded next by the encoding apparatus 10A is intra-frame encoded or inter-frame encoded (in other words, For example, which frame is the key frame and which frame is the difference frame). Then, the control unit 111 controls the state of the switching unit 112 according to the determination result.

  The determination by the control unit 111 is mainly based on scroll detection and scene switching detection in the frame. Here, scrolling means that the pixels included in the frame move in one direction as a whole. For example, a character string image generated by computer graphics moves in the vertical direction or the horizontal direction in the frame. In the content shot by the camera, the image in the frame moves in the direction opposite to the direction of the camera because the shooting direction of the camera shakes in the horizontal direction or the vertical direction (pan, tilt, or camera shake), etc. (See FIG. 8). When the frame is inter-frame encoded, the control unit 111 detects as scrolling that the data size of the inter-frame encoded frame has continued to exceed the threshold A for a predetermined value B times. When the scroll is detected, the control unit 111 performs intra-frame encoding on a predetermined number of A frames. The threshold value A corresponds to the first threshold value. Further, the predetermined value B and the predetermined value A correspond to a first predetermined number and a second predetermined number, respectively.

  The threshold A can be the data size of the latest intra-frame encoded frame, for example. The predetermined value B can take a natural number of 1 or more, and can be arbitrarily determined based on the number of consecutive frames in which no scroll is detected even if the data size of the encoded frame suddenly increases. The predetermined value B will be described in detail later.

  In addition, scene switching means that the contents of continuous frames are switched discontinuously. For example, a change to a different scene in a movie or a drama, or a scene based on a live action is generated by animation or computer graphics. This refers to changes to the scene (see FIG. 9). The control unit 111 detects, as scene switching, that the data size of the intra-coded frame has changed by a predetermined value C or more when a plurality of frames are continuously intra-coded. The predetermined value C corresponds to a third predetermined value.

  In addition, the control unit 111 performs control so that a series of frames included in the content is intra-frame encoded or inter-frame encoded in one of two encoding modes, an intra mode and an inter mode. Here, the intra mode is an encoding mode in which each of a series of frames is intra-frame encoded. In inter mode, a part of a series of frames is intra-frame encoded, and a frame other than the above part is encoded with a difference from the intra-frame encoded frame (that is, between frames). This is an encoding mode. The control unit 111 sets the encoding mode to the intra mode based on information indicating whether the frame included in the encoded data generated by the encoding unit 106 is a key frame and the data size of the encoded data. Switch back and forth to inter mode. The conditions for switching the encoding mode will be described in detail later.

  Note that the control unit 111 can also detect the scroll earlier than the scroll detection in the above description in the scroll detection process. As described above, the control unit 111 performs control such that after a scroll is detected, a predetermined number of frames are intra-frame encoded and subsequent frames are inter-frame encoded. Here, when the scroll in the content is continuously generated, it is assumed that the scroll is generated even after a predetermined number of frames are intra-frame encoded. Therefore, after the predetermined number of frames are intra-coded, the data size of the inter-coded frames having a predetermined value D (here, D <B) exceeds the threshold A. , Detecting that scrolling has occurred. By detecting scrolling early in this way, there is an advantage that an increase in data size due to interframe coding can be suppressed.

  Whether the switching unit 112 sets the frame provided to the quantization unit 105 in the subsequent stage as a frame after the conversion unit 103 performs the conversion process or the difference frame generated by the difference unit 104A through the difference process. It is the switching part which switches. A state in which the switching unit 112 provides the frame after the conversion unit 103 performs the conversion process to the quantization unit 105, that is, a state in which the terminal a and the terminal c in FIG. A state in which the switching unit 112 provides the difference frame generated by the difference processing by the difference unit 104A to the quantization unit 105, that is, a state in which the terminal b and the terminal c are connected in the drawing is referred to as a state B. Whether the switching unit 112 takes the state A or the state B is controlled by the control unit 111.

  Note that the operations of the difference unit 104A and the switching unit 112 described above calculate the difference when the encoding target frame is not a key frame by the difference unit 104 of the encoding device 10, and when the frame is a key frame, This is similar to the operation of not calculating the difference. That is, the difference unit 104A and the switching unit 112 are more specifically described by dividing the difference unit 104 of the encoding device 10 into two components.

  The processing of the encoding apparatus 10A configured as described above will be described using a flowchart.

  FIG. 10 is a flowchart showing processing of the encoding device 10A according to the present embodiment.

  As illustrated in FIG. 10, the encoding device 10A performs an encoding process S301 and performs a transmission process S302 that transmits the encoded data generated by the encoding process S301. Thereafter, the encoding control process S303 is executed according to the result of the executed encoding process S301. Either the transmission process S302 or the encoding control process S303 may be performed first.

  The encoding process S301 and the encoding control process S303 will be described in detail below.

  FIG. 11 is a flowchart showing an encoding process S301 of the encoding device 10A according to the embodiment.

  The processing of steps S101 to S108 in FIG. 11 is substantially the same as the processing of steps S101 to S108 shown in FIG. Here, step S104A is a more specific content of the process of step S104.

  In step S <b> 104 </ b> A, the control unit 111 switches whether the encoding target frame is intra-frame encoded or inter-frame encoded according to the state of the switching unit 112. That is, when the switching unit 112 is in the state A (“intraframe encoding” in step S104A), a series of processes (steps S105 to S107) for intraframe encoding of the encoding target frame is performed. On the other hand, when the switching unit 112 is in the state B (“interframe encoding” in step S104A), a series of processes (steps S111 to S112) for interframe encoding of the encoding target frame is performed.

  FIG. 12A is a first flowchart illustrating an encoding control process S303 of the encoding device 10A according to the embodiment. FIG. 12B is a second flowchart illustrating the encoding control process S303 of the encoding device 10A according to the embodiment.

  As shown in FIG. 12A, in step S401, the control unit 111 determines whether or not the frame encoded by the encoding unit 106 is a key frame, in other words, the encoding unit 106 encodes the frame in the frame. It is determined whether or not interframe coding has been performed. If it is determined that the frame encoded by the encoding unit 106 is a key frame (Yes in step S401), the process proceeds to step S402. If not (No in step S401), the terminal A is It progresses to step S431 (refer FIG. 12B).

  In step S402, the control unit 111 branches the process based on whether the current encoding mode is the intra mode or the inter mode. When the current encoding mode is the intra mode (“intra mode” in step S402), the process proceeds to step S403, and when the current encoding mode is the inter mode (“inter mode” in step S402). The process proceeds to step S421.

  In step S403, the control unit 111 determines whether scene switching is detected in the encoded frame. If a scene change is detected (Yes in step S403), the process proceeds to step S411. If not (No in step S403), the process proceeds to step S404.

  In step S404, the control unit 111 increments the counter A (adds 1).

  In step S405, the control unit 111 determines whether the counter A is equal to or greater than a predetermined value A. If it is determined that the counter A is greater than or equal to the predetermined value A, the process proceeds to step S411, and if not, the process proceeds to step S406.

  In step S406, the control unit 111 performs control so that the next frame in the series of frames is intra-frame encoded. Specifically, the control unit 111 maintains the switching unit 112 in the state A.

  In step S407, the control unit 111 stores the data size of the encoded frame (key frame) for later processing. When the process of step S407 is finished, the series of processes shown in the flowchart is finished.

  In step S411, the control unit 111 makes a transition from the intra mode to the inter mode.

  In step S412, the control unit controls to inter-code the next frame of the series of frames. Specifically, the control unit 111 changes the switching unit 112 to the state B. When the process of step S412 is completed, the process proceeds to step S407.

  In step S421, the control unit 111 controls to inter-code the next frame of the series of frames. Specifically, the control unit 111 changes the switching unit 112 to the state B. When the process of step S421 is completed, the series of processes shown in this flowchart is ended.

  As illustrated in FIG. 12B, in step S431, the control unit 111 determines whether or not the data size of the encoded data generated by the encoding unit 106 by encoding is larger than the threshold value A. If it is determined that the data size of the encoded data is larger than the threshold A (Yes in step S431), the process proceeds to step S432, and if not (No in step S431), the process proceeds to step S451.

  In step S432, the control unit 111 increments the counter B.

  In step S433, the control unit 111 determines whether the counter B is equal to or greater than a predetermined value B. If the counter B is greater than or equal to the predetermined value B (Yes in step S433), the process proceeds to step S435, and if not (No in step S433), the process proceeds to step S441.

  In step S434, the control unit 111 transitions from the inter mode to the intra mode.

  In step S435, the control unit 111 controls the next frame in the series of frames to be intra-coded. Specifically, the control unit 111 changes the switching unit 112 to the state A. When the process of step S421 is completed, the series of processes shown in this flowchart is terminated via the terminal B.

  In step S441, the control unit 111 reads: “The inter mode that is the current encoding mode has been changed from the inter mode when the condition“ counter A ≧ predetermined value A ”in step S405 is satisfied, and the counter B is It is determined whether or not the condition “is equal to or greater than a predetermined value D” is satisfied. When this condition is satisfied, scrolling is detected early. If it is determined that the above condition is satisfied (Yes in step S441), the process proceeds to step S442. If not (No in step S441), the process proceeds to step S435.

  In step S442, the control unit 111 transitions from the inter mode to the intra mode. When the process of step S442 is completed, the process proceeds to step S435.

  In step S451, the control unit 111 determines whether or not the data size of the encoded data is larger than the threshold value B (corresponding to the second threshold value). If it is determined that the data size of the encoded data is larger than the threshold value B (Yes in step S451), the process proceeds to step S452, and if not (No in step S451), the process proceeds to step S453.

  In step S452, the control unit 111 controls the next frame in the series of frames to be intra-coded. Specifically, the control unit 111 changes the switching unit 112 to the state A. When the process of step S452 is finished, the series of processes shown in this flowchart is finished via the terminal B.

  In step S453, the control unit 111 controls to inter-code the next frame of the series of frames. Specifically, the control unit 111 maintains the switching unit 112 in the state B. When the process of step S453 is completed, the series of processes shown in this flowchart is terminated via the terminal B.

  Through the series of processes described above, the encoding device 10A can reduce the size of the encoded data based on scroll detection and scene detection.

  In the following, simulation evaluation of the effect of reducing the data size of encoded data based on scroll detection and scene detection performed by the encoding device 10A will be described together with processing related to scroll detection and scene detection. In the graphs in the following description, a horizontal axis indicates a frame (frame number) when a series of frames are arranged in the order of reproduction, and a vertical axis indicates a frame size generated by encoding each frame. Further, a straight line connecting symbols corresponding to the data size or the like in each mode on the graph is an explanatory straight line for easily showing the transition of the same type of symbol. As for the data size and the like, it is assumed that only the symbols corresponding to the integer values on the horizontal axis are meaningful, and the portion between adjacent integer values in the straight line has no meaning.

  In the following description, an operation mode called “scroll detection mode” is used. The scroll detection mode corresponds to a state in which step S403 included in the scene switching detection process is excluded from the flowcharts of FIGS. 12A and 12B. A flowchart of the scroll detection mode is shown in FIG. Each processing step shown in FIG. 13 performs the same processing as the processing step of the same name in FIG. 12A or FIG.

  First, simulation results for scroll detection will be described.

  14-17 is explanatory drawing explaining the effect of the scroll detection by 10 A of encoding apparatuses based on this Embodiment.

  First, the case of only the intra mode and the case of only the inter mode will be described with reference to FIG. Here, “intra mode only” means an operation mode that always operates in the intra mode and does not transition from the intra mode to another mode. “Inter mode only” refers to an operation mode that always operates in the inter mode and does not transition from the inter mode to another mode.

  FIG. 14 shows the data size of the encoded data in each of the intra mode only and the inter mode only when the motion of the image in the frame is relatively small.

  In the case of only the intra mode, the data size of the encoded frame 1 is about 300 bytes, and the data size gradually increases as the frame number advances. This increase in data size depends on the content of the image included in the frame, and in general, such an increase in data size is observed when the motion of the image included in the frame gradually increases. .

  In the case of only the inter mode, first, frame 1 is intra-frame encoded as a key frame. Thereafter, the frames after frame 2 are sequentially encoded using the difference from frame 1, which is the latest key frame in the past, in the order of reproduction. When the frames after frame 2 are sequentially inter-coded in the order of reproduction, the data size of the encoded frames gradually increases. In FIG. 14, a broken line indicating the threshold value B for each frame is drawn. Here, the threshold value B is 4/5 of the data size of the key frame immediately before the frame.

  When the data size of the encoded frame does not exceed the threshold B (frames 2 to 6), the next frame of the frame is inter-frame encoded (No in step S451, step S453). When the data size of the encoded frame 7 exceeds the threshold B, the next frame 8 is intra-frame encoded (Yes in step S451, step S452).

  As shown in FIG. 14, when encoding the content, the encoding device 10A may reduce the data size by encoding only in the inter mode rather than encoding only in the intra mode. it can.

  14 corresponds to the eighth frame, and frame 8 corresponds to the ninth frame.

  FIG. 15 shows the data size of the encoded data in each of the intra mode only and the inter mode only when scrolling occurs in the frame.

  In the case of only the intra mode, the data size of the encoded frame 1 is about 300 bytes, and the data size gradually increases as the frame number advances. Similar to the description in FIG. 14, the increase in the data size depends on the contents of the image included in the frame.

  In the case of only the inter mode, first, the frame 1 is intra-frame encoded as a key frame, and then the frames after the frame 2 are sequentially encoded using the difference from the frame 1 in order of reproduction. In FIG. 15, a broken line indicating the threshold value A for each frame is drawn. Here, the threshold A is the data size of the key frame immediately before the frame.

  If the data size of the encoded frame 2 exceeds the threshold A, the next frame 3 is intra-frame encoded (Yes in step S431, step S435). Thereafter, the frame 4 is intra-frame encoded, and the next frame 5 is intra-frame encoded when the data size exceeds the threshold. Thereafter, the same is repeated.

  As shown in FIG. 15, when encoding content in which scrolling has occurred, the encoding apparatus 10A has a data size that is encoded only in the intra mode rather than encoded only in the inter mode. Can be reduced.

  In other words, depending on whether scrolling has occurred in the content, an encoding mode in which the data size can be reduced among the intra mode only and the inter mode only is used.

  FIG. 16 shows only the inter mode when the content is the same as FIG. 15 (scrolling is occurring in the frame), and a mode for transitioning to the intra mode based on scroll detection in the inter mode (hereinafter, scroll detection). The data size of the encoded data in each case) is also shown. The predetermined value B (step S433 in FIG. 12B) used for the scroll detection process is set to 2.

  The data size of the encoded frame in the case of only the inter mode is the same as that shown in FIG.

  In the case of the scroll detection mode, the data size up to frame 4 is the same as the data size only in the inter mode, but the data size from frame 5 is the same as the data size only in the intra mode.

  This is because, when the frame 4 is inter-coded, the data size of each of two consecutive frames (that is, frames 2 and 4) that end with the frame 4 among the frames that have already been inter-coded is the threshold value. This is due to the transition from the inter mode to the intra mode based on exceeding A.

  As described above, in the inter mode, when the mode is changed to the intra mode based on the scroll detection, there is an effect that the total amount of the data size can be made smaller than that in the case of only the inter mode.

  Note that frames 1, 2, 4, and 5 in the scroll detection mode of FIG. 16 correspond to the first frame, the second frame, the third frame, and the fourth frame, respectively.

  The effect of being able to set the predetermined value B to a value different from 1 in the scroll detection process will be described below.

  FIG. 17 shows encoded data in only the intra mode, the scroll detection mode (predetermined value B = 1), and the scroll detection mode (predetermined value B = 2) when a sudden large movement occurs in the content. Indicates the data size. Assume that a sudden large movement occurs in the frame 6. The description “scroll detection mode (predetermined value B = 1)” means a scroll detection mode in which the predetermined value B is 1. This description will be used hereinafter.

  In the scroll detection mode (predetermined value B = 1), the transition from the inter mode to the intra mode is performed based on a single increase in the data size in the frame 6, and the subsequent frames are encoded in the intra mode.

  On the other hand, in the scroll detection mode (predetermined value B = 2), the transition to the intra mode is not made only by a single increase in the data size in the frame 6, and the subsequent frames are encoded in the inter mode.

  If the movement in the content is sudden, the magnitude of the subsequent movement in the content is expected to be about the same as before the sudden increase. Therefore, it is desirable to maintain the inter mode even if the data size increases only once. In the scroll detection mode (predetermined value B = 2), there is an effect that the inter mode can be maintained even if the data size is increased only once.

  Note that the size of the predetermined value B can be determined arbitrarily, for example, 5 or 30, depending on the number of consecutive frames that allow the data size to suddenly increase.

  Next, the effect of scene change detection will be described.

  18-20 is explanatory drawing explaining the effect of the scene switch detection by 10 A of encoding apparatuses based on this Embodiment.

  FIG. 18 shows the data size of the encoded data in the case of only the intra mode and the inter mode when scene switching is included in the content. It is assumed that scene switching is included in the frame 10. Further, it is assumed that scrolls are included in the contents in frames 1 to 9 before the scene change, and no scrolls are included in the frames 11 and after after the scene change.

  In the case of only the intra mode, the data size of the encoded frame 1 is about 300 bytes, and the data size gradually increases as the frame number advances. Further, the data size is further increased in the frame 10 including the scene switching.

  In the case of only the inter mode, for the frame before the scene change, the data size of the frames 2, 4, 6, 8, and 10 that are inter-frame encoded exceeds the data size of the key frame immediately before the frame. Yes.

  In the case of only the intra mode and the case of only the inter mode, there is room for reducing the respective data sizes. Specifically, focusing on the case of the intra mode only, there is room for reducing the data size by inter-frame encoding the frames after frame 12. Focusing on the case of only the inter mode, there is room for reducing the data size if the frames 2, 4, 6, 8, and 10 are intra-frame encoded.

  FIG. 19 shows the data size of the encoded data in each of the intra mode only, the inter mode only, and the scroll detection mode when scene switching is included in the content. The scene switching position and the like are the same as those in FIG. Further, the graph of only the intra mode and the inter mode is the same as that in FIG.

  In the scroll detection mode, the data size after frame 6 (more specifically, frames 6 and 8) is reduced compared to the case of only the inter mode. However, after scene switching (specifically, after frame 12), a phenomenon occurs in which the data size becomes larger than in the case of only the inter mode. This is because, after the scroll is detected in the scroll detection mode, since the frame is continuously encoded regardless of whether the scroll is included or not, the data size becomes larger than the case of the inter-frame encoding. It is.

  FIG. 20 shows the data size of the encoded data in each of the scroll detection mode and the scene change detection mode when scene switching is included in the content. The scene switching position and the like are the same as those in FIG. The graph of the scroll detection mode is the same as FIG. The scene switching detection mode is a mode in which all processes shown in the flowcharts shown in FIGS. 12A and 12B are performed.

  In the scene switching detection mode shown in FIG. 20, when the scene switching is detected (that is, when the frame 10 is encoded), the mode is changed to the inter mode, and the next frame 11 is inter-frame encoded (Yes in step S403). , Steps S411 and S412). Thereby, intra-frame encoding is continued after scene switching. As a result, it is possible to avoid an increase in the data size (see FIG. 19) by continuing the intra-frame encoding as compared to the inter-frame encoding.

  Note that frames 10, 9, and 11 in the scene detection mode of FIG. 20 correspond to a fifth frame, a sixth frame, and a seventh frame, respectively.

  Next, the effect of the early determination of scroll detection will be described.

  21 and 22 are explanatory diagrams for explaining the effect of early determination of scroll detection by the encoding device 10A according to the present embodiment.

  FIG. 21 shows the data size of the encoded data in each of the intra mode only and the scroll detection mode when scrolling occurs in the frame. The content is the same as in FIG. Here, the predetermined value A in step S405 (FIG. 12A) is set to 5.

  In this case, the encoding unit 106 makes a transition from the inter mode to the intra mode based on the fact that the data size of frames (frames 2 and 4) continuously inter-frame encoded in the scroll detection mode exceeds the threshold A. Then, the five frames 5 to 9 are intra-coded. Thereafter, the mode transits to the inter mode (Yes in S405, S411), and transits from the inter mode to the intra mode based on the fact that the continuously inter-frame encoded frames (frames 10 and 12) exceed the threshold A. The frames after frame 13 are intra-coded.

  Here, when scrolling occurs continuously, the data size of the encoded frame can be suppressed by making an early transition from the inter mode to the intra mode. More specifically, if the frame 12 is subjected to intraframe coding, the data size can be reduced as compared with the case where the frame is interframe coded.

  FIG. 22 shows the data size of the encoded data in each of the intra mode only, the scroll detection mode, and the scroll early detection mode when scrolling occurs in the content. The content is the same as in FIG. Here, the predetermined value D in step S441 (FIG. 12B) is set to 1.

  In FIG. 22, when the frames 5 to 9 are intra-coded in the intra mode and then transitioned to the intra mode, the data size exceeds the threshold A when the next frame 10 is inter-coded. Accordingly, when the condition for early detection of scrolling is satisfied (Yes in S441), the mode is changed to the intra mode, and the frames after 11 are encoded.

  Thereby, the encoding unit 106 performs intra-frame encoding of the frame 12. Thereby, there is an effect that the data size can be made smaller than in the scroll detection mode in which the frame 12 is inter-coded.

  Note that the encoding device 10A may further include a transmission unit (not shown) that transmits encoded data generated by the encoding unit 106 by performing intraframe encoding or interframe encoding. In that case, if the frame included in the encoded data is an inter-frame encoded frame and the data size of the frame is greater than or equal to the threshold, the transmission unit transmits the frame. A transmission control unit (not shown) that prohibits the transmission may be provided. The threshold value can be determined according to, for example, the data size of the frame transmitted immediately before by the transmission unit. More specifically, the threshold value is 2 of the data size of the frame transmitted immediately before by the transmission unit. Can be doubled.

  If the transmission of such a frame is prohibited, this frame cannot be reproduced by the decoding device, but it is considered that the reproduction of the content is not greatly affected as long as one frame is not reproduced. As a result, it is possible to suppress the influence on the content played back by the decryption device and to suppress the traffic on the network.

(Modification of the embodiment)
In the present modification, a technique for further enhancing the data size compression effect in an encoding device or the like that reduces the size of encoded data will be described.

  FIG. 23 is a block diagram illustrating a configuration of an encoding device 10B according to a modification of the embodiment.

  As illustrated in FIG. 23, the encoding device 10B includes a frame holding unit 100, an acquisition unit 101, a blocking filter 102, a conversion unit 103, a difference unit 104A, a quantization unit 105, and an encoding unit 106. An inverse quantization unit 107, a reference holding unit 108, a control unit 111A, a switching unit 112, a re-encoding holding unit 121, and a switching unit 122.

  Here, since the constituent elements excluding the control unit 111A, the re-encoding holding unit 121, and the switching unit 122 have the same functions as the constituents with the same names in the above-described embodiment, detailed description thereof is omitted.

  When the encoding unit 106 intra-frame encodes the first frame of the series of frames included in the content, the control unit 111A determines the second frame next to the first frame among the series of frames between frames. Control to encode. In addition, when the encoding unit 106 inter-frame encodes the third frame of the series of frames, the control unit 111A includes a predetermined number of consecutive inter-frame encoded frames including the third frame. When the data size of the third frame is equal to or larger than the first threshold value, the third frame is controlled to be intra-coded.

  The control unit 111A controls the switching unit 122, the quantization unit 105, and the encoding unit 106 in addition to the control of the switching unit 112 in the above embodiment. Similar to the control unit 111 in the above embodiment, the control unit 111A includes information indicating whether or not the frame included in the encoded data generated by the encoding unit 106 is a key frame, and the data size of the encoded data. And get. Then, a determination based on the information and the data size is performed, and a new encoding target frame to be encoded next by the encoding device 10B is intra-frame encoded or inter-frame encoded, or The encoding unit 106 determines again whether to encode the already encoded frame within the frame. Then, the control unit 111A controls the states of the switching unit 112 and the switching unit 122 according to the determination result. In addition, when it is determined that the frame is to be intra-frame encoded again, the quantization unit 105 and the encoding unit 106 execute processing.

  The re-encoding storage unit 121 is a storage unit that stores the encoding target frame that has been subjected to the conversion process by the conversion unit 103. The frame held by the re-encoding holding unit 121 is a frame immediately before the frame output by the conversion unit 103 out of a series of frames included in the content.

  The switching unit 122 sets a frame provided to the difference unit 104A and the switching unit 112 as a frame after the conversion unit 103 performs the conversion process, or a frame held by the re-encoding holding unit 121 It is a switching part which switches whether or not. The state in which the switching unit 122 provides the frame held by the re-encoding holding unit 121 to the difference unit 104A and the switching unit 112, that is, the state in which the terminal d and the terminal f are connected in FIG. That's it. Further, the state in which the switching unit 122 provides the difference frame generated by the difference processing by the difference unit 104A to the difference unit 104A and the switching unit 112, that is, the state in which the terminal e and the terminal f are connected in the drawing is referred to as a state E. . Whether the switching unit 122 takes the state D or the state E is controlled by the control unit 111A.

  FIG. 24 is a flowchart showing processing of the encoding device 10B according to the modification of the present modification.

  As illustrated in FIG. 24, the encoding device 10B executes the encoding process S301, and executes the encoding control process S303A according to the result of the executed encoding process S301. And the transmission process S302 which transmits the encoding frame produced | generated by encoding process S301 and encoding control process S303A is performed.

  Since the encoding process S301 and the transmission process S302 are the same as those in the above embodiment, the description thereof is omitted. The encoding control process S303A will be described in detail below.

  FIG. 25 is a flowchart showing processing of the encoding device 10B according to the present modification.

  In step S501, the control unit 111A determines whether the frame encoded by the encoding unit 106 is a key frame, in other words, whether the encoding unit 106 has intra-coded the frame or an inter-frame code. It is determined whether or not. If it is determined that the frame encoded by the encoding unit 106 is a key frame (Yes in step S501), the process proceeds to step S502. Otherwise (No in step S501), the process proceeds to step S511.

  In step S502, control unit 111A stores the data size of the encoded frame (key frame) for later processing. When the process of step S502 is finished, a series of processes shown in the flowchart is finished.

  In step S511, the control unit 111A determines whether or not the data size of the encoded data generated by the encoding unit 106 by encoding is larger than the threshold value A. If it is determined that the data size of the encoded data is larger than the threshold A (Yes in step S502), the process proceeds to step S512. If not (No in step S502), the process proceeds to step S521.

  In step S512, the control unit 111A performs control so that the frame encoded by the encoding unit 106 (that is, the frame subjected to the determination in step S501) is newly encoded within the frame. Specifically, the control unit 111A sets the switching unit 122 to the state D and sets the switching unit 112 to the state A, and then causes the quantization unit 105 and the encoding unit 106 to execute each process. When the process of step S512 is finished, the series of processes shown in the flowchart is finished. In the transmission process performed after this, the frame newly generated by the intra-frame coding in step S512 is transmitted.

  In step S521, the control unit 111A determines whether or not the data size of the encoded data is larger than the threshold value B. If it is determined that the data size of the encoded data is larger than the threshold value B (Yes in step S521), the process proceeds to step S522. If not (No in step S521), the process proceeds to step S523.

  In step S522, control unit 111A controls to intraframe-code the next frame of the series of frames. Specifically, control unit 111A changes switching unit 122 to state E and changes switching unit 112 to state A. When the process of step S522 is finished, the series of processes shown in the flowchart is finished.

  In step S523, the control unit 111A controls to interframe-code the next frame of the series of frames. Specifically, control unit 111A changes switching unit 122 to state E and changes switching unit 112 to state B. When the process of step S523 is finished, a series of processes shown in the flowchart is finished.

  Accordingly, the encoding device 10B can further enhance the data size compression effect in an encoding device or the like that reduces the size of encoded data.

  As described above, the encoding apparatus according to the present embodiment reduces the size of encoded data by inter-frame encoding a frame after intra-frame encoding. In addition, when the data size of the inter-frame encoded frame is relatively large continuously, the encoding device reduces the data size of the frame by intra-frame encoding the subsequent frames. The data size of an inter-frame encoded frame may be larger than the data size of an intra-frame encoded frame when the motion of the entire image in the frame is relatively large. At this time, if a new frame is encoded in the frame, the data size of the frame becomes smaller than that in the case of inter-frame encoding. With these controls, the encoding device can reduce the size of the encoded data.

  Further, when the data size of the inter-frame encoded frames is continuously relatively large, the encoding apparatus performs intra-frame encoding on a predetermined number of subsequent frames. In a series of frames of content, when the entire image in the frame moves continuously (for example, when there is scrolling or when there is continuous camera shake at the time of shooting), an inter-frame encoded frame Therefore, the effect of differential encoding by inter-frame encoding becomes relatively small. In that case, the encoding apparatus can make the data size of those frames smaller than the case of inter-frame encoding by performing a predetermined number of intra-frame encodings after the new frame.

  In addition, when the data size of the encoded frame changes relatively greatly when the encoding device continues the intra-frame encoding by continuously moving the entire image in the frame, Then, the intra-frame encoding is stopped and the frame is inter-frame encoded. One reason for the relatively large change in data size as described above may be that the scene in the frame has been switched. In such a case, there is a possibility that the continuous movement of the entire image in the frame may be lost after that, so that the size of the encoded data can be reduced by stopping the intra-frame encoding and performing the inter-frame encoding. Can be reduced.

  The encoding device encodes a new frame in the frame when the data size of the inter-frame encoded frame is relatively large. When the difference between the inter-frame encoded frame and the key frame is relatively large, the difference between the subsequent frames and the new key frame is calculated by intra-frame encoding using the new frame as the new key frame. This is because it can be made relatively small.

  In addition, the encoding device can suppress the amount of communication in the network by prohibiting transmission of a frame having a relatively large data size among the frames that are inter-frame encoded. If the frame transmission is prohibited in this manner, the frame cannot be reproduced by the decoding device. However, it is considered that the reproduction of the content is not greatly affected as long as one frame is not reproduced.

  Also, the encoding device reduces the size of encoded data by performing inter-frame encoding after intra-frame encoding. Further, the encoding device reduces the data size of the frame by performing intra-frame encoding on the encoded frame again when the data size of the inter-frame encoded frame is relatively large continuously. With these controls, the encoding device can reduce the size of the encoded data.

  As described above, the encoding apparatus, the content reproduction system, and the like of the present invention have been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. .

  The present invention can be used for an encoding apparatus and a content reproduction system that reduce the size of encoded data. Specifically, it is installed in a public space or a commercial facility, and can be used for a playback device that plays back images and the like, a transmission device that provides content to the playback device, and the like.

DESCRIPTION OF SYMBOLS 1 Transmitter 2 Reproduction | regeneration apparatus 3 Network 10, 10A, 10B Encoding apparatus 11 CPU
12 Main memory 13 Storage 20 Decoding device 100 Frame holding unit 101 Acquisition unit 102 Blocking filter 103 Conversion unit 104, 104A Difference unit 105 Quantization unit 106 Encoding unit 107, 203 Inverse quantization unit 108, 204 Reference holding unit 111, 111A control unit 112, 122 switching unit 121 re-encoding holding unit 201 decoding unit 202 addition unit 205 inverse conversion unit 206 deblocking filter 207 generation unit S content reproduction system

Claims (8)

An encoding unit for intra-frame encoding or inter-frame encoding of frames included in a series of frames;
(A) When the encoding unit encodes the first frame in the series of frames, the second frame next to the first frame in the series of frames is inter-frame encoded. Control and
(B) When the encoding unit performs inter-frame encoding on a third frame of the series of frames, a continuous first frame that ends with the third frame among frames that have already been inter-frame encoded. A control unit that controls, when the data size of each of the predetermined number of frames is equal to or larger than a first threshold, to intra-frame encode a fourth frame next to the third frame in the series of frames. apparatus. The controller is
(C) In (b), when the data size of each of the frames subjected to inter-frame coding is equal to or larger than the first threshold value, a continuous second frame starting from the fourth frame in the series of frames. The encoding apparatus according to claim 1, wherein control is performed so that two predetermined number of frames are intra-frame encoded. The control unit in (c),
Sequentially encoding the second predetermined number of frames in the order of playback;
When the fifth frame of the second predetermined number of frames is intra-frame encoded, the data size of the fifth frame that is intra-frame encoded is immediately before the fifth frame of the series of frames. 3. The code according to claim 2, wherein when the data size of the sixth frame that has been intra-frame encoded has changed by a third predetermined value or more, the seventh frame that follows the fifth frame is controlled to be inter-coded. Device. In the control unit (a),
When the encoding unit intra-encodes the first frame, the second and subsequent frames of the series of frames are controlled to be sequentially inter-frame encoded in the reproduction order,
When the data size of the eighth frame inter-coded at the time when the eighth frame of the frames after the second frame is inter-frame encoded is greater than or equal to the second threshold, The encoding device according to any one of claims 1 to 3, wherein control is performed so that the ninth frame following the eighth frame is intra-frame encoded. The encoding device further includes:
A transmission unit that transmits encoded data generated by the encoding unit performing intraframe encoding or interframe encoding; and
When the frame included in the encoded data is an inter-frame encoded frame and the data size of the frame is greater than or equal to a third threshold, the transmission unit is prohibited from transmitting the frame. The encoding apparatus of any one of Claims 1-4 provided with the transmission control part to perform. An encoding unit for intra-frame encoding or inter-frame encoding of frames included in a series of frames;
(D) When the encoding unit encodes the first frame of the series of frames, the second frame next to the first frame of the series of frames is inter-frame encoded. Control and
(E) When the encoding unit performs inter-frame encoding on the third frame of the series of frames, each data of a predetermined number of consecutive inter-frame encoded frames including the third frame And a control unit that controls the third frame to be subjected to intra-frame encoding when the size is equal to or larger than a first threshold. An encoding step of intra-frame encoding or inter-frame encoding of frames included in a series of frames;
(A) When the first frame of the series of frames is intra-frame encoded in the encoding step, the second frame next to the first frame of the series of frames is inter-frame encoded. Control and
(B) When the third frame of the series of frames is inter-frame encoded in the encoding step, a continuous first frame ending with the third frame among the frames already inter-frame encoded. And a control step of controlling the fourth frame next to the third frame in the series of frames when the data size of each of the predetermined number of frames is equal to or greater than a first threshold value. Method. An encoding step of intra-frame encoding or inter-frame encoding of frames included in a series of frames;
(D) When the first frame of the series of frames is intra-coded in the encoding step, the second frame next to the first frame of the series of frames is inter-frame encoded. Control and
(E) When a third frame of the series of frames is inter-frame encoded in the encoding step, each data of a consecutive predetermined number of inter-frame encoded frames including the third frame And a control step of controlling the third frame to be intra-coded when the size is equal to or larger than a first threshold.

Images