JP7570877B2 - Packet generation device and program - Google Patents

Description

The present invention relates to a packet generation device and program that compresses video signals of video frames and packetizes the compressed signals.

The Society of Motion Picture and Television Engineers (SMPTE) has been working on standardizing IP program production systems that convert video signals into IP packets and transmit them over general-purpose Ethernet (registered trademark) networks.

In the IP packetization standard for video signals standardized by SMPTE, a standard for IP packetizing and transmitting compressed video signals is known (see, for example, Non-Patent Document 1). In addition, an international standard for a video signal compression method suitable for IP program production systems has been standardized (see, for example, Non-Patent Document 2).

Figure 9 is a diagram explaining the packetization process using the JPEG-XS standard of Non-Patent Document 2. As shown in Figure 9, the Nth video frame is divided into a picture header and an area (hereinafter referred to as a "slice") in which multiple video lines (hereinafter referred to as "lines") are used as compression units, and compression processing is performed. N is an integer equal to or greater than 1.

The picture header and compressed signal are then packetized and converted into a data sequence called a packet. Specifically, the picture header and compressed signal are mapped into the payload of an RTP packet.

The SMPTE ST2110-22 standard in the aforementioned non-patent document 1 imposes restrictions on compression and packetization so that the amount of compressed signal and packet data generated from one video frame, as well as the number of packets (number of RTP packets), are constant. To satisfy this restriction, the packetization process allows the amount of data (and number of packets) to be adjusted by adding padding.

In the SMPTE ST2110-22 standard in the aforementioned non-patent document 1, the number of packets generated from the Nth video frame must be the same as the number of packets generated from the N+1th video frame.

However, the SMPTE ST2110-22 standard does not prescribe any specific method for ensuring that the number of packets generated from different video frames is the same.

Therefore, a method has been proposed to keep the number of packets generated for each video frame constant (see Patent Document 1). This method is called slice mapping, and it maps compressed data into the RTP payload in units of slices of video frames.

JP 2020-22087 A

SMPTE ST2110-22 JPEG-XS

However, in the method of Patent Document 1, when compressed data is treated as a unit of slice, the amount of data may vary for each slice. In the slice mapping method, packetization is performed for each slice, so if the amount of data varies for each slice, it is necessary to add padding to appropriately adjust the amount of data in order to satisfy the SMPTE ST2110-22 standard of Non-Patent Document 1 mentioned above.

As a method for adjusting the amount of data by adding padding, a method has been proposed by the same applicant as this patent application, described in Japanese Patent Application No. 2019-219809, which was unpublished at the time of filing this patent application.

This method adjusts the number of packets for each video frame based on the number of packets after packetization of slice data generated from the first video frame to be transmitted, in order to adjust the amount of data and number of packets for each video frame to a constant amount. Specifically, this method sets the number of packets for the first video frame plus a fixed adjustment value as a target value (Npacket_first), and uses adjustment packets to adjust the number of packets for the second and subsequent video frames so that the number of packets matches the target value.

This allows the number of packets generated from each video frame after packetization to match a target value, resulting in a constant number of packets for each video frame. However, the target value, which is a constant number of packets per video frame, varies depending on the first video frame.

The number of packets per slice generated by applying the slice mapping method varies depending on the amount of data in each slice. The amount of data in a slice, in turn, varies depending on the content of the video being compressed. In general, complex video tends to have a large amount of data, while monotonous video tends to have a small amount of data.

As a result, with this method, the target number of packets per video frame changes depending on the type of video the first video frame is. In other words, the target value is the target transmission rate, but even if the compression rate for each video frame is the same, the transmission rate changes depending on whether the video is complex or monotonous.

If the target value changes depending on the video, it becomes difficult to determine the necessary transmission rate in advance when operating in an environment with transmission bandwidth constraints. Also, for example, if the first video frame is a complex video, the target value will be large, and when transmitting a later monotonous video frame, the number of adjustment packets will increase, resulting in an unnecessary increase in transmission rate.

The present invention has been made to solve the above problems, and its purpose is to provide a packet generation device and program that can prevent unnecessary increases in transmission rate by making the amount of data and number of packets generated for each video frame constant in the slice mapping method and by making the target value for the number of packets per video frame constant and independent of the first video frame.

In order to solve the above problem, the packet generation device of claim 1 is a packet generation device that packetizes a compressed signal obtained by compressing a video frame, the packet generation device includes a compression signal generation unit that compresses the video frame for each slice and generates the compressed signal for each slice, and a packet generation unit that packetizes the compressed signal for each slice generated by the compression signal generation unit for each video frame to generate generated packets, generates adjustment packets of a predetermined adjustment packet reference number Adj, and increases or decreases the number of the adjustment packets so that the total number of the generated packets and the adjustment packets for the video frame is the same as a predetermined total packet number target value Npacket_target, and the packet generation unit determines the data amount Lcod of the compressed signal per video frame, the number of generated packets per video frame obtained from the data amount Lcod that is constant between the video frames as the generated packet reference number Np, the number of slices per video frame as the adjustment packet reference number Adj, and calculates the total packet number target value Npacket_target by adding the adjustment packet reference number Adj to the generated packet reference number Np.

The packet generating device of claim 2 is the packet generating device of claim 1, characterized in that the packet generating unit obtains the generated packet reference number Np by dividing the data amount Lcod of the compressed signal per video frame by a preset payload length of the generated packets, obtains the adjustment packet reference number Adj by dividing the vertical line number Hf of the video frame by the line number Hsl of the slice, and obtains the total packet number target value Npacket_target by adding the adjustment packet reference number Adj to the generated packet reference number Np.

The packet generation device of claim 3 is the packet generation device of claim 1, wherein the compressed signal generation unit generates a picture header from the video frame for each of the video frames, the picture header including the amount of data Lcod of the compressed signal per video frame, the number of lines Hf in the vertical direction of the video frame, and the number of lines Hsl of the slice, and generates a slice header and the compressed signal for each of the slices of the video frame. The packet generation unit obtains the amount of data Lcod of the compressed signal per video frame, the number of lines Hf in the vertical direction of the video frame, and the number of lines Hsl of the slice from the picture header generated by the compressed signal generation unit, and divides the amount of data Lcod of the compressed signal per video frame by a payload length of the generated packet that is set in advance, thereby generating the compressed signal. The method is characterized in that the adjustment packet reference number Adj is obtained by calculating a generation packet reference number Np, and the adjustment packet reference number Adj is obtained by dividing the number of lines Hf in the vertical direction of the video frame by the number of lines Hsl of the slice, and the total packet number target value Npacket_target is obtained by adding the adjustment packet reference number Adj to the generation packet reference number Np, and the generated packets are generated by packetizing the picture header generated by the compressed signal generation unit and the slice header and the compressed signal for each slice for each video frame, and the adjustment packets are generated for the adjustment packet reference number Adj, and the number of the adjustment packets is increased or decreased so that the total number of the generated packets and the adjustment packets for the video frame is the same as the total packet number target value Npacket_target.

Furthermore, the program of claim 4 is characterized in that it causes a computer to function as a packet generation device according to any one of claims 1 to 3.

As described above, according to the present invention, in the slice mapping method, the amount and number of packets generated for each video frame can be made constant, and the target value for the number of packets per video frame can be made constant, independent of the first video frame.

1 is a block diagram showing a configuration of a packet generating device according to an embodiment of the present invention. 11 is a flowchart illustrating an example of processing by a compressed signal generating unit. FIG. 13 is a diagram showing the structure of a picture header. 4 is a block diagram showing an example of the configuration of a packet generating unit; FIG. 13 is a flowchart showing an example of processing by a packet generating unit; 13 is a flowchart showing adjustment packet increase/decrease processing (step S519). 1 is a diagram for explaining the total packet number target value Npacket_target, 2 is a diagram for explaining a case where the total packet number Npacket is smaller than the total packet number target value Npacket_target, and 3 is a diagram for explaining a case where the total packet number Npacket is larger than the total packet number target value Npacket_target. FIG. 13 is a diagram showing a simulation result. 1A and 1B are diagrams for explaining a packetization process using the conventional JPEG-XS standard.

The following describes in detail the embodiments of the present invention with reference to the drawings. The present invention focuses on the fact that the amount of data Lcod of the compressed signal per video frame is constant (the same) between video frames, regardless of whether the video is complex or monotonous, as long as the compression parameters (compression rate, number of video pixels, etc.) are not changed.

The present invention is characterized in that it calculates a target number of packets per video frame (target total number of packets Npacket_target) by adding the number of slices per video frame (number of adjustment packets) to the amount of data Lcod of the compressed signal per video frame, and then increases or decreases the number of adjustment packets so that the number of packets per video frame is the same as the target total number of packets Npacket_target.

As a result, the total packet count target value Npacket_target remains constant even when the image changes from complex to monotonous, or vice versa.

Therefore, in the slice mapping method, the amount of data and number of packets generated for each video frame can be kept constant, and the target value for the total number of packets, Npacket_target, can be kept constant and independent of the first video frame.

[Packet Generation Device]
1 is a block diagram showing the configuration of a packet generating device according to an embodiment of the present invention. The packet generating device 1 comprises a compressed signal generating unit 10 and a packet generating unit 11.

The compressed signal generating unit 10 inputs video data in units of video frames, for example, from a camera. Then, for each video frame, the compressed signal generating unit 10 generates a picture header based on the video frame, and divides the video frame into video data for each slice (hereinafter referred to as "slice data").

The compressed signal generation unit 10 compresses the slice data for each slice in accordance with the JPEG-XS standard, generates a slice header based on the slice data of that slice, and generates a compressed signal based on the slice data of that slice.

The compressed signal generation unit 10 outputs a picture header, slice headers for each slice, and a compressed signal for each video frame to the packet generation unit 11.

The packet generation unit 11 inputs a picture header and a slice header and compressed signal for each slice from the compressed signal generation unit 10 for each video frame. Then, the packet generation unit 11 packetizes the picture header for each video frame, and packetizes the slice header and compressed signal for each slice in the video frame, to generate packets (generated packets). The packet generation unit 11 also packetizes specified adjustment data to generate adjustment packets.

Here, the packet generator 11 monitors its processing so as to comply with the constraints of the SMPTE ST2110-22 standard. That is, the packet generator 11 monitors for each video frame whether the number of packets generated from the video frame is the same. If the packet generator 11 determines that the number of packets is not the same, it adjusts the number of adjustment packets for that video frame, equivalent to the difference, so that the number of packets becomes the same.

Specifically, based on predetermined data included in the picture header, the packet generation unit 11 calculates the number of RTP packets corresponding to the compressed signal per video frame as the reference number of packets to be generated in the video frame (reference number of packets to be generated Np), and calculates the number of slices in the video frame as the reference number of adjustment packets in the video frame (reference number of adjustment packets Adj). The packet generation unit 11 then calculates the target total number of packets Npacket_target by adding the reference number of adjustment packets Adj to the reference number of packets to be generated Np.

The packet generation unit 11 increases or decreases the number of adjustment packets for each video frame so that the sum of the number of generated packets and adjustment packets generated from that video frame (total number of packets Npacket) is the same as the target value for the total number of packets Npacket_target.

By making the total number of packets per video frame, Npacket, the same as the target value for the total number of packets, Npacket_target, it is possible to comply with the constraints of the SMPTE ST2110-22 standard.

Note that the number of generated packets including picture headers is the same for all video frames, and is therefore excluded from the calculation here. The packet generation unit 11 may increase or decrease the number of adjustment packets for each video frame so that the total number of generated packets including picture headers, generated packets including slice headers and compressed signals, packets including compressed signals, and adjustment packets is the same as the total packet number target value Npacket_target.

The packet generator 11 transmits generated packets and adjustment packets for each video frame. There are three types of generated packets: packets containing a picture header, packets containing a slice header and a compressed signal, and packets containing a compressed signal.

(Compressed signal generator 10)
Next, a detailed description will be given of the compressed signal generating unit 10 shown in Fig. 1. Fig. 2 is a flowchart for explaining an example of processing performed by the compressed signal generating unit 10.

The compressed signal generator 10 sequentially inputs video data constituting a video frame (step S201) and performs the following steps S202 to S208 for each video frame.

The compressed signal generator 10 determines whether the video data of the sequentially input video frames is the beginning of a video frame (step S202).

If the compressed signal generator 10 determines in step S202 that the video data is not the beginning of a video frame (step S202: N), it ends the process and waits until the beginning video data of the next video frame is input.

If the compressed signal generator 10 determines in step S202 that the video data is the beginning of a video frame (step S202: Y), it determines that it has detected the beginning of a video frame and generates a picture header based on that video frame. The compressed signal generator 10 then outputs the picture header to the packet generator 11 (step S203).

3 is a diagram showing the structure of a picture header, which is described in Table A.6 (pages 18 and 19) of the following document:
[Non-patent literature] INTERNATIONAL STANDARD ISO/IEC21122-1, Information technology-JPEG XS low-latency lightweight image coding system-Part1:Core coding system, Reference number ISO/IEC21122-1:2019(E)

The picture header is composed of (1) the amount of compressed signal data per video frame (Lcod), (2) the number of vertical lines in the video frame (Hf), (3) the number of lines in a slice (Hsl), etc. The amount of compressed signal data per video frame Lcod, the number of vertical lines in the video frame Hf, and the number of lines in a slice Hsl are the same values regardless of whether the video is complex or monotonous, as long as the compression parameters are not changed. The same is true for the other data that make up the picture header.

The receiving device that receives the packets sent from the packet generating device 1 reconstructs the video frame based on the various data that make up this picture header.

Returning to FIG. 2, the compressed signal generator 10 proceeds from step S203 to slice the video data for each predetermined number of lines in the video frame (step S204). In other words, the compressed signal generator 10 treats each predetermined number of lines as one slice and performs the processes of steps S204 to S208 for each slice. The sliced video data for the predetermined number of lines is treated as slice data.

The compressed signal generating unit 10 compresses the slice data for each slice in accordance with the JPEG-XS standard (step S205). The compressed signal generating unit 10 then generates a slice header based on the slice data of the slice, and generates a compressed signal based on the slice data of the slice (step S206).

The compressed signal generation unit 10 outputs the slice header and the compressed signal to the packet generation unit 11 (step S207).

The compressed signal generator 10 determines whether or not processing of all slices of the input video frame has been completed (step S208). If the compressed signal generator 10 determines in step S208 that processing of all slices has not been completed (step S208: N), it proceeds to step S204 and performs the processing of steps S204 to S208 on the next slice.

On the other hand, if the compressed signal generator 10 determines in step S208 that processing of all slices has been completed (step S208: Y), the compressed signal generator 10 ends processing of the video frame. Then, the compressed signal generator 10 processes the next video frame.

In this way, the processing of steps S201 to S208 is performed for each video frame, and the processing of steps S204 to S208 is performed for each slice. Furthermore, the compression processing is performed for each slice, and a compressed signal is generated. Then, within one video frame, the picture header and the slice headers and compressed signals for each slice (for all slices) are output to the packet generation unit 11.

(Packet Generation Unit 11)
Next, a detailed description will be given of the packet generator 11 shown in Fig. 1. Fig. 4 is a block diagram showing an example of the configuration of the packet generator 11, and Fig. 5 is a flowchart showing an example of the process of the packet generator 11.

The packet generation unit 11 includes a picture header processing unit 20, a slice header and data processing unit 21, and an adjustment unit 22. The adjustment unit 22 includes an adjustment packet processing unit 23 and an adjustment packet increase/decrease unit 24.

As mentioned above, the packet generator 11 monitors its own processing so as to comply with the constraints of the SMPTE ST2110-22 standard. The compressed signal generator 10 compresses slice data for each slice of a video frame to generate a compressed signal. In this case, the amount of data in the compressed signal may differ for each slice. This results in a change in the number of packets for each video frame, and in order to comply with the constraints of the SMPTE ST2110-22 standard, it is necessary to adjust the number of packets between video frames.

Therefore, the packet generation unit 11 adjusts the number of adjustment packets so that the number of packets generated by the slice mapping method for each video frame is constant. Specifically, the packet generation unit 11 increases or decreases the number of adjustment packets for each video frame based on the number of adjustment packets common between video frames.

The packet generator 11 inputs the picture header, slice header, and compressed signal from the compressed signal generator 10 (step S501). The packet generator 11 determines whether the data input in step S501 is a picture header (step S502).

If the packet generation unit 11 determines in step S502 that the input data is a picture header (step S502: Y), the process proceeds to step S503, and the picture header processing unit 20 performs the processes of steps S503 to S510.

On the other hand, if the packet generation unit 11 determines in step S502 that the input data is not a picture header (step S502: N), the process proceeds to step S511, the slice header and data processing unit 21 performs the processes of steps S511 to S515, and the adjustment unit 22 performs the processes of steps S516 to S520.

The picture header processing unit 20 proceeds from step S502 (Y) and obtains the number of vertical lines Hf of the video frame, the number of lines Hsl of the slice, and the amount of data Lcod of the compressed signal per video frame from the picture header (step S503).

The picture header processing unit 20 obtains the number of slices in the video frame by dividing the number of lines in the vertical direction Hf in the video frame by the number of lines in the slice Hsl using the following formula, and sets this as the adjustment packet reference number Adj (step S504).
[Equation 1]
Adj=ceil(Hf/Hsl)...(1)

Based on this adjustment packet reference number Adj, the adjustment packetization process is performed in step S518, which will be described later, and the number of adjustment packets transmitted in the video frame is increased or decreased in step S519, which will be described later.

Here, in the compression process using the JPEG-XS standard by the compressed signal generation unit 10, the compression process is performed for each slice, so the length of the compressed signal will differ for each slice, but the difference will be one packet. In other words, the difference in the number of packets generated for each video frame will be at most the number of slices in the video frame, i.e., the adjustment packet reference number Adj.

Therefore, in the processing of steps S518 and S519 described below, the adjustment packets are increased or decreased based on the adjustment packet reference number Adj, and it is possible to increase or decrease the adjustment packets up to the adjustment packet reference number Adj, which is the number of slices in a video frame.

In this way, in step S504, the adjustment packet reference number Adj, which indicates the number of slices in a video frame, is calculated, and the adjustment packet reference number Adj is used as a reference for increasing or decreasing the number of adjustment packets. This results in the generation, adjustment, and transmission of the minimum number of adjustment packets that satisfies the constraints of the SMPTE ST2110-22 standard. In other words, the efficiency of processing and transmission by the packet generation device 1 can be improved.

The above formula (1) uses the ceil function, but it is also possible to use the floor function or other functions.

After the process of step S504, the picture header processing unit 20 inputs the preset RTP payload length RTPlen (step S505). Then, the picture header processing unit 20 divides the data amount Lcod of the compressed signal per video frame by the RTP payload length RTPlen using the following formula to obtain the number of RTP packets corresponding to the compressed video signal per video frame, and sets this as the generation packet reference number Np (step S506). Here, the data amount Lcod of the compressed signal per video frame is constant between video frames.
[Equation 2]
Np=ceil(Lcod/RTPlen)...(2)

The above formula (2) uses the ceil function, but it is also possible to use the floor function or another function.

The picture header processing unit 20 calculates a target total number of packets Npacket_target indicating a target value for the total number of packets to be generated in one video frame by adding the reference number of adjustment packets Adj calculated in step S504 to the reference number of generated packets Np calculated in step S506 using the following formula (step S507).
[Equation 3]
Npacket_target=Np+Adj...(3)

This total packet count target value, Npacket_target, is used as the target number of packets for all video frames.

In step S506, the amount of compressed signal data per video frame Lcod and the RTP payload length RTPlen used in calculating the reference number of packets to be generated Np are values that do not change unless the compression parameters are changed. Similarly, in step S504, the number of vertical lines Hf and the number of lines Hsl in a slice used in calculating the reference number of adjustment packets Adj are values that do not change unless the compression parameters are changed. For this reason, the target value Npacket_target for the total number of packets calculated in step S507 is also a value that does not change unless the compression parameters are changed.

In other words, the total packet count target value Npacket_target is a constant value that does not change unless the compression parameters are changed, regardless of whether the video is complex or monotonous, and regardless of the level of complexity. As a result, by using the total packet count target value Npacket_target, the required transmission rate can be considered in advance, solving the conventional problem.

The picture header processing unit 20 maps the picture header to the RTP payload (step S508), and performs packetization processing using UDP, IP, and MAC processing to generate a packet (step S509). The picture header processing unit 20 transmits the generated packet as the generated packet (step S510), and proceeds to step S502.

Here, the picture header processing unit 20 calculates the adjustment packet reference number Adj, the generation packet reference number Np, and the total packet count target value Npacket_target for each video frame in steps S504, S506, and S507. In other words, the adjustment packet reference number Adj, the generation packet reference number Np, and the total packet count target value Npacket_target are calculated for each video frame regardless of whether the compression parameters of the video frame have been changed. This allows the packet generating device 1 to realize continuous processing without interrupting the packet generation process, even if the compression parameters of the video frame have been changed.

If the input data is not a picture header, the slice header and data processing unit 21 proceeds from step S502 (N) and maps the slice header and compressed signal of the slice to the RTP payload (step S511). Then, the slice header and data processing unit 21 generates a packet by performing packetization processing using UDP, IP, and MAC processing (step S512). The slice header and data processing unit 21 transmits the generated packet as a generated packet (step S513).

The slice header and data processing unit 21 determines whether or not processing of all slices for the video frame has been completed (step S514). If the slice header and data processing unit 21 determines in step S514 that processing of all slices has not been completed (step S514: N), it performs the processes of steps S511 to S514 for the next slice (step S515).

On the other hand, if the slice header and data processing unit 21 determines that processing of all slices for the video frame has been completed (step S514: Y), it proceeds to step S516.

The adjustment packet processing unit 23 of the adjustment unit 22 proceeds from step S514 (Y) and acquires the number of generated packets Npacket_tmp indicating the number of slice headers and compressed signal packets generated for the video frame in step S512 (step S516).

The adjustment packet processing unit 23 calculates the total number of packets Npacket indicating the total number of packets generated in one video frame (the total number of generated packets and adjustment packets) by adding the adjustment packet reference number Adj calculated in step S504 to the number of generated packets Npacket_tmp obtained in step S516 using the following formula (step S517).
[Equation 4]
Npacket=Npacket_tmp+Adj...(4)

The adjustment packet processing unit 23 maps the preset adjustment data to the RTP payload. Then, the adjustment packet processing unit 23 performs packetization processing for the reference number Adj of adjustment packets calculated in step S504 using UDP, IP, and MAC processing, and generates adjustment packets for the reference number Adj of adjustment packets (step S518).

The preset adjustment data is a set of padding data (padding data group) with a value of 0 as specified in the RFC3550 standard. When generating an adjustment packet, the adjustment packet processing unit 23 sets the padding flag P in the RTP header to 1 to indicate that the RTP packet, which is the adjustment packet, contains padding data.

When an embodiment of the present invention is applied to the technology of Patent Document 1, there are two types of packets for which the padding flag P is set to 1: packets that contain a compressed signal and padding data in the same packet, and adjustment packets that contain only padding data.

In this case, the number of padding data is set at the end of the padding data group so that the receiving device can distinguish between these two types of packets.

If the number set at the end of the padding data matches the total number of padding data mapped to the RTP payload (a preset number), the receiving device identifies the received packet as a calibration packet. On the other hand, if the numbers do not match, the receiving device identifies the packet as containing a compressed signal and padding data.

The adjustment packet increase/decrease unit 24 of the adjustment unit 22 proceeds from step S518 to perform adjustment packet increase/decrease processing (step S519). Details of the adjustment packet increase/decrease processing in step S519 will be described later.

After performing the adjustment packet increase/decrease process, the adjustment packet increase/decrease unit 24 transmits the adjustment packets after the increase/decrease (step S520).

(Adjustment packet increase/decrease process/step S519)
6 is a flowchart showing the adjustment packet increase/decrease process (step S519). The adjustment packet increase/decrease unit 24 compares the total number of packets Npacket calculated in step S517 with the target value Npacket_target of the total number of packets calculated in step S507 (step S601).

If the adjustment packet increase/decrease unit 24 determines in step S601 that the total number of packets Npacket is smaller than the total number of packets target value Npacket_target (step S601: Npacket < Npacket_target), it performs processing to increase the adjustment packets in steps S602 and S603.

Specifically, the adjustment packet adding/reducing unit 24 maps preset adjustment data to the RTP payload, performs packetization processing for one packet by UDP, IP, and MAC processing, and adds one adjustment packet to the generated adjustment packet (step S602). The adjustment packet adding/reducing unit 24 increments the total number of packets Npacket by one (step S603), and proceeds to step S601. This increases the number of adjustment packets sent in step S520 of FIG. 5.

On the other hand, if the adjustment packet increase/decrease unit 24 determines in step S601 that the total number of packets Npacket is greater than the target value Npacket_target for the total number of packets (step S601: Npacket>Npacket_target), it performs processing to decrease the adjustment packets in steps S604 and S605.

Specifically, the adjustment packet increase/decrease unit 24 reduces one adjustment packet from the adjustment packets that have already been generated (step S604), decrements the total number of packets Npacket by one (step S605), and proceeds to step S601. As a result, the number of adjustment packets sent in step S520 of FIG. 5 is reduced.

On the other hand, if the adjustment packet increase/decrease unit 24 determines in step S601 that the total number of packets Npacket and the target value for the total number of packets Npacket_target are equal (step S601: Npacket_target = Npacket), it terminates the process.

In this way, the adjustment packets are increased or decreased based on the adjustment packet reference number Adj already generated in step S518 of FIG. 5 until it is determined that the total number of packets Npacket and the target value for the total number of packets Npacket_target are equal. Then, the number of adjustment packets after the increase or decrease process when it is determined that the total number of packets Npacket and the target value for the total number of packets Npacket_target are equal is transmitted in step S520 of FIG. 5.

Next, the total packet count target value Npacket_target, the total packet count Npacket, the adjustment packet reference number Adj, and the number of adjustment packets actually sent in the adjustment packet increase/decrease process shown in Figure 6 will be explained. Figure 7 (1) is a diagram explaining the total packet count target value Npacket_target, (2) is a diagram explaining a case where the total packet count Npacket is smaller than the total packet count target value Npacket_target, and (3) is a diagram explaining a case where the total packet count Npacket is larger than the total packet count target value Npacket_target.

Referring to (1), the target value for the total number of packets, Npacket_target, consists of a reference number of packets to be generated, Np, which corresponds to the number of packets to be generated, and a reference number of adjustment packets, Adj, which corresponds to the number of adjustment packets.

As mentioned above, the compressed signal generator 10 shown in FIG. 1 divides a video frame into slice data for each slice, and compresses the slice data for each slice in accordance with the JPEG-XS standard in the aforementioned non-patent document 2 to generate a compressed signal.

Here, according to the JPEG-XS standard, the slice data amount, which is the amount of data in the compressed signal for each slice, varies for each slice, but the frame data amount, which is the total amount of compressed signal for each slice in one video frame, is constant between video frames. In other words, the amount of data Lcod of the compressed signal per video frame is constant between video frames. For this reason, the reference number of packets to be generated Np and the target value for the total number of packets Npacket_target, which are calculated using the amount of data Lcod of the compressed signal per video frame, etc., are also constant between video frames.

Thus, the amount of frame data is constant between video frames, but the amount of slice data varies from slice to slice, so the number of generated packets Npacket_tmp generated from all slice data in a video frame varies from video frame to video frame.

In other words, as shown in (2), the number of generated packets Npacket_tmp may be smaller than the reference number of generated packets Np, and as shown in (3), the number of generated packets Npacket_tmp may be larger than the reference number of generated packets Np.

Referring to (2), if the total number of packets Npacket is smaller than the target value for the total number of packets Npacket_target (step S601: Npacket < Npacket_target), the number of adjustment packets is increased by the number equivalent to the difference. As a result, the number of adjustment packets transmitted in the video frame is the number obtained by adding the difference to the adjustment packet reference number Adj.

Referring to (3), if the total number of packets Npacket is greater than the target value for the total number of packets Npacket_target (step S601: Npacket>Npacket_target), the number of adjustment packets is reduced by the number equivalent to the difference. As a result, the number of adjustment packets transmitted in the video frame becomes the number obtained by subtracting the difference from the adjustment packet reference number Adj.

[Simulation results]
Next, the results of a computer simulation will be described. Fig. 8 is a diagram showing the results of the simulation, which shows the results of the method described in Japanese Patent Application No. 2019-219809 (a method for determining a target value in the first video frame) that was not yet published at the time of filing this patent application, and the results of an embodiment of the present invention.

In the graph on the left at the bottom of Figure 8, the horizontal axis indicates the frame number, and the vertical axis indicates the difference between the target value Npacket_first for the total number of packets and the number of generated packets Npacket_first_tmp when using the method described in Japanese Patent Application No. 2019-219809, i.e., the number of adjustment packets.

In addition, in the graph on the lower right side of Figure 8, the horizontal axis indicates the frame number, and the vertical axis indicates the difference between the target value for the total number of packets Npacket_target and the number of generated packets Npacket_tmp when an embodiment of the present invention is used, i.e., the number of adjustment packets.

a shows the characteristics when the RTP payload length RTPlen = 64 bytes, the target value Npacket_first = 16303 of the method described in Japanese Patent Application No. 2019-219809, and the target value Npacket_target of the total number of packets in the embodiment of the present invention = 16267. b, c, and d show the characteristics when the RTP payload length RTPlen, the target value Npacket_first of the method described in Japanese Patent Application No. 2019-219809, and the target value Npacket_target of the total number of packets in the embodiment of the present invention are respectively as shown in the table at the top of Figure 8.

From Figure 8, it can be seen that for all characteristics a, b, c, and d, the difference between the target value Npacket_target for the total number of packets and the number of generated packets Npacket_tmp in the embodiment of the present invention is smaller than the difference between the target value Npacket_first and the number of generated packets Npacket_first_tmp in the method described in Japanese Patent Application No. 2019-219809.

These differences correspond to the number of adjustment packets generated, so the number of adjustment packets generated for each video frame is smaller in the embodiment of the present invention than in the method described in Japanese Patent Application No. 2019-219809. As a result, the embodiment of the present invention can transmit generated packets and adjustment packets at a lower transmission rate than the method described in Japanese Patent Application No. 2019-219809. Therefore, the embodiment of the present invention can suppress unnecessary increases in transmission rate compared to the method described in Japanese Patent Application No. 2019-219809.

As described above, according to the packet generation device 1 of the embodiment of the present invention, the compressed signal generation unit 10 generates a picture header for each video frame based on the video frame. The compressed signal generation unit 10 also divides the video frame into slice data for each slice, compresses the slice data according to the JPEG-XS standard, and generates a slice header and a compressed signal based on the slice data of the slice.

The packet generation unit 11 obtains an adjustment packet reference number Adj indicating the number of slices in a video frame by dividing the number of vertical lines Hf of the video frame included in the picture header by the number of lines Hsl of the slice, obtains a generated packet reference number Np by dividing the data amount Lcod of the compressed signal per video frame included in the picture header by a preset RTP payload length RTPlen, and obtains a total packet number target value Npacket_target by adding the adjustment packet reference number Adj to the generated packet reference number Np.

The packet generation unit 11 increases or decreases the number of adjustment packets for each video frame so that the sum of the number of generated packets and adjustment packets generated from that video frame (total number of packets Npacket) is the same as the target value for the total number of packets Npacket_target.

As a result, even if the image changes from complex to monotonous, or vice versa, the target value for the total number of packets, Npacket_target, will not change unless the compression parameters are changed. In other words, in the slice mapping method, the amount and number of packets generated for each video frame can be kept constant, and the target value for the number of packets per video frame can be kept constant, independent of the first video frame.

In addition, because the number of packets generated for each video frame in the slice mapping method is constant, it is possible to realize packet transmission of a compressed signal that satisfies the constraints of the SMPTE ST2110-22 standard in the aforementioned non-patent document 1. And because the amount and number of packet data for each video frame is constant regardless of the degree of complexity of the video, the transmission rate can be fixed and the transmission rate can be considered in advance.

For example, in the method described in Japanese Patent Application No. 2019-219809 (a method of determining a target value in the first video frame), when the first video is complex, the target value becomes large, and when a monotonous video is transmitted later, the number of adjustment packets increases, resulting in an unnecessary increase in the transmission rate. In an embodiment of the present invention, the number of packets generated for each video frame is constant regardless of the degree of complexity of the video, so the number of unnecessary adjustment packets does not increase, and the transmission rate does not increase.

The present invention has been described above using embodiments, but the present invention is not limited to the above embodiments and can be modified in various ways without departing from the technical concept thereof.

The hardware configuration of the packet generation device 1 according to the embodiment of the present invention can be a normal computer. The packet generation device 1 is configured by a computer equipped with a CPU, a volatile storage medium such as RAM, a non-volatile storage medium such as ROM, and an interface, etc.

The functions of the compressed signal generator 10 and packet generator 11 (picture header processor 20, slice header and data processor 21, and adjustment unit 22) provided in the packet generator 1 are each realized by having the CPU execute a program that describes these functions.

These programs are stored in the storage medium and are read and executed by the CPU. In addition, these programs can be distributed by storing them on storage media such as magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), and semiconductor memories, and can also be transmitted and received via a network.

REFERENCE SIGNS LIST 1 Packet generation device 10 Compressed signal generation unit 11 Packet generation unit 20 Picture header processing unit 21 Slice header and data processing unit 22 Adjustment unit 23 Adjustment packet processing unit 24 Adjustment packet increase/decrease unit
Lcod Data volume of compressed signal per video frame
Hf The number of vertical lines in a video frame
Number of lines in the Hsl slice
RTPlen RTP payload length
Np: Reference number of packets generated (number of RTP packets corresponding to the compressed signal per video frame)
Adj: Adjustment packet reference number (number of slices in a video frame)
Npacket_target Total number of packets target value
Npacket_tmp Number of packets generated
Npacket Total number of packets

Claims (4)

A packet generation device for packetizing a compressed signal obtained by compressing a video frame, comprising:
a compressed signal generating unit that compresses the video frame for each slice and generates the compressed signal for each slice;
packetizing the compressed signal for each slice generated by the compressed signal generating unit for each video frame to generate generated packets, and generating adjustment packets of a predetermined adjustment packet reference number Adj;
a packet generation unit that increases or decreases the number of the adjustment packets so that a total number of the generated packets and the adjustment packets for the video frame becomes equal to a predetermined total packet number target value Npacket_target,
The packet generation unit
The data amount Lcod of the compressed signal per video frame is a constant data amount Lcod between the video frames, and the number of generated packets per video frame obtained from the data amount Lcod is a generated packet reference number Np, and the number of slices per video frame is the adjustment packet reference number Adj,
A packet generating device, comprising: a packet generating unit that calculates the target value Npacket_target for the total number of packets by adding the adjustment packet reference number Adj to the generated packet reference number Np. 2. The packet generating device according to claim 1,
The packet generation unit
obtain the generated packet reference number Np by dividing the data amount Lcod of the compressed signal per video frame by a preset payload length of the generated packets;
The adjustment packet reference number Adj is obtained by dividing the number of lines Hf in the vertical direction of the video frame by the number of lines Hsl of the slice;
A packet generating device, comprising: a packet generating unit that calculates the target value Npacket_target for the total number of packets by adding the adjustment packet reference number Adj to the generated packet reference number Np. 2. The packet generating device according to claim 1,
The compressed signal generation unit
generating a picture header from the video frame, the picture header including a data amount Lcod of the compressed signal per video frame, a vertical line number Hf of the video frame, and a line number Hsl of the slice, and generating a slice header and the compressed signal for each slice of the video frame;
The packet generation unit
obtain, from the picture header generated by the compressed signal generation unit, a data amount Lcod of the compressed signal per video frame, a vertical line number Hf of the video frame, and a line number Hsl of the slice;
obtain the generated packet reference number Np by dividing the data amount Lcod of the compressed signal per video frame by a preset payload length of the generated packets;
The adjustment packet reference number Adj is obtained by dividing the number of lines Hf in the vertical direction of the video frame by the number of lines Hsl of the slice;
The total packet number target value Npacket_target is calculated by adding the adjustment packet reference number Adj to the generated packet reference number Np;
packetizing the picture header generated by the compressed signal generating unit, and the slice header and the compressed signal for each slice, to generate the generated packets, and generating the adjustment packets of the adjustment packet reference number Adj;
A packet generating device, comprising: a packet generating unit configured to increase or decrease the number of the adjustment packets so that the total number of the generated packets and the adjustment packets for the video frame becomes equal to the total packet number target value Npacket_target. A program for causing a computer to function as a packet generating device according to any one of claims 1 to 3.

Images