JP5330183B2 - Packet insertion / deletion method and call system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce deterioration in quality of a video image or voice when a packet is inserted or deleted in or from a fluctuation absorbing buffer in order to adjust buffer size of the number of accumulated packets of the fluctuation absorbing buffer. <P>SOLUTION: When one packet is deleted from the fluctuation absorbing buffer 22, if two or more effective packets including the voice or the video image continuously exist, a buffer size change part 24 performs overlap addition of the two continuous effective packets located in the middle to delete the two continuous effective packets among the continuous effective packets. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for adjusting the buffer size of a fluctuation absorbing buffer.

  In recent years, systems such as VoIP that transmit voice packets in real time are known. In such a system, the terminal device on the transmission side transmits packets at a constant period of about 10 msec to 40 msec. The reception-side terminal device also reproduces the audio signal included in the packet at the same cycle as the transmission-side terminal device. Thereby, the sound is continuously reproduced.

  However, if packet fluctuations occur due to packet collisions on the network, deterioration of transmission conditions in the wireless section, congestion of routers and bridges, etc., the terminal device on the receiving side sends the packets at a fixed period. It becomes impossible to receive. Therefore, in order to absorb fluctuations in transmission delay and reproduce voices at a constant period, a terminal device on the receiving side is generally provided with a fluctuation absorbing buffer. That is, by accumulating a certain number of packets in the fluctuation absorbing buffer, the exhaustion of packets is prevented and continuous reproduction of audio signals is realized.

  In Patent Document 1, when the number of stored packets in the fluctuation absorbing buffer reaches the start accumulated packet number, the arrival of received packets larger than the starting accumulated packet number in the fluctuation absorbing buffer control method that starts reading the received packets. When timing fluctuations occur, the starting accumulated packet number and the maximum accumulated packet number are increased and changed. On the other hand, when the fluctuation is stable at a value lower than the starting accumulated packet number, the starting accumulated packet number and the maximum accumulated packet number A technique for reducing the change is disclosed.

  Further, Patent Document 2 starts reproduction of a received packet when the number of received packets exceeds a reference value, and monitors the number of packets in a reception buffer (fluctuation absorbing buffer) at a predetermined time interval. A packet processing device is disclosed that changes the reference value depending on whether the number of packets increases or decreases with time.

Japanese Patent No. 3874112 Japanese Patent No. 3506960

  However, the techniques of Patent Documents 1 and 2 both adjust the reference value itself such as the number of starting accumulated packets, and insert a packet into the fluctuation absorbing buffer in order to adjust the buffer size of the fluctuation absorbing buffer. It does not delete packets from the fluctuation absorbing buffer.

  An object of the present invention is to provide a technique capable of reducing deterioration in video or audio quality when a packet is inserted into or deleted from a fluctuation absorbing buffer in order to adjust the buffer size of the number of accumulated packets in the fluctuation absorbing buffer. That is.

(1) In the packet insertion / deletion method according to one aspect of the present invention, the receiving device that receives an audio or video packet to be continuously reproduced from a transmitting device connected via an IP network and stores the packet in a fluctuation absorbing buffer is provided. A packet insertion / deletion method for inserting or deleting a packet accumulated in a fluctuation absorbing buffer, the counting step counting the number of accumulated packets of the packet stored in the fluctuation absorbing buffer, and a count result by the counting step And a buffer size changing step for inserting or deleting a packet in the fluctuation absorbing buffer. When the buffer size changing step deletes one packet from the fluctuation absorbing buffer, an effective packet including audio or video is included. If there are two or more in succession, these Of the effective packets continue, the effective packet located in the center, and deletes two valid packets overlap-add to the effective packet positioned consecutively to the valid packet.

A call system according to another aspect of the present invention is a call system including a transmission device and a reception device connected via an IP network, and the reception device receives audio to be continuously reproduced from the transmission device. Alternatively, a fluctuation absorbing buffer for storing video packets, a counting means for counting the number of packets stored in the fluctuation absorbing buffer, and a packet inserted into the fluctuation absorbing buffer based on a count result by the counting means Or a buffer size changing means for deleting, and when the buffer size changing means deletes one packet from the fluctuation absorbing buffer, if there are two or more valid packets containing audio or video in succession, these buffer size changing means of the effective consecutive packets, the effective packet positioned in the center, the And deletes the two valid packets between the effective packet positioned continuously in effect packets overlap-add to.

  According to this configuration, when one packet is deleted from the fluctuation absorbing buffer, one packet is generated by overlapping and adding two packets located in the middle (middle) of a section where two or more valid packets are continuous. Therefore, it is possible to reduce the deterioration of audio and video quality.

  (2) The buffer size changing step inserts an invalid packet that does not include audio or video when inserting a packet into the fluctuation absorbing buffer, and inserts a past packet when deleting a packet from the fluctuation absorbing buffer. If there is an invalid packet, it is preferable to delete the invalid packet.

  According to this configuration, when a packet is deleted from the fluctuation absorbing buffer, if there is an invalid packet inserted in the past, the invalid packet is deleted. As a result, deletion of valid packets is prevented, and quality degradation of audio and video can be reduced as compared with the case of deleting valid packets.

  (3) When inserting a packet into the fluctuation absorbing buffer, the buffer size changing step preferably inserts an invalid packet between these two valid packets if there are two consecutive valid packets.

  According to this configuration, since the invalid packet is sandwiched between two valid packets, when the packet loss concealment process is performed on the invalid packet, the invalid packet can be concealed from the preceding and succeeding valid packets. The continuity of the video is maintained, and the audio and video can be reproduced smoothly.

  (4) In the buffer size changing step, an upper limit value of the number of packets that can be inserted or deleted at a time is preferably determined in advance.

  According to this configuration, the number of invalid packets to be inserted and the number of overlap-added packets are within the upper limit value, and quality degradation of audio and video can be reduced.

  (5) It is preferable to further include a concealment processing step for executing packet loss concealment processing on the invalid packet inserted by the buffer size changing step.

  According to this configuration, since the packet loss concealment process is executed for the inserted invalid packet, it is possible to reduce audio degradation and video degradation.

(6) In the buffer size changing step, when an invalid packet is deleted, when a valid packet corresponding to the deleted invalid packet is received later, if there is an invalid packet next to the deleted invalid packet , the next step It is preferable to replace the invalid packet and the received valid packet.

  According to this configuration, when an invalid packet is inserted and then a valid packet corresponding to the invalid packet is received, the valid packet is restored, so that quality degradation of audio and video can be reduced.

  (7) Preferably, the packet is a voice packet, and the buffer size changing step inserts an invalid packet between two consecutive packets storing vowel voice.

  According to this configuration, since the invalid packet is inserted between two consecutive packets including the vowel voice, the voice generated by executing the packet loss concealment process on the inserted invalid packet is Can be continuously connected to the voice included in the packet, and quality degradation of the voice can be reduced.

  According to the present invention, it is possible to change the jitter buffer size, which is the size of the fluctuation absorbing buffer, so that the quality degradation of audio and video is reduced.

1 shows a network configuration diagram of a call system according to an embodiment of the present invention. FIG. FIG. 2 shows a block diagram of the receiving apparatus shown in FIG. 1. It is explanatory drawing of the calculation process of the packet count value by the count part shown in FIG. It is a figure for demonstrating the role of a fluctuation | variation absorption buffer. An example of the graph of the transmission delay characteristic which shows the relationship between transmission delay and occurrence frequency is shown. It is a figure for demonstrating the optimal buffer size of a fluctuation | variation absorption buffer. It is a flowchart which shows the process of the receiver shown in FIG. It is a flowchart which shows the detail of the calculation process of a packet count value. It is the graph which showed the relationship between the packet count value and the calculation time of a packet count value. (A) is the schematic diagram which showed the process at the time of packet insertion by the buffer size change part, (b) is the schematic diagram which showed the process at the time of packet deletion by the buffer size change part. It is explanatory drawing of the process which a buffer size change part deletes one packet by overlap addition. It is explanatory drawing of the process in which a buffer size change part deletes one invalid packet. It is explanatory drawing of the process which a buffer size change part inserts one packet by overlap addition. It is a figure for demonstrating the process in the case of inserting five packets at a time into a fluctuation | variation absorption buffer. It is a figure explaining the process at the time of receiving the valid packet corresponding to the deleted invalid packet after deleting an invalid packet. It is a figure explaining the process in case a buffer size change part inserts the packet by which concealment processing was carried out instead of the invalid packet in a fluctuation | variation absorption buffer. It is the flowchart which showed the deletion process by the buffer size change part. It is the flowchart which showed the insertion process by the buffer size change part.

  FIG. 1 shows a network configuration diagram of a call system according to an embodiment of the present invention. The call system shown in FIG. 1 includes a transmission device 1 and a reception device 2. The transmission device 1 and the reception device 2 are connected via an IP network. Here, the IP network is a network interconnected using Internet protocol technology, and various data are transmitted and received using a TCP (Transmission Control Protocol) / IP (Internet Protocol) communication protocol.

  The transmission device 1 is a communication device having a call function based on VoIP (Voice over Internet Protocol), for example, SIP (Session Initiation Protocol), H.264. Communication is performed with the receiving apparatus 2 using a communication protocol such as H.323, IPv4, or IPv6. Then, the transmission apparatus 1 packetizes the audio or video using RTP (Real-time Transport Protocol) which is a predetermined protocol for stream transmission of audio and video, and transmits the packet to the reception apparatus 2.

  In the present embodiment, it is assumed that the transmission device 1 packetizes an audio signal by RTP and transmits the packet to the reception device 2 at a predetermined first period (for example, 20 msec). Therefore, the packet transmitted by the transmission apparatus 1 includes a 20 msec digital audio signal. Note that, as the audio signal, for example, a signal sampled at a sampling frequency of 8 kHz by PCM μ-Law or the like can be employed. The transmission device 1 may packetize the video signal.

  The receiving device 2 includes a communication device having the same configuration as that of the transmitting device 1. In FIG. 1, for convenience of explanation, the transmitting device 1 and the receiving device 2 are described, but both devices can transmit and receive packets to and from each other.

  FIG. 2 shows a block diagram of the receiving apparatus 2 shown in FIG. The receiving apparatus 2 includes a packet transmitting / receiving unit 21, a fluctuation absorbing buffer 22, a counting unit 23, a buffer size changing unit 24, a reception time recording unit 25, a reference value storage unit 26, a concealment processing unit 27, a reproduction unit 28, and an observation history holding unit. 29, a speaker 30, a display unit 31, a microphone 33, and a packet generation unit 34.

  The packet transmission / reception unit 21 receives a packet transmitted from the transmission device 1 in the first period, and accumulates the packet in the fluctuation absorbing buffer 22. Since this packet is a packet compliant with RTP as described above, it includes an RTP header. Therefore, the packet transmitting / receiving unit 21 accumulates the received packets in the fluctuation absorbing buffer 22 in chronological order according to the sequence number included in the RTP header.

  Note that the RTP header includes a sequence number and the like. The sequence number indicates the transmission order of voice packets.

  Further, the packet transmitting / receiving unit 21 transmits the packet generated by the packet generating unit 34 to the IP network at the first period Ta.

  The packet transmitting / receiving unit 21 records the reception time of the received packet in the reception time recording unit 25. In this case, the packet transmitting / receiving unit 21 may record the sequence number of the received packet in the reception time recording unit 25 in association with the reception time.

  The fluctuation absorbing buffer 22 is configured by, for example, a ring buffer, and accumulates packets received by the packet transmitting / receiving unit 21 in chronological order. As a result, fluctuations in the transmission delay of the packet transmitted from the transmission device 1 are absorbed. As the size of the fluctuation absorbing buffer 22, a size larger than a reference value described later is adopted.

  The count unit 23 calculates a packet count value by counting the number of accumulated packets accumulated in the fluctuation absorbing buffer 22 in a predetermined second period equal to or less than the first period, and observes the calculated packet count value The history holding unit 29 holds it. The observation history holding unit 29 is configured by, for example, a volatile storage device, and holds the packet count value of the past N (N is a positive integer) times calculated by the counting unit 23.

  FIG. 3 is an explanatory diagram of packet count value calculation processing by the count unit 23. As shown in FIG. 3, the count unit 23 calculates a packet count value at the second period Tb.

  Here, the counting unit 23 sets the count value to a value obtained by ΔT / Ta for the packet PS received in the past in the first period Ta from the calculation time Tk which is the calculation timing of the packet count value. For the packet PL received before the first period Ta from the calculation time Tk, the packet count value is calculated by setting the count value to 1. That is, the packet count value of the packet PS decreases as the difference ΔT decreases as the reception time approaches the calculation time Tk.

  Here, for the packet PS, since the reception time is used to calculate the packet count value, it is necessary to hold the reception time. On the other hand, for the packet PL, when calculating the packet count value, the reception time is not necessary, so it is not necessary to record the reception time.

  Therefore, when the calculation process of the packet count value is completed, the count unit 23 receives the packet received before the difference (= Ta−Tb) between the first period Ta and the second period Tb from the calculation time Tk. Deletes the reception time from the reception time recording unit 25.

  Thereby, at the time Tk + 1 which is the next calculation time of the packet count value, the reception time of the packet received in the first period Ta in the past is held in the reception time recording unit 25. As a result, the counting unit 23 At Tk + 1, the reception time of the packet received in the past and the first cycle Ta can be acquired. By doing so, the capacity of the reception time recording unit 25 can be saved.

  The buffer size changing unit 24 reads the past N packet count values of the packet count value calculated by the counting unit 23 from the observation history holding unit 29, and the nth smallest packet from the read N packet count values. The count value is calculated as a representative value of the packet count value. If the calculated representative value is larger than a predetermined reference value, the packet stored in the fluctuation absorbing buffer 22 is deleted. If the representative value is smaller than the reference value, the fluctuation is calculated. A packet is inserted into the absorption buffer 22.

  Here, when the representative value is smaller than the reference value, the buffer size changing unit 24 may insert a packet into the fluctuation absorbing buffer 22 so that the representative value is not less than the reference value and less than the reference value + 1.

  For example, when the representative value is 2.1 and the reference value is 4, two packets are inserted into the fluctuation absorbing buffer 22 so that the representative value is 4.1.

  In addition, when the representative value is larger than the reference value, the buffer size changing unit 24 may delete the packet from the fluctuation absorbing buffer 22 so that the representative value is not less than the reference value and less than the reference value + 1.

  For example, when the representative value is 4.2 and the reference value is 2, two packets are deleted from the fluctuation absorbing buffer 22 so that the representative value is 2.2.

  As n, it is preferable to adopt a value rounded to an integer value by N × α. Further, as the reference value, a value determined in advance based on the call delay time allowed by the call system when the transmitting apparatus 1 and the receiving apparatus 2 perform two-way communication is employed. That is, if the number of packets stored in the fluctuation absorbing buffer 22 is larger than the reference value, the number of packets waiting to be output in the fluctuation absorbing buffer 22 increases, so that a call delay occurs. Therefore, in this call system, when the representative value that is the nth packet count value is larger than the reference value (specifically, when it is larger than the reference value + 1), the packet is deleted from the fluctuation absorbing buffer 22. Thereby, a call delay can be prevented.

  On the other hand, when the representative value that is the nth packet count value is smaller than the reference value, the packet is inserted into the fluctuation absorbing buffer 22. Thereby, the probability that the number of stored packets is equal to or less than the reference value can be set to α (= n / N)%.

  The reception time recording unit 25 is configured by, for example, a volatile storage device, and records the reception time of the packet received by the packet transmission / reception unit 21. The reference value storage unit 26 is configured by, for example, a nonvolatile storage device, and stores a reference value.

  The concealment processing unit 27 performs packet loss concealment processing on invalid packets inserted into the fluctuation absorbing buffer 22 and performs packet loss concealment processing when packets are depleted in the fluctuation absorbing buffer 22. Here, as the packet loss concealment process, for example, the basic period of the audio signal is detected from the audio signal in the past from the invalid packet, and in the audio signal of the valid packet immediately before the invalid packet, A technique may be employed in which a speech waveform obtained by extracting a speech waveform and repeating this speech waveform for a period of 20 ms is generated as a speech signal of an invalid packet.

  In addition, as a method for detecting the basic period, an audio signal having a time width from the current time to the past is set as a reference signal, the reference signal is slid from the current time to the past, and the reference signal and A method of detecting the basic period by obtaining a correlation with the audio signal, and a method of increasing the time width of the reference signal as the slide amount of the reference signal increases may be adopted. Further, as a technique for detecting the fundamental period, a technique described in Japanese Patent No. 3321868 may be employed instead of the above technique. The invalid packet does not include a voice signal. The valid packet includes a voice signal.

  When the number of packets stored in the fluctuation absorbing buffer 22 exceeds the reference value, the reproducing unit 28 reads out the packets from the fluctuation absorbing buffer 22 in the first cycle Ta in time series, and extracts the audio signal from the read packets. The audio signal is subjected to inverse encoding processing and digital / analog conversion processing, and then output to the speaker 30. Here, when the packet extracted from the fluctuation absorbing buffer 22 is an invalid packet that does not include the audio signal, the reproducing unit 28 causes the concealment processing unit 27 to execute the packet loss concealment process, and the audio signal after the execution process is transmitted to the speaker 30. Output to.

  The observation history holding unit 29 is configured by, for example, a non-volatile storage device, and holds the past N packet count values calculated by the counting unit 23.

  The speaker 30 outputs sound according to the sound signal output from the playback unit 28. The display unit 31 is configured by, for example, a liquid crystal display panel, and displays a video according to a video signal output from the playback unit 28.

  The microphone 33 converts the voice of the caller into an audio signal that is an electrical signal. The packet generation unit 34 divides the audio signal output from the microphone 33 every 20 ms, packetizes the divided audio signal according to RTP, and outputs the packetized signal to the packet transmitting / receiving unit 21.

  FIG. 4 is a diagram for explaining the role of the fluctuation absorbing buffer 22. As illustrated in FIG. 4, the transmission device 1 performs encoding processing and analog-digital conversion processing on the audio signal VS, and then packetizes and transmits the audio signal VS at a cycle of 20 ms. In FIG. 4, it can be seen that 8 packets numbered 1 to 8 are output from the transmission apparatus 1 at intervals of 20 ms.

  A packet transmitted from the transmission device 1 is received by the reception device 2 via the IP network. However, depending on a difference in characteristics of each transmission path constituting the IP network and a traffic situation of the IP network that changes every moment. The transmission delay, which is the time taken for a packet transmitted from the transmission device 1 with a period of 20 ms to reach the reception device 2, is greatly different for each packet, and transmission delay fluctuations occur. Therefore, the reception intervals of packets by the reception device 2 are unequal intervals.

  In order to absorb this delay fluctuation, a fluctuation absorbing buffer 22 is provided. In FIG. 4, the buffer size of the fluctuation absorbing buffer 22 is set to three packets. In addition, the receiving device 2 starts reproduction by performing decoding processing and digital-analog conversion processing on the first packet at time T1 when the delay time Td has elapsed since receiving the first packet.

  In the case of FIG. 4, the fluctuation absorbing buffer 22 stores the second packet at time T2, which is the reproduction time of the second packet 20 ms after time T1. Therefore, the receiving device 2 can reproduce the second packet at time T2.

  On the other hand, since the third packet has an extremely large transmission delay, it does not reach the receiving device 2 at time T3, and the fluctuation absorption buffer 22 has run out of packets. For this reason, the receiving device 2 cannot reproduce the third packet at time T3, and sound loss occurs.

  Further, the third to seventh packets reach the receiving device 2 continuously in a short time after the congestion is eliminated. When the seventh packet reaches the receiving device 2, the fluctuation absorbing buffer 22 has 5, 6 The seventh packet exists, but the seventh buffer is not discarded but is stored in the fluctuation absorbing buffer 22 because the fluctuation absorbing buffer 22 is empty. Therefore, the seventh packet is reproduced at time T7.

  Therefore, at time T7, since the seventh packet is not accumulated in the fluctuation absorbing buffer 22, sound omission occurs.

  As described above, since the characteristics of the transmission delay fluctuation dynamically change, if the buffer size of the fluctuation absorbing buffer 22 is set to a fixed size, it must be made sufficiently longer than the assumed transmission delay fluctuation. Further, if the buffer size of the fluctuation absorbing buffer 22 is made sufficiently long and the delay time Td is made sufficiently long, the occurrence of sound omission can be prevented, but if the delay time Td is long, the fluctuation absorbing buffer 22 outputs Waiting packets increase and call delay occurs.

  FIG. 5 shows an example of a transmission delay characteristic graph showing the relationship between the transmission delay and the frequency of occurrence of the transmission delay. In FIG. 5, the vertical axis represents the occurrence frequency, and the horizontal axis represents the transmission delay. FIG. 6 is a diagram for explaining the optimum buffer size of the fluctuation absorbing buffer 22. In FIG. 5, dmin indicates the minimum transmission delay, and dmax indicates the maximum transmission delay. In FIG. 6, the transmission delay of the (k−1) th packet is dmin, the transmission delay of the kth packet is d, and the transmission delay of the (k + 1) th packet is dmax.

  In this case, the optimum output waiting time by the receiving apparatus 2 is as follows. i) A packet received at dmax is immediately output. ii) A packet received at dmin is output after waiting for dmax-dmin. iii) The packet received at d is output after waiting for dmax-d.

  Therefore, in order to avoid packet depletion in the fluctuation absorbing buffer 22, the buffer size buf of the fluctuation absorbing buffer 22 may be set to buf ≧ dmax−dmin. However, if dmax of the transmission delay characteristic becomes extremely large, that is, When the tail at the right end of the graph of FIG. 5 becomes extremely long, the value of the buffer size buf becomes large.

  Further, as shown in the graph of FIG. 5, since the frequency of occurrence decreases as the transmission delay increases, in order to observe the true dmax, it is necessary to observe the transmission delay of a huge number of packets.

  Therefore, in the graph of FIG. 5, not the true dmax, but a value that rounds down the top few percent of the distribution of transmission characteristics is regarded as dmax. In this case, when a transmission delay of dmax or more occurs, packet depletion occurs.

  Therefore, in order to prevent packet depletion, it is preferable to set the buffer size buf to be larger than dmax. However, when dmax is large, as a result, the number of waiting packets to be output in the fluctuation absorbing buffer 22 increases. An output delay occurs. Since such an output delay appears as a call delay in an IP telephone, it is preferable to suppress it as much as possible. Therefore, in the call system, the above-described processing is executed to prevent packet depletion and to prevent call delay.

  FIG. 7 is a flowchart showing processing of the receiving device 2 shown in FIG. 2 is assumed to be transmitting a packet to the receiving device 2 at the first period Ta. First, in step S <b> 1, the counting unit 23 determines whether or not the second period Tb has elapsed since the calculation timing of the packet count value last time and the calculation timing of the packet count value has come. When the count unit 23 determines that the packet count value calculation timing has come (YES in step S1), the count unit 23 counts the number of accumulated packets that are currently accumulated in the fluctuation absorbing buffer 22 (step S2). . On the other hand, when the count unit 23 determines that the packet count value calculation timing has not come (NO in step S1), the count unit 23 returns the process to step S1.

  Next, the count unit 23 executes a packet count value calculation process to calculate a packet count value (step S3).

  FIG. 8 is a flowchart showing details of the packet count value calculation process. First, the count unit 23 specifies the current time as the calculation time of the packet count value (step S21). Here, since the receiving device 2 has a clock, the calculated time is specified using this clock.

  Next, the counting unit 23 determines the reception time of each packet received in the past and the first period Ta from the calculation time Tk as shown in FIG. 3 among the packets stored in the fluctuation absorbing buffer 22. Specify (step S22). In this case, the count unit 23 specifies the reception time of each packet by specifying the sequence number associated with the reception time recorded in the reception time recording unit 25.

  Next, the count unit 23 calculates a difference ΔT between the calculation time Tk and the reception time for each packet received in the past and within the first period Ta from the calculation time Tk (step S23). Next, the count unit 23 calculates ΔT / Ta for each packet received in the past in the first period Ta, and sets this ΔT / Ta as the count value of each packet (step S24).

  Next, the count unit 23 sets a count value as 1 for packets received from the calculation time Tk in the past and before the first period Ta among the packets stored in the fluctuation absorbing buffer 22 ( Step S25).

  Next, the count unit 23 calculates the packet count value by counting the number of accumulated packets in the fluctuation absorbing buffer 22 using the count value set in steps S24 and S25 (step S26). For example, the number of packets received from the calculation time Tk in the past and before the first cycle Ta is one, and the number of packets received in the past and the first cycle Ta from the calculation time Tk is two. If the reception times of these two packets are Ti and Tj, the packet count value is 1+ (Tk−Ti) / Ta + (Tk−Tj) / Ta.

  Next, the count unit 23 deletes the reception time from the reception time recording unit 25 for packets received in the past and before Ta-Tb from the calculation time Tk (step S27).

  Returning to FIG. 7, in step S <b> 4, the count unit 23 causes the observation history holding unit 29 to hold the packet count value at the calculation time Tk. In this case, the count unit 23 deletes the oldest packet count value from the observation history holding unit 29 so that the number of packet count values held in the observation history holding unit 29 is N.

  Next, the buffer size changing unit 24 specifies the nth smallest packet count value among the N packet count values stored in the observation history holding unit 29 as a representative value (step S5).

  FIG. 9 is a schematic diagram showing the relationship between the packet count value and the calculation time of the packet count value. The vertical axis shows the packet count value, and the horizontal axis shows the calculation time of the packet count value. In FIG. 9, N = 9 and n = 3. Accordingly, the buffer size changing unit 24 specifies the packet count value at time Tk-7 as the representative value because the packet count value at the second time Tk-7 from the left end shown in FIG. 9 is the third smallest.

  Next, the buffer size changing unit 24 determines whether or not the representative value is greater than the reference value. If representative value ≧ reference value + 1 (YES in step S6), the representative value is greater than or equal to the reference value and less than the reference value + 1. Are deleted from the fluctuation absorbing buffer 22 (step S7).

  Next, the buffer size changing unit 24 subtracts the number of packets deleted in step S7 from each of the N packet count values held in the observation history holding unit 29, and updates the N packet count values. The observation history is updated (step S8). For example, assuming that the number of deleted packets is 1, 1 is subtracted from all N packet count values. Thereby, the fact that the packet is deleted from the fluctuation absorbing buffer 22 is reflected in the observation history.

  On the other hand, in step S6, when the representative value is less than the reference value +1 (NO in step S6) and the representative value is greater than or equal to the reference value (NO in step S9), the buffer size changing unit 24 uses the fluctuation absorbing buffer 22. The packet is not deleted or inserted (step S10).

  On the other hand, if the representative value is smaller than the reference value (YES in step S9), the buffer size changing unit 24 inserts the number of packets whose representative value is greater than or equal to the reference value and less than the reference value + 1 into the fluctuation absorbing buffer 22 (step S11). ).

  Next, the buffer size changing unit 24 adds the number of packets inserted in step S11 to each of the N packet count values held in the observation history holding unit 29, and updates the N packet count values. Then, the observation history is updated (step S12). For example, if the number of inserted packets is 1, 1 is added to all N packet count values. Thereby, the fact that the packet is inserted into the fluctuation absorbing buffer 22 is reflected in the observation history.

  When the process of step S8, S10, or S12 is completed, the process returns to step S1, and when the next packet count value calculation time arrives, the processes after step S2 are executed.

  FIG. 10A is a schematic diagram showing processing at the time of packet insertion by the buffer size changing unit 24, and FIG. 10B is a schematic diagram showing processing at the time of packet deletion by the buffer size changing unit 24. In the example of FIG. 10A, the buffer size changing unit 24 inserts an invalid packet between the fourth packet and the fifth packet which are valid packets. In the example of FIG. 10B, the buffer size changing unit 24 overlaps the fourth packet and the fifth packet, which are valid packets, so that the two packet lengths become one packet length. One packet is deleted.

  Next, a process for inserting or deleting a packet in the fluctuation absorbing buffer 22 will be specifically described. When the buffer size changing unit 24 shown in FIG. 2 deletes one packet from the fluctuation absorbing buffer 22, if there are two or more effective packets continuously including an audio signal or a video signal, Of these, two consecutive valid packets located in the middle are overlap-added and deleted.

  FIG. 11 is an explanatory diagram of a process in which the buffer size changing unit 24 deletes one packet by overlap addition. (A) shows the fluctuation absorbing buffer 22 before deletion, and (b) shows fluctuation absorption after deletion. A buffer 22 is shown.

  The read pointer RP shown in FIG. 11 indicates the start address of the fluctuation absorbing buffer 22 having a ring buffer structure, and the write pointer WP indicates the end address of the fluctuation absorbing buffer 22. Further, in FIG. 11, each 升 indicates one packet, and the numbers in 升 indicate the time-series order of the packets. In addition, a white wrinkle indicates an invalid packet, and a gray wrinkle indicates a valid packet.

  In the case of FIG. 11A, the fifth and sixth valid packets located in the fourth to seventh valid packet sections are not the first to second valid packet sections. As shown in FIG. 5, one packet is collected by overlapping addition, and one packet is deleted.

  Here, if overlap addition is performed in the interval of the first to second valid packets shown in FIG. 11A, an invalid packet exists after one packet generated by overlap addition. There is a possibility that voice degradation will increase when packet loss concealment processing is performed. On the other hand, if the fifth valid packet and the sixth valid packet are overlap-added, the packets before and after one packet generated by overlap addition are valid packets, so that the voice deterioration due to the packet loss concealment process is reduced. can do.

  In other words, if two or more valid packets are consecutive, one packet can be deleted by overlap addition, but packet loss concealment processing is performed when overlap addition is performed in a section where there are many consecutive valid packets. It is possible to reduce voice deterioration when

  Therefore, in the fluctuation absorbing buffer 22, when there are a plurality of sections in which valid packets are continuous, overlap addition is performed using a valid packet in the middle of a section in which the number of consecutive valid packets is large.

  Here, as the overlap addition, overlap addition using triangular window functions RF1 and RF2 can be adopted as shown in FIG. Specifically, the buffer size changing unit 24 performs window function processing using the triangular window function RF1 on the audio signal of the fifth packet, and applies the triangular window to the audio signal of the sixth packet. The window function processing using the function RF2 is performed, the two audio signals after the window function processing are added to generate one audio signal, and this is packetized into one to perform overlap addition.

  Here, as the triangular window function RF1, a linear function having a time width of 20 ms, a maximum value of 1, a minimum value of 0, and a value that decreases as time passes can be employed. As the triangular window function RF2, a linear function having a time width of 20 ms, a maximum value of 1 and a minimum value of 0 and increasing in value as time passes can be employed.

  Further, when deleting a packet from the fluctuation absorbing buffer 22, the buffer size changing unit 24 deletes the invalid packet if there is an invalid packet inserted in the past.

  FIG. 12 is an explanatory diagram of a process in which the buffer size changing unit 24 deletes one invalid packet. (A) shows the fluctuation absorbing buffer 22 before deletion, and (b) shows the fluctuation absorbing buffer 22 after deletion. Show.

  In FIG. 12A, the third and fourth packets are invalid packets. Therefore, the buffer size changing unit 24 deletes one packet by deleting either the third or the fourth packet. Here, when there are a plurality of invalid packets in the fluctuation absorbing buffer 22, for example, one invalid packet may be selected at random and the selected invalid packet may be deleted. Alternatively, when two or more invalid packets are continuously present, the buffer size changing unit 24 preferentially extracts invalid packets in a continuous area, and randomly selects one invalid packet from the extracted invalid packets. A packet may be selected and deleted.

  Further, when inserting a packet into the fluctuation absorbing buffer 22, the buffer size changing unit 24 inserts an invalid packet between these two valid packets if there are two consecutive valid packets.

  FIG. 13 is an explanatory diagram of processing in which the buffer size changing unit 24 inserts one packet. FIG. 13A shows the fluctuation absorbing buffer 22 before insertion, and FIG. 13B shows the fluctuation absorbing buffer 22 after insertion. ing.

  As shown in FIGS. 13A and 13B, one invalid packet is inserted between the fifth valid packet and the sixth valid packet. This is because inserting one invalid packet between the fifth valid packet and the sixth valid packet increases the number of consecutive valid packets.

  For example, even if an invalid packet is inserted between the first valid packet and the second valid packet, the packet concealing process can be performed because the valid packet exists before and after the inserted invalid packet.

  However, before and after the second valid packet become invalid packets, the number of consecutive valid packets becomes small. On the other hand, if an invalid packet is inserted between the fifth valid packet and the sixth valid packet, all the valid packets are continuous. Here, when packet loss concealment processing is performed, voice deterioration can be reduced as the number of consecutive valid packets increases. Therefore, when there are a plurality of sections where valid packets continue in the fluctuation absorbing buffer 22, the buffer size changing unit 24 inserts invalid packets in the middle of a section where the number of consecutive valid packets is large.

  The buffer size changing unit 24 has a predetermined upper limit value for the number of packets that can be inserted or deleted at a time.

  FIG. 14 is a diagram for explaining processing when five packets are inserted into the fluctuation absorbing buffer 22 at a time. FIG. 14A shows the fluctuation absorbing buffer 22 before insertion, and FIG. 14B shows the state after insertion. A fluctuation absorbing buffer 22 is shown. 14A and 14B, five invalid packets are inserted between the first valid packet and the second valid packet. In this case, since there are continuous invalid packets, there is a risk that voice deterioration will increase. Therefore, in this embodiment, an upper limit is provided for the number of invalid packets inserted. Here, “at a time” refers to one process executed when the second period Tb described above is reached.

  For example, in FIG. 14A, if the upper limit value is set to 3, even if it is necessary to insert five invalid packets, only three invalid packets are inserted.

  As a result, the number of consecutive invalid packets is prevented from exceeding a certain number, and voice degradation due to packet loss concealment processing can be reduced.

  Further, when the invalid packet is deleted, the buffer size changing unit 24 receives a valid packet corresponding to the deleted invalid packet later, and if there is an invalid packet in addition to the deleted invalid packet, Replace the packet with the received valid packet.

  FIG. 15 is a diagram for explaining processing when a valid packet corresponding to a deleted invalid packet is received after deleting the invalid packet. FIG. 15A shows the fluctuation absorbing buffer 22 before deletion, and FIG. Shows the fluctuation absorbing buffer 22 after deletion, and (c) shows the fluctuation absorbing buffer 22 after replacement.

  As shown in FIGS. 15A and 15B, the third invalid packet is deleted. Thereafter, as shown in FIG. 15C, the third valid packet corresponding to the third invalid packet is received.

  In this case, since the fourth packet next to the third invalid packet is an invalid packet, the buffer size changing unit 24 replaces the fourth invalid packet with the received third valid packet. As a result, the third valid packet can be restored, and voice deterioration can be reduced.

  Here, when a packet is accumulated in the fluctuation absorbing buffer 22, the buffer size changing unit 24 determines whether or not an invalid packet corresponding to the accumulated packet is accumulated in the fluctuation absorbing buffer 22. Then, when the corresponding invalid packet is accumulated in the fluctuation absorbing buffer 22, the buffer size changing unit 24 determines whether the invalid packet is stored next to the invalid packet, and the invalid packet is stored. In such a case, the next invalid packet may be replaced with the received invalid packet by deleting the next invalid packet and inserting the received valid packet at the deleted location.

  On the other hand, when the invalid packet corresponding to the packet accumulated in the fluctuation absorbing buffer 22 is not accumulated in the fluctuation absorbing buffer 22, or the invalid packet is stored next to the corresponding invalid packet. If not, the above replacement is not performed. The buffer size changing unit 24 may determine that a valid packet corresponding to an invalid packet has been received when a packet having the same sequence number as the invalid packet sequence number is accumulated in the fluctuation absorbing buffer 22.

  In addition, when inserting a packet between two consecutive valid packets, the buffer size changing unit 24 causes the concealment processing unit 27 to execute a packet loss concealment process using the previous valid packet, thereby concealing. A processed packet may be generated and inserted into the fluctuation absorbing buffer 22.

  FIG. 16 is a diagram for explaining processing when the buffer size changing unit 24 inserts a concealed packet in place of the invalid packet into the fluctuation absorbing buffer 22, and FIG. 16A shows the fluctuation absorbing buffer 22 before insertion. (B) shows the fluctuation absorbing buffer 22 after insertion.

  As shown in FIGS. 16A and 16B, a concealed packet is inserted between the third valid packet and the fourth valid packet.

  Thereby, when the reproducing unit 28 reads a packet from the fluctuation absorbing buffer 22, it is not necessary to execute the packet loss concealment process, and the processing delay of the packet loss concealment process at the time of reproduction can be reduced.

  In addition, when inserting an invalid packet, the buffer size changing unit 24 preferably inserts an invalid packet between two consecutive packets including vowel sounds. Thereby, the voice generated by executing the packet loss concealment process on the inserted invalid packet is continuously connected to the voice included in the preceding and succeeding packets, and voice deterioration can be reduced.

  FIG. 17 is a flowchart showing the deletion process by the buffer size changing unit 24.

  First, in step S51, the buffer size changing unit 24 determines whether or not the number of packet deletion requests is equal to or less than a predetermined maximum packet deletion number (upper limit), and the number of deletion requests is equal to or less than the upper limit value. If so (YES in step S51), the deletion count value DN is set to the number of deletion requests (step S52). On the other hand, when the number of deletion requests is larger than the upper limit value (NO in step S51), the deletion count value DN is set to the upper limit value (step S53).

  Next, when the maximum number of consecutive valid packets is 2 or more in the fluctuation absorbing buffer 22 (2 or more in step S54), the buffer size changing unit 24 sets the maximum number of continuous packets to 2 or more times the deletion count value DN. It is determined whether or not there is (step S55). Here, it is determined whether or not the maximum continuous number is twice the deletion count value DN. When one packet is deleted, since two packets are overlap-added, the valid packet is deleted. This is because twice the value DN is required.

  If the buffer size changing unit 24 determines that the maximum continuous number is twice or more the deletion count value DN (YES in step S55), the buffer size changing unit 24 deletes packets corresponding to the deletion count value DN by overlap addition, and deletes the deletion count. The number of deleted packets is subtracted from the value DN, and the deletion count value DN is updated (step S58).

  On the other hand, when the maximum continuous number is less than twice the deletion count value DN in step S55 (NO in step S55), the buffer size changing unit 24 deletes the deleteable packet by overlap addition, and deletes the deletion count value. The number of deleted packets is subtracted from the DN, the deletion count value DN is updated (step S56), and the process returns to step S54.

  For example, when the maximum consecutive number is 7 and the deletion count value DN (= 4) × 2 is 8, 6 valid packets are overlapped and added out of 7 consecutive valid packets. Delete the packet. Then, the deletion count value DN is updated to DN = 1 (= 4-3).

  On the other hand, in step S54, when the maximum number of consecutive valid packets is 1 or less (1 or less in step S54), invalid packets are deleted, and the deleted count value DN is subtracted from the deleted count value DN. Is updated (step S57).

  For example, if the deletion count value DN is 4 and the number of invalid packets is 3, three invalid packets are deleted and updated to DN = 1 (= 4-3).

  In step S59, the buffer size changing unit 24 determines whether or not the deletion count value DN is 0. If the deletion count value DN is 0 (YES in step S59), the process ends.

  On the other hand, in step S59, if the deletion count value DN is not 0 (NO in step S59), the buffer size changing unit 24 deletes the effective packet and processes if there is a valid packet (YES in step S60). Is finished (step S61). In this case, since the valid packet to be deleted is not continuous with other valid packets, it is simply deleted regardless of overlap addition. On the other hand, if there is no valid packet (NO in step S60), the process is terminated as it is.

  FIG. 18 is a flowchart showing insertion processing by the buffer size changing unit 24.

  First, in step S71, the buffer size changing unit 24 determines whether or not the number of packet insertion requests is equal to or less than a predetermined maximum packet insertion number (upper limit), and the number of deletion requests is equal to or less than the maximum number of insertions. If there is (YES in step S71), the number of insertions is set to the number of insertion requests (step S72). On the other hand, when the number of insertion requests is larger than the maximum number of insertions (NO in step S71), the number of insertions is set to the maximum number of insertions (step S73).

  Next, when the maximum number of consecutive valid packets in the fluctuation absorbing buffer 22 is 0 (0 in step S74), the buffer size changing unit 24 inserts invalid packets corresponding to the number of insertions from the beginning of the fluctuation absorbing buffer 22. (Step S75), and the process ends.

  In addition, when the maximum number of consecutive valid packets is 2 or more in the fluctuation absorbing buffer 22 (2 or more in step S74), the buffer size changing unit 24 invalidates the number of insertions in the middle of the continuous valid packet section. A packet is inserted (step S76), and the process ends.

  When the maximum number of consecutive valid packets is 1 in the fluctuation absorbing buffer 22 (1 in step S74), the buffer size changing unit 24 inserts invalid packets for the number of insertions immediately after the valid packets (step S74). S77), the process is terminated.

  As described above, according to the call system according to the present embodiment, when one packet is deleted from the fluctuation absorbing buffer, two packets positioned in the middle of a section where two or more valid packets are continuous are overlap-added. Since one packet is deleted by generating one packet, it is possible to reduce quality degradation of audio and video.

  Further, when a packet is inserted into the fluctuation absorbing buffer 22, if there are two consecutive valid packets, an invalid packet is inserted between these two valid packets. When packet loss concealment processing is executed for this invalid packet, the invalid packet can be concealed from the previous and subsequent valid packets, and the continuity of the audio and video is maintained, and the audio and video are reproduced smoothly. I can do it.

DESCRIPTION OF SYMBOLS 1 Transmission apparatus 2 Reception apparatus 21 Packet transmission / reception part 22 Fluctuation absorption buffer 23 Count part 24 Buffer size change part 25 Reception time recording part 26 Reference value storage part 27 Concealment processing part 28 Reproduction part 29 Observation history holding part 30 Speaker 31 Display part 33 Microphone 34 packet generator

Claims (8)

A packet in which a receiving device that receives audio or video packets to be continuously reproduced from a transmitting device connected via an IP network and stores it in a fluctuation absorbing buffer inserts or deletes a packet accumulated in the fluctuation absorbing buffer. An insertion and deletion method,
A counting step of counting the number of accumulated packets of the packets stored in the fluctuation absorbing buffer;
A buffer size changing step for inserting or deleting packets in the fluctuation absorbing buffer based on the counting result of the counting step;
In the buffer size changing step, when one packet is deleted from the fluctuation absorbing buffer, if there are two or more valid packets including audio or video in succession, the buffer size changing step is located at the center of the successive valid packets. If two valid packets, a valid packet and a valid packet positioned consecutively with the valid packet, are overlapped and deleted , and there are multiple consecutive valid packets, the number of consecutive valid packets A packet insertion / deletion method characterized in that overlap addition is performed using two valid packets, ie, a valid packet located in the center and valid packets consecutively located in the valid packet in a section with a large amount of data .   The buffer size changing step inserts an invalid packet that does not include audio or video when inserting a packet into the fluctuation absorbing buffer. When deleting a packet from the fluctuation absorbing buffer, the invalid packet inserted in the past is inserted. 2. The packet insertion / deletion method according to claim 1, wherein if there is, the invalid packet is deleted.   The buffer size changing step inserts an invalid packet between the two valid packets if there are two consecutive valid packets when the packet is inserted into the fluctuation absorbing buffer. Or the packet insertion / deletion method according to 2;   4. The packet insertion / deletion method according to claim 1, wherein an upper limit value of the number of packets that can be inserted or deleted at a time is predetermined in the buffer size changing step.   4. The packet insertion / deletion method according to claim 2, further comprising a concealment processing step for executing a packet loss concealment process on the invalid packet inserted by the buffer size changing step. The buffer size changing step, when you delete an invalid packet, if received later the effective packet corresponding to invalid packet deleting, if there is invalid packet to the next invalid packets deleted, the next invalid packets 3. The packet insertion / deletion method according to claim 2, wherein the received valid packet is exchanged. The packet is a voice packet;
4. The packet insertion / deletion method according to claim 3, wherein the buffer size changing step inserts an invalid packet between two consecutive packets storing vowel sounds. A call system comprising a transmission device and a reception device connected via an IP network,
The receiving device is:
A fluctuation absorbing buffer for storing audio or video packets to be continuously reproduced from the transmission device;
Counting means for counting the number of accumulated packets of the packets stored in the fluctuation absorbing buffer;
Buffer size changing means for inserting or deleting packets in the fluctuation absorbing buffer based on the count result by the counting means,
When the buffer size changing unit deletes one packet from the fluctuation absorbing buffer, if there are two or more valid packets including audio or video in succession, the buffer size changing unit is located at the center of the successive valid packets. If two valid packets, a valid packet and a valid packet positioned consecutively with the valid packet, are overlapped and deleted , and there are multiple consecutive valid packets, the number of consecutive valid packets A call system characterized in that overlap addition is performed using two effective packets, ie, an effective packet located in the center and an effective packet positioned consecutively in the effective packet in a section with a large amount of data.

Images