JP4190163B2 - Multiplexed data separation device for multimedia communication, multimedia communication device, multimedia communication system, and recording medium storing program for realizing multiplexed data separation device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a multimedia communication device, a multimedia communication system, and a multiplexed data separation device used for transmitting audio, video, data, and the like between distant multipoints.
[0002]
[Prior art]
Conventionally, in order to perform so-called multimedia communication for transmitting voice, video, data, etc., multiplexing for transferring any combination of digital voice / audio, digital video / video, and data information over a single communication link Protocol is used.
As a standard of this multiplexing protocol, ITU-T H.264 is used. 225.0 and ITU-TH. The former is a multimedia communication system standard ITU-T H.264. In 323, the latter is also ITU-T H.264. At 324, it is specified to be used.
[0003]
Here, when one terminal conforms to one system standard, it responds only to a multiplexing protocol used in the conforming system standard and media multiplexed by another multiplexing protocol. Even if information is transmitted, it may be discarded as nonstandard information.
[0004]
For example, ITU-T H.I. ITU-T H.323 for terminals conforming to H.323. When information multiplexed according to the 225.0 standard is transmitted, the terminal must appropriately separate this information and perform media playback processing. On the other hand, for the same terminal, ITU-TH. Even if information multiplexed according to the H.223 standard is transmitted, it is non-standard multiplexed information for the terminal, and is therefore ignored as incompatible information.
[0005]
[Problems to be solved by the invention]
However, when one terminal conforms to a plurality of system standards, that is, when there are a plurality of multiplexing protocols for information to be received, the terminal multiplexes the information to be received by what protocol. You must know in advance what is being done.
[0006]
In order to inform the receiving side of the type of multiplexing protocol, for example, ITU-T H.264. In the H.323 system, at the start of communication, ITU-T Q.3 which is a standard for call control. A call connection using a message defined in 931 is performed, and multiplexing protocol information is added to the information element of the message. However, on the other hand, ITU-TH In the 324 system, ITU-T H.264. It operates on the premise that it is multiplexed by H.223, and the multiplexing protocol type notification means is not particularly defined.
[0007]
For this reason, for example, the communication system standard ITU-T H.264. 323 and ITU-TH. If there is a single terminal that complies with both of H.324, the user must select which standard to use to connect at the start of multimedia communication. If no such selection is made, the terminal receives the received information as ITU-TH. Is it multiplexed by 225.0, ITU-TH. It is impossible to determine whether the data is multiplexed by H.223.
[0008]
However, in such a terminal conforming to a plurality of standards, the user can use ITU-TH. When the H.323 standard is selected, the ITU-T H.264 Despite having the ability to communicate with 324-compliant terminals, ITU-TH. Even if information is received from a 324-compliant terminal, the ITU-TH. Since it operates as a H.323 compliant terminal, the received information is discarded and communication is impossible.
Similarly, the user can use ITU-T H.264. When the 324 standard is selected, the ITU-T H.264 standard is selected. Cannot communicate with a H.323 compliant terminal.
[0009]
The present invention has been proposed in view of the above-described circumstances, and automatically identifies a multiplexing protocol based on multiplexed data received from a communication network and separates the multiplexed data into a plurality of information source encoded data. An object of the present invention is to provide a multiplexed data separation device, multimedia communication device, and multimedia communication system for multimedia communication.
[0010]
[Means for Solving the Problems]
A multiplexed data separation device for multimedia communication according to the present invention is for multimedia communication for separating data generated by multiplexing a plurality of information source encoded data into a plurality of information source encoded data. In the multiplexed data separator,
Data separation means for selectively using a plurality of demultiplexing protocols to separate multiplexed data into a plurality of information source encoded data, and data from which the input multiplexed data can be separated by the data separating means A demultiplexing protocol identifying means for identifying a demultiplexing protocol of the multiplexed data using a part or all of the multiplexed data.
When the demultiplexing protocol identifying means identifies the demultiplexing protocol of the input multiplexed data, the data demultiplexing means converts the input multiplexed data into a plurality of information source encoded data. It is characterized by separating.
[0011]
A multimedia communication apparatus according to the present invention is configured using the multiplexed data demultiplexing apparatus.
[0012]
In addition, the multimedia communication system according to the present invention is configured using the multimedia communication apparatus.
[0013]
Further, it is possible to record a program for realizing the above-described means on a recording medium using a computer.
In this case, the computer is a concept including a substrate on which a chip having functions such as a CPU, a ROM, and a RAM, a so-called one-chip microcomputer, and the like, in addition to a computer configured by a plurality of devices such as a personal computer. .
[0014]
Since the multiplexed data demultiplexing device, multimedia communication device, and multimedia communication system according to the present invention have the above-described configuration, they operate as follows.
That is, the multiplexed data demultiplexing apparatus, multimedia communication apparatus, and multimedia communication system according to the present invention identify and use the multiplexing protocol in the demultiplexing unit in the terminal instead of being selected by the user. Switch the multiplexing protocol to be used. For example, ITU-TH. 323 and ITU-TH. When there is a terminal conforming to the two standards of H.324, the terminal reads information received from the communication network, and the format is ITU-T H.264. 225.0, ITU-TH. The multiplexing protocol to be used is switched depending on which of H.223.
Here, the received information is ITU-T H.264. If the format conforms to 225.0, the ITU-T H.264 standard. The information is separated according to the 225.0 standard. On the other hand, the received information is ITU-T H.264. If the format conforms to H.223, ITU-T H.264 is used. Separate information according to the H.223 standard.
In this way, multimedia communication can be performed using a terminal that complies with a plurality of communication system standards and corresponding to the connection that the terminal can respond to.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a multiplexed data demultiplexing device, a multimedia communication device, and a multimedia communication system according to the present invention will be described below based on specific examples shown in the drawings.
[0016]
FIG. 1 is a block diagram showing a schematic configuration of a multimedia communication apparatus according to an embodiment of the present invention.
As shown in FIG. 1, a multimedia communication apparatus 1 according to an embodiment of the present invention negotiates connection with another multimedia communication apparatus via a system control unit 2 and a communication network 8 for controlling the entire apparatus. A terminal-to-terminal signal control unit 3 for controlling a connection control signal to be transmitted / received at the time of transmission, an audio codec 4 for encoding / decoding of audio data to be transmitted / received, The video codec 5, the connection control signal, the encoded audio data, the multiplexing unit 6 for multiplexing the encoded video data, the connection control signal, the encoded audio data, and the encoded video data sent after multiplexing. It comprises a separation unit 14 for separation and a communication network interface unit 7.
In addition, a user interface 9 is connected to the system control unit 2, a speaker 10 and a microphone 11 are connected to the audio codec 4, and a monitor 12 and a camera 13 are connected to the video codec 5. Yes.
[0017]
FIG. 2 is a block diagram showing a schematic configuration of the separation unit 14 shown in FIG.
As shown in FIG. 225.0 separation unit 15, multiplexing protocol identification unit 16, H.264. The 223 separation unit 17 is configured.
[0018]
The multiplexing protocol identification unit 16 identifies the protocol of multiplexed data transmitted from another multimedia communication device via the communication network interface unit 7. If the data is multiplexed by the 225.0 protocol, the received data is converted to H.264. 225.0 is transmitted to the separation unit 15, and If the data is multiplexed by the H.223 protocol, the received data is converted to H.264. 223 transmission to the separation unit 17.
[0019]
H. The 225.0 separation unit 15 applies the H.264 to the data received from the multiplexing protocol identification unit 16. In accordance with the protocol of the 225.0 protocol, it is separated into a connection control signal, encoded audio data and encoded video data, and transmitted to the inter-terminal signal control unit 3, audio codec 4 and video codec 5, respectively.
H. The H.223 separation unit 17 applies the H.264 to the data received from the multiplexing protocol identification unit 16. In accordance with the protocol of the H.223 protocol, the connection control signal, the encoded audio data, and the encoded video data are separated and transmitted to the inter-terminal signal control unit 3, the audio codec 4, and the video codec 5, respectively.
[0020]
Next, a procedure for identifying the protocol of the received multiplexed data using the multiplexing protocol identifying unit 16 will be described with reference to FIG.
FIG. 3 is a flowchart showing a procedure for identifying a protocol of received multiplexed data using the multiplexing protocol identifying unit 16.
In the example shown in FIG. 3, the ITU-T recommendation H.264 is used as the multiplexing protocol. 223 or H.264. 225.0 is used.
[0021]
In order to identify the protocol of the received multiplexed data using the multiplexing protocol identification unit 16, the multiplexing protocol identification unit 16 sets the first octet of the multiplexed data received from the line as shown in FIG. Extract (S1).
The received data is ITU-T recommendation H.264. When multiplexed according to 225.0, the first two bits indicate the RTP version. RTP is a protocol (Realtime Transport Protocol) for transferring data in real time described in RFC (Internet Request for Comments) 1889 and 1890. ITU-T recommendation The version of RTP used in 225.0 is defined as “2”.
[0022]
Therefore, it is determined whether or not the first 2 bits are “10 (binary number: the same applies hereinafter)” (S2). If the first 2 bits are “10”, the received data is multiplexed. H. It is determined that the received data is H.225.0. The data is transmitted to the 225.0 separation unit 15 (S4).
[0023]
On the other hand, if the first 2 bits are not “10”, the received data is H.264. It is determined whether it is H.223 multiplexed data. That is, H.I. Data multiplexed by the H.223 protocol is transmitted by a packet called MUX-PDU. H. The 223MUX-PDU must have a flag composed of a unique bit pattern “01111110 (binary number: the same applies hereinafter)” before and after the 223MUX-PDU. Therefore, the multiplexing protocol identification unit 16 checks the first one octet of the received data and determines whether or not it is “01111110” (S3).
[0024]
Here, if the first octet is “01111110”, the multiplexing protocol of the received data is set to H.264. 223, and the received data is H.264. It is transmitted to the H.223 separation unit 17 (S5).
On the other hand, if the first octet is not “01111110”, the received data is discarded (S6).
[0025]
Next, an operation in the case where multimedia communication is performed by the multimedia communication apparatus 1 using the multiplexed data separation apparatus configured as described above will be described.
First, ITU-T H.264 as a multiplexing protocol. The operation when 225.0 is used will be described.
[0026]
The multimedia communication device 1 receives the ITU-TH .. communication from another remote communication terminal via the communication network 8. At least one of audio and video information multiplexed according to 225.0 is received. In addition, the multimedia communication device 1 sets the multiplexing protocol of received information to ITU-T H.264 based on the procedure shown in FIG. It is determined that the received information is 225.0, and the received information is separated into at least one of audio encoded data and video encoded data, and transmitted to at least one of the audio codec 4 and the video codec 5.
[0027]
The audio codec 4 and the video codec 5 decode the received encoded data, and output the received audio / video information using the speaker 10 or the monitor 12.
As a multiplexing protocol, ITU-T H.264. The operation when using H.223 is basically the same as that when using ITU-TH 225.0.
ITU-TH. In 223, audio / image / inter-terminal signal information is transmitted by MUX-PDU.
[0028]
The multimedia communication device 1 receives the ITU-T H.264 from a remote communication terminal via the communication network 8. The inter-terminal signal information multiplexed according to H.223 and at least one of audio or video information is received. In addition, the multimedia communication device 1 sets the multiplexing protocol of received information to ITU-T H.264 based on the procedure shown in FIG. 223, and the received information is separated into inter-terminal signal information and at least one of audio encoded data or video encoded data, and transmitted to the inter-terminal signal control unit 3 and at least one of the audio codec 4 or the video codec 5. To do.
[0029]
The audio codec 4 and the video codec 5 decode the received encoded data, and output the received audio / video information using the speaker 10 or the monitor 12.
The inter-terminal signal control unit 3 reads, for example, the audio / video encoding / decoding capability of the transmitting terminal from the received inter-terminal signal and notifies the system control unit 2 of the read / output signal. Upon receiving the notification, the system control unit 2, for example, transmits audio / video from the multimedia communication device 1 as well, so that the inter-terminal signal control unit 3 is notified so as to notify its own coding / decoding capability. A request is made or a request is made to the audio codec 4 and the video codec 5 to encode the audio / video in accordance with the notified capability.
[0030]
Up to this point, the multiplexed data separation apparatus and bidirectional communication terminal for bidirectionally communicating a plurality of media such as audio and video have been described. However, the multiplexed data demultiplexer according to the present invention can be applied to a client terminal of a streaming system that multiplexes voice, image, text data, etc., transmits the data from the server to the client in one direction, and sequentially reproduces the data at the client. .
[0031]
Hereinafter, such a multiplexed data separator and a client terminal using the multiplexed data separator will be described. In the example shown below, a case will be described in which data multiplexed by an ASF (Advanced Streaming Format), which is one of the multiplexing protocols for information distributed on the streaming system, is separated.
[0032]
Data multiplexed by ASF is composed of a plurality of objects. The first 16 bytes of each object is a unique ID for identifying an object called uuid. Since the head of the data multiplexed by ASF is always a header object, uid indicating the header object is attached to the head 16 bytes.
[0033]
FIG. 4 shows a schematic configuration of a multiplexed data separation device for separating data multiplexed by ASF.
As shown in FIG. 225.0 separation unit 15, multiplexing protocol identification unit 19, H.264. 223 separation unit 17 and ASF separation unit 20. The configuration of the multimedia communication apparatus using the separation unit 14 shown in FIG. 4 is the same as that shown in FIG.
[0034]
The multiplexing protocol identification unit 19 is basically the same as the multiplexing protocol identification unit 16 shown in FIG. The difference from the multiplexing protocol identification unit 16 shown in FIG. 223 and ITU-TH. In addition to 225.0, the data multiplexed by the ASF is identified, and if the data is multiplexed by the ASF, the received data is transmitted to the ASF separation unit 20.
[0035]
H. 225.0 separator 15, H.22. The 223 separator 17 has the same function as that shown in FIG.
The ASF separation unit 20 separates the data received from the multiplexing protocol identification unit 19 into encoded audio data and encoded video data in accordance with the ASF rules, and transmits them to the audio codec 4 and the video codec 5, respectively.
[0036]
Next, a procedure for identifying the protocol of the received multiplexed data using the multiplexing protocol identifying unit 19 will be described with reference to FIG.
FIG. 5 is a flowchart showing a procedure for identifying the protocol of the received multiplexed data using the multiplexing protocol identifying unit 19.
[0037]
In order to identify the protocol of the received multiplexed data using the multiplexing protocol identifying unit 19, as shown in FIG. 5, the multiplexing protocol identifying unit 19 sets the first octet of the multiplexed data received from the line. Extract (S11).
Subsequently, it is determined whether or not the first two bits are “10 (binary number: the same applies hereinafter)” (S12). If the first two bits are “10”, the received data is multiplexed. H. It is determined that the received data is H.225.0. The data is transmitted to the 225.0 separation unit 15 (S14).
[0038]
On the other hand, if the first two bits are not “10”, the multiplexing protocol identification unit 16 checks the first one octet of the received data and determines whether it is “01111110 (binary number: the same applies hereinafter)”. It is determined whether or not (S13).
Here, if the first octet is “01111110”, the multiplexing protocol of the received data is set to H.264. 223, and the received data is H.264. It is transmitted to the H.223 separation unit 17 (S15).
[0039]
For the data for which the multiplexing protocol has not been identified by the above-described procedures (S11 to S13), the first 16 octets are extracted (S16), and it is determined whether or not the first 16 octets are the uuid of the ASF header object. (S17).
Here, if the first 16 octets are uuid of the ASF header object, the received data is transmitted to the ASF separation unit 20.
On the other hand, if the first 16 octets are not uid of the ASF header object, the received data is discarded (S19).
[0040]
Next, the operation when the multimedia communication device 1 using the multiplexed data separation device configured as described above receives multiplexed information from a server and reproduces it as a client terminal of a streaming system will be described. Here, a case where an ASF stream is received and reproduced will be described.
[0041]
When receiving the multiplexed data from the server, the multimedia communication device 1 determines that the multiplexing protocol of the received information is ASF based on the procedure shown in FIG. Thereafter, the information received from the server is separated by the ASF separation unit 20, separated into at least one of audio encoded data and video encoded data, and the data is transmitted to at least one of the audio codec 4 and the video codec 5. .
The audio codec 4 and the video codec 5 decode the received encoded data, and output the received audio / video information using the speaker 10 or the monitor 12.
[0042]
The data multiplexed by ASF is not only transmitted in real time via the communication network 8, but may be provided as a download file. FIG. 6 shows a schematic configuration of a multimedia communication apparatus considering the download ASF file reproduction. In FIG. 6, parts having the same functions as those of the multimedia communication apparatus 1 shown in FIG.
[0043]
As shown in FIG. 6, the multimedia communication device 21 considering the reproduction of the downloaded ASF file is negotiated with another multimedia communication device via the system control unit 22 for controlling the entire device and the communication network 8. Inter-terminal signal control unit 3 for controlling connection control signals to be transmitted / received when performing transmission, audio codec 4 for encoding / decoding audio data to be transmitted / received, and video for encoding / decoding video data to be transmitted / received The codec 5, the connection control signal, the encoded audio data, the multiplexing unit 6 for multiplexing the encoded video data, and the connection control signal, the encoded audio data, and the encoded video data transmitted after multiplexing are separated. And the communication network interface unit 7 and the storage device 23.
In addition, a user interface 9 is connected to the system control unit 2, a speaker 10 and a microphone 11 are connected to the audio codec 4, and a monitor 12 and a camera 13 are connected to the video codec 5. Yes.
[0044]
Next, the operation when the downloaded ASF file is reproduced by the multimedia communication device 21 using the multiplexed data separation device configured as described above will be described.
[0045]
The downloaded ASF file is stored in the storage device 23 of the multimedia communication device 21. Further, the multimedia communication device 21 reads the information of the file from the storage device 23, and determines that the multiplexing protocol of the information stored in the file is ASF based on the procedure shown in FIG. Thereafter, the information read from the storage device 23 is separated by the ASF separation unit 20, separated into at least one of the audio encoded data and the video encoded data, and the data is transmitted to at least one of the audio codec 4 or the video codec 5. .
The audio codec 4 and the video codec 5 decode the received encoded data, and output the received audio / video information using the speaker 10 or the monitor 12.
[0046]
In addition, a program for realizing the function of the multiplexed data separation device described above is recorded on a recording medium, the program is read into the computer using the recording medium, and the computer is operated according to the program, thereby multiplexing the program. A data separation device and a multimedia data communication device using the data separation device can be configured. Note that various recording media such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, MO, PD, ZIP, and hard disk storage device can be used as the recording medium.
[0047]
【The invention's effect】
As described above, the multiplexed data demultiplexer according to the present invention automatically identifies the multiplexing protocol based on the received data from the communication network without relying on user control, and converts it into data such as audio and video. Can be separated.
Therefore, in a terminal conforming to a plurality of communication system standards, the multiplexing protocol can be automatically switched, and multimedia communication can be performed in an appropriate manner corresponding to the connection that can be made by the terminal.
[0048]
Further, by using the multiplexed data separation device according to the present invention for the multimedia communication device and the multimedia communication system, the same effects can be achieved in the multimedia communication device and the multimedia communication system.
[0049]
Also, by recording a program for realizing the above-described means on a recording medium using a computer, the program is read by the computer using the recording medium, and the computer is operated according to the program. Thus, a multiplexed data separator and a multimedia data communication device using the same can be configured.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a multimedia communication apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a schematic configuration of a separation unit shown in FIG. 1;
FIG. 3 is a flowchart showing a procedure for identifying a protocol of received multiplexed data by a multiplexing protocol identifying unit shown in FIG. 2;
FIG. 4 is a block diagram showing a schematic configuration of a separation unit having a multiplexed data separation function in a client terminal of the streaming system.
FIG. 5 is a flowchart showing a procedure for identifying a protocol of received multiplexed data by a multiplexing protocol identifying unit shown in FIG. 4;
FIG. 6 is a block diagram showing a schematic configuration of a multimedia communication apparatus having a multimedia communication function and a streaming system client function.
[Explanation of symbols]
1, 21 Multimedia communication device 2, 22 System control unit 3 Inter-terminal signal control unit 4 Audio codec 5 Video codec 6 Multiplexing unit 7 Communication network interface unit 8 Communication network 9 User interface 10 Speaker 11 Microphone 12 Monitor 13 Camera 14 Separation Part 15 H. 225.0 Separation unit 16 Multiplexing protocol identification unit 17 223 Separation Unit 18 Separation Unit 19 Multiplexing Protocol Identification Unit 20 ASF Separation Unit 23 Storage Device

Claims (4)

In a multiplexed data separator for multimedia communication for separating data generated by multiplexing a plurality of information source encoded data into a plurality of information source encoded data,
Data separation means for selectively using a plurality of demultiplexing protocols to separate the multiplexed data into a plurality of source encoded data;
When the input multiplexed data is data that can be separated by the data separating means, a multiplexing for identifying a multiplexing / demultiplexing protocol of the multiplexed data using a part or all of the multiplexed data And separation protocol identification means,
When the demultiplexing protocol identifying means identifies the demultiplexing protocol of the input multiplexed data, the data demultiplexing means converts the input multiplexed data into a plurality of information source encoded data. A multiplexed data separation device for multimedia communication, characterized by separation. Data separation means for selectively using a plurality of demultiplexing protocols to separate the multiplexed data into a plurality of source encoded data;
When the input multiplexed data is data that can be separated by the data separating means, a multiplexing for identifying a multiplexing / demultiplexing protocol of the multiplexed data using a part or all of the multiplexed data A multiplexed data separation device having a demultiplexing protocol identification means,
When the demultiplexing protocol identifying means identifies the demultiplexing protocol of the input multiplexed data, the data demultiplexing means converts the input multiplexed data into a plurality of information source encoded data. A multimedia communication apparatus that performs multimedia communication by separating. Data separation means for selectively using a plurality of demultiplexing protocols to separate the multiplexed data into a plurality of source encoded data;
When the input multiplexed data is data that can be separated by the data separating means, a multiplexing for identifying a multiplexing / demultiplexing protocol of the multiplexed data using a part or all of the multiplexed data A multimedia communication system comprising: a multimedia communication device comprising a multiplexed data separation device for multimedia communication having a demultiplexing protocol identifying means; Using a computer
Data separation means for selectively using a plurality of demultiplexing protocols to separate the multiplexed data into a plurality of source encoded data;
When the input multiplexed data is data that can be separated by the data separating means, a multiplexing for identifying a multiplexing / demultiplexing protocol of the multiplexed data using a part or all of the multiplexed data Demultiplexing protocol identifying means, and when the demultiplexing protocol of the input multiplexed data is identified by the demultiplexing protocol identifying means, the demultiplexing protocol inputted by the data demultiplexing means A recording medium on which a program having a function for separating data into a plurality of information source encoded data is recorded.

Images