JP7578239B2 - Multipoint conference control device, signal conversion device, terminal device, and programs therefor - Google Patents

Description

The present invention relates to a conference system that realizes a conference that includes tactile sensations via communication.

There are many systems in practical use that realize a multi-point conference using sound and video via communication such as the Internet, and a representative one is the multi-point conference system described in Non-Patent Document 1. In a multi-point conference system such as Non-Patent Document 1, audio signals, video signals, and screen information from each connected point are selected to generate audio signals, video signals, and screen information for playback, and the generated audio signals, video signals, and screen information are played back at each connected point, thereby realizing a multi-point conference via communication.

Non-Patent Document 2 discloses a technology for transmitting vibrations via communication in addition to the conventionally transmitted sound and video. Non-Patent Document 2 discloses a technology for realizing a conference including tactile sensation via communication by having a terminal device at each point connected to a communication line acquire sound, video, and vibrations at that point, respectively, and output them to the communication line as audio signals, video signals, and vibration signals, and reproducing sound, video, and vibrations from the audio signals, video signals, and vibration signals input from the communication line, respectively. Non-Patent Document 2 discloses that, in transmission via a communication line, one of two channels prepared for transmitting audio signals is used for transmitting the audio signal, and the other of the two channels prepared for transmitting audio signals is used for transmitting the vibration signal.

Cisco and/or its affiliates, "Cisco Webex", [online], [searched June 28, 2021], Internet <URL: https://www.webex.com/ja/all-new-webex.html> Hirohiko Hayakawa and 7 others, "Proposal of a bidirectional audiovisual and haptic medium for remote communication with a high sense of reality: Public haptic legends," Transactions of the Virtual Reality Society of Japan

In the multipoint conference system of Non-Patent Document 1, a conference can be held at multiple points using audio signals, video signals, and screen information, but the transmission of vibration signals for conveying tactile sensations is not taken into consideration, and a conference at multiple points including vibration signals cannot be held. If the technology of Non-Patent Document 2 is applied to the multipoint conference system of Non-Patent Document 1, a conference at multiple points including vibration signals can be held. However, in a multipoint conference in which the technology of Non-Patent Document 2 is applied to the multipoint conference system of Non-Patent Document 1, one of the multiple channels that are originally for audio signals is used as a channel for vibration signals, so if a terminal device that normally plays back audio signals from multiple channels is used, the terminal device will output the vibration signal of the channel for vibration signals to the speaker, which may cause problems such as the speaker emitting unexpected sounds, breaking down, or being destroyed. The present invention aims to realize a multipoint conference in which one or more of the multiple channels that are originally for audio signals are used for vibration signals, while suppressing the occurrence of problems during playback as described above.

In order to solve the above-mentioned problems, the present invention is a multipoint conference system that includes multiple terminal devices and a multipoint conference control device connected to each terminal device via a communication path, and substitutes one or more of multiple channels that are originally for audio signals for vibration signals. A device that inputs signals to the terminal devices converts the signals for terminal devices that cannot present vibration signals or terminal devices that do not present vibration signals so that the channels substituted for vibration signals do not include vibration signals, and converts the signals for terminal devices that cannot present audio signals or terminal devices that do not present audio signals so that the channels for audio signals that are not substituted for vibration signals do not include audio signals.

In addition, in the present invention, the terminal device displays on the display screen a presentation signal selection user interface area including a media group switching button that, when pressed, performs a selection by cyclic state switching including at least three states, a state in which sound is presented without presenting vibration, a state in which both sound and vibration are presented, and a state in which vibration is presented without presenting sound, and a media display group that, when one of the states is selected by pressing the media group switching button, displays so that it is visible whether the state is a state in which sound is presented without presenting vibration, a state in which both sound and vibration are presented, or a state in which vibration is presented.

According to the present invention, in a configuration in which one or more of the multiple channels that are originally used for audio signals are used for vibration signals, it is possible to realize a conference at multiple locations while suppressing the occurrence of problems during playback.

FIG. 1 is a block diagram illustrating an example of a multipoint conferencing system 10, 12. FIG. 2 is a block diagram showing an example of a terminal device 200-m (a tactile-enabled terminal device 200-m). FIG. 3 is a flow chart showing an example of the processing of the sending device 210-m (haptically-enabled sending device 210-m). FIG. 4 is a diagram showing an example of channels constituting a two-channel stereo and signals stored in each channel by the transmitting device 210-m (haptic-compatible transmitting device 210-m). FIG. 5 is a diagram showing an example of channels constituting 5.1 channel surround and signals stored in each channel by the transmitting device 210-m (haptic-enabled transmitting device 210-m). FIG. 6 is a block diagram illustrating an example of a multipoint conference control device 100. As shown in FIG. FIG. 7 is a flowchart showing an example of the process of the multipoint conference control device 100. As shown in FIG. FIG. 8 is a flowchart showing an example of the process of the conference signal generating unit 110. As shown in FIG. FIG. 9 is a flow chart showing an example of the processing of the receiving device 220-m (haptically-enabled receiving device 220-m). FIG. 10 is a block diagram showing an example of a multipoint conference system 11. As shown in FIG. FIG. 11 is a block diagram showing an example of a tactile non-enabled terminal device 400-n. FIG. 12 is a flow diagram illustrating an example of the processing of a non-haptically enabled sender device 410-n. FIG. 13 is a block diagram illustrating an example of a multipoint conference control device 101. As shown in FIG. FIG. 14 is a flowchart showing an example of the process of the multipoint conference control unit 101. FIG. 15 is a flowchart showing an example of the process of the conference signal generating unit 111. FIG. 16 is a flow diagram illustrating an example of the processing of a non-haptic enabled receiver device 420-n. FIG. 17 is a block diagram illustrating an example of a multipoint conference control device 102. FIG. 18 is a flowchart showing an example of the processing of the multipoint conference control device 102 and the signal conversion device 604-m. FIG. 19 is a flowchart showing an example of the process of the conference signal generating unit 112. FIG. 20 is a diagram showing an example of a screen 2211-m of the video presentation unit 221-m. FIG. 21 is a diagram showing an example of a screen 2211-m of the video presentation unit 221-m. FIG. 22 is a block diagram showing an example of a multipoint conference system 13. As shown in FIG. FIG. 23 is a block diagram illustrating an example of the multipoint conference control device 103. FIG. 24 is a flowchart showing an example of the process of the multipoint conference control unit 103. FIG. 25 is a block diagram showing an example of a signal conversion device 603-n. FIG. 26 is a flow chart showing an example of the process of the signal conversion device 603-n. FIG. 27 is a flowchart showing an example of the process of the conference signal conversion unit 613-n. FIG. 28 is a block diagram showing an example of a multipoint conference system 14. FIG. 29 is a block diagram showing an example of signal conversion device 604-m. FIG. 30 is a diagram illustrating an example of the functional configuration of a computer that realizes each device.

<Reference form>
Before describing the embodiment of the present invention, a multipoint conference system that is the premise of the present invention will be described as a reference embodiment. The multipoint conference system of the reference embodiment is a multipoint conference system of the non-patent document 1 to which the technology of the non-patent document 2 is applied, and uses one or more of the multiple channels prepared for audio signals in an existing audio transmission method to transmit a signal representing vibration to be perceived as a touch (hereinafter referred to as a "vibration signal"), and uses a multipoint conference control device prepared for multipoint connection of video signals and audio signals to perform multipoint connection of vibration signals in addition to video signals and audio signals, thereby enabling the realization of a multipoint conference including tactile information. That is, the multipoint conference system of the reference embodiment uses one or more predetermined channels of the multiple channels of a specified audio transmission method as channels for audio signals, and uses one or more predetermined channels different from the channels for audio signals of the multiple channels of the specified audio transmission method as channels for vibration signals. The multipoint conference system of the reference embodiment and each device included in the system will be described below.

As shown in FIG. 1, the multipoint conference system 10 of the reference embodiment includes a multipoint conference control device 100 and terminal devices 200-m. In the multipoint conference system 10, M is an integer of 2 or more, and m is an integer of 1 to M. The multipoint conference control device 100 and each terminal device 200-m are connected via each transmission path 300-m of a communication network. The communication network is, for example, the Internet, and is capable of transmitting signals between multiple devices in almost real time. In the following, the point where the terminal device 200-m is located is referred to as point m. Note that reference numbers with "-" are written by connecting two numbers with "-" and do not represent a number subtracted from another number.

[Terminal device 200-m]
The terminal device 200-m is a terminal device located at a point m, and is composed of, for example, a PC and peripheral devices connected to the PC, and includes a transmitting device 210-m and a receiving device 220-m as shown in Fig. 2. Here, the transmitting device 210-m will be explained first. The receiving device 220-m will be explained after the multipoint conference control device 100 is explained.

[Transmitting device 210-m]
The transmitting device 210-m includes a video acquisition unit 211-m, an audio acquisition unit 212-m, a vibration acquisition unit 213-m, and a transmission signal generation unit 214-m, as illustrated in Fig. 2. The transmitting device 210-m performs each process of steps S211, S212, S213, and S214 illustrated in Fig. 3 for each predetermined time interval. The predetermined time interval is a time interval for generating a terminal transmission signal SA-m determined by the encoding processing unit or the packetization processing unit, and is, for example, an interval of several tens of ms in length.

[Video acquisition unit 211-m]
The image acquisition unit 211-m is, for example, a camera built into a PC or a camera connected to a PC. The image acquisition unit 211-m captures an image of a point m and outputs an image corresponding to the captured image. The signal is output to the transmission signal generating unit 214-m (step S211).

[Sound acquisition unit 212-m]
The sound acquisition unit 212-m is, for example, a microphone built into the PC or a microphone connected to the PC. The sound acquisition unit 212-m collects a sound generated at a point m and outputs the collected sound as An acoustic signal corresponding to the sound is output to the transmission signal generating unit 214-m (step S212).

[Vibration acquisition unit 213-m]
The vibration acquisition unit 213-m is, for example, a vibration measurement sensor connected to a PC. The vibration acquisition unit 213-m acquires vibrations generated at a point m, and outputs a vibration signal corresponding to the acquired vibration. The signal is output to the transmission signal generator 214-m (step S213).

[Output signal generation unit 214-m]
The transmission signal generating unit 214-m receives the video signal output by the video acquiring unit 211-m, the audio signal output by the audio acquiring unit 212-m, and the vibration signal output by the vibration acquiring unit 213-m. The transmission signal generating unit 214-m generates a video bit string, which is a bit string of a predetermined video transmission method including the input video signal, and transmits a multi-channel signal including the input acoustic signal and the input vibration signal. and generates an audio bit stream, which is a bit stream of a predetermined audio transmission method corresponding to the video bit stream and the audio bit stream, and outputs the generated audio bit stream and the terminal transmission signal SA-m including the generated video bit stream and the generated audio bit stream to the multipoint conference control device 100. (step S214).

The specified video transmission method is, for example, a standard for transmitting one channel of video signal, and may be lossy encoding such as MPEG-2 Visual or MPEG-4 Visual, lossless encoding, or uncompressed. The transmission signal generation unit 214-m generates a video bitstream by, for example, encoding the input video signal in accordance with the specified video transmission method or storing it in a bitstream.

The predetermined acoustic transmission method corresponding to multiple channels is, for example, a standard for transmitting a 2-channel stereo acoustic signal or a 5.1-channel surround acoustic signal, and may be lossy encoding such as MPEG-4 AAC, lossless encoding such as MPEG-4 ALS, or uncompressed. The transmission signal generating unit 214-m generates an acoustic vibration bit string by converting the input acoustic signal into a signal of a predetermined channel (hereinafter also referred to as a "channel for acoustic signals") among the multiple channels of the predetermined acoustic transmission method, converting the input vibration signal into a signal of a predetermined channel other than the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method (hereinafter also referred to as a "channel for vibration signals"), and performing encoding processing corresponding to the predetermined acoustic transmission method or storing processing into a bit string on the multiple channel signals. Note that although the acoustic vibration bit string is actually a bit string of a predetermined acoustic transmission method as described above, it is a bit string containing both an acoustic signal and a vibration signal, so it is conveniently referred to as an "acoustic vibration bit string" in this specification. An example using a 2-channel stereo audio transmission method and an example using a 5.1-channel surround audio transmission method will be described later, but in short, the transmission signal generation unit 214-m sets one or more predetermined channels out of the multiple channels of the specified audio transmission method as channels for audio signals, sets one or more predetermined channels out of the multiple channels of the specified audio transmission method different from the channels for audio signals as channels for vibration signals, stores audio signals in the channels for audio signals, and outputs a terminal transmission signal SA-m in which a vibration signal is stored in the channel for vibration signals to the multipoint conference control device 100.

[Example of using a two-channel stereo audio transmission system]
When a two-channel stereo acoustic transmission method is used, for example, a left channel (L channel) that is a predetermined one of the two channels is used as a channel for acoustic signals, and a right channel (R channel) that is a predetermined one of the two channels different from the channel for acoustic signals is used as a channel for vibration signals. In other words, when a two-channel stereo acoustic transmission method is used, the transmission signal generating unit 214-m may use the acoustic signal as a left channel signal and the vibration signal as a right channel signal, as exemplified in FIG. 4.

When using a two-channel stereo sound transmission method, the channel for the sound signal itself is one channel, but the sound acquisition unit 212-m may be one microphone or two or more microphones. That is, the number of microphones included in the sound acquisition unit 212-m and the number of channels for the sound signal may be the same or different. When the sound acquisition unit 212-m is one microphone, the transmission signal generation unit 214-m may directly use the single input sound signal as the left channel signal. When the sound acquisition unit 212-m is two or more microphones, for example, the transmission signal generation unit 214-m may use a signal obtained by mixing two or more input sound signals as the left channel signal, or may use a signal selected from the two or more input sound signals as the left channel signal.

Similarly, when using a two-channel stereo acoustic transmission method, the channel for the vibration signal itself is one channel, but the vibration acquisition unit 213-m may be one vibration measurement sensor or two or more vibration measurement sensors. In other words, the number of vibration measurement sensors included in the vibration acquisition unit 213-m and the number of channels for the vibration signal may be the same or different. When the vibration acquisition unit 213-m is one vibration measurement sensor, the transmission signal generation unit 214-m may directly use the input vibration signal as the right channel signal. When the vibration acquisition unit 213-m is two or more vibration measurement sensors, for example, the transmission signal generation unit 214-m may use a signal obtained by mixing two or more input vibration signals as the right channel signal, or may use a signal selected from the two or more input vibration signals as the right channel signal.

[Example of using a 5.1 channel surround sound transmission method]
When using a 5.1 channel surround sound transmission method, for example, two predetermined channels out of six channels, the left front channel and the right front channel, are used as channels for sound signals, and four predetermined channels out of six channels different from the channels for sound signals, the front channel, the left rear channel, the right rear channel, and the deep bass channel, are used as channels for vibration signals. That is, when using a 5.1 channel surround sound transmission method, as illustrated in FIG. 5, the transmission signal generating unit 214-m sets the two sound signals of the left channel and the right channel as left front channel and right front channel signals, respectively, and sets the first to fourth four vibration signals as front channel signals, left rear channel signals, right rear channel signals, and deep bass channel signals, respectively. When using a 5.1 channel surround sound transmission method, the number of microphones included in the sound acquisition unit 212-m and the number of channels for sound signals may be the same or different, and the number of vibration measurement sensors included in the vibration acquisition unit 213-m and the number of channels for vibration signals may be the same or different, as in the case of using a 2 channel stereo sound transmission method. Note that the transmission signal generation unit 214-m may not store a signal in any of the multiple channels for sound signals, and may not store a signal in any of the multiple channels for vibration signals.

[Whether or not video is transmitted]
It is not essential that the sending signal generating unit 214-m include a video bitstream in the terminal sending signal SA-m. For example, the sending side device 210-m may not include the video acquiring unit 211-m, and the sending side device 210-m may not perform step S211, and the sending signal generating unit 214-m may output a terminal sending signal SA-m that does not include a video bitstream to the multipoint conference control device 100. Also, for example, as in the multipoint conference system of Non-Patent Document 1, the user may be allowed to select whether or not the sending side device 210-m sends out video. In other words, the transmitting side device 210-m may be provided with a transmission signal selection unit 215-m including a switch (hereinafter referred to as a "video transmission ON/OFF switch") that accepts user input regarding whether or not the transmitting side device 210-m will transmit video, and the transmission signal generation unit 214-m may operate to determine whether or not to include a video bit string in the terminal transmission signal SA-m in accordance with the input accepted by the video transmission ON/OFF switch of the transmission signal selection unit 215-m.

The switch may be a hardware switch provided in the terminal device 200-m, but it may also be a software switch provided within a screen displayed on a display device built into or connected to the terminal device 200-m, and may be operated by the user using a mouse provided in the terminal device 200-m, or by touching a display device that is a touch screen. This also applies to each of the following switches.

[Sound transmission availability]
It is not essential for the transmission signal generating unit 214-m to include an acoustic signal in the acoustic vibration bit string. For example, the transmitting device 210-m may not include the acoustic acquiring unit 212-m, and the transmitting device 210-m may not perform step S212. The transmission signal generating unit 214-m may generate an acoustic vibration bit string that includes the input vibration signal but does not include an acoustic signal, and output a terminal transmission signal SA-m that includes the generated acoustic vibration bit string to the multipoint conference control device 100. Also, for example, as in the multipoint conference system of Non-Patent Document 1, the user may be allowed to select whether or not the transmitting device 210-m transmits sound. That is, the transmitting device 210-m may be provided with a transmission signal selection unit 215-m including a switch (hereinafter referred to as an "audio transmission ON/OFF switch") that accepts a user's input regarding whether the transmitting device 210-m transmits sound, and the transmission signal generation unit 214-m may operate to determine whether or not to include an acoustic signal in an acoustic vibration bit string according to the input accepted by the acoustic transmission ON/OFF switch of the transmission signal selection unit 215-m. Note that, when the transmission signal generation unit 214-m does not transmit sound, it may include an all-zero signal in a predetermined channel for an acoustic signal among multiple channels of a predetermined acoustic transmission method, and then perform an encoding process or a storage process into a bit string corresponding to the predetermined acoustic transmission method on the signals of the multiple channels to generate an acoustic vibration bit string, and include the generated acoustic vibration bit string in the terminal transmission signal SA-m.

[Whether or not vibration is sent]
It is not essential for the sending signal generating unit 214-m to include a vibration signal in the acoustic vibration bit string. For example, the sending side device 210-m may not include the vibration acquiring unit 213-m, and the sending side device 210-m may not perform step S213. The sending signal generating unit 214-m may generate an acoustic vibration bit string that includes the input acoustic signal but does not include a vibration signal, and output a terminal sending signal SA-m that includes the generated acoustic vibration bit string to the multipoint conference control device 100. Also, for example, the sending side device 210-m may be configured so that a user can select whether or not to send vibration. That is, the transmitting device 210-m may be provided with a transmission signal selection unit 215-m including a switch (hereinafter referred to as a "vibration transmission ON/OFF switch") that accepts a user's input regarding whether or not the transmitting device 210-m transmits vibration, and the transmission signal generation unit 214-m may operate to determine whether or not to include a vibration signal in an acoustic vibration bit string according to the input accepted by the vibration transmission ON/OFF switch of the transmission signal selection unit 215-m. Note that, when the transmission signal generation unit 214-m does not transmit vibration, it may include an all-zero signal in a predetermined channel for a vibration signal among multiple channels of a predetermined acoustic transmission method, and then perform encoding processing or storage processing into a bit string corresponding to the predetermined acoustic transmission method on the signals of the multiple channels to generate an acoustic vibration bit string, and include the generated acoustic vibration bit string in the terminal transmission signal SA-m.

[When neither sound nor vibration is sent]
When the transmission signal generating unit 214-m is not transmitting sound or vibration, it may include all-zero signals in both a channel for a predetermined acoustic signal among the multiple channels of a predetermined acoustic transmission method and a channel for a predetermined vibration signal among the multiple channels of a predetermined acoustic transmission method, and then generate an acoustic vibration bit string by performing an encoding process or a storage process into a bit string corresponding to the predetermined acoustic transmission method on the signals of the multiple channels, and include the generated acoustic vibration bit string in the terminal transmission signal SA-m, or it may not include the acoustic vibration bit string in the terminal transmission signal SA-m to be output.

[Multipoint conference control device 100]
The multipoint conference control device 100 includes a conference signal generating unit 110, as shown in Fig. 6. Terminal transmission signals SA-1 to SA-M output from the terminal devices 200-1 to 200-M connected via the transmission paths 300-1 to 300-M are input to the multipoint conference control device 100. The multipoint conference control device 100 obtains and outputs a terminal-directed signal SB by performing the process of step S110 shown in Fig. 7 for each predetermined time period.

[Conference signal generating unit 110]
Terminal transmission signals SA-1 to SA-M are input to the conference signal generating unit 110. The conference signal generating unit 110 performs the processes of steps S110-1 to S110-6 shown in Fig. 8, for example, to obtain a terminal-directed signal SB from the input terminal transmission signals SA-1 to SA-M and outputs the signal (step S110).

The conference signal generating unit 110 generates a video signal for output including the video signal included in the video bit string of each point included in each of the terminal transmission signals SA-1 to SA-M (step S110-1). For example, similar to the multipoint conference system of Non-Patent Document 1, the conference signal generating unit 110 may obtain a reduced image by reducing the size of the video of each point, generate one image including the reduced image of each point, and generate a video signal corresponding to the generated image. If there is a terminal transmission signal that does not include a video bit string in the terminal transmission signals SA-1 to SA-M, the conference signal generating unit 110 may generate a video signal for output including the video signal of each point only from the terminal transmission signals that include a video bit string. After performing step S110-1, the conference signal generating unit 110 generates a video bit string that is a bit string of a predetermined video transmission method including the video signal obtained in step S110-1 (step S110-2).

The conference signal generating unit 110 generates an output acoustic signal including the acoustic signal included in the acoustic vibration bit string of each point included in each of the terminal transmission signals SA-1 to SA-M (step S110-3). The conference signal generating unit 110 may generate one acoustic signal by mixing the acoustic signals of each point, as in the multipoint conference system of Non-Patent Document 1, for example. If there is a terminal transmission signal in which the acoustic vibration bit string of the terminal transmission signals SA-1 to SA-M does not include an acoustic signal, the conference signal generating unit 110 may generate an output acoustic signal including the acoustic signal of each point only for the terminal transmission signal in which the acoustic vibration bit string includes an acoustic signal. If there are multiple channels for acoustic signals, the conference signal generating unit 110 may generate an output acoustic signal including the acoustic signal of each point for each channel.

The conference signal generating unit 110 also generates an output vibration signal including the vibration signal included in the acoustic vibration bit string of each point included in each of the terminal sending signals SA-1 to SA-M (step S110-4). The conference signal generating unit 110 may, for example, generate one vibration signal by mixing the vibration signals of each point. If there is a terminal sending signal that does not include a vibration signal in the acoustic vibration bit string of the terminal sending signals SA-1 to SA-M, the conference signal generating unit 110 may generate an output vibration signal including the vibration signal of each point only for terminal sending signals that include a vibration signal in the acoustic vibration bit string. If there are multiple channels for vibration signals, the conference signal generating unit 110 may generate an output vibration signal including the vibration signal of each point for each channel.

After performing steps S110-3 and S110-4, the conference signal generating unit 110 stores the acoustic signal obtained in step S110-3 in a predetermined channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, and stores the vibration signal obtained in step S110-4 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating an acoustic vibration bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S110-5). After performing steps S110-2 and S110-5, the conference signal generating unit 110 obtains a terminal-oriented signal SB including the video bit string generated in step S110-2 and the acoustic vibration bit string generated in step S110-5, and outputs it to each terminal device, that is, outputs it to each of the terminal devices 200-1 to 200-M via each transmission path 300-1 to 300-M (step S110-6).

If none of the terminal transmission signals includes a video signal, the conference signal generation unit 110 performs step S110-5 without performing steps S110-1 and S110-2, and then performs step S110-5 in step S110-6, obtaining a terminal-directed signal SB including the acoustic vibration bit string generated in step S110-5 and outputting it to each terminal device. In short, the conference signal generation unit 110 stores an acoustic signal based on an acoustic signal input from each point in a channel for an acoustic signal that is one or more predetermined channels out of the multiple channels of a predetermined acoustic transmission method, and outputs a terminal-directed signal SB to each terminal device in which a vibration signal based on a vibration signal input from each point is stored in a channel for a vibration signal that is one or more predetermined channels different from the channel for an acoustic signal out of the multiple channels of a predetermined acoustic transmission method.

[Receiving device 220-m]
The receiving device 220-m includes a presentation signal generating unit 224-m, a video presentation unit 221-m, an audio presentation unit 222-m, and a vibration presentation unit 223-m, as illustrated in Fig. 2. The receiving device 220-m receives a terminal-directed signal SB output by the multipoint conference control device 100 connected via each transmission path 300-m. The receiving device 220-m performs each process of steps S224, S221, S222, and S223 illustrated in Fig. 9 for each predetermined time period.

[Presentation signal generation unit 224-m]
The terminal-directed signal SB is input to the presentation signal generator 224-m. The presentation signal generator 224-m derives a video signal from a video bit string, which is a bit string of a predetermined video transmission method included in the terminal-directed signal SB. and outputs the signal to the video presentation unit 221-m, and obtains a signal of a channel for an acoustic signal from an acoustic vibration bit string, which is a bit string of a predetermined acoustic transmission method included in the terminal-directed signal SB, and outputs the signal to the audio presentation unit 222-m. and outputs the signal of the channel for the vibration signal from the acoustic vibration bit string, which is a bit string of a predetermined acoustic transmission method included in the terminal-directed signal SB, to the vibration presentation unit 223-m as a vibration signal. Output (step S224).

[Video presentation section 221-m]
The video presentation unit 221-m is, for example, a display device built into a PC or a display device connected to a PC. The image presenting unit 221-m displays an image corresponding to the input image signal, that is, presents the image (step S221).

[Sound presentation section 222-m]
The audio presentation unit 222-m is, for example, a speaker built in the PC or a speaker connected to the PC. The audio presentation unit 222-m receives the audio signal output by the presentation signal generation unit 224-m. The sound presentation unit 222-m generates a sound corresponding to the input sound signal, that is, presents the sound (step S222).

[Vibration presentation unit 223-m]
The vibration presentation unit 223-m is, for example, a vibrator connected to a PC. The vibration presentation unit 223-m receives a vibration signal output from the presentation signal generation unit 224-m. -m generates vibration corresponding to the input vibration signal, that is, presents vibration (step S223).

[Whether or not video is shown]
It is not essential that the receiving device 220-m presents the video. For example, the receiving device 220-m may not include the video presentation unit 221-m, the presentation signal generation unit 224-m may not obtain a video signal from the terminal-directed signal SB, and the receiving device 220-m may not perform step S221. Also, for example, the receiving device 220-m may be configured to allow the user to select whether or not to present the video. That is, the receiving device 220-m may be configured to include a presentation signal selection unit 225-m including a switch (hereinafter referred to as a "video presentation ON/OFF switch") that accepts an input by the user regarding whether or not the receiving device 220-m presents the video, and the presentation signal generation unit 224-m may operate to output or not output a video signal according to the input accepted by the video presentation ON/OFF switch of the presentation signal selection unit 225-m.

[Sound presentation or not]
It is not essential that the receiving device 220-m presents the sound. For example, the receiving device 220-m may not include the audio presentation unit 222-m, the presentation signal generation unit 224-m may not obtain an audio signal from the terminal-directed signal SB, and the receiving device 220-m may not perform step S222. Also, for example, the receiving device 220-m may be configured to allow the user to select whether or not the receiving device 220-m presents the sound. That is, the receiving device 220-m may be configured to include a presentation signal selection unit 225-m including a switch (hereinafter referred to as an "audio presentation ON/OFF switch") that accepts an input by the user regarding whether or not the receiving device 220-m presents the sound, and the presentation signal generation unit 224-m may operate to output or not output an audio signal according to the input accepted by the audio presentation ON/OFF switch of the presentation signal selection unit 225-m.

[Whether or not vibration is displayed]
It is not essential that the receiving device 220-m presents vibration. For example, the receiving device 220-m may not include the vibration presentation unit 223-m, the presentation signal generation unit 224-m may not obtain a vibration signal from the terminal-directed signal SB, and the receiving device 220-m may not perform step S223. Also, for example, the receiving device 220-m may be configured to allow the user to select whether or not to present vibration. The receiving device 220-m may be configured to include a presentation signal selection unit 225-m including a switch (hereinafter referred to as a "vibration presentation ON/OFF switch") that accepts a user's input regarding whether or not the receiving device 220-m presents vibration, and the presentation signal generation unit 224-m may operate to output or not output a vibration signal according to the input accepted by the vibration presentation ON/OFF switch of the presentation signal selection unit 225-m.

First Embodiment
In the multipoint conference system 10 of the reference embodiment, a channel originally prepared for an acoustic signal in a predetermined acoustic transmission method is substituted as a channel for a vibration signal. Therefore, if a terminal device that reproduces a multi-channel acoustic signal according to a predetermined acoustic transmission method is used as a terminal device of the multipoint conference system 10 of the reference embodiment, instead of the above-mentioned terminal device 200-m, the terminal device will output a vibration signal of a channel for a vibration signal to a speaker for an acoustic signal, and the speaker may emit an unexpected sound, break down, or be destroyed. In order to eliminate these possibilities, the multipoint conference control device outputs a terminal-directed signal suitable for each terminal device in the multipoint conference system of the first embodiment. Below, the multipoint conference system of the first embodiment and each device included in the system will be described, focusing on the differences from the reference embodiment.

As shown in FIG. 10, the multipoint conference system 11 of the first embodiment includes a multipoint conference control device 101, a haptic-enabled terminal device 200-m, and a non-haptic-enabled terminal device 400-n. In the first embodiment, M is an integer of 1 or more, m is an integer of 1 to M, N is an integer of 1 or more, and n is an integer of M+1 to M+N. The multipoint conference control device 101 and each haptic-enabled terminal device 200-m are connected via each transmission path 300-m of the communication network. The multipoint conference control device 101 and each non-haptic-enabled terminal device 400-n are connected via each transmission path 500-n of the communication network. The communication network is, for example, the Internet, and can transmit signals between multiple devices in almost real time. In the following, the point where the haptic-enabled terminal device 200-m is located is called point m, and the point where the non-haptic-enabled terminal device 400-n is located is called point n.

[Tactile-enabled terminal device 200-m]
The haptic-enabled terminal device 200-m is similar to the terminal device 200-m of the reference embodiment, and the configuration and operation of each part of the haptic-enabled terminal device 200-m are similar to those described in the description of the terminal device 200-m of the reference embodiment. The name "haptic-enabled terminal device" is adopted to clearly distinguish it from the "non-haptic-enabled terminal device" included in the multipoint conference system 11 of the first embodiment. For the same reason, the transmitting side device 210-m of the reference embodiment is called the haptic-enabled transmitting side device 210-m in the first embodiment, the receiving side device 220-m of the reference embodiment is called the haptic-enabled receiving side device 220-m in the first embodiment, the terminal sending signal SA-m in the reference embodiment is called the haptic-enabled terminal sending signal SA-m in the first embodiment, and the terminal-directed signal SB in the reference embodiment is called the haptic-enabled terminal-directed signal SB in the first embodiment. That is, the haptic-enabled transmitting device 210-m outputs a haptic-enabled terminal sending signal SA-m to the multipoint conference control device 101 via a transmission path 300-m. Also, a haptic-enabled terminal-directed signal SB is input to the haptic-enabled receiving device 220-m from the multipoint conference control device 101 via the transmission path 300-m.

[Haptic-non-compatible terminal device 400-n]
The haptic non-capable terminal device 400-n is a terminal device located at point n, and is composed of, for example, a PC and a peripheral device connected to the PC, or a PC alone, and includes a haptic non-capable transmitting side device 410-n and a haptic non-capable receiving side device 420-n, as shown in Fig. 11. Here, the haptic non-capable transmitting side device 410-n will be described first. The haptic non-capable receiving side device 420-n will be described after the multipoint conference control device 101 has been described.

[Non-haptic-enabled transmitting device 410-n]
The haptic incompatible transmitting device 410-n includes a video acquiring unit 411-n, a sound acquiring unit 412-n, and a transmission signal generating unit 414-n, as illustrated in Fig. 11. The haptic incompatible transmitting device 410-n performs the processes of steps S411, S412, and S414 illustrated in Fig. 12 for each predetermined time interval.

[Video acquisition unit 411-n]
The image acquisition unit 411-n is, for example, a camera built into a PC or a camera connected to a PC. The image acquisition unit 411-n captures an image of a point n and outputs an image corresponding to the captured image. The signal is output to the transmission signal generating unit 414-n (step S411).

[Sound acquisition unit 412-n]
The sound acquisition unit 412-n is, for example, a microphone built into the PC or a microphone connected to the PC. The sound acquisition unit 412-n collects a sound generated at the point n and outputs the collected sound. An acoustic signal corresponding to the sound is output to the transmission signal generator 414-n (step S412).

[Output signal generation unit 414-n]
The transmission signal generation unit 414-n receives as input the video signal output by the video acquisition unit 411-n and the audio signal output by the audio acquisition unit 412-n. A video bit string is generated which is a bit string of a predetermined video transmission method including the input video signal, and an audio bit string is generated which is a bit string of a predetermined audio transmission method corresponding to a plurality of channels including the input audio signal, The haptic non-capable terminal output signal SC-n containing the obtained video bit string and the generated audio bit string is output to the multipoint conference control unit 101 (step S414).

Similar to the transmission signal generation unit 214-m of the haptic-compatible transmission device 210-m, the transmission signal generation unit 414-n generates a video bitstream by performing encoding processing corresponding to a specified video transmission method or storing processing into a bitstream on the input video signal.

The transmission signal generation unit 414-n generates an audio bit string by treating the input audio signal as a signal for the channel for the above-mentioned audio signal among the multiple channels of the predetermined audio transmission method, and performing encoding processing or storing processing in a bit string corresponding to the predetermined audio transmission method on the signal of the multiple channels without including a substantial signal in the channel for the above-mentioned vibration signal among the multiple channels of the predetermined audio transmission method. For example, in the case of generating an audio bit string that always includes information on multiple channels for which a predetermined audio transmission method is set, the transmission signal generation unit 414-n generates an audio bit string by treating the input audio signal as a signal for the channel for the above-mentioned audio signal among the multiple channels of the predetermined audio transmission method, and treating an all-zero signal as a signal for the channel for the above-mentioned vibration signal among the multiple channels of the predetermined audio transmission method, and performing encoding processing or storing processing in a bit string corresponding to the predetermined audio transmission method on the signal of the multiple channels. Also, for example, if a specific audio transmission method is used to generate an audio bitstream that includes information about only the required channel, the transmission signal generation unit 414-n converts the input audio signal into a signal for the channel for audio signals described above among the multiple channels of the specific audio transmission method, performs encoding processing corresponding to the specific audio transmission method or storage processing into a bitstream on the signal for the channel for audio signals, and generates an audio bitstream by including information indicating that the channel for audio signals is the only required channel.

[Whether or not video is transmitted]
It is not essential for the transmission signal generating unit 414-n to include a video bit string in the haptic non-capable terminal transmission signal SC-n. For example, the haptic non-capable transmission side device 410-n may not include the video acquisition unit 411-n, and the haptic non-capable transmission side device 410-n may not perform step S411, and the transmission signal generating unit 414-n may output the haptic non-capable terminal transmission signal SC-n that does not include a video bit string to the multipoint conference control device 101. Also, for example, as in the multipoint conference system of Non-Patent Document 1, the user may be allowed to select whether or not the haptic non-capable transmission side device 410-n transmits video. In other words, the non-tactile transmitting side device 410-n may be provided with a transmission signal selection unit 415-n including a "video transmission ON/OFF switch" which is a switch that accepts input from the user as to whether or not the non-tactile-enabled transmitting side device 410-n will transmit video, and the transmission signal generation unit 414-n may operate to determine whether or not to include a video bit string in the non-tactile terminal transmission signal SC-n in accordance with the input accepted by the video transmission ON/OFF switch of the transmission signal selection unit 415-n.

[Sound transmission availability]
It is not essential for the transmission signal generating unit 414-n to include an acoustic bit string in the haptic non-capable terminal transmission signal SC-n. For example, the haptic non-capable transmitting side device 410-n may not include the acoustic acquisition unit 412-n, and the haptic non-capable transmitting side device 410-n may not perform step S412, and the transmission signal generating unit 414-n may output the haptic non-capable terminal transmission signal SC-n that does not include an acoustic bit string to the multipoint conference control device 101. Also, for example, as in the multipoint conference system of Non-Patent Document 1, the user may be allowed to select whether or not the haptic non-capable transmitting side device 410-n transmits sound. That is, the non-haptic capable transmitting side device 410-n may be provided with a transmitting signal selection unit 415-n including an audio transmitting ON/OFF switch that is a switch that receives a user's input as to whether or not the non-haptic capable transmitting side device 410-n transmits sound, and the transmitting signal generation unit 414-n may operate to determine whether or not to include an audio bit string in the non-haptic capable terminal transmitting signal SC-n according to the input received by the audio transmitting ON/OFF switch of the transmitting signal selection unit 415-n. Note that, when sound is not to be transmitted, the transmitting signal generation unit 414-n may include an all-zero signal in the channel for the audio signal, and then perform encoding processing or storage processing into a bit string corresponding to a predetermined audio transmission method on the signals of the multiple channels to generate an audio bit string, and include the generated audio bit string in the non-haptic capable terminal transmitting signal SC-n.

[Multipoint conference control device 101]
The multipoint conference control device 101 includes a conference signal generating unit 111, as shown in Fig. 13. The multipoint conference control device 101 receives haptic-compatible terminal output signals SA-1 to SA-M output from the haptic-compatible terminal devices 200-1 to 200-M connected via the transmission paths 300-1 to 300-M, and non-haptic-compatible terminal output signals SC-(M+1) to SC-(M+N) output from the non-haptic-compatible terminal devices 400-(M+1) to 400-(M+N) connected via the transmission paths 500-(M+1) to 500-(M+N). The multipoint conference control device 101 performs the process of step S111 shown in Fig. 14 for each predetermined time period.

[Conference signal generating unit 111]
Haptic-compatible terminal transmission signals SA-1 to SA-M and haptic non-compatible terminal transmission signals SC-(M+1) to SC-(M+N) are input to the conference signal generation unit 111. The conference signal generation unit 111 performs the processes of steps S111-1 to S111-8 shown in Fig. 15, for example, to obtain and output a haptic-compatible terminal-directed signal SB and a non-haptic terminal-directed signal SD from the input haptic-compatible terminal transmission signals SA-1 to SA-M and the non-haptic terminal transmission signals SC-(M+1) to SC-(M+N) (step S111).

The conference signal generating unit 111 generates a video signal for output that includes the video signal included in the video bit strings of each point included in the haptic-enabled terminal sending signals SA-1 to SA-M and the haptic-incompatible terminal sending signals SC-(M+1) to SC-(M+N) (step S111-1). For example, similar to the multipoint conference system of Non-Patent Document 1, the conference signal generating unit 111 obtains reduced images by reducing the size of the images of each point, generates one image including the reduced images of each point, and generates a video signal corresponding to the generated images. If there is a haptic-enabled terminal sending signal that does not include a video bit string among the haptic-enabled terminal sending signals SA-1 to SA-M, or if there is a non-haptic-enabled terminal sending signal that does not include a video bit string among the non-haptic-enabled terminal sending signals SC-(M+1) to SC-(M+N), the conference signal generating unit 111 only needs to generate a video signal for output that includes a video signal at each point from the haptic-enabled terminal sending signals that include a video bit string and the non-haptic-enabled terminal sending signals. After performing step S111-1, the conference signal generating unit 111 generates a video bit string that is a bit string of a predetermined video transmission method that includes the video signal obtained in step S111-1 (step S111-2).

The conference signal generator 111 also generates an output acoustic signal that includes the acoustic signals included in the acoustic vibration bit strings of each point included in each of the haptic-enabled terminal transmission signals SA-1 to SA-M, and the acoustic signals included in the acoustic bit strings of each point included in each of the non-haptic-enabled terminal transmission signals SC-(M+1) to SC-(M+N) (step S111-3). The conference signal generator 111 may mix the acoustic signals of each point to generate a single acoustic signal, for example, as in the multipoint conference system of Non-Patent Document 1. When there are haptic-enabled terminal sending signals SA-1 to SA-M whose acoustic vibration bit strings do not include an acoustic signal, or when there are haptic-enabled terminal sending signals SC-(M+1) to SC-(M+N) whose acoustic bit strings do not include an acoustic bit string (i.e., haptic-enabled terminal sending signals that do not include an acoustic signal), the conference signal generating unit 111 only needs to generate an output acoustic signal that includes an acoustic signal at each point for haptic-enabled terminal sending signals whose acoustic vibration bit strings include an acoustic signal and haptic-enabled non-terminal sending signals that include an acoustic bit string (i.e., only haptic-enabled terminal sending signals that include an acoustic signal and haptic-enabled non-terminal sending signals that include an acoustic signal). When there are multiple channels for acoustic signals, the conference signal generating unit 111 only needs to generate an output acoustic signal that includes an acoustic signal at each point for each channel.

The conference signal generating unit 111 also generates an output vibration signal including the vibration signal included in the acoustic vibration bit string of each point included in each of the haptic-compatible terminal sending signals SA-1 to SA-M (step S111-4). The conference signal generating unit 111 may, for example, generate one vibration signal by mixing the vibration signals of each point. If there is a haptic-compatible terminal sending signal in which the acoustic vibration bit string of the haptic-compatible terminal sending signals SA-1 to SA-M does not include a vibration signal, the conference signal generating unit 111 may generate an output vibration signal including the vibration signal of each point only for the haptic-compatible terminal sending signal in which the acoustic vibration bit string includes a vibration signal. If there are multiple channels for vibration signals, the conference signal generating unit 111 may generate an output vibration signal including the vibration signal of each point for each channel.

After performing steps S111-3 and S111-4, the conference signal generating unit 111 stores the acoustic signal obtained in step S111-3 in a predetermined channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, and stores the vibration signal obtained in step S111-4 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating an acoustic vibration bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S111-5). After performing steps S111-2 and S111-5, the conference signal generating unit 111 obtains a haptic-compatible terminal-oriented signal SB including the video bit string generated in step S111-2 and the acoustic vibration bit string generated in step S111-5, and outputs it to each haptic-compatible terminal device, that is, outputs it to each of the haptic-compatible terminal devices 200-1 to 200-M via each transmission path 300-1 to 300-M (step S111-6).

Steps S111-1 to S111-6 performed by the conference signal generating unit 111 are the same as steps S110-1 to S110-6 performed by the conference signal generating unit 110 of the reference embodiment, but the conference signal generating unit 111 also performs the following steps S111-7 and S111-8. After performing step S111-3, the conference signal generating unit 111 stores the acoustic signal obtained in step S111-3 in a predetermined channel for acoustic signals among the multiple channels of a predetermined acoustic transmission method, and stores an all-zero signal or the acoustic signal obtained in step S111-3 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating an acoustic bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S111-7). After performing steps S111-2 and S111-7, the conference signal generating unit 111 obtains a signal SD for a non-haptic terminal including the video bit string generated in step S111-2 and the audio bit string generated in step S111-7, and outputs it to each non-haptic terminal device, that is, outputs it to each of the non-haptic terminal devices 400-(M+1) to 400-(M+N) via each transmission path 500-(M+1) to 500-(M+N) (step S111-8). Note that an all-zero signal is a signal whose amplitude is zero at all times, and is an example of a signal that does not substantially generate sound or vibration even when input to the audio presentation unit or vibration presentation unit. In other words, the conference signal generating unit 110 may use a signal that is equivalent to substantially no signal, such as a signal with a small amplitude, instead of an all-zero signal, as long as the signal does not substantially generate sound or vibration even when input to the audio presentation unit or vibration presentation unit. The same applies to the subsequent all-zero signals.

If no video signal is included in any of the tactile-enabled terminal output signals and the non-tactile-enabled terminal signals, the conference signal generating unit 111 does not perform steps S111-1 and S111-2, but instead performs step S111-5, and then, in step S111-6, obtains a signal SB for the tactile-enabled terminal including the acoustic vibration bit string generated in step S111-5 and outputs it to each tactile-enabled terminal device, and after performing step S111-7, obtains a signal SD for the non-tactile terminal including the acoustic bit string generated in step S111-7 and outputs it to each non-tactile terminal device, in step S111-8. In short, the conference signal generating unit 111 outputs to each tactile-enabled terminal device a signal SB for a tactile-enabled terminal in which an acoustic signal based on an acoustic signal input from each point is stored in a channel for an acoustic signal, which is one or more predetermined channels among the multiple channels of a predetermined acoustic transmission method, and a vibration signal based on a vibration signal input from each point is stored in a channel for a vibration signal, which is one or more predetermined channels different from the channel for an acoustic signal among the multiple channels of a predetermined acoustic transmission method, and outputs to each tactile-non-enabled terminal device a signal SD for a tactile-non-enabled terminal in which an acoustic signal based on an acoustic signal input from each point is stored in a channel for an acoustic signal, and an all-zero signal or an acoustic signal based on an acoustic signal input from each point is stored in a channel for a vibration signal.

[Non-haptic-enabled receiving device 420-n]
The non-haptic-enabled receiving device 420-n includes a presentation signal generating unit 424-n, a video presentation unit 421-n, and an audio presentation unit 422-n, as illustrated in Fig. 11. The non-haptic-enabled receiving device 420-n receives a signal SD for a non-haptic terminal output by the multipoint conference control device 101 connected via each transmission path 500-n. The non-haptic-enabled receiving device 420-n performs the processes of steps S424, S421, and S422 illustrated in Fig. 16 for each predetermined time period.

[Presentation signal generation unit 424-n]
The signal SD for non-haptic-enabled terminals is input to the presentation signal generation unit 424-n. The presentation signal generation unit 424-n generates a video signal, which is a bit string of a predetermined video transmission method included in the signal SD for non-haptic-enabled terminals. A video signal is obtained from the bit string and output to the video presentation unit 421-n, and a signal of a channel for the audio signal is obtained from an audio bit string, which is a bit string of a predetermined audio transmission method included in the signal SD for a non-tactile-enabled terminal. The audio output unit 422-n outputs the audio signal (step S424).

The haptic non-compatible terminal device 400-n may be a terminal device that reproduces a multi-channel acoustic signal according to a predetermined acoustic transmission method. When the haptic non-compatible terminal device 400-n is a terminal device that reproduces a multi-channel acoustic signal according to a predetermined acoustic transmission method, the presentation signal generation unit 424-n obtains a channel signal for an acoustic signal from an acoustic bit string that is a bit string of a predetermined acoustic transmission method included in the signal SD for a haptic non-compatible terminal and outputs the signal as an acoustic signal to the audio presentation unit 422-n in step S424, and obtains a channel signal for a vibration signal from an acoustic bit string that is a bit string of a predetermined acoustic transmission method included in the signal SD for a haptic non-compatible terminal and outputs the signal as an acoustic signal to the audio presentation unit 422-n.

[Video presentation section 421-n]
The video presentation unit 421-n is, for example, a display device built into a PC or a display device connected to a PC. The video presentation unit 421-n displays a video corresponding to the input video signal, that is, presents the video (step S421).

[Sound presentation unit 422-n]
The audio presentation unit 422-n is, for example, a speaker built into the PC or a speaker connected to the PC. The audio presentation unit 422-n receives the audio signal output by the presentation signal generation unit 424-n. The sound presentation unit 422-n generates a sound corresponding to the input sound signal, that is, presents the sound (step S422).

Even if the haptic non-compatible terminal device 400-n is a terminal device that reproduces audio signals of multiple channels according to a specified audio transmission method, the audio presentation unit 422-n generates a sound corresponding to the input audio signal in step S422. That is, in this case, the audio presentation unit 422-n generates a sound corresponding to the signal of the channel for the vibration signal in addition to the sound corresponding to the signal of the channel for the audio signal included in the audio bit string. However, in the channel for the vibration signal, the conference signal generation unit 111 of the multipoint conference control device 101 stores an all-zero signal or an audio signal based on the audio signal input from each point. Therefore, even if the haptic non-compatible terminal device 400-n is a terminal device that reproduces audio signals of multiple channels according to a specified audio transmission method, the speaker does not emit unexpected sounds, and there is no possibility that the speaker will malfunction or be destroyed.

[Whether or not video is shown]
It is not essential that the non-haptic receiving device 420-n presents an image. For example, the non-haptic receiving device 420-n may not include the image presenting unit 421-n, the presentation signal generating unit 424-n may not obtain an image signal from the signal SD for the non-haptic terminal, and the non-haptic receiving device 420-n may not perform step S421. Also, for example, the user may be able to select whether the non-haptic receiving device 420-n presents an image. That is, the non-haptic receiving device 420-n may be provided with a presentation signal selecting unit 425-n including an image presentation ON/OFF switch that is a switch that accepts an input by the user regarding whether the non-haptic receiving device 420-n presents an image, and the presentation signal generating unit 424-n may operate to output or not output a video signal according to the input accepted by the image presentation ON/OFF switch of the presentation signal selecting unit 425-n.

[Sound presentation or not]
It is not essential that the non-haptic receiving device 420-n presents sound. For example, the non-haptic receiving device 420-n may not include the audio presentation unit 422-n, the presentation signal generation unit 424-n may not obtain an audio signal from the signal SD for the non-haptic terminal, and the non-haptic receiving device 420-n may not perform step S422. Also, for example, the user may be able to select whether the non-haptic receiving device 420-n presents sound. That is, the non-haptic receiving device 420-n may be provided with a presentation signal selection unit 425-n including an audio presentation ON/OFF switch that is a switch that accepts an input by the user regarding whether the non-haptic receiving device 420-n presents sound, and the presentation signal generation unit 424-n may operate to output or not output an audio signal according to the input accepted by the audio presentation ON/OFF switch of the presentation signal selection unit 425-n.

<First Modification of First Embodiment>
In order for the multipoint conference control device 101 to perform the above-mentioned operation, the multipoint conference control device 101 needs to be able to identify whether each terminal device connected via each transmission path is a tactile-enabled terminal device or a tactile-incompatible terminal device. The multipoint conference control device 101 may identify whether each terminal device is a tactile-enabled terminal device or a tactile-incompatible terminal device based on identification information or initial settings in a connection via the Internet, or may identify based on information output by each terminal device. An example in which the multipoint conference control device 101 identifies whether each terminal device is a tactile-enabled terminal device or a tactile-incompatible terminal device based on information output by each terminal device will be described as a first modification of the first embodiment, with a difference from the first embodiment.

[Haptic-enabled receiving device 220-m, supported media information sending unit 226-m]
The haptic-enabled receiving device 220-m also includes a supported media information transmitter 226-m, as indicated by a dashed line in Fig. 2. The supported media information transmitter 226-m outputs information indicating that the terminal device m is a haptic-enabled terminal device, i.e., information indicating that the receiving device m is a receiving device capable of presenting a vibration signal, as supported media information Im to the multipoint conference control device 101 (step S226).

[Haptic incompatible receiving side device 420-n, compatible media information transmitting unit 426-n]
The haptic incompatible receiving device 420-n also includes a compatible media information transmitter 426-n, as indicated by a dashed line in Fig. 11. The compatible media information transmitter 426-n outputs information indicating that the terminal device n is a haptic incompatible terminal device, that is, information indicating that the receiving device n is a terminal device that cannot present a vibration signal, as compatible media information In to the multipoint conference control device 101 (step S426).

[Multipoint conference control device 101, conference signal generating unit 111]
The multipoint conference control device 101 also receives corresponding media information I-k (k is an integer between 1 and M+N) output by each terminal device connected via each transmission path. At the time when the corresponding media information is input to the multipoint conference control device 101, the multipoint conference control device 101 cannot determine whether each terminal device is a tactile-compatible terminal device or a tactile-incompatible terminal device, so the corresponding media information is represented here as I-k rather than I-m and I-n. The corresponding media information I-k input to the multipoint conference control device 101 is input to a conference signal generation unit 111. For each k between 1 and M+N, if the corresponding media information I-k is information indicating that terminal device k is a tactile-compatible terminal device, i.e., information indicating that the receiving device k is a receiving device capable of presenting a vibration signal, the conference signal generating unit 111 identifies terminal device k as a tactile-compatible terminal device and identifies the terminal transmission signal input from terminal device k via the transmission path as a tactile-compatible terminal transmission signal; if the corresponding media information I-k is information indicating that terminal device k is a tactile-incompatible terminal device, i.e., information indicating that the receiving device k is a receiving device that cannot present a vibration signal, the conference signal generating unit 111 identifies terminal device k as a tactile-incompatible terminal device and identifies the terminal transmission signal input from terminal device k via the transmission path as a tactile-incompatible terminal transmission signal (step S111-0), and then performs the processing of step S111 described in the first embodiment.

Second Embodiment
In a multipoint conference system in which a channel originally prepared for an audio signal in a predetermined audio transmission method is substituted for a channel for a vibration signal, a user of each terminal device may be allowed to select which of sound and vibration he or she wishes to reproduce, and the multipoint conference control device may generate a terminal-directed signal corresponding to the selection in each terminal device and output it to each terminal device. This embodiment is taken as the second embodiment, and the multipoint conference system of the second embodiment and each device included in the system will be described, focusing on the differences from the reference embodiment.

As shown in FIG. 1, the multipoint conference system 12 of the second embodiment includes a multipoint conference control device 102 and terminal devices 200-m. In the second embodiment, M is an integer of 2 or more, and m is an integer of 1 to M. The multipoint conference control device 102 and each terminal device 200-m are connected via each transmission path 300-m of a communication network. The communication network is, for example, the Internet, and is capable of transmitting signals between multiple devices in almost real time. In the following, the point where the terminal device 200-m is located is referred to as point m.

[Terminal device 200-m]
The terminal device 200-m is a terminal device located at a point m, and is configured, for example, by a PC and peripheral devices connected to the PC, and includes a transmitting device 210-m and a receiving device 220-m as shown in FIG.

[Transmitting device 210-m]
The transmitting device 210-m is the same as the transmitting device 210-m in the reference embodiment, except that the transmitting device 210-m outputs a terminal sending signal SA-m to the multipoint conference control device 102 via a transmission path 300-m.

[Receiving device 220-m]
2, the receiving device 220-m includes a presentation signal generating unit 224-m, a video presentation unit 221-m, an audio presentation unit 222-m, a vibration presentation unit 223-m, a presentation signal selecting unit 225-m, and a corresponding media information transmitting unit 226-m. Here, the presentation signal selecting unit 225-m and the corresponding media information transmitting unit 226-m will be described first. The presentation signal generating unit 224-m, the video presentation unit 221-m, the audio presentation unit 222-m, and the vibration presentation unit 223-m will be described after the multipoint conference control device 102 is described.

[Presentation signal selection unit 225]
The presentation signal selection unit 225 accepts a user's selection of any of the options including at least three options for sound and vibration: the receiving device 220-m presents both sound and vibration, the receiving device 220-m presents sound without presenting vibration, and the receiving device 220-m presents vibration without presenting sound, and outputs information about the selected option to the corresponding media information transmission unit 226-m. Of course, the options may also include the receiving device 220-m presenting neither sound nor vibration. In the above example using a two-channel stereo sound transmission method, one of the two channels is a channel for sound signals, and one of the two channels different from the channel for sound signals is a channel for vibration signals, and there is one channel for sound signals and one channel for vibration signals, so that the options only need to include three options: the receiving device 220-m presents both sound and vibration, the receiving device 220-m presents sound without presenting vibration, and the receiving device 220-m presents vibration without presenting sound. In the above-mentioned example using a 5.1 channel surround sound transmission method, two of the six channels are for acoustic signals, and four of the six channels different from the channels for acoustic signals are for vibration signals, and there are multiple channels for acoustic signals and vibration signals, so the options include the three options of the receiving device 220-m presenting both sound and vibration, the receiving device 220-m presenting sound without presenting vibration, and the receiving device 220-m presenting vibration without presenting sound, and may also include which of the two channels for acoustic signals to present when the receiving device 220-m presents sound, and which of the four channels for vibration signals to present when the receiving device 220-m presents vibration.

The presentation signal selection unit 225 may also allow the user to select whether or not the receiving device 220-m will present video. In other words, the presentation signal selection unit 225 may also include a video presentation ON/OFF switch that accepts input from the user regarding whether or not the receiving device 220-m will present video, and may also output information accepted by the video presentation ON/OFF switch to the compatible media information transmission unit 226-m.

[Compatible media information transmission unit 226-m]
The corresponding media information transmitter 226-m outputs corresponding media information I-m including information corresponding to the above-mentioned options for sound and vibration input from the presentation signal selector 225 to the multipoint conference control device 102. That is, when there are three options, that is, the receiving device 220-m presents both sound and vibration, the receiving device 220-m presents sound without presenting vibration, and the receiving device 220-m presents vibration without presenting sound, the corresponding media information transmitter 226-m outputs corresponding media information I-m including any of information indicating "the receiving device 220-m presents both sound and vibration", information indicating "the receiving device 220-m presents sound without presenting vibration", and information indicating "the receiving device 220-m presents vibration without presenting sound". In addition, if there are four options, including the option that the receiving device 220-m does not present either sound or vibration, the compatible media information transmitting unit 226-m outputs compatible media information I-m that includes one of four pieces of information, the three pieces of information mentioned above plus information that "the receiving device 220-m does not present either sound or vibration."

When the information received by the video presentation ON/OFF switch is also input from the presentation signal selection unit 225, the compatible media information transmission unit 226-m outputs compatible media information I-m including the information received by the video presentation ON/OFF switch to the multipoint conference control unit 102. In other words, the compatible media information transmission unit 226-m outputs compatible media information I-m including either information indicating that "the receiving device 220-m presents video" or information indicating that "the receiving device 220-m does not present video" to the multipoint conference control unit 102.

The compatible media information transmission unit 226-m must output the compatible media information I-m at least when the terminal device 200-m is connected to the multipoint conference control device 102 and when the compatible media information I-m of the receiving device 220-m changes, but may also output the compatible media information I-m at other times. For example, the compatible media information transmission unit 226-m may output the compatible media information I-m at each predetermined time interval, or may output the compatible media information I-m at time intervals longer than the predetermined time interval.

[Multipoint Conference Control Device 102]
The multipoint conference control device 102 includes a conference signal generating unit 112, as shown in Fig. 17. The multipoint conference control device 102 receives corresponding media information I-1 to I-M output by each of the terminal devices 200-1 to 200-M connected via each of the transmission paths 300-1 to 300-M, and terminal transmission signals SA-1 to SA-M output by each of the terminal devices 200-1 to 200-M connected via each of the transmission paths 300-1 to 300-M. The multipoint conference control device 102 performs the process of step S112 shown in Fig. 18 for each predetermined time period.

[Conference signal generating unit 112]
Corresponding media information I-1 to IM and terminal transmission signals SA-1 to SA-M are input to the conference signal generation unit 112. The conference signal generation unit 112 performs the processes of steps S112-1 to S112-7 in Fig. 19, for example, to obtain and output terminal-directed signals SE-1 to SE-M (step S112).

The conference signal generating unit 112 generates a video signal for output including the video signal included in the video bit string of each point included in each of the terminal transmission signals SA-1 to SA-M (step S112-1). For example, similar to the multipoint conference system of Non-Patent Document 1, the conference signal generating unit 112 obtains a reduced image by reducing the size of the video of each point, generates one video including the reduced video of each point, and generates a video signal corresponding to the generated video. If there is a terminal transmission signal that does not include a video bit string in the terminal transmission signals SA-1 to SA-M, the conference signal generating unit 112 generates a video signal for output including the video signal of each point only from the terminal transmission signals that include a video bit string. After performing step S112-1, the conference signal generating unit 112 generates a video bit string that is a bit string of a predetermined video transmission method including the video signal obtained in step S112-1 (step S112-2). After performing step S112-2, the conference signal generating unit 112 also obtains a terminal-directed signal SE-G that includes the video bitstream generated in step S112-2 (step S112-6G).

The conference signal generating unit 112 generates an output acoustic signal including the acoustic signal included in the acoustic vibration bit string of each point included in each of the terminal transmission signals SA-1 to SA-M (step S112-3). The conference signal generating unit 112 may generate one acoustic signal by mixing the acoustic signals of each point, as in the multipoint conference system of Non-Patent Document 1, for example. If there is a terminal transmission signal in which the acoustic vibration bit string of the terminal transmission signals SA-1 to SA-M does not include an acoustic signal, the conference signal generating unit 112 may generate an output acoustic signal including the acoustic signal of each point only for the terminal transmission signal in which the acoustic vibration bit string includes an acoustic signal. If there are multiple channels for acoustic signals, the conference signal generating unit 112 may generate an output acoustic signal including the acoustic signal of each point for each channel.

The conference signal generating unit 112 also generates an output vibration signal including the vibration signal included in the acoustic vibration bit string of each point included in each of the terminal sending signals SA-1 to SA-M (step S112-4). The conference signal generating unit 112 may, for example, generate one vibration signal by mixing the vibration signals of each point. If there is a terminal sending signal that does not include a vibration signal in the acoustic vibration bit string of the terminal sending signals SA-1 to SA-M, the conference signal generating unit 112 may generate an output vibration signal including the vibration signal of each point only for terminal sending signals whose acoustic vibration bit string includes a vibration signal. If there are multiple channels for vibration signals, the conference signal generating unit 112 may generate an output vibration signal including the vibration signal of each point for each channel.

After performing steps S112-3 and S111-4, the conference signal generating unit 112 stores the acoustic signal obtained in step S112-3 in a predetermined channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, and stores the vibration signal obtained in step S112-4 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating an acoustic vibration bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S112-5A). After performing steps S112-2 and S112-5A, the conference signal generating unit 112 obtains a terminal-directed signal SE-A including the video bit string generated in step S112-2 and the acoustic vibration bit string generated in step S112-5A (step S112-6A). After performing step S112-5A, the conference signal generating unit 112 also obtains a terminal-directed signal SE-D that includes the acoustic vibration bit string generated in step S112-5A (step S112-6D).

After performing step S112-3, the conference signal generating unit 112 stores the acoustic signal obtained in step S112-3 in a predetermined channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, and stores the all-zero signal or the acoustic signal obtained in step S112-3 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method to generate an acoustic bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S112-5B). After performing steps S112-2 and S112-5B, the conference signal generating unit 112 obtains a terminal-directed signal SE-B including the video bit string generated in step S112-2 and the acoustic bit string generated in step S112-5B (step S112-6B). After performing step S112-5B, the conference signal generating unit 112 obtains a terminal-directed signal SE-E including the acoustic bit string generated in step S112-5B (step S112-6E). When storing an acoustic signal in a channel for vibration signals, if there are multiple predetermined channels for acoustic signals, the conference signal generating unit 112 may store the acoustic signal of any one of the channels in the channel for vibration signals, or may store a mixture of the acoustic signals of multiple channels in the channel for vibration signals.

After performing step S112-4, the conference signal generating unit 112 stores the all-zero signal or the vibration signal obtained in step S112-4 in a predetermined channel for the acoustic signal among the multiple channels of the predetermined acoustic transmission method, and stores the vibration signal obtained in step S112-4 in a predetermined channel for the vibration signal different from the channel for the acoustic signal among the multiple channels of the predetermined acoustic transmission method to generate a vibration bit string that is a bit string of the predetermined acoustic transmission method corresponding to the multiple channels (step S112-5C). After performing steps S112-2 and S112-5C, the conference signal generating unit 112 obtains a terminal-directed signal SE-C including the video bit string generated in step S112-2 and the vibration bit string generated in step S112-5C (step S112-6C). After performing step S112-5C, the conference signal generating unit 112 obtains a terminal-directed signal SE-F including the vibration bit string generated in step S112-5C (step S112-6F). When storing vibration signals in a channel for acoustic signals, if there are multiple predetermined channels for vibration signals, the conference signal generating unit 112 may store the vibration signal of any one of the channels in the channel for acoustic signals, or may store a mixture of the vibration signals of multiple channels in the channel for acoustic signals.

The conference signal generating unit 112 also obtains a terminal-directed signal SE-H that does not include a video bit string, an acoustic vibration bit string, an audio bit string, or a vibration bit string (step S112-6H). Note that even if the terminal-directed signal SE-H does not include a video bit string or an acoustic vibration bit string, the conference signal generating unit 112 needs to obtain the terminal-directed signal SE-H if the terminal-directed signal SE-H includes information that is neither video, sound, nor vibration. However, if there is no information that needs to be included in and sent in the terminal-directed signal SE-H, it is not necessary to obtain the terminal-directed signal SE-H, and it is not necessary to perform the process described below, i.e., the process of outputting the terminal-directed signal SE-H to the terminal device as the terminal-directed signal SE-m.

After performing steps S112-6A to S112-6H, the conference signal generation unit 112 obtains signals SE-1 to SE-M for each terminal in one of the following eight ways (A) to (H) and outputs them to each terminal device, that is, outputs them to each of the terminal devices 200-1 to 200-M via each of the transmission paths 300-1 to 300-M (step S112-7).

(A) When the corresponding media information Im includes information indicating that "the receiving device 220-m presents both sound and vibration" and information indicating that "the receiving device 220-m presents video," the conference signal generating unit 112 converts the terminal-directed signal SE-A into the terminal-directed signal SE-m.
(B) When the corresponding media information Im includes information indicating that "the receiving device 220-m presents sound without presenting vibration" and information indicating that "the receiving device 220-m presents video", the conference signal generating unit 112 converts the terminal-directed signal SE-B into the terminal-directed signal SE-m.
(C) When the corresponding media information I-m includes information indicating that "the receiving side device 220-m presents vibration without presenting sound" and information indicating that "the receiving side device 220-m presents video", the conference signal generating unit 112 converts the terminal-directed signal SE-C into the terminal-directed signal SE-m.
(D) When the corresponding media information I-m includes information indicating that "the receiving device 220-m presents both sound and vibration" and information indicating that "the receiving device 220-m does not present video," the conference signal generating unit 112 converts the terminal-directed signal SE-D into the terminal-directed signal SE-m.
(E) When the corresponding media information I-m includes information indicating that "the receiving device 220-m presents sound without presenting vibration" and information indicating that "the receiving device 220-m does not present video", the conference signal generating unit 112 sets the terminal-directed signal SE-E to the terminal-directed signal SE-m.
(F) When the corresponding media information Im includes information indicating that "the receiving device 220-m presents vibration without presenting sound" and information indicating that "the receiving device 220-m does not present video", the conference signal generating unit 112 converts the terminal-directed signal SE-F into the terminal-directed signal SE-m.
(G) When the corresponding media information Im includes information indicating that "the receiving device 220-m presents neither sound nor vibration" and information indicating that "the receiving device 220-m presents video", the conference signal generating unit 112 converts the terminal-directed signal SE-G into the terminal-directed signal SE-m.
(H) When the corresponding media information Im includes information indicating that "the receiving device 220-m does not present sound or vibration" and information indicating that "the receiving device 220-m does not present video", the conference signal generating unit 112 converts the terminal-directed signal SE-H into the terminal-directed signal SE-m.

Note that if none of the supported media information I-1 to I-M applies among the above (A) to (H), the conference signal generating unit 112 does not need to generate terminal-directed signals SE-A to SE-H that do not apply to any of the supported media information I-1 to I-M. In this case, the conference signal generating unit 112 does not need to perform unnecessary steps among steps S112-6A to S112-6H and the steps before them. Given that the terminal-directed signal that the conference signal generating unit 112 actually outputs is SE-m, the terminal-directed signals SE-A to SE-H are provisional terminal-directed signals within the conference signal generating unit 112, and the terminal-directed signals SE-A to SE-H can also be said to be "provisional terminal-directed signals SE-A to SE-H".

[The presentation signal generating unit 224-m, the video presentation unit 221-m, the audio presentation unit 222-m, and the vibration presentation unit 223-m of the receiving device 220-m]
The terminal-directed signal SE-m is input to the presentation signal generation unit 224-m. The presentation signal generation unit 224-m, the video presentation unit 221-m, the audio presentation unit 222-m, and the vibration presentation unit 223-m perform one of the following eight operations according to the selection received by the presentation signal selection unit 225.

(A) When the selection accepted by the presentation signal selection unit 225 is a selection to present sound, vibration, and video The presentation signal generation unit 224-m obtains a video signal from a video bit string, which is a bit string of a predetermined video transmission method included in the terminal-directed signal SE-m, and outputs it to the video presentation unit 221-m, obtains a signal of a channel for an audio signal from an audio vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as an audio signal to the audio presentation unit 222-m, and obtains a signal of a channel for a vibration signal from an audio vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as a vibration signal to the vibration presentation unit 223-m. The video presentation unit 221-m presents a video corresponding to the input video signal. The audio presentation unit 222-m presents a sound corresponding to the input audio signal. The vibration presentation unit 223-m presents a vibration corresponding to the input vibration signal.

(B) When the selection accepted by the presentation signal selection unit 225 is a selection to present a sound and not a vibration, the presentation signal generation unit 224-m obtains a video signal from a video bit string, which is a bit string of a predetermined video transmission method included in the terminal-directed signal SE-m, and outputs it to the video presentation unit 221-m, and obtains a signal of a channel for an audio signal from an audio bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as an audio signal to the audio presentation unit 222-m. The video presentation unit 221-m presents a video corresponding to the input video signal. The audio presentation unit 222-m presents a sound corresponding to the input audio signal. The presentation signal generation unit 224-m may obtain a signal of a channel for a vibration signal from the audio bit string and output it as an audio signal to the audio presentation unit 222-m. That is, the receiving side device 220-m may present a signal of a channel for a vibration signal included in the terminal-directed signal SE-m as a sound. Since no signal is input to the vibration presentation unit 223-m, the vibration presentation unit 223-m does not present any vibration.

(C) When the selection accepted by the presentation signal selection unit 225 is a selection to present a vibration and a video without presenting a sound The presentation signal generation unit 224-m obtains a video signal from a video bit string, which is a bit string of a predetermined video transmission method included in the terminal-directed signal SE-m, and outputs it to the video presentation unit 221-m, and obtains a signal of a channel for a vibration signal from a vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as a vibration signal to the vibration presentation unit 223-m. The video presentation unit 221-m presents a video corresponding to the input video signal. The vibration presentation unit 223-m presents a vibration corresponding to the input vibration signal. The presentation signal generation unit 224-m may obtain a signal of a channel for an audio signal from the vibration bit string and output it as a vibration signal to the vibration presentation unit 223-m. That is, the receiving side device 220-m may present a signal of a channel for an audio signal included in the terminal-directed signal SE-m as a vibration. Since no signal is input to the audio presentation unit 222-m, the audio presentation unit 222-m does not present any sound.

(D) When the selection accepted by the presentation signal selection unit 225 is a selection to present sound, vibration, and no video, the presentation signal generation unit 224-m obtains a signal of a channel for an audio signal from an audio vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as an audio signal to the audio presentation unit 222-m, and obtains a signal of a channel for a vibration signal from an audio vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs it as a vibration signal to the vibration presentation unit 223-m. The audio presentation unit 222-m presents a sound corresponding to the input audio signal. The vibration presentation unit 223-m presents a vibration corresponding to the input vibration signal. Since no signal is input to the video presentation unit 221-m, the video presentation unit 221-m does not present a video.

(E) When the selection accepted by the presentation signal selection unit 225 is a selection to present sound, not present vibration, and not present video The presentation signal generation unit 224-m obtains a signal of a channel for audio signals from an audio bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs the signal as an audio signal to the audio presentation unit 222-m. The audio presentation unit 222-m presents a sound corresponding to the input audio signal. The presentation signal generation unit 224-m may obtain a signal of a channel for vibration signals from the audio bit string and output the signal as an audio signal to the audio presentation unit 222-m. That is, the receiving device 220-m may present a signal of a channel for vibration signals included in the terminal-directed signal SE-m as sound. Since no signal is input to the video presentation unit 221-m, the video presentation unit 221-m does not present video. Since no signal is input to the vibration presentation unit 223-m, the vibration presentation unit 223-m does not present vibration.

(F) When the selection accepted by the presentation signal selection unit 225 is a selection to present vibration without presenting sound and without presenting video The presentation signal generation unit 224-m obtains a signal of a channel for vibration signals from a vibration bit string, which is a bit string of a predetermined audio transmission method included in the terminal-directed signal SE-m, and outputs the signal as a vibration signal to the vibration presentation unit 223-m. The vibration presentation unit 223-m presents a vibration corresponding to the input vibration signal. The presentation signal generation unit 224-m may obtain a signal of a channel for audio signals from the vibration bit string and output the signal as a vibration signal to the vibration presentation unit 223-m. That is, the receiving device 220-m may present a signal of a channel for audio signals included in the terminal-directed signal SE-m as vibration. Since no signal is input to the video presentation unit 221-m, the video presentation unit 221-m does not present video. Since no signal is input to the audio presentation unit 222-m, the audio presentation unit 222-m does not present sound.

(G) When the selection received by the presentation signal selection unit 225 is a selection to present a video without presenting a sound or vibration, the presentation signal generation unit 224-m obtains a video signal from a video bit string, which is a bit string of a predetermined video transmission method included in the terminal-directed signal SE-m, and outputs the video signal to the video presentation unit 221-m. The video presentation unit 221-m presents a video corresponding to the input video signal. Since no signal is input to the audio presentation unit 222-m, the audio presentation unit 222-m does not present a sound. Since no signal is input to the vibration presentation unit 223-m, the vibration presentation unit 223-m does not present a vibration.

(H) When the selection received by the presentation signal selection unit 225 is a selection of not presenting sound, not presenting vibration, and not presenting video, the presentation signal generation unit 224-m outputs no video signal, no audio signal, and no vibration signal. Since no signal is input to the video presentation unit 221-m, the video presentation unit 221-m does not present video. Since no signal is input to the audio presentation unit 222-m, the audio presentation unit 222-m does not present sound. Since no signal is input to the vibration presentation unit 223-m, the vibration presentation unit 223-m does not present vibration.

<Modifications 1 and 2 of the Second Embodiment>
The presentation signal selection unit 225 may include, for example, a presentation signal selection user interface area that is displayed on the screen of the video presentation unit 221-m of the terminal device 200-m, accepts a user's input, and displays the result. The user's input may be accepted by a well-known means such as a mouse provided on the terminal device 200-m or a touch operation on a display device that is a touch screen. An example of a case where a 2-channel stereo sound transmission method is used will be described as Variation 1, and an example of a case where a 5.1-channel surround sound transmission method is used will be described as Variation 2.

<Modification 1 of the second embodiment>
20A to 20C, the screen 2211-m of the video presentation unit 221-m includes a video display area 2212-m that is an area for displaying video, and a presentation signal selection user interface area 2251-m. The presentation signal selection user interface area 2251-m includes a media group switching button group 2252-m using a "+" button and a "-" button for cyclically switching between Fig. 20A, which is a display of a state in which sound is presented without presenting vibration, Fig. 20B, which is a display of a state in which both sound and vibration are presented, and Fig. 20C, which is a display of a state in which vibration is presented without presenting sound, and a media display group 2253-m that displays each media presented in each state.

When the "+" button is pressed while screen 2211-m is displaying FIG. 20A, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20B, and when the "+" button is pressed while screen 2211-m is displaying FIG. 20B, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20C, and when the "+" button is pressed while screen 2211-m is displaying FIG. 20C, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20A. When the "-" button is pressed while screen 2211-m is displaying FIG. 20A, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20C; when the "-" button is pressed while screen 2211-m is displaying FIG. 20C, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20B; and when the "-" button is pressed while screen 2211-m is displaying FIG. 20B, the presentation signal selection unit 225 causes screen 2211-m to display FIG. 20A.

The media group switching button group 2252-m, which consists of a "+" button and a "-" button, allows easy cyclic switching between three possible combinations of sound and vibration presentation in a terminal device of a multipoint conference system in which a channel originally prepared for an audio signal in a specified audio transmission method is substituted as a channel for a vibration signal. The media group switching button group 2252-m may have only the above-mentioned "+" button and not the above-mentioned "-" button, or may have only the above-mentioned "-" button and not the above-mentioned "+" button. If the media group switching button group 2252-m has only one button, it is not a "group," and can therefore be said to be a media group switching button 2252-m.

The media display group 2253-m is for displaying the selection result by the media group switching button group 2252-m. For example, the media display group 2253-m in FIG. 20A, which includes "Video", "Sound L", and "Sound R", is a display that allows visual confirmation that video is presented, sound is presented, and vibration is not presented. The display of "Sound L" and "Sound R" in FIG. 20A means that the audio presentation unit 222-m presents sound in two channels. In this case, for example, the presentation signal generation unit 224-m obtains a signal of a channel for an audio signal from the audio vibration bit string and outputs it to the audio presentation unit 222-m as an audio signal for the left channel, and obtains a signal of a channel for a vibration signal from the audio vibration bit string and outputs it to the audio presentation unit 222-m as an audio signal for the right channel, and the audio presentation unit 222-m presents sounds corresponding to the audio signals of the two input channels. As described above, when the receiving device 220-m presents sound but not vibration, the signal in the channel for the vibration signal is either an all-zero signal or the same acoustic signal as the signal in the channel for the acoustic signal, so there is no problem in presenting a sound corresponding to the signal in the channel for the vibration signal.

For example, the media display group 2253-m in Fig. 20B, which includes "Video", "Sound L", and "Vibration 1", is a display that allows visual confirmation that video, sound, and vibration are being presented. The display "Sound L" in Fig. 20B means that the audio presentation unit 222-m presents sound of one channel, and in this case, the presentation signal generation unit 224-m obtains a signal of a channel for the audio signal from the audio vibration bit string and outputs it as an audio signal to the audio presentation unit 222-m, and the audio presentation unit 222-m presents sound corresponding to the input audio signal of one channel. The indication "Vibration 1" in FIG. 20B means that the acoustic presentation unit 222-m presents vibration of one channel. In this case, the presentation signal generation unit 224-m obtains a signal of a channel for the vibration signal from the acoustic vibration bit string and outputs it as a vibration signal to the vibration presentation unit 223-m, and the vibration presentation unit 223-m presents vibration corresponding to the input vibration signal of one channel.

For example, the media display group 2253-m in Fig. 20C, which includes "video", "vibration 1" and "vibration 2", is a display that allows visual confirmation that video is presented, sound is not presented and vibration is presented. The displays of "vibration 1" and "vibration 2" in Fig. 20C mean that the vibration presentation unit 223-m presents vibrations of two channels. In this case, the presentation signal generation unit 224-m obtains a signal of a channel for vibration signals from the acoustic vibration bit string and outputs it to the vibration presentation unit 222-m as a vibration signal of a first channel, and obtains a signal of a channel for acoustic signals from the acoustic vibration bit string and outputs it to the vibration presentation unit 223-m as a vibration signal of a second channel, and the vibration presentation unit 223-m presents vibrations corresponding to the vibration signals of the two input channels. As described above, when the receiving device 220-m does not present sound but presents vibration, the signal in the channel for the acoustic signal is either an all-zero signal or the same vibration signal as the signal in the channel for the vibration signal, so there is no problem in presenting a vibration corresponding to the signal in the channel for the acoustic signal.

The media display group 2253-m may function as a button for switching whether or not to present each media or each channel in the selection result by the media group switching button group 2252-m. For example, the display of "Video" in the media display group 2253-m may function as a button for accepting input from the user as to whether or not the receiving device 220-m will present video. The same applies to "Sound L", "Sound R", "Vibration 1", and "Vibration 2". The displays of "Sound L" and "Sound R" in the media display group 2253-m may function as buttons for accepting input from the user as to whether or not the receiving device 220-m will present sound, and the displays of "Vibration 1" and "Vibration 2" in the media display group 2253-m may function as buttons for accepting input from the user as to whether or not the receiving device 220-m will present vibration. Also, for example, the displays "Sound L" and "Sound R" in the media display group 2253-m may function as buttons that accept user input regarding whether or not the receiving device 220-m will present sound on each channel. Similarly, the displays "Vibration 1" and "Vibration 2" in the media display group 2253-m may function as buttons that accept user input regarding whether or not the receiving device 220-m will present vibration on each channel. When the media display group 2253-m is made to function as buttons for switching whether or not each media or each channel is presented, it is preferable to display whether or not a video is presented, whether or not a sound is presented, or whether or not a vibration is presented by the color and/or brightness of each button. Note that when the media display group 2253-m functions as a button for switching whether or not each media or each channel is presented, it can be said to be a media ON/OFF button group 2253-m.

<Modification 2 of the Second Embodiment>
The presentation signal selection user interface area 2251-m may include a media group switching button group 2252-m or a media group switching button 2252-m, and a media display group 2253-m or a media ON/OFF button group 2253-m so that it functions in the same way as variant example 1 when a 5.1 channel surround audio transmission method is used and when a 2 channel stereo audio transmission method is used.

For example, the media group switching button group 2252-m shown in Figures 21A to 21C allows easy and cyclic switching between three possible combinations of sound and vibration presentation and, when vibration is presented, which of the four vibration signal channels is to be presented. For example, when the media group switching button group 2252-m receives the press of the "+" button while the screen 2211-m is displaying FIG. 21A, it displays a screen with "vibration 1" added to the media display group 2253-m of FIG. 21A, and when the press of the "+" button is received while the screen is displaying the screen with "vibration 1" added to the media display group 2253-m of FIG. 21A, it displays FIG. 21B. When the press of the "+" button is received while the screen 2211-m is displaying FIG. 21B, it displays a screen with "vibration 3" added to the media display group 2253-m of FIG. 21B, and when the press of the "+" button is received while the screen is displaying the screen with "vibration 3" added to the media display group 2253-m of FIG. 21B, it displays FIG. 21C. The media display group 2253-m illustrated in Figures 21A to 21C is for displaying the results of selection made by the media group switching button group 2252-m, and is similar to the media display group 2253-m in Modification Example 1. In other words, the media display group 2253-m may be made to function as a button for switching whether or not to present each media or each channel, and may be made into the media ON/OFF button group 2253-m.

Third Embodiment
A multipoint conference system that functions similarly to the first embodiment can be realized by providing a signal conversion device between the multipoint conference control device and each terminal device, and using a multipoint conference control device that obtains and outputs one type of terminal-oriented signal, similar to the multipoint conference system 10 of the reference embodiment. This embodiment will be referred to as the third embodiment, and the differences from the first embodiment will be mainly described.

As illustrated in FIG. 22, the multipoint conference system 13 of the third embodiment includes a multipoint conference control device 103, a tactile-enabled terminal device 200-m, a non-tactile-enabled terminal device 400-n, and a signal conversion device 603-n. In the third embodiment, M is an integer of 1 or more, m is an integer of 1 to M, N is an integer of 1 or more, and n is an integer of M+1 to M+N. The multipoint conference control device 103 and each tactile-enabled terminal device 200-m are connected via each transmission path 300-m of the communication network. The multipoint conference control device 103 and each signal conversion device 603-n are connected via each transmission path 501-n of the communication network. Each signal conversion device 603-n and each non-tactile-enabled terminal device 400-n are connected via each transmission path 502-n. Each transmission path 502-n does not necessarily have to be a transmission path of a communication network, and may be a transmission path such as a LAN that connects a signal conversion device 603-n and a non-tactile terminal device 400-n located at the same point n, or may be a path that transmits a signal inside a device in which the signal conversion device 603-n and the non-tactile terminal device 400-n are integrated. The multipoint conference control device 103 and each non-tactile terminal device 500-n may be connected via each transmission path 500-n of a communication network, as shown by the dashed lines in FIG. 22.

[Tactile-enabled terminal device 200-m]
The haptic-compatible terminal device 200-m is the same as the haptic-compatible terminal device 200-m of the first embodiment. However, the haptic-compatible transmitting device 210-m outputs a haptic-compatible terminal sending signal SA-m to the multipoint conference control device 103 via a transmission path 300-m. Also, a haptic-compatible terminal-directed signal SB is input to the haptic-compatible receiving device 220-n from the multipoint conference control device 103 via the transmission path 300-m.

[Haptic-non-compatible terminal device 400-n]
The haptic non-compatible terminal device 400-n is the same as the haptic non-compatible terminal device 400-n of the first embodiment. However, the haptic non-compatible transmitting side device 410-n outputs a haptic non-compatible terminal sending signal SC-n to the multipoint conference control device 103 via the transmission path 502-n, the signal conversion device 603-n, and the transmission path 501-n, or outputs it to the multipoint conference control device 103 via the transmission path 500-n without via the signal conversion device 603-n. Also, the haptic non-compatible receiving side device 420-n receives a haptic non-compatible terminal-directed signal SD from the signal conversion device 603-n via the transmission path 502-n.

[Multipoint Conference Control Device 103]
The multipoint conference control device 103 includes a conference signal generating unit 113, as shown in Fig. 23. The multipoint conference control device 103 receives the tactile-compatible terminal sending signals SA-1 to SA-M output from the tactile-compatible terminal devices 200-1 to 200-M, and the non-tactile-compatible terminal sending signals SC-(M+1) to SC-(M+N) output from the non-tactile-compatible terminal devices 400-(M+1) to 400-(M+N). The multipoint conference control device 103 obtains and outputs a tactile-compatible terminal-directed signal SB by performing the process of step S113 shown in Fig. 24 for each predetermined time period.

[Conference signal generating unit 113]
The conference signal generating unit 113 receives haptic-compatible terminal sending signals SA-1 to SA-M and non-haptic-compatible terminal sending signals SC-(M+1) to SC-(M+N). The conference signal generating unit 113 obtains and outputs a haptic-compatible terminal-oriented signal SB from the input haptic-compatible terminal sending signals SA-1 to SA-M and non-haptic-compatible terminal sending signals SC-(M+1) to SC-(M+N) (step S113). The process of the conference signal generating unit 113 obtaining a haptic-compatible terminal-oriented signal SB from the input haptic-compatible terminal sending signals SA-1 to SA-M and non-haptic-compatible terminal sending signals SC-(M+1) to SC-(M+N) is the same as that of the conference signal generating unit 111 of the first embodiment. The conference signal generating unit 113 does not obtain or output a non-haptic-compatible terminal-oriented signal SD. That is, the conference signal generating unit 113 performs steps S111-1 to S111-6, which are the same as those of the conference signal generating unit 111 of the first embodiment, but does not perform steps S111-7 and S111-8, which are performed by the conference signal generating unit 111 of the first embodiment. Also, the conference signal generating unit 113 outputs the haptic-compatible terminal-oriented signal SB to each haptic-compatible terminal device and each signal conversion device. That is, the conference signal generating unit 113 outputs the haptic-compatible terminal-oriented signal SB to each of the haptic-compatible terminal devices 200-1 to 200-M via each of the transmission paths 300-1 to 300-M, and to each of the signal conversion devices 603-(M+1) to 603-(M+N) via each of the transmission paths 501-(M+1) to 501-(M+N).

[Signal conversion device 603-n]
The signal converter 603-n includes a conference signal converter 613-n as shown in Fig. 25. The signal converter 603-n receives a signal SB for a tactile-sense-compatible terminal output by the multipoint conference control device 103 connected via the transmission path 501-n. The signal converter 603-n performs the process of step S613 shown in Fig. 26 for each predetermined time period.

The conference signal conversion unit 613-n receives the signal SB for the tactile-enabled terminal. The conference signal conversion unit 613-n performs the processes shown in steps S613-1 to S613-4 in FIG. 27, for example, to obtain and output the signal SD for the non-tactile-enabled terminal from the input signal SB for the tactile-enabled terminal (step S613).

The conference signal conversion unit 613-n extracts an acoustic signal contained in the acoustic vibration bit string contained in the haptic terminal-oriented signal SB (step S613-1). After performing step S613-1, the conference signal conversion unit 613-n stores the acoustic signal obtained in step 613-1 in a predetermined channel for acoustic signals among the multiple channels of a predetermined acoustic transmission method, and stores an all-zero signal or the acoustic signal obtained in step S613-1 in a predetermined channel for vibration signals that is different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating an acoustic bit string that is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels (step S613-2).

The conference signal conversion unit 613-n extracts the video bit string contained in the haptic-compatible terminal-oriented signal SB (step S613-3). After performing steps S613-2 and S613-3, the conference signal conversion unit 613-n obtains a haptic-non-compatible terminal-oriented signal SD containing the audio bit string obtained in step 613-2 and the video bit string obtained in step 613-3, and outputs it to the corresponding haptic-non-compatible terminal device, i.e., outputs it via the transmission path 502-n to the haptic-non-compatible terminal device 400-n connected via the transmission path 502-n (step S613-4).

In addition, for convenience of connection to the transmission path, as shown by the dashed line in Figure 25, the signal conversion device 603-n may be configured to input a non-tactile terminal transmission signal SC-n from a non-tactile terminal device 400-n connected via a transmission path 502-n, and the input non-tactile terminal transmission signal SC-n may be output directly to the multipoint conference control device 103 connected via the transmission path 501-n. If signal conversion device 603-n does not input or output non-tactile-capable terminal transmission signal SC-n, signal conversion devices 603-(M+1) to 603-(M+N) all perform the same processing, so multipoint conference system 13 does not need to have signal conversion devices 603-(M+1) to 603-(M+N) corresponding to each of tactile-capable non-terminal devices 400-(M+1) to 400-(M+N), and may have one signal conversion device for multiple tactile-capable non-terminal devices, or may have only one signal conversion device.

Fourth Embodiment
A multipoint conference system that functions similarly to the second embodiment can be realized by providing a signal conversion device between the multipoint conference control device and each terminal device, and using a multipoint conference control device that obtains and outputs one type of terminal-oriented signal, similar to the multipoint conference system 10 of the reference embodiment. This embodiment will be described as the fourth embodiment, focusing on the differences from the second embodiment and also using the differences and similarities with the reference embodiment as appropriate.

As shown in FIG. 28, the multipoint conference system 14 of the fourth embodiment includes a multipoint conference control device 100, a terminal device 200-m, and a signal conversion device 604-m. In the fourth embodiment, M is an integer of 2 or more, and m is an integer of 1 to M. The multipoint conference control device 100 and each signal conversion device 604-m are connected via each transmission path 301-m of the communication network. Each signal conversion device 604-m and each terminal device 200-m are connected via each transmission path 302-m. Each transmission path 302-m does not necessarily have to be a transmission path of the communication network, and may be a transmission path such as a LAN that connects the signal conversion device 604-m and the terminal device 200-m located at the same point m, or may be a path that transmits a signal inside a device in which the signal conversion device 604-m and the terminal device 200-m are integrated. The multipoint conference control device 100 and each terminal device 200-m may be connected via each transmission path 300-m of the communication network, as shown by the dashed lines in FIG. 28.

[Terminal device 200-m]
The terminal device 200-m is the same as the terminal device 200-m of the second embodiment. However, the terminal device 200-m outputs a terminal sending signal SA-m to the multipoint conference control device 100 via the transmission path 302-m, the signal conversion device 604-m, and the transmission path 301-m, or outputs the terminal sending signal SA-m to the multipoint conference control device 100 via the transmission path 300-m without via the signal conversion device 604-m. Also, the terminal device 200-m outputs corresponding media information I-m to the signal conversion device 604-m via the transmission path 302-m. Also, the terminal device 200-m receives a terminal-directed signal SE-m from the signal conversion device 604-m via the transmission path 302-m.

[Multipoint conference control device 100]
The multipoint conference control device 100 is the same as the multipoint conference control device 100 of the reference embodiment. However, the multipoint conference control device 100 outputs a terminal-directed signal SB to each of the signal conversion devices 604-1 to 604-M via each of the transmission paths 301-1 to 301-M. That is, in this embodiment, the "terminal-directed signal SB" is not a signal directed to a terminal device but a signal directed to a signal conversion device, and therefore can also be said to be a "signal SB directed to a signal conversion device."

[Signal conversion device 604-m]
The signal conversion device 604-m includes a conference signal conversion unit 614-m as shown in Fig. 29. The signal conversion device 604-m receives the terminal-directed signal SB output by the multipoint conference control device 100 and the corresponding media information I-m output by the terminal device 200-m. The signal conversion device 604-m obtains the terminal-directed signal SE-m by performing the process of step S614 shown in Fig. 18 for each predetermined time period, and outputs the signal to the terminal device 200-m, that is, outputs the signal to the terminal device 200-m via each transmission path 301-m.

The conference signal conversion unit 614-m receives the terminal-directed signal SB and the corresponding media information I-m. The conference signal conversion unit 614-m obtains and outputs the terminal-directed signal SE-m from the input terminal-directed signal SB by performing any one of the processes in steps S614-A to S614-H below (step S614).

(A) If the corresponding media information I-m includes information indicating that "the receiving device 220-m presents both sound and vibration" and information indicating that "the receiving device 220-m presents video", the conference signal conversion unit 614-m outputs the terminal-directed signal SB as the terminal-directed signal SE-m (step S614-A).

(B) If the corresponding media information I-m includes information indicating that "the receiving device 220-m presents sound without vibration" and information indicating that "the receiving device 220-m presents video", the conference signal conversion unit 614-m performs steps S614-B1 to S614-B3 below to obtain and output a terminal-directed signal SE-m (step S614-B).

The conference signal conversion unit 614-m obtains an acoustic signal, which is a signal for an acoustic signal channel, from the acoustic vibration bit string included in the terminal-directed signal SB (step S614-B1). After performing step S614-B1, the conference signal conversion unit 614-m stores the acoustic signal obtained in step S614-B1 in a predetermined channel for acoustic signals among the multiple channels of a predetermined acoustic transmission method, and generates an acoustic bit string, which is a bit string of a predetermined acoustic transmission method corresponding to the multiple channels, by storing an all-zero signal or the acoustic signal obtained in step S614-B1 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method (step S614-B2). After performing step S614-B2, the conference signal conversion unit 614-m obtains and outputs a terminal-directed signal SE-m including the video bit string included in the terminal-directed signal SB and the acoustic bit string obtained in step S614-B2 (step S614-B3).

(C) If the corresponding media information I-m includes information indicating that "the receiving device 220-m presents vibration without presenting sound" and information indicating that "the receiving device 220-m presents video", the conference signal conversion unit 614-m performs steps S614-C1 to S614-C3 below to obtain and output a terminal-directed signal SE-m (step S614-C).

The conference signal conversion unit 614-m obtains a vibration signal, which is a signal for a channel for vibration signals, from the acoustic vibration bit string included in the terminal-directed signal SB (step S614-C1). After performing step S614-C1, the conference signal conversion unit 614-m stores the all-zero signal or the vibration signal obtained in step S614-C1 in a predetermined channel for acoustic signals among the multiple channels of a predetermined acoustic transmission method, and stores the vibration signal obtained in step S614-C1 in a predetermined channel for vibration signals different from the channel for acoustic signals among the multiple channels of the predetermined acoustic transmission method, thereby generating a vibration bit string, which is a bit string for a predetermined acoustic transmission method corresponding to the multiple channels (step S614-C2). After performing step S614-C2, the conference signal conversion unit 614-1 obtains and outputs a terminal-directed signal SE-m including the video bit string included in the terminal-directed signal SB and the vibration bit string obtained in step S614-C2 (step S614-C3).

(D) If the corresponding media information I-m includes information that "the receiving device 220-m presents both sound and vibration" and information that "the receiving device 220-m does not present video", the conference signal conversion unit 614-m obtains and outputs a terminal-directed signal SE-m that does not include a video bit string but includes the acoustic vibration bit string included in the terminal-directed signal SB (step S614-D).

(E) If the corresponding media information I-m includes information that "the receiving device 220-m presents sound without vibration" and information that "the receiving device 220-m does not present video", the conference signal conversion unit 614-m performs the above-mentioned steps S614-B1 and S614-B2, and after performing step 614-B2, obtains and outputs a terminal-directed signal SE-m that does not include a video bitstream but includes the audio bitstream obtained in step S614-B2 (step S614-E).

(F) If the corresponding media information I-m includes information that "the receiving device 220-m does not present sound but presents vibration" and information that "the receiving device 220-m does not present video", the conference signal conversion unit 614-m performs the above-mentioned steps S614-C1 and S614-C2, and after performing step 614-C2, obtains and outputs a terminal-directed signal SE-m that does not include a video bit string but includes the vibration bit string obtained in step S614-C2 (step S614-F).

(G) If the corresponding media information I-m includes information indicating that "the receiving device 220-m presents neither sound nor vibration" and information indicating that "the receiving device 220-m presents video", the conference signal conversion unit 614-m obtains and outputs a terminal-directed signal SE-m that does not include an acoustic vibration bit string, an audio bit string, or a vibration bit string, but includes a video bit string included in the terminal-directed signal SB (step S614-G).

(H) When the corresponding media information I-m includes information that "the receiving device 220-m does not present sound or vibration" and information that "the receiving device 220-m does not present video", the conference signal conversion unit 614-m obtains and outputs a terminal-directed signal SE-m that does not include a video bit string, an acoustic vibration bit string, an audio bit string, or a vibration bit string (step S614-H). Note that even if the terminal-directed signal SE-m does not include a video bit string, an acoustic vibration bit string, an audio bit string, or a vibration bit string, the conference signal conversion unit 614-m needs to obtain and output the terminal-directed signal SE-m if the terminal-directed signal SE-m includes information that is not video, sound, or vibration, but if there is no information that needs to be included in and sent in the terminal-directed signal SE-m, it is not necessary to obtain and output the terminal-directed signal SE-m.

<Program and recording medium>
The processing of each of the above-mentioned multipoint conference control devices, each terminal device, and each signal conversion device may be realized by a computer, in which case the processing contents of the functions that each device should have are described by a program. Then, by loading this program into the storage unit 1020 of the computer 1000 shown in Fig. 30 and operating the arithmetic processing unit 1010, input unit 1030, output unit 1040, etc., various processing functions of the above-mentioned devices are realized on the computer.

The program describing this processing can be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a non-transitory recording medium, specifically, a magnetic recording device, an optical disk, etc.

The program may be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from a server computer in its own non-transient storage device, the auxiliary recording unit 1050. Then, when executing the process, the computer reads the program stored in its own non-transient storage device, the auxiliary recording unit 1050, into the storage unit 1020, and executes the process according to the read program. In addition, as another execution form of this program, the computer may read the program directly from the portable recording medium into the storage unit 1020 and execute the process according to the program, or, further, each time a program is transferred from the server computer to this computer, the computer may execute the process according to the received program one by one. In addition, the server computer may not transfer the program to this computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes the processing function only by issuing an execution instruction and obtaining the result. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct command to a computer but has properties that dictate computer processing).

In addition, in this embodiment, the device is configured by executing a specific program on a computer, but at least a portion of the processing may be realized by hardware.

Needless to say, other modifications are possible without departing from the spirit of this invention.

Claims (17)

A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the acoustic signal, and no signal or an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the vibration signal, as a signal for a terminal device among the terminal devices that can present sound but cannot present vibration. A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which a vibration signal input from the terminal device via the communication path is set as a signal on a channel for the vibration signal, and no signal or a vibration signal input from the terminal device via the communication path is set as a signal on the channel for the acoustic signal, as a signal for a terminal device among the terminal devices that can present vibration but cannot present sound. A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a signal of the acoustic transmission method is obtained, as a signal for a terminal device among the terminal devices that can present sound but cannot present vibration, in which an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the acoustic signal, and no signal or an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the vibration signal;
a signal of the acoustic transmission method is obtained, as a signal for a terminal device among the terminal devices that can present vibration but cannot present sound, in which a vibration signal input from the terminal device through the communication path is set as a signal of a channel for the vibration signal, and no signal or a vibration signal input from the terminal device through the communication path is set as a signal of the channel for the acoustic signal;
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which an acoustic signal input from the terminal device via the communication path is made a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is made a signal of a channel for the vibration signal, as a signal for a terminal device among the terminal devices that can present both sound and vibration. A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the acoustic signal, and no signal or an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the vibration signal, as a signal for a terminal device among the terminal devices that presents sound but not vibration. A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which a vibration signal input from the terminal device via the communication path is set as a signal for the channel for the vibration signal, and no signal or a vibration signal input from the terminal device via the communication path is set as a signal for the channel for the acoustic signal, as a signal for a terminal device among the terminal devices that presents vibration but does not present sound. A multipoint conference control device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
a signal of the acoustic transmission method is obtained, as a signal for a terminal device among the terminal devices that presents sound but does not present vibration, in which an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the acoustic signal, and no signal or an acoustic signal input from the terminal device via the communication path is set as a signal for the channel for the vibration signal;
a signal of the acoustic transmission method is obtained, as a signal for a terminal device among the terminal devices that presents vibration but does not present sound, in which a vibration signal input from the terminal device via the communication path is set as a signal on a channel for the vibration signal, and no signal or a vibration signal input from the terminal device via the communication path is set as a signal on a channel for the acoustic signal;
a conference signal generation unit for obtaining a signal of the acoustic transmission method, in which an acoustic signal input from the terminal device via the communication path is made a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is made a signal of a channel for the vibration signal, as a signal for a terminal device that presents both sound and vibration among the terminal devices. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
the multipoint conference control device receives and outputs a signal of the acoustic transmission method in which an acoustic signal input from the terminal device via the communication path is treated as a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal of a channel for the vibration signal,
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the acoustic signal, and no signal or an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the vibration signal, as a signal for a terminal device among the terminal devices that can present sound but cannot present vibration. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
the multipoint conference control device receives and outputs a signal of the acoustic transmission method in which an acoustic signal input from the terminal device via the communication path is treated as a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal of a channel for the vibration signal,
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is used as a signal for a terminal device among the terminal devices that can present vibration but cannot present sound, and a no signal or a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is used as a signal for the channel for the acoustic signal. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference control device receives and outputs signals of an acoustic transmission method in which an acoustic signal input from the terminal device via the communication path is treated as a signal on a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal on a channel for the vibration signal,
the multipoint conference system uses one or more of the multiple channels of the acoustic transmission method as channels for acoustic signals, and uses one or more of the multiple channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
obtaining a signal of the acoustic transmission method as a signal for a terminal device among the terminal devices that can present sound but cannot present vibration, the signal being an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device as a signal on a channel for the acoustic signal, and no signal or an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device as a signal on a channel for the vibration signal;
obtaining a signal of the acoustic transmission method, as a signal for a terminal device among the terminal devices that can present vibration but cannot present sound, in which a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the vibration signal, and no signal or a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the acoustic signal;
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is converted into a signal on a channel for the acoustic signal, and a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is converted into a signal on a channel for the vibration signal, as a signal for a terminal device among the terminal devices that can present both sound and vibration. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
the multipoint conference control device receives and outputs a signal of the acoustic transmission method in which an acoustic signal input from the terminal device via the communication path is treated as a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal of a channel for the vibration signal,
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the acoustic signal, and no signal or an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the vibration signal, as a signal for a terminal device among the terminal devices that presents sound but does not present vibration. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
the multipoint conference control device receives and outputs a signal of the acoustic transmission method in which an acoustic signal input from the terminal device via the communication path is treated as a signal of a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal of a channel for the vibration signal,
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the vibration signal, and no signal or a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is made a signal on a channel for the acoustic signal, as a signal for a terminal device among the terminal devices that presents vibration but does not present sound. In a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path, a signal conversion device is disposed between the terminal devices and the multipoint conference control device,
the multipoint conference control device receives and outputs signals of an acoustic transmission system in which an acoustic signal input from the terminal device via the communication path is treated as a signal on a channel for the acoustic signal, and a vibration signal input from the terminal device via the communication path is treated as a signal on a channel for the vibration signal,
the multipoint conference system uses one or more of the multiple channels of the acoustic transmission method as channels for acoustic signals, and uses one or more of the multiple channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
obtaining a signal of the acoustic transmission method, as a signal for a terminal device among the terminal devices that presents sound but not vibration, in which an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the acoustic signal, and no signal or an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the vibration signal;
obtaining a signal of the acoustic transmission method, as a signal for a terminal device among the terminal devices that presents vibration but does not present sound, in which a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the vibration signal, and no signal or a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is set as a signal on a channel for the acoustic signal;
a conference signal conversion unit that obtains a signal of the acoustic transmission method, in which an acoustic signal included in the signal of the acoustic transmission method output by the multipoint conference control device is converted into a signal on a channel for the acoustic signal, and a vibration signal included in the signal of the acoustic transmission method output by the multipoint conference control device is converted into a signal on a channel for the vibration signal, as a signal for a terminal device that presents both sound and vibration among the terminal devices. A terminal device included in a multipoint conference system including a plurality of terminal devices and a multipoint conference control device connected to each of the terminal devices via a communication path,
the multipoint conference system uses one or more channels of a plurality of channels of a predetermined acoustic transmission method as channels for acoustic signals, and uses one or more channels of the plurality of channels of the acoustic transmission method different from the channels for the acoustic signals as channels for vibration signals,
Equipped with a display screen,
On the display screen,
a media group switching button which, when pressed, performs a selection by cyclic state switching including at least three states: a state in which sound is presented without vibration, a state in which both sound and vibration are presented, and a state in which vibration is presented without sound;
a media display group that, when one of the states is selected by pressing the media group switching button, displays in a visually recognizable manner whether the state is a state in which sound is presented without vibration, a state in which both sound and vibration are presented, or a state in which vibration is presented without sound;
A terminal device comprising: a display unit configured to display a presentation signal selection user interface area including: The terminal device according to claim 13,
The media display group includes:
The color and/or brightness of a button (hereinafter referred to as the "sound button") indicates whether or not a sound is to be presented,
The button (hereinafter, referred to as the "vibration button") indicates whether or not the device is in a state in which vibration is to be presented by changing the color and/or brightness of the button.
The terminal device
When the sound button is pressed, a switch is made as to whether or not to present a sound;
A terminal device, characterized in that, when the vibration button is pressed, a switch is made as to whether or not vibration is to be presented. A program for causing a computer to function as a multipoint conference control device according to any one of claims 1 to 6. A program for causing a computer to function as a signal conversion device according to any one of claims 7 to 12. A program for causing a computer to function as the terminal device according to claim 13 or 14.

Images