JP7647017B2 - COMMUNICATION MANAGEMENT SYSTEM, COMMUNICATION ...MANAGEMENT DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a communication management system, a communication system, a communication management device, an image processing method, and a program.

Conference systems that hold conferences between multiple terminal devices via a communication network such as the Internet are becoming widespread. In such conference systems, a user may share stroke images by sending and receiving stroke data between terminal devices, which is used to reproduce stroke images drawn by a user with an electronic pen or the user's hand, using a terminal device such as an electronic whiteboard.

In addition, in such a conference system, a technique is known in which information regarding operations by users at other locations is displayed on a display or the like so that users at each location can know in advance the operations by users at other locations (see, for example, Patent Document 1).

In a conference where stroke images are shared between multiple terminal devices, if operations are performed on the shared stroke images from multiple terminal devices, there is a problem that operations on the stroke images may be performed differently than intended by the user depending on the timing of the operations.

Note that this problem is not limited to conference systems that hold conferences, but is also common to various communication systems in which sessions for sharing stroke images are held, such as remote classes and remote medical treatments.

The embodiment of the present invention has been made in consideration of the above problems, and aims to prevent operations from being performed on a stroke image from multiple terminal devices that are different from the user's intention, in a session in which stroke images are shared between multiple terminal devices.

In order to solve the above problem, a communication management system according to an embodiment of the present invention is a communication management system that manages a session for sharing a stroke image between a plurality of terminal devices including a first terminal device and a second terminal device, and includes: a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a receiving unit that receives, from the first terminal device, a group operation request that specifies one or more stroke data to be a target of an editing operation among the plurality of stroke data; and an operation limiting unit that limits an editing operation on the one or more stroke data by the second terminal device different from the first terminal device, based on the group operation request that specifies the one or more stroke data to be a target of the editing operation, When the one or more stroke data specified by the group operation request does not include stroke data for which editing operations are restricted by operation restriction processing , the restriction unit starts the operation restriction processing, thereby making it possible to accept editing operations on the one or more stroke data from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device ; and when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are restricted by the operation restriction processing, the restriction unit does not make editing operations on the one or more stroke data acceptable from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device.

According to one embodiment of the present invention, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from multiple terminal devices from performing operations on a stroke image that are different from the user's intention.

FIG. 2 is a schematic diagram illustrating an example of a communication route in a communication system according to an embodiment. FIG. 1 is a diagram for explaining an image of how an electronic whiteboard is used according to an embodiment; FIG. 2 is a diagram illustrating an example of a hardware configuration of an electronic whiteboard according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. FIG. 1 is a diagram illustrating an example of a system configuration of a communication system according to an embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration of a communication system according to an embodiment. FIG. 13 is a diagram illustrating an example of an authentication management table according to an embodiment. FIG. 11 is a diagram illustrating an example of a terminal management table according to an embodiment. FIG. 13 is a diagram illustrating an example of a destination list management table according to an embodiment. FIG. 13 is a diagram illustrating an example of a session management table according to an embodiment. FIG. 13 is a diagram illustrating an example of a relay device management table according to an embodiment. FIG. 11 is a diagram illustrating stroke information according to an embodiment. FIG. 11 is a sequence diagram illustrating an example of a preparatory process according to an embodiment. FIG. 11 is a sequence diagram illustrating an example of a session start process according to an embodiment. FIG. 11 is a sequence diagram illustrating an example of a stroke image sharing process according to an embodiment. FIG. 11 is a sequence diagram illustrating an example of a stroke image moving process according to an embodiment. FIG. 13 is a diagram illustrating an example of a display screen according to an embodiment. FIG. 11 is a sequence diagram (1) illustrating an example of an operation restriction process according to the first embodiment. FIG. 11 is a diagram illustrating an example of group operation information according to an embodiment. FIG. 13 is a sequence diagram (2) illustrating an example of the operation restriction process according to the first embodiment. FIG. 11 is a sequence diagram illustrating an example of an operation restriction process according to the second embodiment. FIG. 11 is a diagram illustrating an example of a display screen according to the second embodiment. FIG. 13 is a sequence diagram illustrating an example of an operation restriction process according to the third embodiment. FIG. 13 is a diagram illustrating an example of a display screen according to the third embodiment. FIG. 11 is a diagram illustrating another example of a usage scene according to an embodiment.

The following describes in detail an embodiment of the present invention with reference to the drawings.

[Outline of the communication system]
<Communication route>
First, a communication system 100 for conducting a video conference while drawing on a plurality of electronic whiteboards 1a and 1b will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing communication routes of the present embodiment. Also, a "video conference" may be called a "television conference" or a "remote conference", etc.

Note that a "video conference," "television conference," or "remote conference" is an example of a session in which stroke images are shared among multiple terminal devices. For example, a session in which stroke images are shared may be a session other than a conference, such as a remote lesson, a remote medical examination, a meeting, or a simple conversation.

The communication system 100 is constructed, for example, from multiple electronic whiteboards 1a, 1b, a relay device 3, a communication management system 5, and an image storage device 7.

The electronic whiteboards (terminal devices) 1a and 1b are terminal devices that communicate with each other for content data such as video data and audio data for calls, image data for sharing, and stroke data, and are also called IWBs (Interactive White Boards).

Note that the communication system 100 is not limited to having two electronic whiteboards 1a and 1b, and may have three or more electronic whiteboards. In the following description, the term "electronic whiteboard 1" will be used to refer to any one of the multiple electronic whiteboards included in the communication system 100.

The electronic whiteboard 1 is an example of a terminal device according to this embodiment. The terminal device may be another terminal device having a communication function, a drawing function, a display function, etc. For example, the terminal device may be an information terminal such as a PC (Personal Computer), a tablet terminal, or a smartphone on which an application program corresponding to the communication system 100 is installed.

Stroke data is data necessary to reproduce (reproduce) a stroke image, and includes coordinate data, line width data, line color data, vector data, etc. In addition, electronic whiteboards 1a and 1b can transmit and receive video data and audio data for a call, reproducing the video and audio of the other party, enabling video calls.

The electronic whiteboards 1a and 1b transmit and receive image data of shared document images, allowing users of the communication system 100 to share the same document images. The document images are images displayed on the display of the electronic whiteboard 1, and may include, for example, meeting documents, background images displayed on the display, and images of captured screens when the display screen is captured.

The electronic whiteboards 1a and 1b also transmit and receive stroke data of a shared stroke image, allowing participants using the communication system to share the same stroke image. A stroke image is an image showing lines or the like drawn by a user using handwritten strokes with an electronic pen or the like. A stroke image is displayed using stroke data that indicates points that specify coordinates on the display.

In FIG. 1, an electronic whiteboard equipped with a video conferencing function is shown as an example of electronic whiteboards 1a and 1b. Note that the images of the video data may be either moving images or still images.

In the following description, the electronic whiteboard 1 that requests the start of a videoconference is called the "start terminal," and the electronic whiteboard 1 that is the destination (relay destination) of the request is called the "destination terminal." For example, in FIG. 1, when electronic whiteboard 1a requests electronic whiteboard 1b to start a videoconference, electronic whiteboard 1a is the start terminal and electronic whiteboard 1b is the destination terminal. Note that each electronic whiteboard 1b, 1b may be used not only for communication between multiple business offices or between different rooms in the same business office, but also for communication within the same room, or communication between outdoors and indoors, or outdoors and outdoors.

The relay device 3 is configured as a computer and performs the process of relaying content data for calls between multiple electronic whiteboards 1a and 1b.

The communication management system 5 is configured by a computer and centrally manages the authentication of logins from the electronic whiteboards 1a and 1b, the management of the communication status of the electronic whiteboards 1a and 1b, the management of the destination list, and the communication status of the relay device 3. The communication management system 5 also relays shared stroke data between multiple electronic whiteboards 1a and 1b.

The image storage device 7 is configured by a computer, and stores image data of shared document images uploaded from the electronic whiteboard 1a and downloads them to the electronic whiteboard 1b. The reverse is also possible. That is, the image storage device 7 stores image data uploaded from the electronic whiteboard 1b and downloads them to the electronic whiteboard 1a.

The relay device 3, the communication management system 5, and the image storage device 7 may be constructed by a single computer, or may be constructed by multiple computers in which each part (function or means) is divided and arbitrarily assigned.

In addition, in the communication system 100, a management information session sei (hereinafter referred to as "session sei") for transmitting and receiving various management information is established between the electronic whiteboards 1a and 1b via the communication management system 5. In addition, four sessions for transmitting and receiving four types of data, high-resolution image data, medium-resolution image data, low-resolution image data, and sound data, are established between the electronic whiteboards 1a and 1b via the relay device 3. In FIG. 1, these four sessions are collectively shown as an image/sound data session sed (hereinafter simply referred to as "session"). Note that the number of sessions does not necessarily have to be four, and the number of sessions may be less or more than four. Also, a session may be established directly between the initiating terminal and the destination terminal without going through the relay device 3.

Furthermore, in the communication system 100, stroke data can be sent and received between the electronic whiteboards 1a and 1b using the session sei.

Here, the image resolution of the image data handled in this embodiment will be described. Low-resolution image data is, for example, 160 pixels wide and 120 pixels high, and serves as a base image. Medium-resolution image data is, for example, 320 pixels wide and 240 pixels high. High-resolution image data is, for example, 640 pixels wide and 480 pixels high. When passing through a narrowband path, low-quality image data consisting only of low-resolution image data serving as a base image is relayed. When the bandwidth is relatively wide, low-quality image data consisting of low-resolution image data serving as a base image and medium-resolution image data is relayed. When the bandwidth is very wide, high-quality image data consisting of low-resolution image data serving as a base image, medium-resolution image data, and high-resolution image data is relayed. Sound data has a smaller amount of data than video data, and can be relayed even through a narrowband path.

<Image of using the electronic whiteboard>
2 is a diagram for explaining an image of use of an electronic whiteboard according to an embodiment. As shown in FIG. 2, the electronic whiteboard 1 has a leg 151 with a plurality of casters on the lower side, a support 152 on the upper side of the leg 151, a main body 153 of the electronic whiteboard 1 on the upper side of the support 152, and a display 160 on the front side of the main body 153. The main body 153 has a built-in CPU 101, which will be described later. A user can input (draw) a stroke image such as a character on the display 160 using an electronic pen 140 or the like.

[Hardware configuration]
Next, the hardware configuration of each device according to this embodiment will be described.

<Hardware configuration of the electronic whiteboard>
3 is a diagram showing an example of a hardware configuration of an electronic whiteboard according to an embodiment. As shown in Fig. 3, the electronic whiteboard 1 includes a central processing unit (CPU) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a solid state drive (SSD) 104, a network interface (I/F) 105, and an external device connection I/F 106.

Of these, CPU 101 is a calculation device that controls the overall operation of electronic whiteboard 1. ROM 102 is a non-volatile memory that stores CPU 101 and programs used to start CPU 101, such as IPL (Initial Program Loader). RAM 103 is a volatile memory used as a work area for CPU 101. SSD 104 is a large-capacity storage device that stores various data such as programs for electronic whiteboard 1.

The network I/F 105 is a communication interface for connecting the electronic whiteboard 1 to a communication network and performing communication. The external device connection I/F 106 is an interface for connecting various external devices. In this case, the external devices include, for example, a USB (Universal Serial Bus) memory 131, external devices (microphone 132, speaker 133, camera 134), etc.

The electronic whiteboard 1 also includes a capture device 111, a GPU (Graphics Processing Unit) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a short-range communication circuit 119, an antenna 119a for the short-range communication circuit 119, a power switch 117, and selection switches 118.

Of these, the capture device 111 captures (acquires) a display screen or the like displayed on a display or the like of an external PC (Personal Computer) 150 as a still image or video. The GPU 112 is a semiconductor chip (processor) that specializes in graphics. The display controller 113 controls and manages the screen display in order to output the output image from the GPU 112 to the display 160 or the like. The contact sensor 114 detects when the electronic pen 140, the user's hand H, or the like touches the display 160. The sensor controller 115 controls the processing of the contact sensor 114.

The contact sensor 114 performs input and detection of coordinates, for example, by an infrared blocking method. The method of inputting and detecting coordinates is that two light receiving and emitting devices installed at both ends of the upper side of the display 160 emit multiple infrared rays parallel to the display 160, and receive the light that is reflected by a reflecting member provided around the display 160 and returns along the same optical path as the light emitted by the light receiving element. The contact sensor 114 outputs the ID of the infrared rays emitted by the two light receiving and emitting devices that are blocked by an object to the sensor controller 115, and the sensor controller 115 specifies the coordinate position that is the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 140 to determine whether the pen tip or the pen tail has touched the display 160. The near-field communication circuit 119 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark).

The power switch 117 is a switch for switching the power of the electronic whiteboard 1 on and off. The selection switches 118 are, for example, a group of switches for adjusting the brightness and color of the display 160.

The electronic whiteboard 1 further includes a bus line 120. The bus line 120 includes an address bus, a data bus, and various control signals for electrically connecting the components such as the CPU 101 shown in FIG. 3.

The contact sensor 114 is not limited to an infrared blocking type, and may be a capacitive touch panel that identifies the contact position by detecting changes in capacitance, or a resistive film touch panel that identifies the contact position by voltage changes across two opposing resistive films. The contact sensor 114 may also use various detection means, such as an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction caused by a contacting object touching the display unit. Furthermore, the electronic pen controller 116 may determine whether or not there has been a touch not only on the tip and end of the electronic pen 140, but also on the part of the electronic pen 140 that the user holds, or on other parts of the electronic pen.

<Hardware configuration of management device, relay device, image storage device, PC, etc.>
The communication management system 5, the relay device 3, the image storage device 7, and the PC 150, etc., each have a hardware configuration of a computer 400 as shown in Fig. 4. Alternatively, the communication management system 5, the relay device 3, and the image storage device 7 may be realized by a plurality of computers 400.

Figure 4 is a diagram showing an example of the hardware configuration of a computer according to one embodiment. The computer 400 includes, for example, a CPU 401, a ROM 402, a RAM 403, a HD (Hard Disk) 404, a HDD (Hard Disk Drive) controller 405, a display 406, an external device connection I/F 407, a network I/F 408, a keyboard 409, a pointing device 410, a DVD-RW (Digital Versatile Disk ReWritable) drive 412, a media I/F 414, and a bus line 415.

Of these, CPU 401 is a calculation device that controls the overall operation of computer 400. ROM 402 is a non-volatile memory that stores programs used to drive CPU 401, such as IPL. RAM 403 is a volatile memory used as a work area for CPU 401. HD 404 is a large-capacity storage device that stores programs such as the OS (Operating System) and applications, as well as various types of data. HDD controller 405 controls the reading and writing of various types of data from HD 404 under the control of CPU 401.

The display 406 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I/F 407 is an interface for connecting various external devices. The network I/F 408 is a communication interface for data communication using a communication network. The keyboard 409 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 410 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc.

The DVD-RW drive 412 controls the reading and writing (storage) of data from the DVD-RW 411. The DVD-RW drive 412 may control the reading and writing (storage) of data from a disc such as a BD-RE (Blu-ray (registered trademark) Disc Rewritable) instead of the DVD-RW 411. The media I/F 414 controls the reading and writing (storage) of data from a recording medium 413 such as a flash memory. The bus line 415 includes an address bus, a data bus, and various control signals for electrically connecting each component such as the CPU 401 shown in FIG. 4.

[Overall configuration of communication system]
Next, a description will be given of the overall configuration of the communication system 100. Fig. 5 is a diagram showing an example of the system configuration of a communication system according to an embodiment.

In FIG. 5, electronic whiteboard 1a is installed at location A, and electronic whiteboard 1b is installed at location B. For example, location A is an office in Tokyo, Japan, and location B is an office in Osaka, Japan. At location A, user A uses electronic whiteboard 1a, and at location B, user B and user C use electronic whiteboard 1b. Electronic whiteboard 1a is an example of a first terminal device, and electronic whiteboard 1b is an example of a second terminal device.

The electronic whiteboards 1a and 1b, the relay device 3, the communication management system 5, and the image storage device 7 can transmit and receive data to and from each other via a communication network 500, such as the Internet or a LAN (Local Area Network). The communication network 500 may include a wireless communication portion. In FIG. 5, the electronic whiteboards 1a and 1b are assumed to be electronic whiteboards capable of transmitting and receiving conference videos.

[Functional configuration of communication system]
Next, the functional configuration of the communication system 100 according to the present embodiment will be described. Fig. 6 is a diagram showing an example of the functional configuration of the communication system according to one embodiment. In Fig. 6, the electronic whiteboard 1b has the same functional configuration as the electronic whiteboard 1a.

<Functional configuration of the electronic whiteboard>
Each electronic whiteboard 1 realizes a transmitting/receiving unit 11, a receiving unit 12, a video/audio processing unit 13, a display control unit 14, an image processing unit 15, an information transmitting unit 16, a short-range communication unit 17, and a storage/reading processing unit 18 by executing a predetermined program with a CPU 101 in Fig. 3. At least a part of the above-mentioned functional configurations may be realized by hardware. The electronic whiteboard 1 also has a storage unit 1001 realized by a RAM 103, an SSD 104, etc. in Fig. 3.

(Functional configuration of the electronic whiteboard 1)
The transmission/reception unit 11 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmission/reception unit 11 also serves as an initiation unit, and performs a login process to the communication management system 5 or the like, and a process of initiating communication with other terminal devices.

The reception unit 12 receives various inputs from the user using the electronic pen 140 or the like. For example, the reception unit 12 receives a designation operation for designating one or more stroke data to be the target of an operation, a change operation related to the stroke data, a drawing operation of a stroke image, etc.

The video/audio processor 13 performs the main processing of the video conferencing function. For example, the video/audio processor 13 performs digital processing such as encoding video data and audio data based on the output signal of the microphone 132 and the output signal of the camera 134. The video/audio processor 13 also generates video signals and audio signals based on the video data and audio data received by the transmission/reception unit 11. The video/audio processor 13 also performs processing such as combining video data with different resolutions.

The display control unit 14 outputs video signals (image signals) etc. to the display 160 and performs control to display the display screen.

The image processing unit 15 performs the main processing of the electronic whiteboard function. For example, the image processing unit 15 performs processing to create a stroke image and stroke data based on strokes of the electronic pen 140 or the like received by the reception unit 12, and processing to create a stroke image based on stroke data received by the transmission/reception unit 11. The image processing unit 15 also performs processing to generate an image signal based on image data of a document image received by the transmission/reception unit 11.

The information transmitting unit 16 transmits the stroke data created by the image processing unit 15 to the communication management system 5. In addition, the information transmitting unit 16 transmits group operation information that specifies one or more stroke data in response to a designation operation that is accepted by the accepting unit 12 and that designates one or more stroke data to be the target of the operation to the communication management system 5.

The short-range communication unit 17 acquires and provides data between each terminal having a short-range communication unit through short-range wireless communication.

The storage/read processing unit 18 performs processes such as storing various data in the storage unit 1001 or a recording medium 1002 such as a USB memory 131, and reading out various data stored in the storage unit 1001 or the recording medium 1002.

In addition, video data and audio data received during communication with other terminals are overwritten and stored in the memory unit 1001 each time they are received. Of these, images are displayed on the display 160 using the video data before being overwritten, and sound is output from the speaker 133 using the audio data before being overwritten.

<Functional configuration of the communication management system>
The communication management system (communication management device) 5, for example, realizes a transmission/reception unit 51, an authentication unit 52, a terminal management unit 53, a session management unit 54, a relay device management unit 55, a stroke information management unit 56, an operation restriction unit 57, and a storage/read processing unit 58 by executing a predetermined program in the CPU 401 of FIG. 4. Each of the above functional configurations may be realized by a program executed by a plurality of computers 400. Also, at least a part of each of the above functional configurations may be realized by hardware. Also, the communication management system 5 has a storage unit 5000 realized by the HD 504 of FIG. 4. For example, an authentication management DB (Database) 5001, a terminal management DB 5002, a destination list management DB 5003, a session management DB 5004, a relay device management DB 5005, and stroke information 5006 are stored in the storage unit 5000.

(Authentication Management Table)
Fig. 7 is a diagram showing an example of an authentication management table according to an embodiment. For example, an authentication management DB 5001 configured by an authentication management table 700 as shown in Fig. 7 is stored in the storage unit 5000. In this authentication management table 700, each password is associated with each terminal ID of all the electronic whiteboards 1 managed by the communication management system 5 and managed. For example, the authentication management table 700 shown in Fig. 7 indicates that the password corresponding to the terminal ID "01aa" of the electronic whiteboard 1a (terminal device) is "aaaa".

The terminal ID is identification information that identifies the electronic whiteboard (terminal device) 1, and is an example of identification information used by the communication management system 5 to manage terminal devices such as the electronic whiteboard 1. For example, instead of the terminal ID, the communication management system 5 may manage the electronic whiteboard 1 using a user ID that identifies the user, a contract ID that identifies the contractor, a communication ID that specifies the source of communication, or the like.

The password is also an example of authentication information for authenticating a terminal device such as the electronic whiteboard 1. The authentication information may be authentication information other than a password, such as an access token.

(Terminal Management Table)
Fig. 8 is a diagram showing an example of a terminal management table according to an embodiment. For example, a terminal management DB 5002 configured by a terminal management table 800 as shown in Fig. 8 is stored in the storage unit 5000. In this terminal management table 800, for each terminal ID for identifying each electronic whiteboard 1 (terminal device), the destination name when each electronic whiteboard 1 is the destination, the operation status of each electronic whiteboard 1, the reception date and time when login request information described later is received by the communication management system 5, and the IP address of each electronic whiteboard 1 (terminal device) are associated and managed.

For example, in the terminal management table 800 shown in FIG. 8, the electronic whiteboard 1a with the terminal ID "01aa" is shown to have the terminal name "Japan Tokyo Office Terminal AA" and an operating status of "Online (communication possible)". The electronic whiteboard 1a with the terminal ID "01aa" is also shown to have the date and time when the login request information was received by the communication management system 5 as "April 10, 2015, 13:40" and the IP address as "1.2.1.3". The terminal ID, destination name, and terminal IP address, etc. are stored when each electronic whiteboard 1 is pre-registered to receive services provided by the communication management system 5.

(Destination list management table)
FIG. 9 is a diagram showing an example of a destination list management table according to an embodiment. For example, a destination list management DB 5003 configured by a destination list management table 900 as shown in FIG. 9 is stored in the storage unit 5000. In this destination list management table 900, all terminal IDs of destination terminals registered as candidates of the electronic whiteboard 1 (destination terminal) are associated and managed with the terminal ID of the electronic whiteboard 1 (start terminal) requesting the start of communication. For example, the destination list management table 900 shown in FIG. 9 shows that the destination terminal candidates that can request the start of communication from the start terminal (electronic whiteboard 1a) with the terminal ID "01aa" include the electronic whiteboard 1b with the terminal ID "01ba". The destination terminal candidates are updated by being added or deleted in response to an addition or deletion request from any start terminal to the communication management system 5.

Note that the destination list is an example of destination information, and the destination information may not be in list format, but may instead be a list of information about the destinations, such as the terminal ID.

(Session Management Table)
FIG. 10 is a diagram showing an example of a session management table according to an embodiment. The storage unit 5000 stores a session management DB 5004 configured by a session management table 1000 as shown in FIG. 10, for example. In this session management table 1000, for each session ID for identifying a session, the device ID of the relay device 3 used, the terminal ID of the starting terminal, the terminal ID of the destination terminal, and the terminal IDs of one or more participating terminals are associated and managed. In addition, in the session management table 1000, for example, information such as the reception delay time (ms) when the video data is received at the destination terminal and the reception date and time when the delay information indicating this delay time is received from the destination terminal are further associated and managed. For example, in the session management table 1000 shown in FIG. 10, in the session with session ID "se01", the relay device 3 with relay device ID "111a" relays communication between the start terminal with terminal ID "01aa" and the destination terminal with terminal ID "01ba". It also indicates that in the session with session ID "se01", the delay time of the video data at "13:41 on Apr. 10, 2015" is 200 (ms).

(Relay device management table)
11 is a diagram showing an example of a relay device management table according to an embodiment. The storage unit 5000 stores a relay device management DB 5005 configured by a relay device management table 1100 as shown in FIG. 11. In this relay device management table 1100, the operation state of each relay device 3, the reception date and time when the state information indicating the operation state was received, the IP address of the relay device 3, and the maximum data transmission speed (Mbps) are associated and managed for each relay device ID of a plurality of relay devices 3. For example, in the relay device management table 1100 shown in FIG. 11, it is shown that the operation state of the relay device 3 with the relay device ID "111a" is "online" and the date and time when the state information was received by the communication management system 5 is "12:00 on April 10, 2015". It is also shown that the relay device 3 with the relay device ID "111a" has an IP address of "1.2.1.2" and a maximum data transmission speed of this relay device 3 is 100 Mbps.

(Stroke information)
12 is a diagram for explaining stroke information according to an embodiment. This diagram shows an example of stroke information 5006 stored in the storage unit 5000.

In the example of FIG. 12, the stroke information 5006 has a hierarchical structure of "Meeting," "Page (Pages 1 to 3)," and "Stroke (Strokes 1 to 5)."

"Strokes" correspond to stroke data according to this embodiment, and are managed for each "page" by the stroke information management unit 56. For example, stroke 1 includes a stroke ID (id=aaaaaa) that identifies stroke 1, information specifying parent page 1 (parent=hijkl), a Gid "g001" that identifies drawing data 1, and drawing data 1. For example, when an operation such as moving is performed on the stroke image represented by stroke 1, the stroke ID (id=aaaaaa) is carried over as is, but drawing data 1 and the Gid "g001" that identifies drawing data 1 are updated.

Drawing data 1 is drawing data for reproducing stroke 1, and includes, for example, coordinate data, line width data, line color data, vector data, etc.

The electronic whiteboard 1 allows pages to be switched, and a "page" corresponds to each page of the electronic whiteboard 1. For example, page 1 includes information such as a page ID (id=hijkl) that identifies page 1, and information for identifying the parent "meeting" (parent=abcdefg).

A "conference" corresponds to a session managed by the communication system 100, and is managed, for example, by a session ID (id=abcdefg) that identifies the session.

For example, the image processing unit 15 of the electronic whiteboard 1 can create a stroke image drawn on page 1 using drawing data 1 to 5 contained in strokes 1 to 5.

Note that the session ID (id=abcdefg) is an example of first identification information that identifies a session (conference), and the stroke ID (id=aaaaaa) is an example of second identification information that identifies stroke data. Furthermore, the Gid "g001" that identifies drawing data 1 is an example of third identification information that identifies drawing data.

(Functional configuration of the communication management system)
Returning now to FIG. 6, the functional configuration of the communication management system 5 will be described.

The transmission/reception unit 51 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. For example, the transmission/reception unit 51 functions as a receiving unit that receives group operation information that specifies one or more stroke data to be the target of an operation from the electronic whiteboard 1.

The authentication unit 52 authenticates the login request received via the transmission/reception unit 51. For example, if the combination of the terminal ID and password included in the login request received from the electronic whiteboard 1 is stored in the authentication management table 700 as shown in FIG. 7, the authentication unit 52 permits login to the electronic whiteboard 1.

The terminal management unit 53 manages the terminal management DB 5002 and updates information such as the operating status, reception date and time, and terminal IP address in the terminal management table 800 as shown in FIG. 8 according to the status of the electronic whiteboard 1, for example.

The terminal management unit 53 also manages the destination list management table 900 and, for example, in response to a request from the electronic whiteboard 1, provides destination list information including the terminal ID of the destination terminal in the destination list management table 900 as shown in FIG. 9.

The session management unit 54 manages sessions held in the communication system 100. For example, the session management unit 54 generates a session ID for identifying a session in response to start request information that requests the start of communication from the electronic whiteboard 1. In addition, the session management unit 54 stores and manages various information related to the session in association with the session ID, for example, in a session management table 1000 as shown in FIG. 10.

The relay device management unit 55 manages the relay device management DB 5005 and selects a relay device 3 to be used to relay the session, for example, using a relay device management table 1100 as shown in FIG. 11. For example, the relay device management unit 55 may select a relay device 3 near the starting terminal based on the IP address of each relay device 3 stored in the relay device management table 1100 and the IP address of the starting terminal. Alternatively, the relay device management unit 55 may select a relay device 3 based on the maximum data transmission speed of each relay device 3 stored in the relay device management table 1100.

In this embodiment, the method for selecting the relay device 3 to be used to relay the session may be any method. Furthermore, the number of relay devices 3 included in the communication system 100 may be one.

The stroke information management unit 56 associates stroke data transmitted, for example, in session sei from an electronic whiteboard 1 participating in a session with a session ID, and stores and manages the stroke data in the storage unit 5000 or the like. For example, the stroke information management unit 56 stores stroke data received from the electronic whiteboard 1 in stroke information 5006 as shown in FIG. 12. In addition, the stroke information management unit 56 transfers stroke data received from the electronic whiteboard 1, for example, to another electronic whiteboard 1 participating in the same session as the electronic whiteboard 1.

The operation restriction unit 57 restricts operations on the specified stroke data by other electronic whiteboards 1 based on group operation information that is received by the transmission/reception unit 51 from the electronic whiteboard 1 and that specifies one or more stroke data to be operated. The processing by the operation restriction unit 57 and the group operation information will be described later.

The storage/read processing unit 58 performs processes such as storing various data in the storage unit 5000 and reading out various data stored in the storage unit 5000.

<Functional configuration of image storage device>
The image storage device 7 realizes a transmission/reception unit 71, a storage/readout processing unit 72, etc., by executing a predetermined program in the CPU 401 in Fig. 4. The image storage device 7 also has a storage unit 7000 realized by the RAM 403, HD 404, etc. in Fig. 4.

(Functional configuration of the image storage device)
The transmission/reception unit 71 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The storage/read processing unit 72 performs processing to store various data in the storage unit 7000 and to read out various data stored in the storage unit 7000.

With the above configuration, the image storage device 7 stores image data uploaded from the electronic whiteboard 1 in a specified URL (Uniform Resource Locator), and provides the image data stored in the specified URL in response to a request from the electronic whiteboard 1.

<Functional configuration of relay device>
The relay device 3 realizes a transmission/reception unit 31, a determination unit 32, a storage/read processing unit 33, etc., by executing a predetermined program in a CPU 401 in Fig. 4. The relay device 3 also has a storage unit 3000 realized by a RAM 403, a HD 404, etc. in Fig. 4.

(Functional configuration of relay device)
The transmitting/receiving unit 31 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmitting/receiving unit 31 also serves as a transfer unit, and transfers, for example, video data and audio data received from the electronic whiteboard 1 to other electronic whiteboards 1 participating in the same session as the electronic whiteboard 1. The determining unit 32 performs various determinations, such as determining the delay state of data, etc.

The storage/read processing unit 33 performs processes such as storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

The functional configuration of the communication system 100 shown in FIG. 6 is an example, and the communication system 100 can be modified in various ways. For example, each functional configuration included in the communication management system 5 may be realized by multiple computers 400. Furthermore, the function of the image storage device 7 may be realized by a storage server or the like external to the communication system 100. Furthermore, at least a portion of each functional configuration of the image storage device 7 and the relay device 3 may be included in the communication management system 5. Furthermore, the electronic whiteboard 1 is an example of a terminal device, and the terminal device may be an information terminal that executes an application corresponding to the communication system 100.

[Processing flow]
Next, the process flow of the communication management method and image processing method according to this embodiment will be described.

<Preparation for remote communication>
13 is a sequence diagram showing an example of a preparatory process according to an embodiment. This process shows an example of a preparatory process executed before the interactive whiteboard 1 participates in a session (video conference) in the communication system 100.

First, when the power switch 117 on the electronic whiteboard 1a is turned on, the reception unit 12 receives the power-on command (step S22).

Next, the transmitter/receiver 11 transmits login request information requesting login to the communication management system 5 via the communication network 500 (step S23). This login request information includes, for example, the terminal ID and password of the electronic whiteboard 1a.

Next, the authentication unit 52 of the communication management system 5 searches the authentication management table 700 as shown in FIG. 7 using the terminal ID and password included in the login request information received via the transmission/reception unit 51 as search keys. Furthermore, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the authentication unit 52 permits login to the electronic whiteboard 1a (step S24). Here, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the processing from step S25 onwards is executed.

When login to the electronic whiteboard 1a is permitted, the terminal management unit 53 updates the information corresponding to the terminal ID "01aa" of the electronic whiteboard 1a in the terminal management table 800 as shown in FIG. 8. For example, the terminal management unit 53 changes the "operation status" information corresponding to the terminal ID "01aa" to "online (communication possible)" and updates the "received date and time" information to the date and time when the login request information was received (step S25). Note that the IP address information of the terminal is not registered in advance, and the IP address sent from the electronic whiteboard 1a in step S23 above may be used. As a result, in the terminal management table 800, for example, as shown in FIG. 8, the terminal ID "01aa" is associated with the operation status "online (communication possible)", the received date and time "2015.4.10.13:40", and the IP address "1.2.1.3", and the like, and managed.

Next, the session management unit 54 adds a new record in which the terminal ID "01aa" of the electronic whiteboard 1a received in step S23 above is set as the "terminal ID of the starting terminal" to the session management table 1000 as shown in FIG. 10 and manages the new record (step S26). Then, the transmission/reception unit 51 of the communication management system 5 transmits authentication result information indicating the authentication result obtained by the processing of step 24 via the communication network 500 to the electronic whiteboard 1a that made the login request (step S27).

When the transmission/reception unit 11 of the electronic whiteboard 1a receives the authentication result information indicating that login is permitted, it transmits destination list request information indicating a request for a destination list to the communication management system 5 via the communication network 500 (step S28). As a result, the transmission/reception unit 51 of the communication management system 5 receives the destination list request information.

Next, the terminal management unit 53 searches the destination list management table 900 as shown in FIG. 9 using the terminal ID "01aa" of the electronic whiteboard 1a as a search key, and reads out the terminal IDs of destination candidates that can communicate with the electronic whiteboard 1a. The terminal management unit 53 also reads out the destination names corresponding to the terminal IDs of the destination candidates from the terminal management table 800 as shown in FIG. 8 (step S29). This extracts the terminal IDs of the destination candidates that correspond to the terminal ID "01aa" of the electronic whiteboard 1a, and the destination names that correspond to these.

Next, the transmission/reception unit 51 of the communication management system 5 reads, for example, the destination list frame data and the icon data indicating the operating status stored in the memory unit 5000 via the storage/readout processing unit 58 (step S30). The transmission/reception unit 51 of the communication management system 5 also transmits "destination list information (destination list frame, icon, terminal ID, destination name)" including the read destination list frame and icon, and the terminal ID and destination name extracted in step S29 to the electronic whiteboard 1a (step S31). As a result, the transmission/reception unit 11 of the electronic whiteboard 1a receives the destination list information, and the storage/readout processing unit 18 stores the received destination list information in the memory unit 1001 (step S32).

In this way, in this embodiment, rather than managing the destination list information on each electronic whiteboard 1, the communication management system 5 centrally manages the destination list information of all terminals. Therefore, even when a new electronic whiteboard 1 is registered in the communication management system 5, when a new model of terminal device is registered in place of an electronic whiteboard 1 already included, when the appearance of the destination list frame is changed, etc., it is possible to eliminate the need to change the destination list information on each electronic whiteboard 1.

The terminal management unit 53 of the communication management system 5 searches the terminal management table 800 as shown in FIG. 8 using the terminal IDs of the destination candidates extracted in step S29 as a search key, and reads out the corresponding operating status for each destination candidate terminal ID. As a result, the terminal management unit 53 obtains the operating status of each electronic whiteboard 1 corresponding to the destination candidate terminal ID (step S33).

Next, the transmitter/receiver 51 of the communication management system 5 transmits "terminal status information" including the terminal ID used as the search key in step S33 and the operating status of each corresponding destination terminal to the electronic whiteboard 1a via the communication network 500 (step S34).

Next, the storage/readout processing unit 18 of the electronic whiteboard 1a sequentially stores the terminal status information received from the communication management system 5 in the storage unit 1001 (step S35). Thus, by receiving the above-mentioned electronic whiteboard 1 status information, the electronic whiteboard 1a can obtain the current operating status of electronic whiteboards 1b, 1c, etc., which are potential destinations that can communicate with the electronic whiteboard 1a.

Next, the display control unit 14 of the electronic whiteboard 1a creates a destination list that reflects the status of the terminal as a destination candidate based on the destination list information stored in the memory unit 1001 and the terminal status information. The display control unit 14 also uses the created destination list to display a destination list screen on the display 160 of the electronic whiteboard 1a (step S36).

Meanwhile, the terminal management unit 53 of the communication management system 5 searches the destination list management table 900 as shown in FIG. 9 based on the terminal ID "01aa" of the electronic whiteboard 1a to extract the terminal IDs of other terminals that have registered the electronic whiteboard 1a as a destination candidate (step S37).

Next, the terminal management unit 53 of the communication management system 5 searches the terminal management table 800 as shown in FIG. 8 based on the terminal ID "01aa" of the electronic whiteboard 1a, and obtains the operating status of the electronic whiteboard 1a (step S38).

Then, the transmitter/receiver 51 transmits "terminal status information" including the terminal ID and operating status of the electronic whiteboard 1a to the electronic whiteboards 1 of the terminal IDs extracted in step S37 that have an operating status of "online" in the terminal management table 800 (step S39). When transmitting the terminal status information to each electronic whiteboard 1, the transmitter/receiver 51 refers to the IP address of each electronic whiteboard 1 managed in the terminal management table 800 based on each terminal ID. This makes it possible to transmit the terminal ID "01aa" and operating status "online" of the electronic whiteboard 1a to each electronic whiteboard 1 that can communicate with the electronic whiteboard 1a as a destination candidate. This makes it possible to display the status of the electronic whiteboard 1a on each electronic whiteboard 1 that is a destination candidate (step S40).

<Session start processing>
14 is a sequence diagram showing an example of a session start process according to an embodiment. This process shows an example of a session start process executed in the communication system 100 when, for example, the electronic whiteboard 1a requests the electronic whiteboard 1b to start a session.

In step S1401, when the electronic whiteboard 1a, which is the starting terminal, accepts the destination selection by user A, the processing from step S1402 onwards is executed. Here, it is assumed that user A has selected electronic whiteboard 1b as the destination terminal.

In step S1402, when the electronic whiteboard 1a accepts the destination selection by user A, the transmission/reception unit 11 of the electronic whiteboard 1a sends start request information requesting the start of a session to the communication management system 5. This start request information includes, for example, the terminal ID "01aa" of the electronic whiteboard 1a which is the starting terminal, the terminal ID "01ba" of the electronic whiteboard 1b which is the destination terminal, and the IP address of the electronic whiteboard 1a which is the starting terminal. As a result, the transmission/reception unit 51 of the communication management system 5 receives the start request information and also receives the IP address of the starting terminal (electronic whiteboard 1a) which is the sender.

In step S1403, the terminal management unit 53 of the communication management system 5 updates the terminal management DB 5002 based on the terminal ID of the starting terminal and the terminal ID of the destination terminal included in the start request information. For example, based on the terminal ID "01aa" of the starting terminal and the terminal ID "01ba" of the destination terminal included in the start request information, the terminal management unit 53 updates the "operating status" corresponding to the terminal IDs "01aa" and "01ba" in the terminal management table 800 to "online (communicating)". Note that in this state, the electronic whiteboard 1a, which is the starting terminal, and the electronic whiteboard 1b, which is the destination terminal, are not participating in the session, but are managed as being in a communicating state.

In step S1404, the session management unit 54 of the communication management system 5 generates a session ID for identifying the session held between the electronic whiteboard 1a and the electronic whiteboard 1b. Here, it is assumed that the session management unit 54 generates the session ID "se01".

In step S1405, the session management unit 54 stores and manages the generated session ID, the terminal ID "01aa" of the initiating terminal, the terminal ID "01ba" of the destination terminal, etc., in association with each other in a session management table 1000 as shown in FIG. 10.

In step S1406, the relay device management unit 55 of the communication management system 5 selects a relay device 3 that will relay the session between the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b). For example, the relay device management unit 55 selects a relay device 3 with an IP address close to the IP address of the initiating terminal (electronic whiteboard 1a) in the terminal management table 800 from among the device IDs of relay devices 3 whose operating status is "online" in the relay device management table 1100. Here, it is assumed that the relay device 3 with device ID "111a" is selected.

In steps S1407a and S1407b, the session management unit 54 of the communication management system 5 transmits relay device selection information to the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) via the transmitting/receiving unit 51. This relay device selection information includes, for example, the IP address of the relay device 3 selected in step S1406, the session ID generated in step S1404, and the IP address of the communication management system 5.

In steps S1408a and S1408b, when the transmitter/receiver 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) receive the relay device selection information, they transmit reception completion information indicating that the relay device selection information has been received to the communication management system 5. This reception completion information includes, for example, the session ID "se01" that was included in the relay device selection information.

In steps S1409a and S1409b, the transmitting/receiving units 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) establish a session with the relay device 3 using the device ID and session ID of the relay device 3 included in the relay device selection information. As a result, the electronic whiteboard 1a and the electronic whiteboard 1b participate in the session with the same session ID "se01" and can mutually transmit and receive video data such as conference video, audio data, and the like via the relay device 3.

The above process enables a video conference to be held between user A of electronic whiteboard 1a and user B of electronic whiteboard 1b, transmitting and receiving conference video and the like.

<Sharing of stroke images>
In the communication system 100, a user A of the electronic whiteboard 1a and users B and C of the electronic whiteboard 1b who are participating in a video conference can display a stroke image drawn with an electronic pen 140 or the like on each electronic whiteboard 1 as a shared image on the other electronic whiteboards 1. Here, an example of a process of sharing a stroke image between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session will be described.

Figure 15 is a sequence diagram showing an example of a stroke image sharing process according to one embodiment. At the start of the process shown in Figure 15, electronic whiteboard 1a and electronic whiteboard 1b are assumed to be participating in the same session, for example, through the session start process shown in Figure 14.

Furthermore, electronic whiteboard 1a and electronic whiteboard 1b share document images via image storage device 7 as necessary, and display the document images on display 160.

In step S1501, user A draws (moves) the electronic pen 140 or hand H on the display 160 of the electronic whiteboard 1a, and the reception unit 12 receives input of the drawn stroke (trajectory).

In step S1502, the image processing unit 15 of the electronic whiteboard 1a creates stroke data for displaying a stroke image corresponding to the stroke accepted by the acceptance unit 12.

In step S1503, the display control unit 14 of the electronic whiteboard 1a causes the display 160 to display a stroke image based on the stroke data created by the image processing unit 15. As a result, the stroke image drawn by the user A on the display 160 using the electronic pen 140 or the like is displayed on the display 160.

In step S1504, the transmission/reception unit 11 of the electronic whiteboard 1a transmits stroke data for reproducing the stroke image drawn by user A to the communication management system 5, for example, in session sei. As a result, the transmission/reception unit 51 of the communication management system 5 receives the stroke data transmitted from the electronic whiteboard 1a.

In step S1505, the stroke information management unit 56 of the communication management system 5 registers the stroke data received from the electronic whiteboard 1a, for example, in stroke information 5006 as shown in FIG. 12.

In step S1506, the stroke information management unit 56 of the communication management system 5 transfers the stroke data to the electronic whiteboard 1b that is participating in the same session as the electronic whiteboard 1a. As a result, the transmission/reception unit 11 of the electronic whiteboard 1b receives the stroke data.

In step S1507, the image processing unit 15 of the electronic whiteboard 1b creates a stroke image based on the stroke data received by the transmission/reception unit 11.

In step S1508, the display control unit 14 of the electronic whiteboard 1b causes the display 160 to display the stroke image created by the image processing unit 15, i.e., the stroke image drawn by user A in step S1501.

In addition, when user B or user C draws on electronic whiteboard 1b using electronic pen 140 or the like, the stroke image can be displayed on electronic whiteboard 1a in a similar manner.

In this way, stroke images can be shared between electronic whiteboard 1a and electronic whiteboard 1b participating in the same session.

<Stroke image movement processing>
In a conference where stroke images are shared between multiple electronic whiteboards (terminal devices) 1a and 1b, the multiple electronic whiteboards 1a and 1b can perform editing processes such as moving processes on the shared stroke images. In this case, for example, when a moving operation or the like is performed on the shared stroke images from the multiple electronic whiteboards 1a and 1b, an operation different from the user's intention may be performed on the stroke images depending on the timing of the operation.

To solve such problems, the communication management system 5 has an operation restriction function that restricts editing operations by user B on a stroke image when user A is performing an editing operation on the stroke image to be operated, for example.

Before explaining the operation restriction function, we will explain an example of the movement process of a stroke image when the operation restriction function is not used.

Figure 16 is a sequence diagram showing an example of a stroke image movement process according to one embodiment. At the start of the process shown in Figure 16, for example, the display screen 1710 including stroke images ST1 to 5 is displayed on the displays of the electronic whiteboard 1a and electronic whiteboard 1b as a result of the stroke image sharing process shown in Figure 15, as shown in Figure 17 (A). In addition, stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5 stores stroke data (strokes 1 to 5) corresponding to stroke images ST1 to 5, as shown in Figure 12, for example.

As an example for the purposes of explanation, assume that user A of electronic whiteboard 1a performs an operation to sequentially move stroke images ST1 to 5 to the left, and user B of electronic whiteboard 1b performs an operation to sequentially move stroke images ST2 to 4 downward.

In step S1601, the information transmission unit 16 of the electronic whiteboard 1a transmits operation information to the communication management system 5 to move the stroke image ST1 (Gid=g001) a predetermined distance to the left in response to a movement operation on the stroke image ST1 by user A. This operation information includes, for example, coordinate information of the stroke image ST1 after it has been moved.

In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 1 and Gid of stroke 1 corresponding to stroke image ST1 (Gid=g001) in stroke information 5006 shown in FIG. 12, for example. This also causes stroke image ST1 shared between electronic whiteboard 1a and electronic whiteboard 1b to move to the left, for example, as shown in display screen 1720 in FIG. 17(B).

In step S1602, the stroke information management unit 56 of the communication management system 5 sends a success notification to the electronic whiteboard 1a indicating that the movement of the stroke image ST1 was successful. This success notification includes the updated gid of the stroke image ST1 (e.g., Gid=g006).

In step S1603, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST2 (Gid = g002) a predetermined distance to the left in response to the movement operation on the stroke image ST2 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST2 (Gid = g002) in the stroke information 5006 shown in FIG. 12, for example. This also causes the stroke image ST2 shared between the electronic whiteboard 1a and the electronic whiteboard 1b to move to the left, for example, as shown in the display screen 1730 in FIG. 17 (C).

In step S1604, the stroke information management unit 56 of the communication management system 5 sends a success notification to the electronic whiteboard 1a indicating that the movement of the stroke image ST2 was successful. This success notification includes the updated gid of the stroke image ST2 (e.g., Gid=g007).

In step S1605, the information transmission unit 16 of the electronic whiteboard 1b transmits operation information to the communication management system 5 to move the stroke image ST2 (Gid = g002) downward a predetermined distance in response to the movement operation on the stroke image ST2 by user B. However, because the gid of the stroke image ST2 has been updated to "g007", the stroke information management unit 56 of the communication management system 5 cannot perform the movement process of the stroke image ST2 (Gid = g002).

In step S1606, the stroke information management unit 56 of the communication management system 5 sends a failure notification to the electronic whiteboard 1b indicating that the movement of the stroke image ST2 has failed.

In step S1607, the information sending unit 16 of the electronic whiteboard 1b sends operation information to the communication management system 5 to move the stroke image ST3 (Gid = g003) downward a predetermined distance in response to the movement operation on the stroke image ST3 by user B. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 3 corresponding to the stroke image ST3 (Gid = g003) in the stroke information 5006 shown in FIG. 12, for example. This also causes the stroke image ST3 shared between the electronic whiteboard 1a and the electronic whiteboard 1b to move downward, for example, as shown in the display screen 1740 in FIG. 17 (D).

In step S1608, the stroke information management unit 56 of the communication management system 5 sends a success notification to the electronic whiteboard 1b indicating that the movement of the stroke image ST3 was successful. This success notification includes the updated gid of the stroke image ST3 (e.g., Gid=g008).

In step S1609, the information sending unit 16 of the electronic whiteboard 1b sends operation information to the communication management system 5 to move the stroke image ST4 (Gid = g004) downward a predetermined distance in response to the movement operation on the stroke image ST4 by user B. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 4 corresponding to the stroke image ST4 (Gid = g004) in the stroke information 5006 shown in FIG. 12, for example. This also causes the stroke image ST4 shared between the electronic whiteboard 1a and the electronic whiteboard 1b to move downward, for example, as shown in the display screen 1750 in FIG. 17 (E).

In step S1610, the stroke information management unit 56 of the communication management system 5 sends a success notification to the electronic whiteboard 1b indicating that the movement of the stroke image ST4 was successful. This success notification includes the updated gid of the stroke image ST4 (e.g., Gid=g009).

In step S1611, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST3 (Gid = g003) a predetermined distance to the left in response to the move operation on the stroke image ST3 by user A. However, because the gid of the stroke image ST3 has been updated to "g008", the stroke information management unit 56 of the communication management system 5 cannot perform the move process on the stroke image ST3 (Gid = g003).

In step S1612, the stroke information management unit 56 of the communication management system 5 sends a failure notification to the electronic whiteboard 1a indicating that the movement of the stroke image ST3 has failed.

In step S1613, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST4 (Gid = g004) a specified distance to the left in response to the move operation on the stroke image ST4 by user A. However, because the gid of the stroke image ST4 has been updated to "g009", the stroke information management unit 56 of the communication management system 5 cannot perform the move process on the stroke image ST4 (Gid = g004).

In step S1614, the stroke information management unit 56 of the communication management system 5 sends a failure notification to the electronic whiteboard 1a indicating that the movement of the stroke image ST4 has failed.

In step S1615, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST5 (Gid = g005) a predetermined distance to the left in response to the movement operation on the stroke image ST5 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST5 (Gid = g005) in the stroke information 5006 shown in FIG. 12, for example. This also causes the stroke image ST5 shared between the electronic whiteboard 1a and the electronic whiteboard 1b to move to the left, for example, as shown in the display screen 1760 in FIG. 17 (F).

In this way, when the operation restriction function is not used, as shown in the display screen 1760 in FIG. 17(F), stroke images ST1, ST2, and ST5 move leftward in response to user A's operation, while stroke images ST3 and ST4 move downward in response to user B's operation. As a result, stroke images ST1 to 5 end up being moved to positions different from those intended by user A and user B.

To solve such problems, the communication management system 5 has an operation restriction function that restricts editing operations by user B on a stroke image when user A is performing an editing operation on the stroke image to be operated, for example.

[First embodiment]
18 is a sequence diagram (1) showing an example of the operation restriction process according to the first embodiment. This process shows an example of the operation restriction process by the operation restriction unit 57 of the communication management system 5.

Note that at the start of the process shown in FIG. 18, for example, the display screen 1710 including stroke images ST1 to 5 as shown in FIG. 17(A) is displayed on the displays of the electronic whiteboard 1a and electronic whiteboard 1b as a result of the stroke image sharing process shown in FIG. 15. Also, for example, stroke data (strokes 1 to 5) corresponding to stroke images ST1 to 5 are stored in the stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5 as shown in FIG. 12.

As an example for the purposes of explanation, it is assumed here that user A of the electronic whiteboard 1a performs an operation to move stroke images ST1 to ST5 in sequence to the left.

In step S1801, the reception unit 12 of the electronic whiteboard 1a receives a group selection operation by user A.

In this embodiment, before performing an editing operation on a shared stroke image, a user can select the stroke image to be operated on and notify the communication management system 5 of a list of the stroke data to be operated on.

In step S1802, the information transmission unit 16 of the electronic whiteboard 1a transmits a group operation request 1 that specifies one or more stroke data to be operated on to the communication management system 5 in response to the group selection operation by user A. Note that the group operation request 1 is an example of group operation information that specifies one or more stroke data to be operated on.

Figure 19(a) shows an example of group operation information 1910 (group operation request 1) that the information sending unit 16 of the electronic whiteboard 1a sends to the communication management system 5. In the group operation information 1910 shown in Figure 19(a), the character string ""lock":"abcdefg"" is information that specifies the identification information that identifies the conference (session) to be operated. The example of Figure 19(a) shows that the stroke information to be operated is, for example, stroke information 5006 of the conference (id=abcdefg) shown in Figure 12.

The string "list": ["aaaaa", "bbbbb", "ccccc", "ddddd", "eeeee"]" is information that specifies the stroke ID of the stroke data to be operated on. In the example of FIG. 19(a), the stroke data to be operated on is shown to be stroke 1, stroke 2, stroke 3, stroke 4, and stroke 5.

In step S1803 of FIG. 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 1 sent from the electronic whiteboard 1a, it determines whether the operation target (stroke data) specified in the group operation request 1 includes an operation target (stroke data) that is under operation restriction.

If the operation targets specified in group operation request 1 do not include operation targets that are under operation restriction, the processing from step S1804 onwards is executed. On the other hand, if the operation targets specified in group operation request 1 include operation targets that are under operation restriction, as described below, the operation restriction unit 57 of the communication management system 5 sends a rejection notice to the electronic whiteboard 1 that originated the request.

In step S1804, the operation restriction unit 57 of the communication management system 5 starts the operation restriction process for the stroke data specified in the group operation request 1. As a result, the operation restriction unit 57 prohibits operations from the other electronic whiteboards 1b regarding the stroke image specified in the group operation request 1 transmitted from the electronic whiteboard 1a.

In step S1805, the operation restriction unit 57 of the communication management system 5 sends an acceptance notification indicating that group operation request 1 has been accepted to the electronic whiteboard 1a that originated the request.

In step S1806, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST1 (Gid=g001) a predetermined distance to the left in response to the move operation on the stroke image ST1 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 1 of the stroke 1 corresponding to the stroke image ST1 (Gid=g001) in the stroke information 5006 shown in FIG. 12, for example, and updates the Gid to "g006".

In step S1807, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST2 (Gid = g002) a predetermined distance to the left in response to the move operation on the stroke image ST2 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 2 of the stroke 2 corresponding to the stroke image ST2 (Gid = g002) in the stroke information 5006 shown in FIG. 12, for example, and updates the Gid to "g007".

Here, for example, in step S1808, the reception unit 12 of the electronic whiteboard 1b receives a group selection operation by user B.

In step S1809, the information transmission unit 16 of the electronic whiteboard 1b transmits a group operation request 2 specifying one or more stroke data to be operated on to the communication management system 5 in response to the group selection operation by user B. Note that the group operation request 2 is another example of group operation information specifying one or more stroke data to be operated on.

Figure 19 (b) shows an example of group operation information 1920 (group operation request 2) that the information sending unit 16 of the electronic whiteboard 1b sends to the communication management system 5. In the group operation information 1920 shown in Figure 19 (b), the character string ""lock":"abcdefg"" indicates that the stroke information to be operated is, for example, stroke information 5006 of the meeting (id=abcdefg) shown in Figure 12.

The string "list": ["bbbbb", "ccccc", "ddddd"] indicates that the stroke data to be operated on is stroke 2, stroke 3, and stroke 4.

In step S1810 of FIG. 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 2 sent from the electronic whiteboard 1b, it determines whether the operation target (stroke data) specified in the group operation request 2 includes an operation target (stroke data) that is under operation restriction.

Here, in step S1804, the group operation information 1910 shown in FIG. 19(a) starts operation restriction processing for stroke data with stroke IDs "aaaaaa," "bbbbbb," "cccccc," "ddddddd," and "eeeeee." In addition, in the group operation information 1920 shown in FIG. 19(b), stroke IDs "bbbbbb," "cccccc," and "ddddddd" are specified as stroke data to be operated.

In this way, if a part or all of the specified operation target (stroke data) is included in the operation target (stroke data) under operation restriction, the operation restriction unit 57 sends a rejection notice (or failure notice) to the requesting electronic whiteboard 1b in step 1811. In this case, users B and C of the electronic whiteboard 1b cannot perform operations on the stroke data ST2, ST3, and ST4 that are the operation targets.

In step S1812, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST3 (Gid = g003) a predetermined distance to the left in response to the move operation on the stroke image ST3 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 3 of the stroke 3 corresponding to the stroke image ST3 (Gid = g003) in the stroke information 5006 shown in FIG. 12, for example, and updates the Gid to "g008".

In step S1813, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST4 (Gid = g004) a predetermined distance to the left in response to the move operation on the stroke image ST4 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 4 of the stroke 4 corresponding to the stroke image ST4 (Gid = g004) in the stroke information 5006 shown in FIG. 12, for example, and updates the Gid to "g009".

In step S1814, the information sending unit 16 of the electronic whiteboard 1a sends operation information to the communication management system 5 to move the stroke image ST5 (Gid = g005) a predetermined distance to the left in response to the move operation on the stroke image ST5 by user A. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 5 of the stroke 4 corresponding to the stroke image ST5 (Gid = g005) in the stroke information 5006 shown in FIG. 12, for example, and updates the Gid to "g010".

In step S1815, the operation restriction unit 57 of the communication management system 5, for example, when it receives operations for all stroke data specified as operation targets, determines that the group operation is complete and ends the operation restriction process.

As another example, the operation restriction unit 57 may determine that the group operation is complete when it receives end information of the group operation from the electronic whiteboard 1a.

Preferably, the operation restriction unit 57 ends the group restriction process when a predetermined time has elapsed since the operation restriction process started. This makes it possible to prevent a situation in which the operation target cannot be edited, for example, when the electronic whiteboard 1a transmits group operation information to the communication management system 5 and leaves the session due to a failure or the like before completing operations on all stroke information that are the operation target.

In step S1816, the operation restriction unit 57 of the communication management system 5 sends completion information indicating that the group operation has been completed to the electronic whiteboard 1a. The Gid of each stroke data item updated by the group operation may be notified to the electronic whiteboard 1a using this completion information, or may be notified to the electronic whiteboard 1a using, for example, a success notification as shown in step S1602 of FIG. 16.

By performing the above process, for example, as shown on the display screen 1770 in FIG. 17(G), the stroke images ST1 to ST5 to be operated are moved to the left as intended by user A.

Figure 20 is a sequence diagram (2) showing an example of operation restriction processing according to the first embodiment. This processing shows another example of operation restriction processing by the operation restriction unit 57 of the communication management system 5. Of the processing shown in Figure 20, steps S1801 to S1807 and S1812 to S1816 are similar to the processing described in Figure 18, so the following explanation will focus on the differences from the processing shown in Figure 18.

In step S2001, the reception unit 12 of the electronic whiteboard 1b receives a movement operation by user B to move stroke image ST3 downward by a predetermined distance during a group operation by user A on stroke images ST1 to ST5.

In step S2002, the information transmission unit 16 of the electronic whiteboard 1b transmits operation information to the communication management system 5 to move the stroke image ST3 (Gid=g003) downward a predetermined distance in response to the movement operation on the stroke image ST3 by user B. However, since the operation of the stroke image ST3 is restricted by the operation restriction process performed by the operation restriction unit 57 of the communication management system 5, the stroke information management unit 56 cannot perform the movement process on the stroke image ST3 (Gid=g003).

In step S2003, the stroke information management unit 56 of the communication management system 5 sends a failure notification to the electronic whiteboard 1b indicating that the movement of the stroke image ST3 has failed.

In this way, when the operation restriction unit 57 in this embodiment receives group operation information sent from an electronic whiteboard 1, it restricts operations from other electronic whiteboards 1 regarding the stroke image specified in the group operation information.

Therefore, according to this embodiment, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from multiple terminal devices from performing operations on the stroke image that are different from the user's intention.

Second Embodiment
21 is a sequence diagram showing an example of an operation restriction process according to the second embodiment. This process shows an example of an operation restriction process by the communication management system 5 and the electronic whiteboard 1.

In the following explanation, it is assumed that the communication system 100 has three electronic whiteboards 1a, 1b, and 1c, and that the three electronic whiteboards 1a, 1b, and 1c are participating in a video conference in which the same stroke image is shared.

In step S2101, the reception unit 12 of the electronic whiteboard 1a receives a group selection operation by user A who is using the electronic whiteboard 1a. Note that, in this example, it is assumed that "Page 1" of the electronic whiteboard 1a has been selected by the group selection operation. In this manner, the group selection operation that specifies one or more stroke data to be the target of the operation may be performed on a page-by-page basis.

In step S2102, the information transmission unit 16 of the electronic whiteboard 1a transmits a group operation request to the communication management system 5, in response to the group selection operation by user A, which includes the information on the selected operation target, "Page 1," and the terminal ID "10aa" of the electronic whiteboard 1a.

In step S2103, when the operation restriction unit 57 of the communication management system 5 receives the group operation request sent from the electronic whiteboard 1a, it determines whether the "Page 1" specified in the group operation request includes an operation target (stroke data, page, etc.) that is under operation restriction.

If the operation targets specified in the group operation request do not include any operation targets that are under operation restriction, the processing from step S2104 onwards is executed. On the other hand, if the operation targets specified in the group operation request include any operation targets that are under operation restriction, the operation restriction unit 57 of the communication management system 5 stops the execution of the processing from step S2104 onwards and sends a rejection notice to the electronic whiteboard 1a that made the request. Here, the following explanation will be given assuming that the operation targets specified in the group operation request do not include any operation targets that are under operation restriction.

In step S2104, the operation restriction unit 57 of the communication management system 5 starts operation restriction processing for page 1 specified in the group operation request.

In step S2105, the operation restriction unit 57 of the communication management system 5 sends an acceptance notification indicating that the group operation request has been accepted to the electronic whiteboard 1a that originated the request.

In addition, in steps S2106 and S2107, the operation restriction unit 57 of the communication management system 5 according to the second embodiment sends an operation restriction notification to all other electronic whiteboards 1b and 1c participating in the video conference, indicating that the operation restriction process has started. This operation restriction notification includes, for example, information on the operation target that is the subject of the operation restriction (e.g., "Page 1"), and information on the electronic whiteboard 1a that has imposed the operation restriction on the operation target (e.g., terminal ID "10aa"). Note that the information on the electronic whiteboard 1a may be the terminal name of the electronic whiteboard 1a, the destination name managed in the terminal management table 800, etc.

In steps S2108 and S2109, the display control unit 14 of the electronic whiteboards 1b and 1c that have received an operation restriction notification from the communication management system 5 displays a display screen 2200 indicating that operation of page 1 is restricted, as shown in FIG. 22, for example.

Fig. 22 is a diagram showing an example of a display screen according to the second embodiment. This diagram shows an example of a display screen 2200 displayed by the electronic whiteboards 1b and 1c in steps S2108 and S2109 of Fig. 21, for example. In Fig. 22, it is assumed that page 1 is displayed on the display screen 2200 of the electronic whiteboards 1b and 1c. In addition, a message 2201 indicating that operation of page 1 is restricted is displayed on the display screen 2200. This message 2201 can prevent operation of page 1 by the electronic whiteboards 1b and 1c. Preferably, this message 2201 includes information (e.g., terminal name, destination name, terminal ID, etc.) about the electronic whiteboard 1a that has locked (restricted operation of) page 1.

Note that message 2201 is an example of display information indicating that operations are restricted. For example, the display information indicating that operations are restricted may be information other than a character string, such as an emoji or an icon.

By performing the above process, user A using the electronic whiteboard 1a can preferentially operate the stroke images contained in the operation target "Page 1."

For example, in step S2111, when user A moves stroke image ST1 on page 1 of electronic whiteboard 1a to the right, the processing from step S2112 onwards is executed.

In step S2112, the information transmission unit 16 of the electronic whiteboard 1a transmits operation information to the communication management system 5 for moving the stroke image ST1 (stroke 1) on page 1 a predetermined distance to the right.

In steps S2113 and S2114, the stroke information management unit 56 of the communication management system 5 transmits operation information for moving the stroke image ST1 (stroke1) a predetermined distance to the right to the electronic whiteboards 1b and 1c.

As a result, in steps S2115 and S2116, the stroke image ST1 on page 1 of electronic whiteboards 1b and 1c is moved to the right, and the same stroke image is displayed on page 1 of electronic whiteboards 1a, 1b, and 1c.

In this way, the communication system 100 according to the second embodiment can prevent operations from being performed on a stroke image that are different from the user's intention due to operations from multiple electronic whiteboards 1a, 1b, and 1c.

[Third embodiment]
Fig. 23 is a sequence diagram showing an example of operation restriction processing according to the third embodiment. Note that the processing of steps S2101 to S2104 in Fig. 23 is similar to the processing according to the second embodiment described in Fig. 21, so the following description will focus on the differences from the second embodiment.

In step S2104 of FIG. 23, after the operation restriction unit 57 of the communication management system 5 starts the operation restriction process, there is a risk that operations may be performed on page 1 on the electronic whiteboards 1b and 1c until an operation restriction notification is sent to the electronic whiteboards 1b and 1c.

For example, in step S2301 of FIG. 23, assume that user B, who is using the electronic whiteboard 1b, moves the stroke image ST1 on page 1 to the left. In this case, the communication management system 5 and the electronic whiteboard 1 execute the processes of steps S2302 to S2304.

In step S2302, the information transmission unit 16 of the electronic whiteboard 1b transmits operation information to the communication management system 5 for moving the stroke image ST1 (stroke 1) on page 1 a predetermined distance to the left.

However, since operation of page 1 has already been restricted by the operation restriction process performed by the operation restriction unit 57 of the communication management system 5, the stroke information management unit 56 cannot move the stroke image ST1 of page 1. In such a case, in step S2303, the stroke information management unit 56 of the communication management system 5 sends a failure notification to the electronic whiteboard 1b indicating that the movement of the stroke image ST1 of page 1 has failed.

In step S2304, when the image processing unit 15 of the electronic whiteboard 1b receives a failure notification from the communication management system 5, it returns the stroke image ST1 of page 1 that was unsuccessfully moved to its original position. This makes it possible to maintain the stroke image of page 1 displayed on the electronic whiteboards 1a, 1b, and 1c in the same state.

In the second embodiment, when the electronic whiteboard 1b is displaying the display screen 2200 as shown in FIG. 22, if a move operation is received on the stroke image ST1 of page 1, the processing of steps S2302 to S2304 in FIG. 23 can also be applied.

Next, in step S2305, the processing that occurs after the operation restriction unit 57 of the communication management system 5 sends a receipt notification indicating that the group operation request has been received to the electronic whiteboard 1a that originated the request will be described.

In steps S2306 and S2307, the operation restriction unit 57 of the communication management system 5
An operation restriction notice indicating that the operation restriction process has started is transmitted to all the other electronic whiteboards 1b and 1c participating in the video conference.

In steps S2308 and S2309, the electronic whiteboards 1b and 1c that have received the operation restriction notification from the communication management system 5 prohibit operations on the stroke image of page 1 that is subject to the operation restriction. For example, the display control units 14 of the electronic whiteboards 1b and 1c display the display screen 2400 as shown in FIG. 24, and the reception units 12 of the electronic whiteboards 1b and 1c prohibit the reception of editing operations on the stroke image of page 1.

Figure 24 is a diagram showing an example of a display screen according to the third embodiment. This figure shows an example of a display screen 2400 displayed by the electronic whiteboards 1b and 1c in steps S2308 and S2309 in Figure 23, for example.

In FIG. 24, page 1 is displayed on the display screen 2400 of the electronic whiteboards 1b and 1c. However, page 1 is displayed, for example, grayed out, so that it cannot be operated. Also, a message 2401 is displayed on the display screen 2400 indicating that operation of page 1 is restricted and that it cannot be operated. Preferably, this message 2401 includes information (for example, terminal name, destination name, terminal ID, etc.) about the electronic whiteboard 1a that has locked page 1 (operation restriction).

As a result of the above process, editing operations cannot be performed on the stroke images on page 1 from electronic whiteboards 1b and 1c, so user A, who uses electronic whiteboard 1a, can operate the stroke images contained on page 1 with priority.

In the second and third embodiments, an example of restricting the operation of the stroke image of page 1 has been described as an example, but the stroke data that is the subject of the operation restriction may be one or more selected stroke images, as in the first embodiment. Also, the stroke data that is the subject of the operation restriction may be stroke images of multiple pages.

As described above, according to each embodiment of the present invention, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from multiple terminal devices from performing operations on a stroke image that are different from the user's intention.

<Additional Information>
Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), and a conventional circuit module designed to execute each function described above.

In addition, in the above-described embodiment, the terminal device is described as being an electronic whiteboard 1, but this is not limited to the present invention. For example, each embodiment of the present invention can also be applied to an interactive projector 2501 as shown in FIG. 25. The interactive projector 2501 is, for example, a terminal device that projects a display screen 2503 onto a projection surface 2502, and allows the user to write stroke images and select menus with a pen, a user's finger, etc., in the same way as the electronic whiteboard 1.

Other examples of terminal devices may include information terminals such as PCs, smartphones, smart watches, and car navigation devices that execute applications compatible with the communication system 100. Terminal devices also include medical equipment. In the case of medical equipment, the document images may include images of patients, etc.

Furthermore, in each of the embodiments described above, the communication system 100 is used for video conferencing, but the present invention is not limited to this and may be used for meetings, general conversations between family members or friends, remote medical treatment, or one-way presentation of information.

1. Electronic whiteboard (an example of a terminal device)
5. Communications management system (communications management device)
51 Transmitting/receiving unit (receiving unit)
56 Stroke information management unit 57 Operation restriction unit 100 Communication system 1910, 1920 Group operation information 5006 Stroke information

JP 2015-60592 A

Claims (12)

A communication management system for managing a session for sharing a stroke image between a plurality of terminal devices including a first terminal device and a second terminal device, comprising:
a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image;
a receiving unit that receives, from the first terminal device, a group operation request that specifies one or more stroke data to be subjected to an editing operation among the plurality of stroke data;
an operation restriction unit that restricts an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request that specifies the one or more stroke data to be the target of the editing operation;
having
and when the one or more stroke data specified by the group operation request include stroke data for which editing operations are restricted by the operation restriction process , the operation restriction unit starts the operation restriction process, thereby allowing editing operations on the one or more stroke data to be accepted from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device . and when the one or more stroke data specified by the group operation request include stroke data for which editing operations are restricted by the operation restriction process, the operation restriction unit does not allow editing operations on the one or more stroke data to be accepted from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device . 2. The communication management system according to claim 1, wherein the operation restriction unit discontinues processing to prohibit the second terminal device from operating the one or more stroke data when the one or more stroke data specified by the group operation request includes stroke data for which operation is being restricted. The operation limiting unit is
3. A communication management system as described in claim 1 or 2, wherein after prohibiting the second terminal device from operating on the one or more stroke data, when the first terminal device performs an operation on each of the one or more stroke data that are the target of the operation, the prohibition on the operation on the one or more stroke data is terminated. the stroke information management unit manages the plurality of stroke data in association with first identification information for identifying the session;
4. The communication management system according to claim 1, wherein the stroke data includes second identification information for identifying the stroke data, drawing data for reproducing the stroke image, and third identification information for identifying the drawing data. The communication management system according to claim 4 , wherein the group operation request includes the first identification information and the second identification information that identifies the one or more stroke data. 6. A communication system including the communication management system according to claim 1 and a plurality of terminal devices participating in a session managed by the communication management system,
The terminal device
a receiving unit that receives a designation operation for designating one or more pieces of stroke data to be operated on and an editing operation for the one or more pieces of stroke data;
an information sending unit that sends a group operation request to the communication management system, the group operation request specifying one or more stroke data items to be subjected to the editing operation in response to the specifying operation;
A communication system comprising: The communication system according to claim 6, wherein the terminal device has a display control unit that displays, in response to an operation restriction notification notified from the communication management system, display information on a display screen indicating that operations related to one or more stroke data are restricted. The communication system according to claim 6 or 7, wherein the reception unit prohibits the reception of editing operations regarding one or more stroke data in response to an operation restriction notification notified from the communication management system, the operation restriction notification restricting operations regarding the one or more stroke data. The operation restriction unit of the communication management system includes:
A communication system according to any one of claims 6 to 8, further comprising: a group operation request received from a first terminal device, the group operation request specifying one or more stroke data, and an operation restriction notification indicating that operations relating to the one or more stroke data are restricted is sent to each of the second terminal devices among the plurality of terminal devices, the second terminal device being different from the first terminal device. A communication management device that manages a session for sharing stroke images between a plurality of terminal devices including a first terminal device and a second terminal device ,
a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image;
a receiving unit that receives, from the first terminal device, a group operation request that specifies one or more stroke data to be subjected to an editing operation among the plurality of stroke data;
an operation restriction unit that restricts an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request that specifies the one or more stroke data to be the target of the editing operation;
having
the operation restriction unit, when the one or more stroke data specified by the group operation request does not include stroke data for which editing operations are restricted by an operation restriction process, starts the operation restriction process, thereby allowing editing operations on the one or more stroke data to be accepted from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device; and, when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are restricted by the operation restriction process, does not allow editing operations on the one or more stroke data from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device. A communication management system that manages a session for sharing a stroke image among a plurality of terminal devices including a first terminal device and a second terminal device ,
A process of managing stroke information including a plurality of stroke data representing the stroke image;
receiving, from the first terminal device, a group operation request for designating one or more stroke data to be subjected to an editing operation among the plurality of stroke data;
a process of restricting an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request specifying the one or more stroke data to be subjected to the editing operation;
Run
The process of restricting the editing operation, when the one or more stroke data specified by the group operation request does not include stroke data for which editing operations are restricted by an operation restriction process, starts the operation restriction process, thereby making it possible to accept editing operations on the one or more stroke data from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device; and when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are restricted by an operation restriction process, does not make it possible to accept editing operations on the one or more stroke data from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device.
Image processing methods. A program for causing a communication management system to execute the image processing method according to claim 11 .

Images