JP7615609B2 - COMMUNICATION TERMINAL, SCREEN SHARING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a communication terminal, a screen sharing method, and a program .

In recent years, with the spread of the Internet, cloud computing has come to be used in a variety of fields. Cloud computing is a form of service usage in which users use a communication terminal connected to the Internet to access services (cloud services) provided by servers on the Internet and pay a fee for those services.

For example, when multiple communication terminals share a screen, technology has been known for some time that displays the screen in a way that corresponds to the communication terminal even if the resolution is different (see, for example, Patent Document 1).

For example, in a screen sharing system in which multiple communication terminals share a screen, each communication terminal displays a shared image according to the screen resolution of its display. Screen resolution is the total number of pixels displayed on a display unit such as a display. Therefore, in conventional screen sharing systems, when the screen resolution of a shared image is greater than the screen resolution of the display unit of a communication terminal, the shared image is reduced for display, resulting in a problem of reduced definition. Note that the technology described in Patent Document 1 does not solve the above-mentioned problem.

The object of the present invention is to provide a communication terminal that displays a shared image without reducing the resolution, even if the screen resolution of the display unit of the communication terminal is smaller than the screen resolution of the shared image.

In order to achieve the above problem, claim 1 of the present application is a communication terminal that receives a shared image with a screen resolution and aspect ratio displayed by a sharing communication terminal via a network and displays it on a display unit, and when the aspect ratio of the received shared image differs from the aspect ratio of the screen of the display unit, the display control means cuts out a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit, and displays the cut-out predetermined area of the shared image on the display unit, wherein the display control means cuts out the predetermined area of the shared image based on the number of pixels in the horizontal direction of the screen of the display unit when the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit .

According to an embodiment of the present invention, even if the screen resolution of the display unit of the communication terminal is smaller than the screen resolution of the shared image, the shared image can be displayed without reducing the resolution.

FIG. 1 is a system configuration diagram of an example of a screen sharing system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication terminal and an extension terminal according to the embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. FIG. 2 is a functional block diagram of an example of a communication terminal according to the present embodiment. 11 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. 11 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG. 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11A and 11B are diagrams illustrating an example of a process for cutting out a predetermined area of a shared image; 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG. 11 is a sequence diagram illustrating an example of a synchronization process of a shared image in the screen sharing system according to the embodiment. FIG.

The following describes an embodiment of the present invention with reference to the drawings.

[First embodiment]
<System Configuration>
Fig. 1 is a system configuration diagram of an example of a screen sharing system according to this embodiment. In the screen sharing system 1 shown in Fig. 1, a distribution management device 2, a communication terminal 5a, a communication terminal 5b, an extension terminal 6, and a web server 8 communicate with each other via a network 9. Note that the communication terminals 5a and 5b may be collectively referred to as communication terminals 5. Also, although Fig. 1 shows an example in which there are two communication terminals 5, there may be three or more communication terminals.

The communication terminal 5 is an example of an information processing terminal operated by a user who receives the services of the screen sharing system 1. The multiple communication terminals 5 receive the shared images via the network 9 and display them on a display, which is an example of a display unit.

For example, communication terminals 5a and 5b are electronic whiteboards that have the function of displaying a shared image and the function of accepting operations by an operator such as a user using a finger or electronic pen. Communication terminal 5a is controllably connected to extension terminal 6. Communication terminal 5a has an extension mode that displays its own display unit and the display of extension terminal 6 as a single large screen. Extension terminal 6 may be any electronic device that can realize the extension mode with communication terminal 5a, and may be, for example, an electronic whiteboard similar to communication terminal 5a. Communication terminal 5a displays a shared image using its own display and the display of extension terminal 6 as a single large screen. Also, for example, communication terminal 5b displays a shared image on its own display.

Note that the communication terminal 5 is not limited to an electronic whiteboard, but may be, for example, a projector, a video conferencing terminal, a PC (Personal Computer), a smartphone, or a tablet terminal.

Network 9 is a communication network capable of data communication, such as the Internet or a LAN (Local Area Network). Network 9 also includes networks using wireless communication, such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

The distribution management device 2 has a so-called cloud browser (hereinafter referred to as "browser") as a web browser that exists on the cloud, and obtains content images by rendering web content on the cloud. The distribution management device 2 can distribute the obtained content images such as H.264 or MPEG-4 to the communication terminal 5. The content images can be used as shared images to be shared by the communication terminal 5 of the screen sharing system 1 in FIG. 1. The distribution management device 2 also provides a real-time communication (RTC: Real-time communication/collaboration) function. The web server 8 provides web content to the browser of the distribution management device 2.

<Hardware configuration of communication terminal and extension terminal>
The hardware configuration of the communication terminal 5 and the expansion terminal 6 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a hardware configuration diagram of an example of a communication terminal and an expansion terminal according to this embodiment. Fig. 2 illustrates an example of a communication terminal 5a connected to the expansion terminal 6. The hardware configuration of the communication terminal 5b is the same as that of the communication terminal 5a.

The communication terminal 5a in FIG. 2 has a display 300, a controller 350, and a display input unit 358. The display 300 has a touch unit 302, a panel unit 304, a video input terminal 306, a power linkage unit 308, and a controller slot 310. The controller 350 has a communication unit 352, a control unit 354, and a storage unit 356.

The touch unit 302 detects operations performed by an operator using a finger or an electronic pen. The panel unit 304 displays video (images). The video input terminal 306 accepts video input from the outside. For example, the video input terminal 306 accepts an HDMI (registered trademark) input, a VGA input, or an internal input from the controller 350.

The power interlocking unit 308 coordinates power control when connected to the expansion terminal 6. The power interlocking unit 308, for example, coordinates the turning on or off of the power of the communication terminal 5a and the expansion terminal 6. The power interlocking unit 308, for example, coordinates the transition to or return from an energy saving mode of the communication terminal 5a and the expansion terminal 6. The controller slot 310 is used to connect to the controller 350.

The display 300 of the communication terminal 5a can function as an extension terminal by itself. For example, as shown in FIG. 2, the display 300 of the communication terminal 5a and the display 200 of the extension terminal 6 have the same configuration. The display 200 of the extension terminal 6 has a touch unit 202, a panel unit 204, a video input terminal 206, a power supply linkage unit 208, and a controller slot 210.

The display 200 can function as a display unit of the expansion terminal 6 by connecting a controller 350 to the controller slot 210. The display 300 can also function as a display unit of the communication terminal 5a by connecting a controller 350 to the controller slot 310.

The controller 350 also has a communication unit 352, a control unit 354, and a storage unit 356. The communication unit 352 is connected to the network 9. The control unit 354 performs various input/output control or calculations. The storage unit 356 is an SSD (Solid State Drive) or HDD (Hard Disk Drive) and stores data. The storage unit 356 obtains target data for screen sharing from the distribution management device 2 and stores it.

The control unit 354 has, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The control unit 354 realizes the functional configuration described below by the CPU using the RAM as a work area and executing a program stored in the ROM or the storage unit 356.

When the extended terminal 6 is connected, the control unit 354 of the communication terminal 5a can acquire position information (described below) from both the touch unit 302 of the communication terminal 5a and the touch unit 202 of the extended terminal 6. In addition, when the extended terminal 6 is connected, the control unit 354 can output video to both the video input terminal 306 of the panel unit 304 of the communication terminal 5a and the video input terminal 206 of the panel unit 204 of the extended terminal 6.

<Hardware configuration of distribution management device and web server>
The distribution management device 2 and web server 8 shown in Fig. 1 are realized, for example, by a computer 500 having a hardware configuration shown in Fig. 3. Furthermore, the communication terminals 5a and 5b shown in Fig. 1 may be realized, for example, by a computer 500 having a hardware configuration shown in Fig. 3. Fig. 3 is a diagram showing an example of the hardware configuration of a computer according to this embodiment.

The computer 500 includes a CPU 501, a ROM 502, a RAM 503, a HD 504, a HDD controller 505, a display 506, an external device connection I/F 508, a network I/F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I/F 516.

The CPU 501 controls the overall operation of the computer 500 according to a program. The ROM 502 stores programs used to drive the CPU 501, such as IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 according to the control of the CPU 501.

The display 506 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I/F 508 is an interface for connecting various external devices. An external device is, for example, a USB memory. The network I/F 509 is an interface for data communication using the network 9. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 514 controls the reading and writing of various data from the DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I/F 516 controls the reading and writing (storing) of data from the recording medium 515, such as a flash memory.

<Functional configuration of communication terminal>
Next, functions of the communication terminal 5 will be described with reference to Fig. 4. Fig. 4 is a functional block diagram of an example of a communication terminal according to the present embodiment. Each function of the communication terminal 5 is a function or means realized by operating any of the components shown in Fig. 2 by an instruction from a CPU according to a program deployed in a work area.

The communication terminal 5 in FIG. 4 includes a receiving unit 20, a transmitting unit 22, a display control unit 24, and a detecting unit 26. Note that FIG. 4 does not illustrate functions that are not necessary for explaining this embodiment.

The receiving unit 20 communicates with the communication unit 352 and receives data from the communication unit 352. For example, the receiving unit 20 receives information specifying target data to be shared on the screen with other communication terminals 5, such as conference materials, and notifies the display control unit 24. The transmitting unit 22 communicates with the communication unit 352 and transmits data to the communication unit 352. The receiving unit 20 and the transmitting unit 22 are realized, for example, by the CPU of the control unit 354 shown in FIG. 2 using the RAM as a work area to execute processing according to a program.

The display control unit 24 controls the display such as drawing processing and image output. The display control unit 24 obtains the target data to be shared on the screen from the storage unit 356 and uses it for control of the display such as drawing processing and image output. The display control unit 24 may have a configuration including a drawing processing unit and an image output unit. The display control unit 24 is realized, for example, by an instruction from the CPU of the control unit 354 shown in FIG. 2, the panel unit 304, etc. The detection unit 26 detects the position information of the operation by the operator's finger or electronic pen on the touch unit 302. The detection unit 26 obtains the coordinate data of the position where the operator's finger or electronic pen touches or approaches the touch unit 302, and detects the handwritten input or operation by the operator from the obtained coordinate data. The detection unit 26 is realized, for example, by an instruction from the CPU of the control unit 354 shown in FIG. 2, the touch unit 302, etc.

<Example of shared image display>
5 is a conceptual diagram of an example of a process for displaying a shared image on a plurality of communication terminals via a distribution management device. At a first base (left side of FIG. 5), a communication terminal 5a, an extension terminal 6, and a communication terminal 7a as a video conference terminal are used. At a second base (right side of FIG. 5), a communication terminal 5b and a communication terminal 7b as a video conference terminal are used.

At the first location, an electronic pen P2 is used to display operation data such as stroked characters on a communication terminal 5a. At the second location, an electronic pen P1 is used to display operation data such as stroked characters on a communication terminal 5b.

At the first base, the video and audio data acquired by the communication terminal 7a is encoded by the encoding unit 40 and then transmitted to the distribution management device 2. The video and audio data is then decoded by the decoding unit 36 of the distribution management device 2 and input to the browser 30. In addition, operation data indicating strokes drawn on the communication terminal 5a by the electronic pen P2 is transmitted to the distribution management device 2 and input to the browser 30.

Meanwhile, at the second base, the video and audio data acquired by the communication terminal 7b is also encoded by the encoding unit 40 and then transmitted to the distribution management device 2. The data is then decoded by the decoding unit 36 of the distribution management device 2 and input to the browser 30. In addition, operation data such as strokes drawn on the communication terminal 5b by the electronic pen P1 is transmitted to the distribution management device 2 and input to the browser 30.

The browser 30, for example, acquires web content data [A] of a background image to be displayed at each of the first and second locations from the web server 8. The browser 30 acquires operation data [p2] and video and audio data [E2] from the first location. The browser 30 also acquires operation data [p1] and video and audio data [E1] from the second location.

The browser 30 combines and renders the web content data [A], the operation data [p1] and [p2], and the video and audio data [E1] and [E2]. The browser 30 generates a shared image with the screen resolution and aspect ratio of area b of the screen in extended mode realized by the communication terminal 5a and the extended terminal 6 at the first location, for example. Screen resolution is the total number of pixels on the screen. For example, the screen resolution and aspect ratio of area b of the screen in extended mode realized by the communication terminal 5a and the extended terminal 6 at the first location become the screen resolution and aspect ratio of the screens of the communication terminal 5a and the extended terminal 6 treated as one large screen.

The encoder bridge unit 32 encodes the shared image generated by the browser 30, and the distribution management device 2 distributes it to each base. At the first base, the shared image in area b is displayed on the extended mode screen using the display 300 of the communication terminal 5a and the display 200 of the extended terminal 6, and sound based on the video and audio data [E1] of the second base is output from the communication terminal 7a.

Meanwhile, at the second location, the shared image of area a is displayed on the screen of the display 300 of the communication terminal 5b, and sound based on the video and audio data [E2] of the first location is output from the communication terminal 7b. The screen sharing system 1 according to this embodiment is effective for remote conferences, etc., because it allows the same information to be shared in real time between remote locations such as the first location and the second location.

In the example of FIG. 5, the shared image in area b is displayed at the first location in accordance with the landscape screen in extended mode. On the other hand, the display 300 of the communication terminal 5b at the second location has a portrait screen. Therefore, in the example of FIG. 5, if it is to be displayed with the same aspect ratio as the shared image in area b displayed at the first location, it is necessary to reduce the shared image to area a so that it fits on the display 300 of the communication terminal 5b. Therefore, the shared image is reduced and displayed at the second location, resulting in a decrease in the definition of the shared image.

In this embodiment, as shown in FIG. 6, the second location cuts out area d from the shared image of area b displayed at the first location as described below, and displays area d of the cut-out shared image on the display 300 of the communication terminal 5b. FIG. 6 is a conceptual diagram of an example of a process for displaying a shared image on multiple communication terminals via a distribution management device. As shown in FIG. 6, when the screen resolution and aspect ratio of the shared image differ from the screen resolution and aspect ratio of the display 300 of the communication terminal 5b, and the entire shared image cannot be displayed without being reduced, the communication terminal 5b at the second location processes as follows.

The communication terminal 5b cuts out a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the display 300 of the communication terminal 5b, and displays the cut-out area d of the shared image on the display 300 of the communication terminal 5b. In the example of FIG. 6, since the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the display 300 of the communication terminal 5b, the area d of the shared image is cut out and displayed based on the number of pixels in the horizontal direction of the display 300 of the communication terminal 5b.

In addition, if the number of vertical pixels of the shared image is greater than the number of vertical pixels of the display 300 of the communication terminal 5b, the communication terminal 5b cuts out and displays the area of the shared image based on the number of vertical pixels of the display 300.

Furthermore, if the number of vertical and horizontal pixels of the shared image is greater than the number of vertical and horizontal pixels of the display 300 of the communication terminal 5b, the communication terminal 5b cuts out and displays the area of the shared image based on the number of vertical and horizontal pixels of the display 300.

In this way, communication terminal 5b receives the entire shared image with the screen resolution and aspect ratio of area b displayed at the first location, and cuts out and displays a predetermined area of the shared image to be displayed on display 300 from the received shared image data. Furthermore, the predetermined area of the shared image cut out by communication terminal 5b changes according to the operation (such as a touch operation or a drag operation) with the operator's finger or electronic pen detected by detection unit 26. If the operator wishes to display an area other than the predetermined area of the shared image displayed on display 300 of communication terminal 5b, he or she can change the display range by operating with a finger or electronic pen.

Figure 7 is a sequence diagram of an example of synchronization processing of a shared image in the screen sharing system according to this embodiment. In the following explanation, an example will be explained in which the communication terminal 5a at the first location is a presenter terminal operated by a presenter, and the communication terminal 5b at the second location is a participant terminal operated by a participant. Although one participant terminal is shown in Figure 7, multiple terminals may be used.

In step S10, the presenter operates the shared screen of the communication terminal 5a, such as zooming in, zooming out, and page forwarding. In step S12, the detection unit 26 of the communication terminal 5a detects a change in the display range. The operation on the display screen is, for example, a scroll operation, a pinch-in/pinch-out operation, a swipe operation, a flick operation, a double-tap operation, and the like. In step S14, the detection unit 26 notifies the transmission unit 22 of the change in the display range.

In step S16, the transmission unit 22 transmits a page synchronization message to the conference server device 101. The page synchronization message includes information that specifies the display range. For example, the transmission and reception of a screen sharing synchronization message such as a page synchronization message may use the XMPP (Extensible Messaging and Presence Protocol) communication protocol or another communication protocol.

In step S18, the distribution management device 2 transfers the received page synchronization message to one or more communication terminals 5b. In step S20, the receiving unit 20 of the communication terminal 5b notifies the display control unit 24 of receipt of the page synchronization message. In step S22, the display control unit 24 requests the display area of the display unit from the detection unit 26. In step S24, the display control unit 24 receives a response regarding the display area of the display unit from the detection unit 26.

In step S26, the display control unit 24 requests the target data for screen sharing from the storage unit 356. In step S28, the display control unit 24 acquires the target data for screen sharing from the storage unit 356. Note that if there is no target data for screen sharing in the storage unit 356 and it is necessary to acquire new target data for screen sharing, there may be a process of receiving the target data for screen sharing from the distribution management device 2.

In step S30, the display control unit 24 of the communication terminal 5b cuts out an area d to be displayed on the display 300 from the shared image of the area b displayed at the first location, as shown in FIG. 6, for example. In step S32, the display control unit 24 displays the area d of the shared image cut out in step S30 on the display 300.

In step S34, the display control unit 24 performs enlargement or reduction processing so that the shared image in area b displayed at the first location is displayed in area c, which is the display area of the display 300 of the communication terminal 5b shown in Figure 6. In the example of Figure 6, reduction processing is performed so that the shared image in area b displayed at the first location can be displayed in area c of the display 300 of the communication terminal 5b.

In step S36, the display control unit 24 superimposes, for example, display area information onto area c so that it is possible to recognize which part of the shared image area d corresponds to. The display area information is, for example, information that represents, by a rectangle or the like, an area corresponding to area d on the reduced shared image displayed in area c. In step S38, the display control unit 24 displays, in area c, the shared image onto which the display area information has been superimposed.

The process of step S30 for cutting out area d to be displayed on display 300 from the shared image of area b displayed at the first location is performed, for example, as follows. Figures 8 to 13 are explanatory diagrams of an example of the process for cutting out a predetermined area of a shared image.

Figure 8 shows an example of the shared image, the camera image of the sharing source, the camera image of the sharing destination, and the screen of the display unit of the second location, which are necessary for explaining the processing of step S30. Figure 8 (a) shows the shared image displayed on the display unit of the sharing source, which is used for screen sharing. Figure 8 (b) shows a camera image (acquired video) taken by communication terminal 7a at the first location, which is the sharing source of the shared image. Figure 8 (c) shows a camera image taken by communication terminal 7b at the second location, which is the sharing destination of the shared image. Also, Figure 8 (d) shows the display screen (display area) of the display 300 of communication terminal 5b at the second location, which is the sharing destination of the shared image.

In the example of the shared image, the camera image from which the image is shared, the camera image of the shared destination, and the screen of the display unit of the second location shown in FIG. 8, in the processing of step S30, a specific area of the shared image is cut out, for example as shown in FIG. 9, and displayed on the display 300 of the communication terminal 5b.

Figure 9 shows an example in which the number of horizontal pixels (horizontal size) of the shared image used for screen sharing is greater than the number of horizontal pixels of the display 300 of the sharing destination communication terminal 5b. The display screen of the display 300 of the sharing destination communication terminal 5b shown in Figure 9(a) has fewer horizontal pixels than the shared image, as indicated by the arrow in Figure 9(b).

Therefore, communication terminal 5b cuts out a predetermined area of the shared image based on the number of pixels in the horizontal direction of display 300 of communication terminal 5b, as indicated by the arrow in FIG. 9(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. Also, in FIG. 9(b), the camera image of the sharing source and the camera image of the sharing destination are displayed below area d of the cut-out shared image.

In addition, in the example of the shared image, the source camera image, the destination camera image, and the screen of the display unit of the second location shown in FIG. 8, in the processing of step S30, a predetermined area of the shared image may be cut out, for example as shown in FIG. 10, and displayed on the display 300 of the communication terminal 5b.

Figure 10 shows an example in which the number of horizontal pixels (horizontal size) of the shared image used for screen sharing is greater than the number of horizontal pixels (horizontal size) of the display 300 of the sharing destination communication terminal 5b. Figure 10(b) shows a similar state to Figure 9(b). There is unused display area on the display screen in Figure 10(b).

Therefore, communication terminal 5b expands a specified area of the shared image in the direction in which the unused display area decreases, as indicated by the arrow in FIG. 10(c), and then cuts it out based on the number of pixels in the horizontal direction of display 300 of communication terminal 5b, and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b.

Figure 11 shows an example in which the number of vertical pixels (horizontal size) of a shared image for screen sharing is the same as the number of vertical pixels of display 300 of sharing destination communication terminal 5b. The display screen of display 300 of sharing destination communication terminal 5b shown in Figure 11(a) has the same number of vertical pixels as the shared image, as shown in Figure 11(b). Also, the display screen of display 300 of sharing destination communication terminal 5b shown in Figure 11(a) has fewer horizontal pixels than the shared image, as shown in Figure 11(b).

Therefore, communication terminal 5b cuts out a predetermined area of the shared image based on the number of pixels in the vertical and horizontal directions of display 300 of communication terminal 5b shown in FIG. 11(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. In FIG. 11(b), the camera image of the sharing source and the camera image of the sharing destination are displayed superimposed on area d of the cut-out shared image.

Figure 12 shows an example of the shared image, the camera image of the sharing source, the camera image of the sharing destination, and the screen of the display unit of the second location, which are necessary for explaining the processing of step S30. Figure 12 (a) shows the shared image displayed on the display unit of the sharing source, which is used for screen sharing. Figure 12 (b) shows a camera image taken by communication terminal 7a at the first location, which is the sharing source of the shared image. Figure 12 (c) shows a camera image taken by communication terminal 7b at the second location, which is the sharing destination of the shared image. Also, Figure 12 (d) shows the display screen of the display 300 of communication terminal 5b at the second location, which is the sharing destination of the shared image.

In the example of the shared image, the source camera image, the destination camera image, and the screen of the display unit of the second location shown in FIG. 12, in the processing of step S30, a predetermined area of the shared image is cut out, for example as shown in FIG. 13, and displayed on the display 300 of the communication terminal 5b.

Figure 13 shows an example in which the number of vertical pixels (vertical size) of the shared image during screen sharing is greater than the number of vertical pixels (vertical size) of the display 300 of the sharing destination communication terminal 5b. The display screen of the display 300 of the sharing destination communication terminal 5b shown in Figure 13(a) has fewer vertical pixels than the shared image, as indicated by the arrow in Figure 13(b).

Therefore, communication terminal 5b cuts out a predetermined area of the shared image based on the number of pixels in the vertical direction of display 300 of communication terminal 5b, as indicated by the arrow in FIG. 13(b), and displays area d of the cut-out shared image on the display screen of display 300 of communication terminal 5b. In FIG. 13(b), the camera image of the sharing source and the camera image of the sharing destination are displayed to the right of area d of the cut-out shared image.

In this way, the process of step S30 for cutting out area d to be displayed on display 300 from the shared image of area b displayed at the first location cuts out and displays a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the display unit of the sharing destination. Since the communication terminal 5b to which the shared image is shared cuts out and displays a predetermined area of the shared image on the display unit without reducing the shared image, the shared image can be displayed without reducing definition even if the screen resolution of the display unit of the sharing destination is smaller than the screen resolution of the shared image.

Figure 14 is a sequence diagram of an example of a synchronization process for a shared image in the screen sharing system according to this embodiment. In step S50, a participant performs a touch operation such as a drag operation on area c or d displayed on the display screen of communication terminal 5b.

In step S52, the detection unit 26 detects a change in the display range based on an operation performed by the participant on area c or d displayed on the display screen. In step S54, the detection unit 26 notifies the display control unit 24 of the detected change in the display range. In step S56, the display control unit 24 calculates the movement distance (movement amount) of the display area based on the change in the display range notified by the detection unit 26.

In step S58, the display control unit 24 requests the target data to be screen-shared from the storage unit 356. In step S60, the display control unit 24 acquires the target data to be screen-shared from the storage unit 356. In step S62, the display control unit 24 of the communication terminal 5b cuts out an area d to be displayed on the display 300 from the shared image. In step S64, the display control unit 24 displays the area d of the shared image cut out in step S62 on the display 300.

In step S66, the display control unit 24 performs a process of reducing the shared image in area b displayed at the first location so that it can be displayed in area c of the display 300 of the communication terminal 5b shown in FIG. 6, for example.

In step S68, the display control unit 24 combines the display area information with area c so that it is possible to perceive which part of the shared image area d corresponds to. In step S70, the display control unit 24 displays the shared image combined with the display area information in area c.

Figure 15 is a sequence diagram of an example of a synchronization process for a shared image in the screen sharing system according to this embodiment. In step S80, a participant performs a touch operation such as a pinch-in or pinch-out operation on area c or d displayed on the display screen of communication terminal 5b.

In step S82, the detection unit 26 detects a change in the display range based on an operation performed by the participant on area c or d displayed on the display screen. In step S84, the detection unit 26 notifies the display control unit 24 of the detected change in the display range. In step S86, the display control unit 24 calculates the enlargement or reduction rate (size) of the display area based on the change in the display range notified by the detection unit 26.

In step S88, the display control unit 24 requests the target data to be screen-shared from the storage unit 356. In step S90, the display control unit 24 acquires the target data to be screen-shared from the storage unit 356. In step S92, the display control unit 24 of the communication terminal 5b cuts out an area d from the shared image to be displayed on the display 300. In step S94, the display control unit 24 displays the area d of the shared image cut out in step S92 on the display 300.

In step S96, the display control unit 24 performs a process of reducing the shared image in area b displayed at the first location so that it can be displayed in area c of the display 300 of the communication terminal 5b shown in FIG. 6, for example.

In step S98, the display control unit 24 combines the display area information with area c so that it is possible to perceive which part of the shared image area d corresponds to. In step S100, the display control unit 24 displays the shared image combined with the display area information in area c.

<Other application examples>
The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

For example, in this embodiment, the process of cutting out a specific area from a shared image is performed on the sharing destination communication terminal 5b, but this may be performed on the distribution management device 2 or on a server other than the distribution management device 2.

Furthermore, each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), and conventional circuit modules designed to execute each function described above.

REFERENCE SIGNS LIST 1 Screen sharing system 2 Distribution management device 5, 5a, 5b Communication terminal 6 Extended terminal 8 Web server 9 Network 20 Receiving unit 22 Transmitting unit 24 Display control unit 26 Detection unit 200, 300 Display 202, 302 Touch unit 204, 304 Panel unit 350 Controller 354 Control unit 356 Storage unit 358 Display input unit

JP 2015-60292 A

Claims (10)

A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing communication terminal via a network and displays the shared image on a display unit,
a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of the shared image on the display unit;
having
When the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit, the display control means cuts out a predetermined area of the shared image based on the number of pixels in the horizontal direction of the screen of the display unit.
A communication terminal characterized by : A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing communication terminal via a network and displays the shared image on a display unit,
a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of the shared image on the display unit;
having
The communication terminal is characterized in that, when the number of vertical pixels of the shared image is greater than the number of vertical pixels of the screen of the display unit and the number of horizontal pixels of the shared image is less than the number of horizontal pixels of the screen of the display unit, the display control means cuts out a predetermined area of the shared image based on the number of vertical pixels of the screen of the display unit. When an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, the display control means enlarges the shared image and then cuts out a predetermined area of the shared image based on the number of pixels in the vertical or horizontal direction of the screen of the display unit.
2. The communication terminal according to claim 1, 4. The communication terminal according to claim 1 , wherein the display control means changes a predetermined area to be cut out from the shared image based on an operation on the display screen received from an operator. The communication terminal according to any one of claims 1 to 4, characterized in that the display control means further causes the display unit to display display area information indicating which part of the shared image the specified area displayed on the display unit corresponds to. A communication terminal as claimed in any one of claims 1 to 5, characterized in that the screen resolution and aspect ratio of the shared image are the screen resolution and aspect ratio of an image displayed as a single display screen using multiple display units by one or more communication terminals that are the source of sharing. A screen sharing method for a communication terminal, which receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit,
a display control step of, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit, and displaying the cropped predetermined area of the shared image on the display unit;
having
The display control step includes, when a number of pixels in a horizontal direction of the shared image is greater than a number of pixels in a horizontal direction of the screen of the display unit and a number of pixels in a vertical direction of the shared image is less than a number of pixels in a vertical direction of the screen of the display unit, cutting out a predetermined area of the shared image based on the number of pixels in a horizontal direction of the screen of the display unit.
A screen sharing method characterized by: A screen sharing method for a communication terminal, which receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit,
a display control step of, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit, and displaying the cropped predetermined area of the shared image on the display unit;
having
The display control step includes, when a number of pixels in a vertical direction of the shared image is greater than a number of pixels in a vertical direction of the screen of the display unit and a number of pixels in a horizontal direction of the shared image is less than a number of pixels in a horizontal direction of the screen of the display unit, cutting out a predetermined area of the shared image based on the number of pixels in a vertical direction of the screen of the display unit.
A screen sharing method characterized by: A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit.
a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of the shared image on the display unit;
Functioning as a
When the number of pixels in the horizontal direction of the shared image is greater than the number of pixels in the horizontal direction of the screen of the display unit and the number of pixels in the vertical direction of the shared image is less than the number of pixels in the vertical direction of the screen of the display unit, the display control means cuts out a predetermined area of the shared image based on the number of pixels in the horizontal direction of the screen of the display unit.
A program characterized by. A communication terminal that receives a shared image having a screen resolution and aspect ratio displayed by a sharing source communication terminal via a network and displays the shared image on a display unit.
a display control means for, when an aspect ratio of the received shared image differs from an aspect ratio of a screen of the display unit, cropping a predetermined area of the shared image based on the number of pixels in at least one of the vertical and horizontal directions of the screen of the display unit and displaying the cropped predetermined area of the shared image on the display unit;
Functioning as a
When the number of pixels in the vertical direction of the shared image is greater than the number of pixels in the vertical direction of the screen of the display unit and the number of pixels in the horizontal direction of the shared image is less than the number of pixels in the horizontal direction of the screen of the display unit, the display control means cuts out a predetermined area of the shared image based on the number of pixels in the vertical direction of the screen of the display unit.
A program characterized by.

Images