JP7472638B2 - Information processing device, information processing system, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing system, an information processing method, and a program.

Electronic whiteboards are used at events such as meetings and classes at companies, educational institutions, or government agencies, where an image is displayed on a display and a user draws stroke images of letters, numbers, figures, and the like on the image (see Patent Document 1).

In order to output large data such as high-quality images without compromising image quality, a technology is known for storing image data in a server machine for this electronic whiteboard (see Patent Document 2).

However, conventional methods store image data in an external storage device such as a server, and do not store the operation history of stroke drawing on the display screen. Therefore, when a meeting or other event being held on an electronic whiteboard is temporarily interrupted, the operation history is lost and divided, resulting in the problem that the display screen cannot be restored.

In order to solve the above-mentioned problems, the invention according to claim 1 is an information processing device that outputs a display screen including a stroke image and a background image, and is provided with: a storage means that stores operation history storage position information indicating a storage position in an external storage device of operation history data indicating an operation history of strokes; a communication means that receives from the external storage device the operation history data corresponding to the operation history storage position information stored in the storage means; and an output means that outputs restored screen data in which the display screen is restored based on the operation history data received from the external storage device.

The present invention has the advantage that the display screen displayed on the electronic whiteboard can be restored by reading the operation history stored in the external storage device.

FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system according to an embodiment; FIG. 1 is a diagram for explaining an image of how an electronic whiteboard according to an embodiment is used; FIG. 2 is a diagram illustrating an example of a hardware configuration of an electronic whiteboard according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a data management device according to an embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration of an information processing system according to an embodiment. FIG. 2 is a diagram for roughly explaining an example of display screen information according to the embodiment; 11 is a flowchart illustrating an example of a display process in the electronic whiteboard according to the embodiment. FIG. 4 is a diagram illustrating an example of a data structure of operation history data according to the embodiment. 1A to 1C are diagrams illustrating an example of a display screen shared among electronic whiteboards. 11 is a flowchart illustrating an example of a storage process of background image data in the electronic whiteboard according to the embodiment. FIG. 11 is a sequence diagram showing an example of a data upload process in the information processing system according to the embodiment. FIG. 4 is a conceptual diagram illustrating an example of screen data according to the embodiment. 11 is a flowchart illustrating an example of a storage process of background image data in the electronic whiteboard according to the embodiment. 1A is a sequence diagram showing an example of a download process for operation history data in an information processing system according to an embodiment, and FIG. 1B is a sequence diagram showing an example of a download process for background image data in an information processing system according to an embodiment. FIG. 11 is a conceptual diagram illustrating an example of restored screen data according to the embodiment. FIG. 11 is a sequence diagram illustrating an example of a download process of background image data in an information processing system according to a first modified example of an embodiment.

Below, we will explain the mode for implementing the invention with reference to the drawings. Note that in the explanation of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.

●System Overview●
System Configuration Fig. 1 is a diagram showing an example of the system configuration of an information processing system according to an embodiment. The information processing system shown in Fig. 1 is a system that displays a background image and a stroke image drawn by a user on an electronic whiteboard 1. Users can conduct meetings, classes, etc. while looking at the display screen of the electronic whiteboard 1.

The information processing system is constructed by an electronic whiteboard 1, a PC 400, and a data management device 5. The electronic whiteboard 1 and the data management device 5 are connected to each other via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, etc. Note that the communication network 100 may include not only wired communication, but also networks using wireless communication such as 5G (5th Generation), Wi-Fi (Wireless Fidelity (registered trademark)), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

The electronic whiteboard 1 is installed at a specific location (e.g., location A) and is a handwriting input device that allows a user (e.g., user A1 or user A2) to handwrite strokes on the display. The electronic whiteboard 1 is connected to a PC (Personal Computer) 400 via a cable or the like. The electronic whiteboard 1 displays an image input from the PC 400 on the display as a background image.

Here, the background image is an image displayed on the display of the electronic whiteboard 1. The background image includes, for example, a document image in which an electronic file such as a meeting document is displayed on the display. The background image data is, for example, a large volume of data such as an A0 size image or a high quality image such as 4K or 8K quality. In addition, stroke data indicating strokes input (drawn) on the display is data required to reproduce (reproduce) a stroke image. The stroke data includes coordinate data, line width data, line color data, vector data, data indicating the pen pressure when the stroke was drawn, and the like. The stroke image is an image indicating lines, etc., drawn by a user with a handwritten stroke using an electronic pen or the like. The stroke image is displayed by stroke data indicating points specifying coordinates on the display.

The electronic whiteboard 1 is an example of an information processing device having a communication function, a drawing function, a display function, and the like. The information processing device may be, for example, a PC on which an application program corresponding to an information processing system is installed, a smartphone, a tablet terminal, a smart watch, a car navigation terminal, a game console, or a telepresence robot. Furthermore, the information processing device may be a medical device. In the case of a medical device, the background image is an image of a patient.

Note that while FIG. 1 shows an example in which one electronic whiteboard 1 is installed at location A, the information processing system may be configured to perform remote communication using electronic whiteboards 1 installed at multiple locations. In this case, the electronic whiteboard 1 set up at location A performs mutual communication between electronic whiteboards installed at other locations, for background image data related to the background image displayed on the display, and stroke data related to strokes input (drawn) on the display. This allows the electronic whiteboard 1 to remotely share the display screen with electronic whiteboards installed at other locations.

The PC 400 is an external input device that is connected to the electronic whiteboard 1 via a cable or the like and inputs background image data for displaying a background image on the electronic whiteboard 1. The background image data input from the PC 400 to the electronic whiteboard 1 is data that is used, for example, with word processing software, spreadsheet software, presentation software, or the like. The PC 400 may be, for example, a tablet terminal, a smartphone, a PDA (Personal Data Assistant), a USB (Universal Serial Bus) memory, or the like.

The cable connecting the electronic whiteboard 1 and the PC 400 is a cable capable of communication according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), and VGA (Video Graphics Array). The electronic whiteboard 1 and the PC 400 may also be connected by wireless communication conforming to standards such as Bluetooth (registered trademark).

The data management device 5 is configured by a computer and is communicatively connected to the electronic whiteboard 1 via the communication network 100. The data management device 5 stores and manages data uploaded from the electronic whiteboard 1. The data management device 5 may be configured by a single computer, or may be configured by multiple computers in which each part (function or means) of the device is divided and arbitrarily assigned.

●Usage image of electronic whiteboard FIG. 2 is a diagram for explaining an example of the use of an electronic whiteboard according to an embodiment. As shown in FIG. 2, the electronic whiteboard 1 is composed of a leg 151 having a plurality of casters on the lower side, a support 152 provided on the upper side of the leg 151, a main body 153 of the electronic whiteboard 1 provided on the upper side of the support 152, and a display 180 provided on the front of the main body 153. The main body 153 includes a CPU 101, which will be described later. A user can input (draw) a stroke image such as a character on the display 180 using the electronic pen 190. The display 180 is an example of a display unit. When the pen tip of the electronic pen 190 is pressed against the display 180, it is pushed into the electronic pen 190 by the repulsive force from the display 180. When the pen tip of the electronic pen 190 is pushed into the electronic pen 190, the electronic pen 190 notifies the corresponding electronic whiteboard 1. Communication between the electronic pen 190 and the corresponding electronic whiteboard 1 is performed by infrared rays or the like.

●Hardware configuration●
Next, the hardware configuration of each device constituting the information processing system will be described with reference to Figures 3 and 4. Note that components may be added or deleted from the hardware configurations shown in Figures 3 and 4 as necessary.

3 is a diagram showing an example of the hardware configuration of the electronic whiteboard according to the embodiment. As shown in Fig. 3, the electronic whiteboard 1 includes a central processing unit (CPU) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a solid state drive (SSD) 104, a network interface (I/F) 105, and an external device connection I/F 106.

Of these, the CPU 101 controls the overall operation of the electronic whiteboard 1. The ROM 102 is a non-volatile memory that stores the CPU 101 and programs used to drive the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory used as a work area for the CPU 101. The SSD 104 is a large-capacity storage device that stores various data such as programs for the electronic whiteboard. The network I/F 105 is a communication interface for connecting the electronic whiteboard 1 to the communication network 100 and communicating. The external device connection I/F 106 is an interface for connecting various external devices. In this case, the external devices are, for example, the PC 400, a storage terminal such as a USB (Universal Serial Bus) memory 300, and external devices (microphone 140, speaker 150, camera 160). The USB memory 300 is an example of an external storage device.

The electronic whiteboard 1 also includes a capture device 111, a GPU (Graphics Processing Unit) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a short-range communication circuit 119, an antenna 119a for the short-range communication circuit 119, a power switch 122, and selection switches 123.

Of these, the capture device 111 displays image data (image information) as a still image or a moving image on the display of the external PC 400. The GPU 112 is a semiconductor chip (processor) that specializes in graphics. The display controller 113 controls and manages the screen display to output the output image from the GPU 112 to the display 180. The contact sensor 114 detects that the electronic pen 190 or the user's hand H has touched the display 180. The sensor controller 115 controls the processing of the contact sensor 114. The contact sensor 114 inputs and detects coordinates using an infrared blocking method. The method of inputting and detecting the coordinates is a method in which two light receiving and emitting devices installed at both ends of the upper side of the display 180 emit multiple infrared rays parallel to the display 180, and receive the light that is reflected by a reflecting member provided around the display 180 and returns along the same optical path as the optical path of the light emitted by the light receiving element. The contact sensor 114 outputs the ID of the infrared light emitted by the two light receiving and emitting devices that was blocked by the object to the sensor controller 115, and the sensor controller 115 identifies the coordinate position of the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 190 to determine whether the tip or the end of the pen has touched the display 180. The short-range communication circuit 119 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 122 is a switch for switching the power of the electronic whiteboard 1 ON/OFF. The selection switches 123 are, for example, a group of switches for adjusting the brightness and color of the display 180.

The electronic whiteboard 1 further includes a bus line 110. The bus line 110 is an address bus, a data bus, etc., for electrically connecting each component such as the CPU 101 shown in FIG. 3.

The contact sensor 114 is not limited to an infrared blocking type, and may use various detection means such as a capacitive touch panel that identifies the contact position by detecting changes in capacitance, a resistive film touch panel that identifies the contact position by voltage changes in two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by a contacting object touching the display unit. The electronic pen controller 116 may also determine whether or not the part of the electronic pen 190 that the user holds or other parts of the electronic pen have been touched, in addition to the tip and end of the electronic pen 190.

●Hardware configuration of the data management device Fig. 4 is a diagram showing an example of the hardware configuration of the data management device according to the embodiment. A computer as an example of the data management device 5 includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I/F (Interface) 508, a network I/F 509, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk ReWritable) drive 514, a media I/F 516, and a bus line 510.

Of these, the CPU 501 controls the operation of the entire data management device 5. The ROM 502 is a non-volatile memory that stores programs used to drive the CPU 501, such as IPL. The RAM 503 is a volatile memory used as a work area for the CPU 501. The HD 504 is a large-capacity storage device that stores various data such as communication management programs. The HDD controller 505 controls the reading or writing of various data from the HD 504 according to the control of the CPU 501. The display 506 displays various information such as a cursor, menu, window, character, or image. The external device connection I/F 508 is an interface for connecting various external devices. The network I/F 509 is an interface for data communication using a communication network 100 such as the Internet. The keyboard 511 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 514 controls the reading or writing (storage) of data from the DVD-RW 513. The DVD-RW 513 may be a DVD-R or the like. The DVD-RW drive 514 may also be a Blu-ray drive or a CD-RW drive that controls the reading or writing (storage) of data from a disc such as a BD-RE (Blu-ray (registered trademark) Disc Rewritable) or a CD-RW (Compact Disc-ReWritable). The media I/F 516 controls the reading or writing (storage) of data from a recording medium 515 such as a flash memory. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 4.

The above programs may be distributed by recording them on a computer-readable recording medium in the form of an installable or executable file. Examples of recording media include CD-Rs (Compact Disc Recordable), DVDs, Blu-ray Discs, SD cards, and USB memories. The recording media may be provided domestically or internationally as a program product. For example, the electronic whiteboard 1 realizes the information processing method according to the present invention by executing the program according to the present invention.

●Function configuration●
Next, the functional configuration of the information processing system according to the present embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an example of the functional configuration of the information processing system according to the embodiment. Note that Fig. 5 shows those devices related to the processes or operations described below among those shown in Fig. 1.

●Functional configuration of the electronic whiteboard First, the functional configuration of the electronic whiteboard 1 will be described with reference to Fig. 5. The electronic whiteboard 1 has a network communication unit 11, a reception unit 12, a display control unit 13, a judgment unit 14, an image processing unit 15, a stroke data creation unit 16, an operation history creation unit 17, an image capture/output unit 18, a network control unit 21, an external device communication unit 22, and a storage/readout unit 19. Each of these units is a function or a functioning means realized by any of the components shown in Fig. 3 operating according to an instruction from the CPU 101 according to a program deployed from the SSD 104 to the RAM 103. The electronic whiteboard 1 also has a storage unit 1000 constructed by the RAM 103, the SSD 104, or the USB memory 300 shown in Fig. 3.

The network communication unit 11 is mainly realized by the processing of the CPU 101 on the network I/F 105, and communicates various data or information with other devices via the communication network 100. The network communication unit 11 performs data communication with, for example, the data management device 5 via the communication network 100.

The reception unit 12 is mainly realized by the processing of the CPU 101 on the contact sensor 114 and the electronic pen controller 116, and receives various inputs from the user using the electronic pen 190, etc.

The display control unit 13 is mainly realized by the processing of the CPU 101 on the display controller 113, and performs control for outputting image signals and the like to the display 180. For example, the display control unit 13 causes the display 180 to display a background image captured by the image capture/output unit 18 and a stroke image generated by the image processing unit 15.

The judgment unit 14 is realized by the processing of the CPU 101 and performs various judgments.

The image processing unit 15 is mainly realized by the processing of the CPU 101 on the capture device 111, performs the main processing of the electronic whiteboard function, and generates various screen data related to the display screen to be displayed on the display 180. The image processing unit 15 generates a stroke image based on, for example, the stroke data created by the stroke data creation unit 16. The image processing unit 15 also generates, for example, low-quality image data having a lower quality than the background image.

The stroke data creation unit 16 is mainly realized by the processing of the CPU 101 on the capture device 111, and creates stroke data based on strokes of the electronic pen 190 etc. accepted by the acceptance unit 12. The operation history creation unit 17 is mainly realized by the processing of the CPU 101, and creates operation history data indicating the operation history of the strokes.

The image capture/output unit 18 is mainly realized by the processing of the CPU 101 on the external device connection I/F 106, and captures image data input from an external input device such as the PC 400. The image capture/output unit 18 also outputs screen data generated by the image processing unit 15 to an external input device such as the PC 400.

The network control unit 21 is mainly realized by the processing of the CPU 101, and controls the connection state of the communication network 100. The external device communication unit 22 is mainly realized by the processing of the CPU 101 for the external device connection I/F 106, and performs data communication with a storage terminal such as a USB memory 300.

The storage/reading unit 19 is mainly realized by the processing of the CPU 101, and stores various data (or information) in the storage unit 1000 and reads various data (or information) from the storage unit 1000.

Display screen information Fig. 6 is a diagram for explaining an example of display screen information according to the embodiment. Fig. 7 shows an example of a state of the display screen displayed on the electronic whiteboard 1 stored in the display screen management DB 1001.

In the example of FIG. 6, the display screen has a hierarchical structure of "meeting", "pages (pages 1-3)", and "strokes (strokes 1-5)". "Strokes" correspond to stroke data according to this embodiment, and are managed for each "page". Furthermore, "page" refers to a state in which image data relating to a screen of one page (display screen) displayed on the display 180 is stored. Furthermore, "meeting" refers to information on a meeting being held by the electronic whiteboard 1. For example, stroke 1 includes a stroke ID (id=aaaaa) that identifies stroke 1, information (parent=page01) for specifying page 1 on which stroke 1 exists, gid (global id) "g001" of drawing data 1, drawing data 1, and the like. For example, when an operation such as moving is performed on the stroke image represented by stroke 1, the stroke ID (id=aaaaa) is inherited as is, but drawing data 1 and gid "g001" that identifies drawing data 1 are updated. Drawing data 1 is drawing data for reproducing stroke 1. The drawing data 1 includes, for example, coordinate data, line width data, line color data, and vector data.

The electronic whiteboard 1 allows pages to be switched, and a "page" corresponds to each page of the electronic whiteboard 1. For example, page 1 includes a page ID (id=page01) that identifies page 1, information (parent=se01) for specifying the "meeting" being held on the electronic whiteboard 1, a gid "g01" that identifies the page (page data), and image data 1. Image data 1 is image data of the background image that constitutes page 1. In other words, image data 1 indicates the background image displayed on page 1. Note that a page does not have to include image data. In this case, a page that includes image data means that it does not have a background image (it has a blank background).

"Meeting" corresponds to a meeting being held on the electronic whiteboard 1, and is managed by a meeting ID for identifying the meeting, the gid "g1" of the meeting (meeting information), information indicating the pages generated by the meeting (pages), and information indicating the pages constituting the display screen currently being displayed on the electronic whiteboard 1 (current page). For example, the image processing unit 15 of the electronic whiteboard 1 can generate a stroke image drawn on page 1 using drawing data 1-3 contained in strokes 1-3. Note that a meeting is an example of an event held using the electronic whiteboard 1, and is not limited to a meeting, and may be a class, a meeting, a gathering, a get-together, a consultation, or a meeting, etc.

●Functional Configuration of Data Management Device Next, the functional configuration of the data management device 5 will be described with reference to Fig. 5. The data management device 5 has a communication unit 51, a judgment unit 52, a creation unit 53, a data processing unit 54, and a storage/reading unit 59. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 4 operates in response to an instruction from the CPU 501 in accordance with the data management program loaded from the HD 504 onto the RAM 503. The data management device 5 also has a storage unit 5000 constructed by the RAM 503, HD 504, or recording medium 515 shown in Fig. 4.

The communication unit 51 is mainly realized by the processing of the CPU 501 on the network I/F 509, and communicates various data or information with other devices via the communication network 100. The communication unit 51 transmits and receives background image data and operation history data to and from the electronic whiteboard 1, for example.

The judgment unit 52 is realized by the processing of the CPU 501, and performs various judgments. The creation unit 53 is mainly realized by the processing of the CPU 501, and creates a URL indicating the storage location of the background image data. The data processing unit 54 is mainly realized by the processing of the CPU 501, and performs compression processing of the background image data uploaded from the electronic whiteboard 1 according to the bandwidth status of the communication network 100.

The storage/reading unit 59 is mainly realized by the processing of the CPU 501, and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000.

Processing or operation of the embodiment
Next, the processing or operation of the information processing system according to this embodiment will be described with reference to FIGS.

7 to 9, a display screen displayed on the electronic whiteboard 1 in a meeting or the like using the electronic whiteboard 1 will be described. Fig. 7 is a flowchart showing an example of a display process in the electronic whiteboard according to the embodiment.

First, the image capture/output unit 18 of the electronic whiteboard 1 captures image data from the PC 400 (step S11). In this case, when the image capture/output unit 18 is connected to the PC 400 via a cable or the like, it captures the image data displayed on the PC 400 into the electronic whiteboard 1. Then, the display control unit 13 displays the image data captured by the image capture/output unit 18 on the display 180 as a background image (step S12).

Next, the user touches and moves the electronic pen 190 or hand H on the display 180 of the electronic whiteboard 1, and the reception unit 12 of the electronic whiteboard 1 receives the input of the stroke (trajectory) of the movement (step S13). Here, the "stroke input (drawing)" is, for example, an event in which the user presses the electronic pen 190 against the display 180, moves the electronic pen 190 in this state, and releases the electronic pen 190 from the display 180. The stroke data creation unit 16 creates stroke data (for example, coordinate data (x, y)) for displaying a stroke image on the two-dimensional display 180 based on the stroke input received by the reception unit 12 (step S14). Then, the display control unit 13 displays the stroke image on the display 180 of the electronic whiteboard 1 based on the stroke data created by the stroke data creation unit 16 (step S15).

Then, the operation history creation unit 17 creates operation history data indicating the operation history performed on the display screen of the electronic whiteboard 1 (step S16). FIG. 8 is a diagram showing an example of the data structure of the operation history data according to the embodiment. The operation history data shown in FIG. 8 stores information on operations performed on the display screen displayed on the electronic whiteboard 1. The operation information shown in FIG. 8 includes a stroke ID (Id) for identifying stroke data, type information (Type) indicating the type of stroke to be operated (marker, pen, etc.), coordinate information (Points) indicating the coordinates of the trajectory of the stroke to be operated, and attribute information (Attributes) indicating the line width and line color of the stroke. Here, the line color of the stroke is expressed in RGBA (Red Green Blue Alpha) data format, with each element being expressed by a numerical value of 0-255. The line width (thickness) of the stroke is expressed in the number of pixels. Furthermore, the vertices of the stroke are expressed by XY coordinates, and when each vertex is connected by a Bezier curve, a line segment indicating the stroke is formed.

The stroke to be operated on may be an object, which is a collection of multiple strokes. This object includes text in which strokes are processed by OCR (Optical Character Reader) to convert them into symbols such as letters or numbers, system-generated characters such as dates or times, predetermined shapes such as triangles, stars, or circles, or lines such as arrows, line segments, or Bezier curves.

Now, with reference to FIG. 9, the display screens displayed on the electronic whiteboard 1 will be described. First, the display screen 200a shown in FIG. 9(A) displays a background image d1 and a stroke image st1. The display screen 200b shown in FIG. 9(B) displays a background image d2 and a stroke image st2. Furthermore, the display screen 200c shown in FIG. 9(C) displays a background image d3 and a stroke image st3.

In this way, the electronic whiteboard 1 shares multiple display screens as shown in Figures 9 (A) to (C). The electronic whiteboard 1 also manages screen data (background image data and stroke data) constituting the display screens to be displayed on the display 180 as the display screen information shown in Figure 6. For example, the display screen 200a shown in Figure 6 (A) is a display screen corresponding to the screen data for "page 1", and the display screen 200b shown in Figure 6 (B) is a display screen corresponding to the screen data for "page 2". Also, for example, the display screen 200c shown in Figure 6 (C) is a display screen corresponding to the screen data for "page 3".

10 to 15, a process for storing the background image data captured in the electronic whiteboard 1 in the electronic whiteboard 1 will be described. Fig. 10 and Fig. 13 are flowcharts showing an example of a process for storing background image data in the electronic whiteboard according to the embodiment.

When user A1 or A2 performs a predetermined input operation, the reception unit 12 of the electronic whiteboard 1 receives a request to save the background image displayed on the display 180 (step S31). Note that the electronic whiteboard 1 may be configured to receive a request to save the background image when image data is imported from the PC 400 in step S11 or when the background image is displayed on the display 180 in step S12.

Next, if the connection state to the communication network 100 is online (YES in step S32), the network control unit 21 transitions the process to step S33. Then, the electronic whiteboard 1 executes a data upload process to the data management device 5 (step S33). Here, the process of uploading the background image data captured in the electronic whiteboard 1 to the data management device 5 will be described with reference to FIG. 11. FIG. 11 is a sequence diagram showing an example of a data upload process in the information processing system according to the embodiment.

The network communication unit 11 of the electronic whiteboard 1 transmits background image data to the data management device 5 (step S331). In this case, the network communication unit 11 transmits the background image data to, for example, a predetermined URL for uploading (for example, "http://xxx.jp/upload"). As a result, the communication unit 51 of the data management device 5 receives the background image data transmitted from the electronic whiteboard 1. The creation unit 53 of the data management device 5 creates a URL indicating the storage location in the data management device 5 of the background image data received by the communication unit 51 (step S332). Here, the URL of the background image data is an example of image storage location information, and the image storage location information also includes a URI (Uniform Resource Identifier). The storage/reading unit 59 stores the background image data received in step S331 in the storage location indicated by the URL created in step S332 in the storage unit 5000 (step S333).

Then, the communication unit 51 transmits an upload completion notification to the electronic whiteboard 1 indicating that the upload of the background image data has been completed (step S334). This upload completion notification includes the URL of the background image data created in step S332. As a result, the network communication unit 11 of the electronic whiteboard 1 receives the upload completion notification transmitted from the data management device 5.

Next, the network communication unit 11 of the electronic whiteboard 1 transmits the operation history data to the data management device 5 (step S335). In this case, the network communication unit 11 transmits the operation history data to, for example, a predetermined URL for uploading (for example, "http://xxx.jp/upload"). As a result, the communication unit 51 of the data management device 5 receives the operation history data transmitted from the electronic whiteboard 1. The creation unit 53 of the data management device 5 creates a URL indicating the storage location in the data management device 5 of the operation history data received by the communication unit 51 (step S336). Here, the URL of the operation history data is an example of operation history storage location information, and the operation history storage location information also includes a URI (Uniform Resource Identifier). In addition, the storage/reading unit 59 stores the operation history data received in step S331 in the storage location indicated by the URL created in step S336 in the storage unit 5000 (step S337).

Then, the communication unit 51 transmits an upload completion notification to the electronic whiteboard 1 indicating that the upload of the operation history data has been completed (step S338). This upload completion notification includes the URL of the operation history data created in step S336. As a result, the network communication unit 11 of the electronic whiteboard 1 receives the upload completion notification transmitted from the data management device 5. The background image data upload process shown in steps S335 to S338 is performed for each page of the meeting as shown in FIG. 6, and when multiple image data exist on one page, it is performed for each image data.

In this way, the electronic whiteboard 1 can reduce the amount of data stored inside the electronic whiteboard 1 by uploading the background image data, which is a large volume of data such as a 4K image, and the operation history data indicating the operation history of strokes, to the data management device 5 via the communication network 100. The order of the processing of steps S331 to S334 and the processing of steps S335 to S338 may be reversed, or may be performed in parallel. Also, while FIG. 11 shows an example in which the upload processing of the operation history data is performed at the timing when a save request from a user is accepted, the operation history data may be configured to be uploaded at the timing when the background image data is imported into the electronic whiteboard 1, or may be configured to be uploaded at a predetermined time interval. In this case, the electronic whiteboard 1 can reduce overhead such as the time required for processing when storing data.

Returning to FIG. 10, the image processing unit 15 of the electronic whiteboard 1 generates low-quality image data having a lower quality than the background image displayed on the display 180 (step S34). Next, the image processing unit 15 generates screen data including the URLs received in steps S334 and S338, and the low-quality image data generated in step S34 (step S35). FIG. 12 is a conceptual diagram showing an example of screen data according to an embodiment. The image data shown in FIG. 12 includes PDF data, a URL for operation history data, stroke data, a URL for low-quality image data, and a URL for background image data.

Of these, the PDF data is image data for outputting the contents of the display screen displayed on the electronic whiteboard 1 to the PC 400 or the like. The URL of the operation history data is storage location information indicating the storage location of the operation history data received in step S338. The stroke data is data for generating a stroke image to be displayed on the electronic whiteboard 1. The low-quality image data is image data to be displayed as a background image on the electronic whiteboard 1 when the uploaded data is read. The URL of the background image data is storage location information indicating the storage location of the background image data received in step S334. In this way, the image processing unit 15 generates screen data to be output in which the operation history data, the URL of the background image data, and the low-quality image data are embedded.

Next, the storage/readout unit 19 stores the screen data generated by the image processing unit 15 in the storage unit 1000 (step S35). Then, the electronic whiteboard 1 outputs the screen data generated by the image processing unit 15 (step S36). Specifically, the image capture/output unit 18 of the electronic whiteboard 1 outputs the PDF data included in the screen data generated by the image processing unit 15 to the PC 400. In addition, the display control unit 13 causes the display 180 to display a low-quality image related to the low-quality image data included in the screen data generated by the image processing unit 15.

On the other hand, in step S32, if the connection state to the communication network 100 is offline (NO in step S32), the network control unit 21 transitions the process to step S38. If the network connection state is offline, the electronic whiteboard 1 stops uploading data (step S38) and ends the process. Note that the electronic whiteboard 1 may be configured to interrupt uploading data when the network connection state is offline, and to resume processing from step S33 when the network state switches to online.

Next, the process of downloading data uploaded to the data management device 5 will be described with reference to Figures 13 to 15. In Figure 13, when user A1 or A2 performs a predetermined input operation, the reception unit 12 of the electronic whiteboard 1 receives a request to read the data uploaded to the data management device 5 (step S51).

Next, the storage/reading unit 19 reads out the image data stored in the storage unit 1000, and obtains the URL of the operation history data indicated in the image data (step S52). If the connection state to the communication network 100 is online (YES in step S53), the network control unit 21 transitions the process to step S54. Then, the electronic whiteboard 1 executes a process of downloading the operation history data to the data management device 5 (step S54). Here, the process of downloading the operation history data uploaded to the data management device 5 will be described with reference to FIG. 14(A). FIG. 14(A) is a sequence diagram showing an example of a process of downloading operation history data in the information processing system according to the embodiment.

The network communication unit 11 of the electronic whiteboard 1 transmits a download request indicating a request to download the operation history data to the URL related to the data management device 5 acquired in step S52 (step S541). As a result, the communication unit 51 of the data management device 5 receives the download request transmitted from the electronic whiteboard 1.

Next, the storage/read unit 59 of the data management device 5 reads the requested operation history data from the storage unit 5000 based on the URL indicated in the download request received by the communication unit 51 (step S542). The communication unit 51 then transmits the requested operation history data to the electronic whiteboard 1 that made the request (step S543). As a result, the network communication unit 11 of the electronic whiteboard 1 downloads (receives) the operation history data.

Returning to FIG. 13, the storage/readout unit 19 of the electronic whiteboard 1 reads out the image data stored in the storage unit 1000 and obtains the URL of the background image data indicated in the image data (step S55). The electronic whiteboard 1 then executes a process of downloading the background image data to the data management device 5 (step S56). Here, the process of downloading the operation history data uploaded to the data management device 5 will be described with reference to FIG. 14(B). FIG. 14(B) is a sequence diagram showing an example of a process of downloading background image data in the information processing system according to the embodiment.

The network communication unit 11 of the electronic whiteboard 1 transmits a download request indicating a request to download background image data to the URL related to the data management device 5 acquired in step S55 (step S561). As a result, the communication unit 51 of the data management device 5 receives the download request transmitted from the electronic whiteboard 1.

Next, the storage/reading unit 59 of the data management device 5 reads the requested background image data from the storage unit 5000 based on the URL indicated in the download request received by the communication unit 51 (step S562). The communication unit 51 then transmits the requested background image data to the electronic whiteboard 1 that made the request (step S563). This causes the network communication unit 11 of the electronic whiteboard 1 to download (receive) the background image data.

Returning to FIG. 13, the image processing unit 15 of the electronic whiteboard 1 generates restored screen data in which the display screen displayed on the display 180 is restored (step S57). FIG. 15 is a conceptual diagram showing an example of restored screen data according to an embodiment. The image data shown in FIG. 15 includes PDF data, operation history data, stroke data, and background image data.

Of these, the operation history data is the operation history data received in step S543. The PDF data is image data for outputting the contents of the display screen restored based on the operation history indicated in the operation history data to the PC 400 or the like. The stroke data is data for generating a stroke image restored based on the operation history indicated in the operation history data. The background image data is the background image data received in step S563.

Then, the electronic whiteboard 1 outputs the restored screen data generated by the image processing unit 15 (step S58). Specifically, the image capture/output unit 18 of the electronic whiteboard 1 outputs the PDF data included in the restored screen data generated by the image processing unit 15 to the PC 400. In addition, the display control unit 13 causes the display 180 to display a background image related to the background image data and a stroke image related to the stroke data included in the restored screen data generated by the image processing unit 15.

On the other hand, in step S53, if the connection state to the communication network 100 is offline (NO in step S53), the network control unit 21 transitions the process to step S59. If the network connection state is offline, the electronic whiteboard 1 stops the data download (step S59) and ends the process. Note that the electronic whiteboard 1 may be configured to interrupt the data download when the network connection state is offline, and to resume the process from step S54 when the network state switches to online.

In this way, when the electronic whiteboard 1 reads data uploaded to the data management device 5, it downloads background image data and operation history data corresponding to each URL based on the URL (storage location information) embedded in the screen data. Then, the electronic whiteboard 1 sequentially replaces the low-quality image data embedded in the screen data with the downloaded background image data. Furthermore, the electronic whiteboard 1 generates restored screen data in which the display screen is restored in order starting from the most recent operation based on the operation history indicated in the downloaded operation history data.

In conventional electronic whiteboards, when large-capacity background image data such as A0-size images or 4K-quality images was loaded into the electronic whiteboard 1 or a local PC, the loading took a long time and put a strain on disk capacity. There was also a method of lowering the image quality when loading, but this resulted in a deterioration in the image quality of high-quality images, making it impossible to reproduce the display screen displayed on the electronic whiteboard 1. Furthermore, in conventional systems, when data related to the display screen displayed on the electronic whiteboard (stroke data, background image data) was stored in an external device, only the content (stroke data, background image data) was stored, and no information on the operation history for the display screen was stored. Therefore, for example, when a meeting using the electronic whiteboard was temporarily interrupted, the operation history was lost and divided, resulting in the problem that the display screen could not be restored.

The electronic whiteboard 1 therefore uploads and stores in the data management device 5 the operation history for the display screen in addition to the stroke images and background image data contained in the display screen. Then, when reading data, the electronic whiteboard 1 restores the display screen by sequentially downloading the operation history data from the most recent based on the stored URL. This allows the electronic whiteboard 1 to restore the display screen using the operation history including data such as deleted strokes. When reading data, the electronic whiteboard 1 also outputs low-quality image data until the download of the background image data is complete, and when the download of the background image data is complete, it sequentially changes to low-quality image data and outputs the background image data, allowing a meeting or the like using the electronic whiteboard 1 to start smoothly.

Modifications of the embodiment
Next, an information processing system according to a modification of the embodiment will be described. Note that the same configurations and functions as those in the above embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

Variation 1
First, a description will be given of an information processing system according to Modification 1. The information processing system according to Modification 1 is a system in which the data management device 5 compresses background image data in accordance with the network bandwidth of the communication network 100, and transmits the processed data to the electronic whiteboard 1.

Figure 16 is a sequence diagram showing an example of a process for downloading background image data in an information processing system according to the first modified example of the embodiment. The process in Figure 16 is executed in place of the process in Figure 14 (B) in the above-described embodiment.

The network communication unit 11 of the electronic whiteboard 1 transmits a download request indicating a request to download background image data to the URL related to the data management device 5 acquired in step S55 (step S561a). This download request includes network bandwidth information indicating the state of the network bandwidth of the communication network 100. As a result, the communication unit 51 of the data management device 5 receives the download request transmitted from the electronic whiteboard 1.

Next, the data processing unit 54 of the data management device 5 determines the compression rate of the background image data based on the network bandwidth information received in step S561a (step S562a). The storage and reading unit 59 also reads the requested background image data from the storage unit 5000 based on the URL indicated in the download request received by the communication unit 51 (step S563a). The data processing unit 54 performs compression processing of the background image data read in step S563a based on the compression rate determined in step S562a (step S564a).

Then, the communication unit 51 transmits the background image data compressed in step S564a to the electronic whiteboard 1 that made the request (step S565a). As a result, the network communication unit 11 of the electronic whiteboard 1 downloads (receives) the background image data.

As a result, in the information processing system of variant example 1, the data management device 5 generates background image data of an appropriate size according to the network bandwidth status of the communication network 100 and transmits it to the electronic whiteboard 1, thereby reducing the time required to download data and enabling meetings, etc. using the electronic whiteboard 1 to be started smoothly.

Variation 2
Next, an information processing system according to Modification 2 will be described. The information processing system according to Modification 2 is a system that stores operation history data and background image data in an external storage device that is a storage terminal such as a USB memory 300, instead of the data management device 5 via the communication network 100. The storage terminal is not limited to the USB memory 300, and may be a PC, a hard disk, or the like connected via an external device connection I/F.

In this case, in FIG. 11, the external device communication unit 22 of the electronic whiteboard 1 transmits operation history data and background image data to the storage terminal. Then, the external device communication unit 22 receives a data path indicating the respective storage locations of the operation history data and the background image data from the storage terminal. This data path is an example of operation history storage location information and image storage location information.

In addition, in Figs. 14(A) and (B), the external device communication unit 22 transmits a read request for the operation history data and background image data to the storage terminal corresponding to each data path, and reads the operation history data and background image data from the storage terminal. Then, the electronic whiteboard 1 outputs restored image data that is restored based on the read operation history data and background image data, similar to the above-described embodiment.

As a result, the information processing system according to variant example 2 can restore the display screen even in an electronic whiteboard 1 that is operated in a stand-alone state and not connected to the communication network 100.

●Summary●
As described above, the electronic whiteboard 1 according to an embodiment of the present invention is an information processing device that outputs a display screen including a stroke image and a background image. The electronic whiteboard 1 stores operation history storage location information (e.g., URL or data path) indicating a storage location in an external storage device (e.g., data management device 5 or storage terminal) of operation history data indicating an operation history of a stroke, receives operation history data corresponding to the stored operation history storage location information from the external storage device, and outputs restored screen data in which the display screen is restored based on the received operation history data. This allows the electronic whiteboard 1 to restore the display screen displayed on the electronic whiteboard by reading the operation history stored in the external storage device.

The electronic whiteboard 1 according to one embodiment of the present invention stores operation history storage location information (e.g., URL or data path) indicating the storage location of the operation history data in an external storage device (e.g., data management device 5 or storage terminal) and image storage location information (e.g., URL or data path) indicating the storage location of the background image data in the external storage device, receives background image data corresponding to the stored image storage location information from the external storage device, and outputs the received background image data as restored screen data on the display screen. As a result, the electronic whiteboard 1 uploads the background image data, which is large data such as a 4K image quality image, and the operation history data indicating the operation history of strokes, to the data management device 5 via the communication network 100, and restores the display screen using the uploaded data, thereby reducing the amount of data stored inside the electronic whiteboard 1.

Furthermore, the electronic whiteboard 1 according to one embodiment of the present invention generates screen data including operation history storage location information (e.g., a URL or data path) indicating the storage location of the operation history data in an external storage device (e.g., a data management device 5 or a storage terminal), image storage location information (e.g., a URL or data path) indicating the storage location of the background image data in an external storage device, and low-quality image data with a lower quality than the background image, outputs the low-quality image data until the background image data is received, and when the background image data is received, outputs the background image data in place of the low-quality image data. As a result, when reading data, the electronic whiteboard 1 outputs low-quality image data until the download of the background image data is complete, and when the download of the background image data is complete, outputs the background image data in place of the low-quality image data, thereby enabling a meeting or the like using the electronic whiteboard 1 to be started smoothly.

In addition, when background image data is imported from the PC 400 (an example of an external input device), the electronic whiteboard 1 according to one embodiment of the present invention transmits the background image data to an external storage device (e.g., the data management device 5 or a storage terminal). In addition, the electronic whiteboard 1 transmits operation history data to an external storage device (e.g., the data management device 5 or a storage terminal) at a predetermined time interval. This enables the electronic whiteboard 1 to reduce overhead, such as the time required for processing when storing data.

Furthermore, in the electronic whiteboard 1 according to one embodiment of the present invention, the operation history storage location information indicating the storage location of the operation history data and the image storage location information indicating the storage location of the background image data are data paths of a storage terminal (an example of an external storage device) in which the operation history data and the background image data are stored, respectively, and the electronic whiteboard 1 receives the operation history data and background image data corresponding to the stored data paths from the storage terminal, and outputs restored screen data including background image data restored based on the received operation history data. This allows the electronic whiteboard 1 to restore the display screen even when it is operated in a stand-alone state not connected to the communication network 100.

The information processing system according to one embodiment of the present invention includes an electronic whiteboard 1 (an example of an information processing device) and a data management device 5 (an example of an external storage device) that can communicate with the electronic whiteboard 1 via a communication network 100. The data management device 5 receives a download request for background image data from the electronic whiteboard 1, compresses the background image data according to the network bandwidth of the communication network 100, and transmits the processed background image data to the electronic whiteboard 1. As a result, the information processing system according to one embodiment of the present invention reduces the time required for downloading data and allows meetings, etc. using the electronic whiteboard 1 to start smoothly by having the data management device 5 generate background image data of an appropriate size according to the state of the network bandwidth of the communication network 100 and transmit it to the electronic whiteboard 1.

●Additional Information●
Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU, and a conventional circuit module designed to execute each function described above.

The various tables in the above-described embodiments may be generated by the learning effect of machine learning, and by classifying the data of each associated item by machine learning, tables may not be used. Here, machine learning is a technology for enabling a computer to acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates algorithms required for judgments such as data identification from learning data that is previously loaded, and applies these to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

So far, we have described an information processing device, information processing system, information processing method, and program according to one embodiment of the present invention, but the present invention is not limited to the above-mentioned embodiment, and can be modified within the scope of what a person skilled in the art can imagine, such as adding, changing, or deleting other embodiments, and any aspect is within the scope of the present invention as long as it achieves the functions and effects of the present invention.

1. Electronic whiteboard (an example of an information processing device)
5. Data management device (an example of an external storage device)
11 Network communication unit (an example of a communication means)
13 Display control unit (an example of an output means)
15 Image processing unit (an example of a generating means)
18 Image capture/output unit (an example of an output means)
22 External device communication unit (an example of a communication means)
51 Communication unit (an example of a receiving means, an example of a transmitting means)
54 Data processing unit (an example of a data processing means)
200a, 200b, 200c Display screen 300 USB memory (an example of an external storage device)
400 PC (an example of an external input device)
1000 Storage unit (an example of a storage means)

JP 2015-70543 A JP 2015-210569 A

Claims (8)

An information processing device that outputs a display screen including a stroke image and a background image,
a storage means for storing operation history storage location information indicating a storage location in an external storage device of operation history data indicating an operation history of strokes, and image storage location information indicating a storage location in the external storage device of background image data ;
a communication means for receiving, from the external storage device, the operation history data corresponding to the operation history storage location information stored in the storage means, and the background image data corresponding to the image storage location information stored in the storage means;
an output means for outputting restored screen data, the restored screen data including the background image data received from the external storage device, the display screen being restored based on the operation history data received from the external storage device ;
a generating means for generating screen data including the operation history storage location information, the image storage location information, and low-quality image data having a lower quality than the background image;
Equipped with
the output means outputs the low quality image data until the background image data is received, and when the background image data is received, outputs the background image data in place of the low quality image data.
Information processing device. 2. The information processing apparatus according to claim 1 , wherein said communication means transmits the background image data to said external storage device when said background image data is input from an external input device. 3 . The information processing apparatus according to claim 1 , wherein the operation history storage location information and the image storage location information are URLs of the external storage device in which the operation history data and the background image data are stored, respectively. the operation history storage location information and the image storage location information are data paths of the external storage device in which the operation history data and the background image data are stored, respectively;
the communication means receives the operation history data and the background image data corresponding to the data path from the external storage device;
The information processing apparatus according to claim 1 , wherein the output means outputs the restored screen data, which is restored based on the received operation history data and includes the background image data. The information processing apparatus according to claim 1 , wherein the communication means transmits the operation history data to the external storage device at a predetermined time interval. 6. An information processing system comprising: the information processing device according to claim 1 ; and the external storage device capable of communicating with the information processing device via a communication network,
The external storage device includes:
a receiving means for receiving a download request for background image data from the information processing device;
a data processing means for performing a compression process on the background image data in accordance with a network bandwidth of the communication network;
a transmitting means for transmitting the processed background image data to the information processing device;
An information processing system comprising: An information processing method executed by an information processing device that outputs a display screen including a stroke image and a background image,
The information processing device includes:
a storage means for storing operation history storage location information indicating a storage location in an external storage device of operation history data indicating an operation history of the stroke , and image storage location information indicating a storage location in the external storage device of background image data ,
a communication step of receiving, from the external storage device, the operation history data corresponding to the operation history storage location information stored in the storage means, and the background image data corresponding to the image storage location information stored in the storage means ;
an output step of outputting restored screen data, the restored screen data including the background image data received from the external storage device, the display screen being restored based on the operation history data received from the external storage device ;
a generating step of generating screen data including the operation history storage location information, the image storage location information, and low-quality image data having a lower quality than the background image;
Run
The output step outputs the low quality image data until the background image data is received, and when the background image data is received, outputs the background image data instead of the low quality image data.
Information processing methods. A program to be executed by an information processing device that outputs a display screen including a stroke image and a background image,
The information processing device includes:
a storage means for storing operation history storage location information indicating a storage location in an external storage device of operation history data indicating an operation history of the stroke , and image storage location information indicating a storage location in the external storage device of background image data ,
a communication step of receiving, from the external storage device, the operation history data corresponding to the operation history storage location information stored in the storage means, and the background image data corresponding to the image storage location information stored in the storage means ;
an output step of outputting restored screen data, the restored screen data including the background image data received from the external storage device, the display screen being restored based on the operation history data received from the external storage device ;
a generating step of generating screen data including the operation history storage location information, the image storage location information, and low-quality image data having a lower quality than the background image;
Run the command ,
The output step outputs the low quality image data until the background image data is received, and when the background image data is received, outputs the background image data instead of the low quality image data.
program.