JP7694938B2 - Information processing system, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing system, an information processing method, and an information processing program.

Conventionally, there are systems that generate subtitle images to be superimposed on video data captured by a camera or the like. For example, Patent Document 1 describes a subtitle display system that generates subtitle image data for displaying a subtitle image within a predetermined display area of a video monitor that has been set in advance. The generated subtitle image data is superimposed on the video data captured by the camera and broadcast on television.

JP 2018-196076 A

However, with conventional technology, companies that produce subtitles had to install a separate device for producing subtitles for each device that would use the subtitle data, and the cost of using the subtitle data could increase as a result of installing the new device.
Furthermore, when the generated subtitle data is transmitted and received via a communication line, the cost of using the subtitle data may increase due to the introduction of equipment to build a dedicated system used for transmitting and receiving the subtitle data.

The present invention has been made in consideration of the above circumstances, and has as its object to provide an information processing system, an information processing method, and an information processing program that can reduce the cost of using subtitle data.

(1) In order to solve the above problem, the information processing system includes a first terminal that is a client terminal in a Web conference system, a second terminal that is another client terminal in the Web conference system and is connected to the first terminal via the Web conference system, and an information processing device that is communicatively connected to the first terminal, the information processing device includes an acquisition unit that acquires material data from the first terminal, a generation unit that generates subtitle data based on the material data acquired by the acquisition unit, and a provision unit that provides the first terminal with image data that includes at least the subtitle data generated by the generation unit, the first terminal includes a first application for using the Web conference system and a first display device that displays the image data provided by the provision unit, and the second terminal includes a second display device that displays the image data acquired via the Web conference system and displayed on the first display device.

(2) In addition, in the information processing system of the embodiment, the first display device may include a main monitor that displays the settings of the subtitle data and a second monitor that displays the image data, and the second terminal may acquire the image data displayed on the second monitor that is shared via the web conference system.

(3) In addition, in the information processing system of the embodiment, the first application may include plug-in software having at least one of a caption data setting function that displays caption data settings on the main monitor to enable the setting of caption data, or an image data display function that displays image data provided from the providing unit.

(4) In addition, in the information processing system of the embodiment, the second terminal may further include a capture unit that captures a display screen of the image data displayed on the second display device, and an output unit that outputs the display screen captured by the capture unit to the third display device.

(5) In addition, in the information processing system of the embodiment, the second terminal may provide input data entered by a user of the second terminal to the first application, and the first application may provide the input data acquired from the second terminal to the information processing device as material data.

(6) In addition, in the information processing system of the embodiment, the second terminal may provide text data input by a user of the second terminal as input data to the first application.

(7) In addition, in the information processing system of the embodiment, the second terminal may provide voice data input by a user of the second terminal as input data to the first application.

(8) In addition, in the information processing system of the embodiment, the second terminal may determine a display area in which image data is displayed corresponding to the second display device.

(9) In order to solve the above problem, the information processing method is an information processing method executed in an information processing system including a first terminal which is a client terminal in a Web conference system, a second terminal which is another client terminal in the Web conference system and is connected to the first terminal via the Web conference system, and an information processing device communicatively connected to the first terminal, and includes an acquisition step executed in the information processing device for acquiring material data from the first terminal, a generation step executed in the acquisition step for generating subtitle data based on the material data acquired in the acquisition step, and a provision step for providing image data including at least the subtitle data generated in the generation step to the first terminal, the information processing device including an activation step executed in the first terminal for activating a first application for using the Web conference system, and a first display step executed in the provision step for displaying the image data provided in the provision step, and a second display device step executed in the second terminal for displaying the image data acquired via the Web conference system and displayed in the first display step.

(10) In order to solve the above problem, the information processing program is executed in an information processing system including a first terminal which is a client terminal in a Web conference system, a second terminal which is another client terminal in the Web conference system and is connected to the first terminal via the Web conference system, and an information processing device which is communicatively connected to the first terminal, and in the information processing device, realizes an acquisition function for acquiring material data from the first terminal, a generation function for generating subtitle data based on the material data acquired in the acquisition function, and a provision function for providing image data including at least the subtitle data generated in the generation function to the first terminal, thereby realizing in the first terminal a first application function for using the Web conference system and a first display function for displaying the image data provided in the provision function, and in the second terminal a second display function for displaying the image data acquired via the Web conference system and displayed in the first display function.

According to one embodiment of the present invention, an information processing device acquires material data from a first terminal, generates subtitle data based on the acquired material data, and provides image data including at least the generated subtitle data to the first terminal, the first terminal uses a web conference system to display the provided image data, and the second terminal communicates with a first application via the web conference system to acquire image data to be displayed on the first display device, and displays the image data, thereby reducing the cost of using the subtitle data.

1 is a block diagram illustrating an example of a system configuration of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating a first display example of a display device used in the information processing system according to the embodiment. FIG. 11 is a diagram illustrating a second display example of the display device used in the information processing system in the embodiment. FIG. 13 is a diagram illustrating a third display example of the display device used in the information processing system in the embodiment. FIG. 13 is a diagram showing a fourth display example of the display device used in the information processing system in the embodiment. FIG. 13 is a diagram showing a fifth display example of a display device used in the information processing system in the embodiment. FIG. 13 is a diagram showing a sixth display example of a display device used in the information processing system in the embodiment. FIG. 13 is a diagram showing a seventh display example of the display device used in the information processing system in the embodiment. 1 is a block diagram showing an example of a hardware configuration of an information processing device according to an embodiment. 5 is a flowchart illustrating an example of an operation of the information processing device according to the embodiment. 10 is a flowchart illustrating an example of an operation of a first terminal in the embodiment. 10 is a flowchart illustrating an example of an operation of a second terminal in the embodiment.

Below, an information processing system, information processing method, and information processing program according to one embodiment of the present invention will be described in detail with reference to the drawings.

First, the configuration of the information processing system will be described using FIG. 1. FIG. 1 is a block diagram showing an example of the system configuration of an information processing system in an embodiment.

In FIG. 1, the information processing system 100 has an information processing device 1, a Web conference server 2, a first terminal 3, and a second terminal 4. The information processing device 1, the Web conference server 2, the first terminal 3, and the second terminal 4 are communicatively connected via a network 9. FIG. 1 illustrates an example of the configuration and connection method of the information processing system 100, and does not limit the configuration and connection method of the information processing system 100.

The information processing device 1 has the following functional units: a communication control unit 11, an authentication unit 12, a storage unit 13, an acquisition unit 14, a UI provision unit 15, a determination unit 16, a generation unit 17, and a provision unit 18. Each of the above functional units of the information processing device 1 in this embodiment will be described as a functional module realized by the information processing program (software) in this embodiment.

The communication control unit 11 controls communication with the first terminal 3 connected via the network 9.

The authentication unit 12 authenticates the user of the first terminal 3 using the communication control unit 11. The user of the first terminal 3 is a person who has the authority to use the functions provided by the information processing device 1 that is communicatively connected to the first terminal 3. The user of the first terminal 3 is, for example, an operator who operates the first terminal 3. The authentication unit 12 authenticates users who have user authority, for example, by a login ID and password assigned to each user. Note that the user in this embodiment does not have to be a specified individual. For example, when one login ID is used by multiple people in sequence, the user can be the person currently using the user ID. Note that the authentication unit 12 may use authentication methods other than authentication by login ID and password (for example, biometric authentication, magnetic card authentication, etc.).

The storage unit 13 stores user information. User information is, for example, authentication information including a login ID and a password, setting information for generating subtitle data set for each user, or material data (described later) stored for each user.

The acquisition unit 14 acquires material data. The material data is data that is the source for generating subtitle data, which will be described later. The subtitle data is generated based on part or all of the material data. That is, the material data may include a portion that is not used as subtitle data. The material data is, for example, text data including character strings, audio data including audio information, graphic data represented by vector data, or image data including image information, or a combination of these. The acquisition unit 14 acquires material data from inside the information processing device 1 (for example, the storage unit 13) or from the first terminal 3. FIG. 1 illustrates an example in which the acquisition unit 14 acquires material data from the first terminal 3. Note that the first terminal 3 may acquire material data from the second terminal 4 and provide the acquired material data to the acquisition unit 14.

In this embodiment, the term "acquisition" of data may refer to pull-type data acquisition, such as downloading, or push-type data acquisition, in which data transmitted from another device is received. For example, the acquisition unit 14 may provide a UI (User Interface) for inputting or selecting material data to be provided to the first terminal 3, and the user of the first terminal 3 may acquire the material data by operating the displayed UI.

The text data is, for example, data manually input from a keyboard (not shown) by the user of the first terminal 3 or the user of the second terminal 4. The data manually input from the keyboard by the user of the second terminal 4 is acquired by the first terminal 3 and provided to the acquisition unit 14. The text data may also include time information, weather information, traffic information, breaking news, subtitles, etc.

The voice data is, for example, data that is acquired by the first terminal 3 from voice input by the user of the second terminal 4 via a microphone (not shown) and provided to the acquisition unit 14. Note that the voice data may also include text information generated by voice recognition.

The image data is, for example, data that is acquired by the first terminal 3 from an image selected or input by a user of the second terminal 4 and provided to the acquisition unit 14. Note that the image data may also include text information generated by image recognition.

The UI providing unit 15 provides a UI for setting telop data. The setting of telop data is a setting for determining the telop data to be generated, such as the selection of material data, the display range of the telop, the character size of the telop, the font or color setting, decoration for the telop, or the display shape of the telop. The UI providing unit 15 may provide a UI to the first application of the first terminal 3, for example, so that the telop data can be set via the first application. The UI providing unit 15 may also be a web server that provides a web page to the first terminal 3. The first terminal 3 can set the telop data by displaying the UI provided by the UI providing unit 15 in a web browser or plug-in software 311 described later. The storage unit 13 may store the setting contents set in the UI provided by the UI providing unit 15.

The UI providing unit 15 may provide the first terminal 3 with a preview screen of the subtitle data generated by the generating unit 17 described below.

The determination unit 16 determines the type of material data acquired by the acquisition unit 14 (text data, audio data, or image data) and recognizes the content of the material data. For example, if the material data is text data, the determination unit 16 determines the number of characters or the type of characters (for example, kanji or hiragana). If the material data is audio data, the determination unit 16 may recognize the content of the audio and generate text data. If the material data is image data, the determination unit 16 may recognize the number of pixels of the image, etc.

The generating unit 17 generates telop data based on the judgment made by the judging unit 16 and the settings made in the UI. For example, if the material data is text data, the generating unit 17 generates telop data by determining the size of the characters of the telop data or the display speed (the number of characters displayed per second) based on the size of the display area of the telop and the number of characters displayed based on the text data. If the material data is voice data, the generating unit 17 may generate telop data by determining the size of the characters of the text data generated by voice recognition or the display speed. If the material data is image data, the generating unit 17 may generate telop data by changing the number of pixels of the image based on the number of pixels of the recognized image and the number of pixels of the display area of the telop. The UI may specify a portion of the material data to be used as telop data. For example, the UI may specify the first 10 characters of the text data or a specific portion of the image data. The generating unit 17 generates the material data of the specified portion as telop data.

Note that in this embodiment, a subtitle is information that is superimposed, inserted or combined (hereinafter sometimes referred to as "superimposed, etc.") with an original image such as a moving image or a still image, and is character information, symbol information, graphic information, audio information or image information, or a combination of these (hereinafter sometimes referred to as "character information, etc."). Subtitle data is data used to display subtitles, and may be, for example, vector data before rasterization, or raster data after rasterization. Furthermore, the subtitle data generated by the generation unit 17 may be image data that at least includes the subtitle data.

The original image on which the telop data is superimposed is a display screen shared in a web conference system described below. The original image may be a television image used for television broadcasting, a video used in a video distribution service, a display image displayed on a large display device, or the like. In this embodiment, the data used for the telop superimposed on the original image is the telop data. Note that telops may also be called captions, subtitles, tickers, alerts, superimposes, or the like depending on the region, content, or format in which they are used, and are included in the telops of this embodiment.

The providing unit 18 provides image data including at least the subtitle data generated by the generating unit 17 to the first terminal 3 connected via the network 9. The image data provided from the providing unit 18 to the first terminal 3 is used in the first terminal 3. For example, the first terminal 3 displays the image data including at least the subtitle data provided from the providing unit 18 on the first display device 32. The first terminal 3 may also use the provided image data for television broadcasting, videos used in video distribution services, or display images displayed on a large display device. The provided data provided by the providing unit 18 is, for example, communication data encoded according to a communication protocol via the network 9, or may be encrypted data encrypted using a predetermined encryption method.

The providing unit 18 may provide, as image data, caption data in a data format corresponding to the input of a transmission device that transmits the caption data to another terminal (not shown). For example, the providing unit 18 may provide a signal conforming to the video electronics standard of the serial digital interface (SDI) or data including that signal. The providing unit 18 may transmit the caption data in accordance with the transmission timing of the transmission device. The providing unit 18 may also transmit the caption data in response to a request from the transmission device.

The Web conference server 2, the first terminal 3, and the second terminal 4 can constitute a Web conference system in this embodiment. A Web conference is a communication means for client terminals connected to a network 9 to exchange or share data such as video, audio, or text in real time. A Web conference system is a system for conducting Web conferences. The Web conference system in this embodiment may be a server-client system or a peer-to-peer system. The Web conference server 2 is a service terminal that provides Web conference services, and the first terminal 3 and the second terminal 4 are client terminals that receive the Web conference service (participate in the Web conference). Note that, although FIG. 1 illustrates an example in which two terminals, the first terminal 3 and the second terminal 4, participate in a Web conference, the number of terminals participating in a Web conference may be three or more.

For example, in a server-client system, the Web conference server 2 acts as a server and provides Web conference services to the first terminal 3 and the second terminal 4, which are client terminals. The first terminal 3 and the second terminal 4 may each log in to the Web conference server 2 and use the Web conference service by exchanging data such as video via the Web conference server 2. The Web conference server 2 may be a cloud server built on the Internet cloud.

In addition, in the peer-to-peer method, the Web conference server 2 may provide destination information such as a destination URL (Uniform Resource Locator) or IP (Internet Protocol) address to the first terminal 3 and the second terminal 4, or may establish communication between the first terminal 3 and the second terminal 4 using the provided destination information. After communication between the first terminal 3 and the second terminal 4 is established, the first terminal 3 and the second terminal 4 may directly exchange data such as video without going through the Web conference server 2.

The first terminal 3 and the second terminal 4 are client terminals participating in a Web conference. The first terminal 3 and the second terminal 4 may be general-purpose devices such as desktop PCs, notebook PCs, tablet PCs, or smartphones, or may be devices dedicated to Web conferences. Among the client terminals of a Web conference, a terminal that provides and shares data such as video is called a host terminal, and a terminal that receives and shares data such as video from the host terminal is called a guest terminal. In FIG. 1, the first terminal 3 is an example of a host terminal that provides data such as video to the second terminal 4 using a Web conference system. The second terminal 4 is an example of a guest terminal that receives data such as video from the first terminal 3 using a Web conference system. For example, in a Web conference, the terminal that holds the conference (starts the Web conference) can be the host terminal. The host terminal and the guest terminal in the Web conference system can be set for each Web conference. In addition, the host terminal and the guest terminal may be switched during the Web conference. Additionally, multiple guest terminals may participate in a web conference, and guest terminals other than the second terminal 4 may also be able to participate.

The first terminal 3 has a first application 31 and a first display device 32. The first application 31 has plug-in software 311. Furthermore, the first display device 32 has a main monitor 321 and a second monitor 322. The second terminal 4 has a second application 41, a second display device 42, a capture unit 43, an output unit 44, and a third display device 45. The second application 41 has plug-in software 411. Furthermore, the second display device 42 has a main monitor 421 and a second monitor 422.

The first application 31 and the second application 41 are application software for using the Web conference system. For example, the first application 31 logs into the Web conference server 2 and starts a Web conference with the first terminal 3 as the host terminal. The second application 41 logs into the Web conference server 2 and participates in the Web conference with the second terminal 4 as the guest terminal. The first application 31 provides data such as video to the second terminal 4, which is a guest terminal participating in the started Web conference. The first application 31 and the second application 41 are application software for using the Web conference system. Note that the first application 31 or the second application 41 can be implemented as a dedicated application executed on, for example, a PC or a smartphone. The first application 31 or the second application 41 may also be implemented by a Web browser.

The plug-in software 311 is software for adding a function for using the information processing device 1 to the first application 31. The plug-in software 311 has at least one of a caption data setting function for displaying caption data settings on the first display device 32 to enable caption data setting, and an image data display function for displaying image data acquired from the providing unit 18 of the information processing device 1. The plug-in software 311 may have other functions. The plug-in software 311 is activated, for example, by a user operating a toolbar or ribbon on the display screen of the first application 31 displayed on the first display device 32. When the first application 31 is implemented by a web browser, the plug-in software 311 may be implemented as a plug-in for the web browser.

The first display device 32 has at least a main monitor 321 and a second monitor 322. The second display device 42 has at least a main monitor 421 and a second monitor 422. The first application 31 and the second application 41 can share the display screens of the first display device 32 and the second display device 42 via a Web conference system. For example, the first application 31 and the second application 41 can share the display screen displayed on the main monitor 321 of the first terminal 3 by displaying it on the main monitor 421 of the second terminal 4. In addition, the first application 31 and the second application 41 can share the display screen displayed on the second monitor 322 of the first terminal 3 by displaying it on the second monitor 422 of the second terminal 4.

In this embodiment, the main monitor 321 and the second monitor 322 are each a display area on the first display device 32. Similarly, the main monitor 421 and the second monitor 422 are each a display area on the second display device 42. The main monitor 321 and the second monitor 322 may each be displayed on a separate display device (e.g., an LCD monitor, etc.), or each area may be displayed on a single display device. Similarly, the main monitor 421 and the second monitor 422 may each be displayed on a separate display device, or each area may be displayed on a single display device.

In this embodiment, the main monitor 321 is a display area that displays the UI of the caption data setting function. The UI of the caption data setting function is provided, for example, by the UI providing unit 15. The second monitor 322 is a display area that displays image data obtained from the providing unit 18. The image data displayed on the second monitor 322 includes the caption data generated by the generating unit 17. By displaying the display screen displayed on the second monitor 322 on the second monitor 422, it is possible to share the display screen that displays the image data including the caption data.

The display areas of the main monitor 321 and the second monitor 322 can be set by the user of the first terminal 3. For example, if the first terminal 3 has one or more display devices, the user may set a part of the display area of the display device to the main monitor 321, and another part of the display area of the display device to the second monitor 322. Also, if the first terminal 3 has two or more display devices, the user may set one of the display devices to the main monitor 321, and the remaining one to the second monitor 322. There may be three or more display areas. For example, there may be a display area of a third monitor (not shown).

Similarly, the display areas of the main monitor 421 and the second monitor 422 can be set by the user of the second terminal 4. For example, if the second terminal 4 has one or more display devices, the user may set a portion of the display area of the display device to the main monitor 421, and set another portion of the display area of the display device to the second monitor 422. Also, if the second terminal 4 has two or more display devices, the user may set any one of the display devices to the main monitor 421, and set the remaining one to the second monitor 422.

<Telop data setting function>
The telop data setting function is a function for setting the telop data generated by the above-mentioned generation unit 17. Setting the telop data includes, for example, selecting the material data used for the telop data, setting the display range of the telop, or setting the size, font, or color of the telop characters. The plug-in software 311 displays a UI for setting telop data on the main monitor 321, allowing the user to use the telop data setting function.

For example, in selecting material data to be used for subtitle data, the user of the first terminal 3, which is the host terminal of the web conference, can select the input data input by the user of the second terminal 4, which is the guest terminal, as the material data. The input data input by the user of the second terminal 4 is, for example, text data, voice data, or image data. The text data is input, for example, by the user operating the keyboard (not shown) of the second terminal 4. The voice data is input, for example, by collecting the user's voice from the microphone (not shown) of the second terminal 4. Also, the image data is input, for example, by the user selecting an image (such as an emoticon) stored in the second terminal 4. The input data can be input, for example, by the chat function of the web conference system. The chat function is a function that provides input data such as text data input by a participant of the web conference (a user of the host terminal or a user of the guest terminal) to the terminals of other participants in real time and shares it. There are various types of chat, for example, text chat, voice chat, and video chat. By selecting the input data entered by the user of the second terminal 4 as material data, it is possible to easily turn input data such as text data provided by participants in the Web conference (users of guest terminals) into subtitle data. Note that the input data entered at the second terminal 4 may be converted into subtitles with different display capabilities. For example, each second terminal 4 may be recognized, and the display color of the subtitle data may be changed depending on the terminal.

In addition, when selecting material data to be used for the subtitle data, the user of the host terminal may select information to be notified to the participants of the Web conference (users of the guest terminal) as material data. Information to be notified to the participants of the Web conference may be, for example, advertisements or announcements for products or services. Information to be notified to the participants of the Web conference may include song lyrics, text data for a text broadcast, or video data in sign language. This allows the user of the host terminal to easily turn information to be notified to the participants of the Web conference into subtitle data.

In addition, when selecting material data to be used for the subtitle data, the user of the host terminal may select information to be displayed on the third display device 45 described below as material data. The information to be displayed on the third display device 45 is, for example, match information showing the progress of a sporting event such as baseball, soccer, volleyball, or basketball. The match information is information that is displayed on a large monitor at the venue. By selecting match information as material data, for example, in a multi-purpose venue where multiple types of sporting events can be held, it becomes possible to easily generate subtitles suitable for each sport.

<Image data display function>
The image data display function is a function for displaying image data provided from the providing unit 18. The plug-in software 311 displays the image data provided from the providing unit 18 on the second monitor 322. For example, the plug-in software 311 acquires the image data provided from the providing unit 18, converts the image data into a number of pixels suitable for display on the second monitor 322, and displays the image data. The user of the first terminal 3 can check the telop data generated from the image data displayed on the second monitor 322. The image data displayed on the second monitor 322 is displayed and shared on the second monitor 422 of the second terminal 4, which is a guest terminal of the Web conference. This allows the second terminal 4 to easily acquire image data including telop data generated in the information processing device 1.

The capture unit 43 captures the display screen of the image data displayed on the second monitor 422 of the second display device 42. Capturing the display screen means recording part or all of the still image or video displayed on the second monitor 422 in a readable manner in a memory or the like (not shown). Capturing the display screen is performed, for example, by an operation of the user of the second terminal 4.

The output unit 44 outputs the video data of the display screen captured by the capture unit 43 to the third display device 45. The third display device 45 is, for example, a large screen or digital signage installed in a public facility, a sports stadium, a commercial facility, or on the street. By outputting the display screen of the image data captured by the capture unit 43 to the third display device 45, it becomes possible to easily use the image data including the subtitle data on the third display device 45.

The network 9 is, for example, the Internet. The network 9 may also include, for example, mobile phone base stations, wireless LAN access points, etc.

In the information processing system 100, the communication control unit 11, authentication unit 12, storage unit 13, acquisition unit 14, UI provision unit 15, judgment unit 16, generation unit 17, and provision unit 18 of the information processing device 1, the first application 31 and first display device 32 of the first terminal 3, and the second application 41, second display device 42, capture unit 43, output unit 44, and third display device 45 of the second terminal 4 are examples of the functional units of the information processing system 100 and do not limit the functions of the information processing system 100. For example, the information processing system 100 does not need to have all of the above functional units, and may have only some of the functional units. The information processing system 100 may also have other functions than those described above. For example, the information processing system 100 may have an input function for inputting information and an output function for notifying the operating status of the device by an LED lamp or the like.

Furthermore, as described above, each of the above-mentioned functional units possessed by the information processing system 100 has been described as being realized by software. However, at least one or more of the above-mentioned functional units possessed by the information processing system 100 may be realized by hardware.

Furthermore, any of the above-mentioned functional units possessed by the information processing system 100 may be implemented by dividing one functional unit into multiple functional units. Furthermore, any two or more of the above-mentioned functional units possessed by the information processing system 100 may be implemented by consolidating them into one functional unit. In other words, FIG. 1 represents the functions possessed by the information processing system 100 in the form of functional blocks, and does not, for example, indicate that each functional unit is composed of a separate program file or the like.

In addition, the information processing device 1 may be a device realized by a single housing, or a system realized by multiple devices connected via a network or the like. For example, the information processing device 1 may have some or all of its functions realized by other virtual devices, such as a cloud service provided by a cloud computing system. That is, the information processing device 1 may have at least one or more of the above-mentioned functional units realized by other devices. In addition, the information processing device 1 may be a general-purpose computer such as a desktop PC, or may be a dedicated device with limited functions.

Next, the display screens displayed in the information processing system will be described with reference to Figs. 2 to 8. In Figs. 2 to 7, the first display device 32 has a main monitor 321 and a second monitor 322, and the second display device 42 has a main monitor 421 and a second monitor 422. The third display device 45 displays a captured image of the second monitor 422. The main monitor 321 and the second monitor 322 exemplify an inputter screen viewed by an inputter who inputs subtitles, and the main monitor 421 and the second monitor 422 exemplify an on-site shared screen that is shared at the site where the third display device 45 is installed.

FIG. 2 is a diagram showing a first display example of a display device used in the information processing system 100 in the embodiment. In FIG. 2, the third display device 45 is a large display device installed in a baseball field. The third display device 45 displays a display screen showing baseball game information. The main monitor 321 displays a UI for setting telops to generate image data to be used on the display screen of the third display device 45. The second monitor 322 displays image data including telop data generated by settings based on the UI displayed on the main monitor 321. In this embodiment, a case where the display screen (full screen) displayed on the second monitor 322 is displayed as telops is illustrated. On the second monitor 322, game information such as the baseball score progress and player names is displayed as telops in a predetermined format. The display screens of the main monitor 321 and the second monitor 322 are shared with the main monitor 421 and the second monitor 422 via a Web conference system. The display screen of the second monitor 422 is captured and displayed on the third display device 45. This makes it possible to reduce the cost of creating telop data at the site where the third display device 45 is installed. Note that, although Figures 2 to 7 show an example in which the display screen of the main monitor 321 is also shared with the main monitor 421, the display screen of the main monitor 321 does not have to be shared.

Figure 3 is a diagram showing a second display example of a display device used in the information processing system 100 in an embodiment. In Figure 3, the third display device 45 is a large display device installed in a baseball stadium, as in Figure 2. The third display device 45 displays a display screen when a home run is hit. The second monitor 322 displays image data including caption data of the text data of "home run" generated by settings based on the UI displayed on the main monitor 321. Note that the text data of "home run" may be provided by the user of the second terminal 4.

Figure 4 is a diagram showing a third display example of a display device used in the information processing system 100 in an embodiment. In Figure 4, the third display device 45 is a large display device installed in a soccer field. The third display device 45 displays soccer match information. The second monitor 322 displays image data including subtitle data of the match information generated by settings based on the UI displayed on the main monitor 321. The match information includes text data of team names, image data of team marks, and text data showing the scoring situation.

Figure 5 is a diagram showing a fourth display example of a display device used in the information processing system 100 in an embodiment. In Figure 5, the third display device 45 is a large display device installed in a soccer field, as in Figure 4. The third display device 45 displays a display screen showing the additional time. The second monitor 322 displays image data including caption data of text data of the additional time generated by settings based on the UI displayed on the main monitor 321. Note that the text data showing the numerical value of the additional time may be provided by the user of the second terminal 4.

Figure 6 is a diagram showing a fifth display example of a display device used in the information processing system 100 in an embodiment. In Figure 5, the third display device 45 is a large display device installed in a gymnasium where a volleyball match is held. Volleyball match information is displayed on the third display device 45. In this embodiment, it is possible to easily display a display screen suitable for each sport even in a place where various sports matches are held, such as a gymnasium.

Figure 7 is a diagram showing a sixth display example of a display device used in the information processing system 100 in an embodiment. In Figure 6, similar to Figure 5, the third display device 45 is a large display device installed in a gymnasium where a volleyball match will be held. The third display device 45 displays volleyball match information different from that shown in Figure 5. In this embodiment, it is possible to easily display various match information.

Figure 8 is a diagram showing a seventh display example of the display device used in the information processing system 100 in the embodiment. In Figure 8, the second display device 42 displays a display screen of a client terminal of a Web conference in which multiple participants (eight in the figure) are participating. In the upper right corner of the display screen, a shared screen shared by the host terminal of the Web conference is displayed. On the shared screen, telop 1001, telop 1002, telop 1003, and telop 1004 are superimposed on the live video of the performance venue. In addition, a comment input field 1011 is displayed on the display screen of the second display device 42. The telop 1001 and the telop 1003 are telop data superimposed on the live video, the material data of which is text data or image data of an advertisement or the like to be notified to the guest terminal, input from the host terminal. The telop 1002 is telop data superimposed on the live video, the material data of which is a comment input from the comment input field 1011 of each guest terminal. The caption 1004 is caption data obtained by converting the audio of the music being played into text data as material information and superimposing it on the live video. The host terminal of the Web conference can reduce the cost of distributing the caption data by sharing image data including the caption data generated by the information processing device 1 with the client terminals in the Web conference. Different caption data may be distributed to each participant on the display screen of the second display device 42.

Next, the hardware configuration of the information processing device 1 will be described with reference to FIG. 9. FIG. 9 is a block diagram showing an example of the hardware configuration of the information processing device 1 in an embodiment.

The information processing device 1 has a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, an I/O device 104, and a communication I/F (Interface) 105. The information processing device 1 is a device that executes the information processing program described in FIG. 1.

The CPU 101 controls the user terminal by executing an information processing program stored in the RAM 102 or the ROM 103. The information processing program is obtained, for example, from a recording medium on which the program is recorded or from a program distribution server via a network, and installed in the ROM 103, and is read and executed by the CPU 101.

The I/O device 104 has an operation input function and a display function (operation display function). The I/O device 104 is, for example, a touch panel. The touch panel enables the user of the information processing device 1 to input operations using a fingertip, a touch pen, or the like. In this embodiment, the I/O device 104 is described as using a touch panel with an operation display function, but the I/O device 104 may have a display device with a display function and an operation input device with an operation input function separately. In that case, the display screen of the touch panel can be implemented as the display screen of the display device, and the operation of the touch panel can be implemented as the operation of the operation input device. The I/O device 104 may be realized in various forms, such as a head-mounted type, a glasses type, or a wristwatch type display.

The communication I/F 105 is an I/F for communication. The communication I/F 105 executes short-range wireless communication such as wireless LAN, wired LAN, infrared, etc. The figure shows only the communication I/F 105 as the communication I/F, but the information processing device 1 may have a communication I/F for each of a plurality of communication methods.

Next, the operation of the information processing device 1 will be described with reference to FIG. 10. FIG. 10 is a flowchart showing an example of the operation of the information processing device 1 in the embodiment. The operations in the flowchart shown below will be described as being performed by the information processing device 1, but each operation can be executed by each functional unit of the information processing device 1 described above.

In FIG. 10, the information processing device 1 provides a UI including a login screen for a user who is to become a host terminal to log in and a caption data setting screen (step S11). The UI can be provided, for example, by providing a web page to a first terminal 3 that has accessed a specific network address of the information processing device 1.

After executing the process of step S11, the information processing device 1 determines whether or not the login of the user of the first terminal 3 has been successful (step S12). Whether or not the login has been successful can be determined based on whether or not authentication has been successful in the authentication unit 12. If it is determined that the login has not been successful (step S12: NO), the information processing device 1 repeats the process of step S12 and waits for the login to be successful.

On the other hand, if it is determined that the login was successful (step S12: YES), the information processing device 1 determines whether or not the material data to be used for the subtitle data has been acquired (step S13). Whether or not the material data has been acquired can be determined, for example, by whether or not the acquisition unit 14 has acquired the material data. If it is determined that the material data has not been acquired (step S13: NO), the information processing device 1 repeats the process of step S13 and waits for the material data to be acquired.

On the other hand, if it is determined that material data has been acquired (step S13: YES), the information processing device 1 determines the type of material data (text data, voice data, or image data) and recognizes the content of each piece of material data (step S14). For example, if the material data is text data, the information processing device 1 recognizes the number of characters in the text data and the type of text (e.g., kanji, hiragana, alphanumeric characters, etc.). If the material data is voice data, the information processing device 1 recognizes the number of characters in the text data generated by voice recognition, etc. If the material data is image data, the information processing device 1 recognizes the number of pixels in the recognized image.

After executing the process of step S14, the information processing device 1 generates subtitle data based on the acquired material data and the subtitle display area, etc. (step S15). The generation of subtitle data can be executed by the function of the generation unit 17 described above. After executing the process of step S15, the information processing device 1 provides the first terminal 3 with image data including at least the subtitle data generated in step S15 (step S16). After executing the process of step S16, the information processing device 1 ends the operation shown in the flowchart.

Note that the illustrated flowchart shows an example of the operation of the information processing device 1 and does not limit the operation of the information processing device 1.

Next, the operation of the first terminal 3 will be described with reference to FIG. 11. FIG. 11 is a flowchart showing an example of the operation of the first terminal 3 in the embodiment. The operations in the flowchart shown below will be described as being executed by the first terminal 3, but each operation can be executed by each functional unit of the first terminal 3 described above.

In FIG. 11, the first terminal 3 starts the Web conference system (step S21). The Web conference system is started, for example, by the first application 31 being started and accessing the Web conference server 2.

After executing the process of step S21, the first terminal 3 determines whether the login of the user of the first terminal 3 has been successful (step S22). Whether the login has been successful can be determined by receiving a notification from the information processing device 1 indicating whether the authentication has been successful in the authentication unit 12. If it is determined that the login has not been successful (step S22: NO), the first terminal 3 repeats the process of step S12 and waits for the login to be successful.

On the other hand, if it is determined that the login was successful (step S22: YES), the first terminal 3 determines whether or not material data to be used for the subtitle data has been acquired from the second terminal 4 (step S23). The material data acquired from the second terminal 4 is, for example, chat data provided by the second terminal 4 in a web conference. If it is determined that material data has not been acquired (step S23: NO), the first terminal 3 repeats the process of step S23 and waits for the material data to be acquired. Note that while FIG. 11 illustrates an example in which material data is acquired from the second terminal 4, the material data may be acquired from the first terminal 3 or another device.

On the other hand, if it is determined that the material data has been acquired (step S23: YES), the first terminal 3 provides the acquired material data to the information processing device 1 (step S24). After executing the process of step S24, the first terminal 3 determines whether or not image data including subtitle data generated in the information processing device 1 has been acquired (step S25). If it is determined that the image data has not been acquired (step S25: NO), the first terminal 3 repeats the process of step S25 and waits to acquire image data.

On the other hand, if it is determined that the image data has been acquired (step S25: YES), the first terminal 3 shares the display screen of the acquired image data with the second terminal 4, which is the client terminal of the Web conference (step S26). The sharing of the display screen can be performed by displaying the image data on the first display device 32, displaying the image data on the first display device 32, and displaying the display screen provided to the second terminal 4 in the Web conference. After executing the process of step S26, the first terminal 3 ends the operation shown in the flowchart.

Note that the illustrated flowchart shows an example of the operation of the first terminal 3 and does not limit the operation of the first terminal 3.

Next, the operation of the second terminal 4 will be described with reference to FIG. 12. FIG. 12 is a flowchart showing an example of the operation of the second terminal 4 in the embodiment. The operations in the flowchart shown below will be described as being performed by the second terminal 4, but each operation can be performed by each functional unit of the second terminal 4 described above.

In FIG. 12, the second terminal 4 starts the Web conference system (step S31). The Web conference system is started, for example, by the second application 41 being started and accessing the Web conference server 2. The Web conference server 2 is accessed, for example, by inputting the conference room number of the Web conference started by the first terminal 3, which is the host terminal. In other words, the second terminal 4 in FIG. 12 can participate in the Web conference as a client terminal by simply inputting the conference room number.

After executing the process of step S31, the second terminal 4 provides the material data to the first terminal 3 (step S32). The provision of the material data is performed, for example, by providing chat data using a chat function of the web conference.

After executing the process of step S31, the second terminal 4 shares the display screen of the first terminal 3, which is the host terminal of the Web conference, as a client terminal (step S33). The sharing of the display screen can be performed by displaying image data on the first display device 32, displaying image data on the first display device 32, and displaying the display screen provided to the second terminal 4 in the Web conference. After executing the process of step S33, the second terminal 4 ends the operation shown in the flowchart.

The illustrated flowchart shows an example of the operation of the second terminal 4, and does not limit the operation of the second terminal 4. For example, the second terminal 4 may capture the display screen shared in step S33 and display it on the third display device 45.

The various processes described above in this embodiment may be performed by recording a program for implementing the functions constituting the device described in this embodiment on a computer-readable recording medium, and having the computer system read and execute the program recorded on the recording medium. Note that the "computer system" referred to here may also include hardware such as the OS and peripheral devices. Furthermore, if a WWW system is used, the "computer system" also includes the homepage providing environment (or display environment). Furthermore, "computer-readable recording medium" refers to a storage device such as a flexible disk, magneto-optical disk, ROM, writable non-volatile memory such as flash memory, portable media such as CD-ROM, or a hard disk built into a computer system.

Furthermore, the term "computer-readable recording medium" includes a memory that holds a program for a certain period of time, such as a volatile memory (e.g., DRAM (Dynamic Random Access Memory)) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. The above program may also be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be a program that realizes part of the above-mentioned functions. Furthermore, it may also be a so-called difference file (difference program) that realizes the above-mentioned functions in combination with a program already recorded in the computer system.

The above describes an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to this embodiment, and various modifications are also included within the scope of the present invention.

For example, in this embodiment, the second terminal 4 has the second application 41 and the second application 41 acquires image data as a method of acquiring image data to be displayed on the first display device acquired via the Web conference system. Here, the second application 41 may be implemented as a dedicated application or plug-in software for a Web browser as described above, or may be implemented as a function of a Web browser that does not require plug-in software. However, the second terminal 4 may not have the second application 41. For example, the second terminal 4 may be a display system that passively receives image data to be displayed on the first display device 32. The second terminal 4 may receive screen data shared by the host terminal through a screen sharing function of the Web conference system and display it on the second display device 42.

REFERENCE SIGNS LIST 100 Information processing system 1 Information processing device 11 Communication control unit 12 Authentication unit 13 Storage unit 14 Acquisition unit 15 UI provision unit 16 Determination unit 17 Generation unit 18 Provision unit 2 Web conference server 3 First terminal 31 First application 311 Plug-in software 32 First display device 321 Main monitor 322 Second monitor 4 Second terminal 41 Second application 411 Plug-in software 42 Second display device 421 Main monitor 422 Second monitor 43 Capture unit 44 Output unit 45 Third display device 9 Network 101 CPU
102 RAM
103 ROM
104 I/O equipment 105 Communication I/F
1001, 1002, 1003, 1004 Caption 1011 Comment input field

Claims (9)

A first terminal which is a client terminal in a Web conference system;
a second terminal which is another client terminal in the Web conference system and is connected to the first terminal via the Web conference system;
An information processing system comprising: an information processing device communicably connected to the first terminal,
The information processing device includes:
An acquisition unit that acquires material data from the first terminal;
a generation unit that generates telop data based on the material data acquired by the acquisition unit;
a providing unit that provides image data including at least the telop data generated by the generating unit to the first terminal,
The first terminal is
a first application for using the Web conference system;
a first display device that displays the image data provided by the providing unit,
The second terminal is
a second display device that displays the image data acquired from the first terminal via the Web conference system and to be displayed on the first display device;
the first display device includes a main monitor that displays the setting of the telop data, and a second monitor that displays the image data;
The second terminal acquires the image data to be displayed on the second monitor shared via the Web conference system.
Information processing system. The information processing system of claim 1, wherein the first application includes plug-in software having at least one of a caption data setting function for displaying the caption data settings on the main monitor to enable the setting of caption data, or an image data display function for displaying the image data provided by the providing unit . The second terminal is
a capture unit that captures a display screen of the image data displayed on the second display device;
3. The information processing system according to claim 1, further comprising: an output unit that outputs the display screen captured by the capture unit to a third display device. The second terminal provides input data input by a user of the second terminal to the first application;
The information processing system according to claim 1 , wherein the first application provides the input data acquired from the second terminal to the information processing device as the material data. The information processing system according to claim 4 , wherein the second terminal provides text data input by a user of the second terminal to the first application as the input data. 6. The information processing system according to claim 4 , wherein the second terminal provides voice data input by a user of the second terminal to the first application as the input data. The second terminal is
The information processing system according to claim 1 , further comprising: determining a display area for displaying the image data in correspondence with the second display device. An information processing method executed in an information processing system including: a first terminal which is a client terminal in a Web conference system; a second terminal which is another client terminal in the Web conference system and is connected to the first terminal via the Web conference system; and an information processing device which is communicatively connected to the first terminal,
Executed in the information processing device,
An acquisition step of acquiring material data from the first terminal;
a generating step of generating telop data based on the material data acquired in the acquiring step;
a providing step of providing image data including at least the telop data generated in the generating step to the first terminal,
Executed on the first terminal,
A start-up step of starting a first application for using the Web conference system;
a first display step of displaying the image data provided in the providing step,
Executed on the second terminal,
a second display step of displaying the image data displayed in the first display step, the image data being acquired from the first terminal via the Web conference system ;
the first terminal includes a main monitor that displays the setting of the telop data, and a second monitor that displays the image data;
The second terminal acquires the image data to be displayed on the second monitor shared via the Web conference system.
Information processing methods. An information processing program executed in an information processing system including: a first terminal which is a client terminal in a Web conference system; a second terminal which is another client terminal in the Web conference system and is connected to the first terminal; and an information processing device which is communicatively connected to the first terminal,
In the information processing device,
An acquisition function for acquiring material data from the first terminal;
a generating function for generating telop data based on the material data acquired by the acquiring function;
a providing function for providing image data including at least the telop data generated by the generating function to the first terminal,
In the first terminal,
a first application function for using the Web conference system;
A first display function that displays the image data provided by the providing function,
In the second terminal,
A second display function is realized to display the image data displayed on the first display function acquired from the first terminal via a Web conference system ,
the first terminal includes a main monitor that displays the setting of the telop data, and a second monitor that displays the image data;
The second terminal acquires the image data to be displayed on the second monitor shared via the Web conference system.
Information processing program.

Images