JP7792870B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

In recent years, with the advancement of information processing technology, online conference technology using multiple terminal devices connected via a network has been attracting attention, and technologies to improve the efficiency of online conferences have been proposed.

For example, Patent Document 1 discloses technology that helps improve the efficiency of online meetings by transmitting an online conference screen that has been laid out so that a video chat screen, a shared screen, and a communication screen other than the video chat screen are displayed side by side on a terminal device.

Japanese Patent Application Laid-Open No. 2022-074159

The above-mentioned conventional technology can support online meetings by transmitting an online conference screen that displays multiple screens to promote smooth two-way communication during online meetings, but there is room for improvement in terms of further improving the efficiency of online meetings.

This application has been made in light of the above, and aims to provide an information processing device, information processing method, and information processing program that can further improve the efficiency of online conferences.

The information processing device according to the present application includes a determination unit and a participation processing unit. The determination unit determines the meeting efficiency, which is the efficiency of an online conference. If the meeting efficiency determined by the determination unit does not satisfy a predetermined condition, the participation processing unit virtually allows a specific user to participate in the online conference.

According to one aspect of the embodiment, it is possible to achieve further efficiency in online meetings.

FIG. 1 is a diagram for explaining an information processing method performed by an information processing apparatus according to an embodiment. FIG. 2 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. FIG. 3 is a diagram illustrating an example of the configuration of the information processing apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of a user information table stored in a user information storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of a conference information table stored in the conference information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of a meeting efficiency information table stored in the meeting efficiency information storage unit according to the embodiment. FIG. 7 is a flowchart illustrating an example of information processing by the processing unit of the information processing apparatus according to the embodiment. FIG. 8 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing apparatus according to the embodiment.

Below, detailed explanations will be given of the information processing device, information processing method, and information processing program (hereinafter referred to as "embodiments") according to the present application, with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. Furthermore, the embodiments can be combined as appropriate to the extent that the processing content is not contradictory. Furthermore, the same components in the following embodiments will be assigned the same reference numerals, and redundant explanations will be omitted.

[1. An example of information processing]
First, an information processing method performed by an information processing apparatus according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an information processing method performed by an information processing apparatus according to an embodiment.

The information processing device 1 shown in Fig. 1 provides an online conference service. Such an online conference service is also called a web conference service. Fig. 1 shows an example in which the online conference service is provided to users _U1 , _U2 , _{and U3} of terminal devices ₂₁ , ₂₂ , and _23. Hereinafter, when each of multiple terminal devices including terminal devices ₂₁ , ₂₂ , and ₂₃ is referred to without distinction, it may be referred to as terminal device 2, and when each of multiple users including users _U1 , _U2 , and _U3 is referred to without distinction, it may be referred to as user U.

Terminal device ₂₁ transmits user information about user _U1 to information processing device 1 (step _S11 ), terminal device ₂₂ transmits user information about user _U2 to information processing device 1 (step S12), and terminal device ₂₃ transmits user information about user _U3 to information processing device 1 (step _S13 ). Each terminal device ₂ is equipped with a microphone, a camera, a display unit, a speaker, etc., and the user information includes real-time information such as voice information of user U collected in real time by the microphone and captured video information that is information about video of user U captured in real time by the camera.

The information processing device 1 generates conference information based on the user information of users _U1 , _U2 , and _U3 transmitted from terminal devices ₂₁ , ₂₂ , and ₂₃ (step S2). The information processing device 1 generates the audio information in the conference information by mixing the audio information included in the user information of each user U. Note that the information processing device 1 can also select one piece of audio information from the audio information included in the user information of multiple users U and use it as the audio information in the conference information.

In addition, the information processing device 1 can convert user U's voice information into text information using voice recognition, and then convert the converted text information into voice information of a character designated by user U using voice synthesis.

In addition, the speech synthesis of the information processing device 1 allows for adjustment of speaking style (volume, speed, intonation (emphasis), speech intervals (pauses), etc.) and sound quality using parameters, and the information processing device 1 can convert text information into audio information through speech synthesis using the speaking style and sound quality specified by the user U.

In addition, the information processing device 1 generates image information for the conference information using captured video information included in the user information of each user U. For example, the information processing device 1 performs image processing on a body image, which is an image of the user U's body included in the captured video information of the user U, to generate video information for the user U in the conference information.

For example, the information processing device 1 uses a learning model or the like to identify the coordinates of each feature point from an image of the user U's body, and generates video information of the user U in the conference information by reconstructing the user U's body in three-dimensional space based on the identification results. The user U's body is, for example, the user U's face and neck, but may also include the user U's torso, hands, arms, legs, etc. in addition to the user U's face and neck.

In addition, the information processing device 1 can also generate video information of user U in the conference information by, for example, transforming a three-dimensional body model of a character designated by user U to match the body shape estimated from an image of user U's body. The character designated by user U is shown as user U's avatar in the conference information. Each user U can designate a desired character from among multiple types of characters.

The information processing device 1 transmits the conference information generated in the processing of step S2 to the terminal devices ₂₁ , ₂₂ , and ₂₃ , and causes the terminal devices ₂₁ , ₂₂ , and ₂₃ to reproduce the conference information (steps _S31 , _S32 , and _S33 ). The information processing device 1 can provide an online conference service to users _U1 , _U2 , and _U3 by repeating the processing of steps _S11 , _S12 , _S13 , _S2 , _S31 , S32, and _S33 .

1A, avatars of users _{U1, U2, and U3 are displayed on the conference screen displayed on each terminal device 2, but this is not limited to this example. For example, instead of the avatars of users U1 , U2 , and U3} , the information processing device 1 can use the captured video information of user U as the video information of user U included in the conference information as is, or can process the captured video information of user U using effect processing or the like to create video information of user U included in the conference information.

While providing the online conference service, the information processing device 1 determines the efficiency of the online conference (step S4). Hereinafter, the efficiency of the online conference may be referred to as conference efficiency.

In the processing of step S4, the information processing device 1 has meeting efficiency information, which is information indicating the relationship between meeting efficiency and the users participating in the online meeting, and determines the meeting efficiency based on this meeting efficiency information.

The meeting efficiency information includes, for example, information on meeting efficiency for each combination of users U for each meeting type. In the example shown in Figure 1, the information processing device 1 determines the meeting efficiency associated with the meeting type, which is a type of online meeting, among the meeting efficiencies associated with the combination of users _U1 , _U2 , and _U3 in the meeting efficiency information as the meeting efficiency of the online meeting shown in Figure 1.

The meeting efficiency information is generated for each meeting type based on the meeting efficiency of past online meetings, for example. Note that the meeting efficiency information may include information on the meeting efficiency index value for each user U, instead of information on the meeting efficiency for each combination of users U. The meeting efficiency index value is a value that serves as an index of meeting efficiency. In this case, for example, the information processing device 1 may determine the sum of the meeting efficiency index values of users _U1 , _U2 , and _U3 as the meeting efficiency of the online conference shown in FIG.

The information processing device 1 can also determine meeting efficiency based on the number of comments per unit time in an online conference. For example, the information processing device 1 determines that the meeting efficiency is lower the fewer the number of comments per unit time in an online conference.

The information processing device 1 can also determine the conference efficiency based on information indicating the efficiency of the online conference that is input by a user U participating in the online conference using the terminal device 2. For example, the information processing device 1 determines the conference efficiency as the average value of the online conference efficiency input by each of users _U1 , _U2 , and _U3 shown in FIG.

The information processing device 1 can also determine the conference efficiency as the result of weighting the online conference efficiencies input by each of the users _U1 , _U2 , and _U3 shown in Fig. 1. The weight in the weighting is, for example, a value determined in advance for each user U, but may be a value that increases as the number of comments per unit time by the user U increases or decreases.

In addition, the information processing device 1 can determine the conference efficiency as the result of weighting two or more of the conference efficiency corresponding to the combination of participating users, the number of comments per unit time, and the conference efficiency input by user U. In this case, the weights used in the weighting are predetermined values, but are not limited to such an example.

Next, the information processing device 1 determines whether the meeting efficiency determined in step S4 does not satisfy a predetermined condition (step S5). The predetermined condition is, for example, that the meeting efficiency is equal to or greater than a predetermined threshold, and the threshold is set for each type of meeting. If the meeting efficiency determined in step S4 is less than the threshold, the information processing device 1 determines that the meeting efficiency determined in step S4 does not satisfy the predetermined condition.

Furthermore, if the meeting efficiency determined in step S4 fluctuates while the online meeting is in progress, the predetermined condition may be, for example, a condition that the state in which the meeting efficiency is equal to or greater than a predetermined threshold does not occur more than a predetermined number of times, or a condition that the average value of the meeting efficiency is equal to or greater than a predetermined threshold.

Next, if the information processing device 1 determines in step S4 that the meeting efficiency does not satisfy the predetermined conditions, it determines another user U who is estimated to improve the efficiency of the online meeting as a specific user (step S6).

In step S6, for example, the information processing device 1 can determine a specific user based on the meeting efficiency information. For example, the information processing device 1 determines as a specific user another user U whose meeting efficiency associated with a combination of multiple users U participating in an online conference is equal to or greater than a threshold, or another user U whose meeting efficiency associated with a combination of multiple users U participating in an online conference is the highest.

For example, in the example shown in FIG. 1, another user U whose meeting efficiency associated with the combination of users _U1 , _U2 , and _U3 is equal to or greater than a threshold value, or another user U whose meeting efficiency associated with the combination of users _U1 , _U2 , and _U3 is the highest, is determined as the specific user.

The information processing device 1 can also determine, for example, a predetermined other user U as a specific user. For example, the information processing device 1 determines, as a specific user, a predetermined other user U for each conference type or each combination of participating users. The information processing device 1 can also determine, as a specific user, a user U whose total or weighted sum of conference efficiency index values with users _U1 , _U2 , and _U3 is equal to or greater than the above-described predetermined threshold.

Next, if the information processing device 1 determines in step S5 that the meeting efficiency does not satisfy the predetermined condition, it virtually allows the specific user to participate in the online meeting (step S7). Virtually allowing the specific user to participate in the online meeting means including information about the specific user in the meeting information as if the specific user were participating in the online meeting, without using the above-mentioned real-time information about the specific user.

For example, the information processing device 1 virtually allows a specific user to participate in an online conference by making the avatar of the specific user appear in the online conference. The avatar of the specific user is, for example, an avatar of a character designated by the specific user. In the example shown in FIG. 1B, the conference screen displayed on each terminal device 2 displays the avatar of the specific user in addition to the avatars of users _U1 , _U2 , and _U3 .

The information processing device 1 stores multiple types of preset information, which is, for example, information indicating actions (body movements and speech) set in advance by specific users. The multiple types of preset information are, for example, preset information for each meeting situation, such as information indicating actions of nodding in response to comments made by participating users, information indicating actions of encouraging participating users to speak, and information indicating actions of praising participating users, but are not limited to these examples.

The information processing device 1 can make a specific user's avatar appear in an online conference by including conversion information, which converts pre-configured information into the behavior of the specific user's avatar, in the conference information. The conversion information includes information such as avatar audio information and video information. Note that the information processing device 1 can also make a specific user appear in an online conference by using the pre-configured information as is without converting it.

The information processing device 1 can also make a specific user's avatar appear in an online conference so that it can be operated by some of the participating users in the online conference. For example, some of the participating users can select their desired preset information from multiple types of preset information, and the information processing device 1 can include conversion information in the conference information that converts the preset information selected by some of the participating users into the behavior of the specific user's avatar. In this case, the information processing device 1 can also make a specific user appear in an online conference using the preset information as is without converting it.

In addition, the information processing device 1 can also include in the conference information conversion information obtained by converting the user information of some participating users into the behavior of a specific user's avatar. The "some participating users" are the participating users who host the conference, but are not limited to this example. For example, they may be the participating users with the most comments, or participating users other than the participant user with the highest conference efficiency index value.

The information processing device 1 also notifies the specific user of virtual participation information, which is information indicating that the specific user is virtually participating in the online conference (step S8). For example, the information processing device 1 notifies the specific user of the virtual participation information by sending the virtual participation information to the specific user's terminal device 2 or the specific user's email address. This allows the specific user to understand that they have been virtually used to improve the efficiency of an online conference in which they are not actually participating.

In this way, the information processing device 1 determines the meeting efficiency, which is the efficiency of an online conference, and if it determines that the meeting efficiency does not satisfy predetermined conditions, it virtually allows a specific user to participate in the online conference. This increases the likelihood that users U participating in the online conference will try to increase the efficiency of the online conference due to the presence of the specific user, allowing the information processing device 1 to further improve the efficiency of online conferences.

The following provides a detailed description of the configuration of an information processing system including an information processing device 1 and a terminal device 2 that perform such processing.

2. Configuration of the information processing system
2 is a diagram showing an example of the configuration of an information processing system according to an embodiment. As shown in FIG. 2, the information processing system 100 according to the embodiment includes an information processing device 1 that provides an online conference service, and a plurality of terminal devices 2 including a plurality of terminal devices 2 ₁ to 2 _n owned by a plurality of users U who participate in the online conference service. n is, for example, an integer equal to or greater than 3.

The information processing device 1 and multiple terminal devices 2 are connected to each other via a network N, either wired or wirelessly, so that they can communicate with each other. Note that the information processing system 100 shown in FIG. 2 may include multiple information processing devices 1. The network N is, for example, a local area network (LAN) or a wide area network (WAN) such as the Internet.

Each terminal device 2 is equipped with a microphone, camera, display, speaker, etc. Each terminal device 2 transmits real-time information, including voice information of the user U collected in real time by the microphone, and captured video information, which is information about videos of the user U captured in real time by the camera, to the information processing device 1 via the network N.

In addition, each terminal device 2 receives conference information transmitted from the information processing device 1 via the network N, displays the video information included in the conference information on a display unit (not shown), and outputs the audio information included in the conference information from a speaker.

Each terminal device 2 may be, for example, a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a smartphone, a mobile phone, or a PDA (Personal Digital Assistant). Each terminal device 2 is operated by a user U. Note that each terminal device 2 is not limited to the above examples and may also be, for example, a wearable device.

In addition, the terminal device 2 can connect to the network N via a wireless communication network such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation: fifth generation mobile communication system), or via short-range wireless communication such as Bluetooth (registered trademark) or wireless LAN, and communicate with the information processing device 1.

3. Configuration of information processing device 1
The following describes an example of the functional configuration of the information processing device 1. Fig. 3 is a diagram showing an example of the configuration of the information processing device 1 according to an embodiment. As shown in Fig. 3, the information processing device 1 has a communication unit 10, a storage unit 11, and a processing unit 12.

3.1. Communication Unit 10
The communication unit 10 is realized by, for example, a network interface card (NIC). The communication unit 10 is connected to a network N via a wired or wireless connection and transmits and receives information to and from various other devices. For example, the communication unit 10 transmits and receives information to and from each of a plurality of terminal devices 2 via the network N.

[3.2. Storage unit 11]
The storage unit 11 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 also has a user information storage unit 20, a meeting information storage unit 21, and a meeting efficiency information storage unit 22.

3.2.1. User Information Storage Unit 20
The user information storage unit 20 stores various types of information related to the user U. Fig. 4 is a diagram showing an example of a user information table stored in the user information storage unit 20 according to the embodiment. In the example shown in Fig. 4, the user information table stored in the user information storage unit 20 includes information on items such as "user ID (identifier),""attributeinformation," and "history information."

"User ID" is an identifier that identifies user U. "Attribute information" is attribute information related to the attributes of user U associated with the "user ID." User U's attributes include, for example, demographic attributes and psychographic attributes. Demographic attributes are demographic attributes such as age, gender, affiliation, occupation, place of residence, annual income, and family composition. Psychographic attributes are psychological attributes such as lifestyle, values, and interests.

"History information" is information such as the user U's usage history of the online conference service associated with the "user ID," and includes information such as the conference ID of online conferences that user U previously participated in, the date and time of the conference, and user U's participation results. User U's participation results include information such as the number of comments made by user U in online conferences and the content of the comments.

The information stored in the user information storage unit 20 is not limited to the above and may include various types of information depending on the purpose. For example, the information stored in the user information storage unit 20 may include information indicating the context history of user U.

3.2.2. Conference Information Storage Unit 21
The conference information storage unit 21 stores various information related to reserved online conferences. Fig. 5 is a diagram illustrating an example of a conference information table stored in the conference information storage unit 21 according to the embodiment.

In the example shown in FIG. 5, the conference information table stored in the conference information storage unit 21 includes information for the following items: "Conference ID," "User ID," "Date and Time of Event," and "Organizer." "Conference ID" is an identifier that identifies the online conference. "User ID" is the user ID of user U who will participate in the online conference associated with the "Conference ID." "Date and Time of Event" is information that indicates the date and time of the online conference associated with the "Conference ID."

"Organizer" is the user ID of the user U who is hosting the online conference associated with the "Conference ID." Note that the information stored in the conference information storage unit 21 is not limited to the above and may include various information depending on the purpose.

3.2.3. Meeting Efficiency Information Storage Unit 22
The meeting efficiency information storage unit 22 stores meeting efficiency information, which is information indicating a relationship between meeting efficiency and participating users of an online meeting. Fig. 6 is a diagram illustrating an example of a meeting efficiency information table stored in the meeting efficiency information storage unit 22 according to the embodiment.

In the example shown in Figure 6, the meeting efficiency information table stored in the meeting efficiency information storage unit 22 includes information under the items "combination ID," "user ID," and "efficiency." "Combination ID" is an identifier that identifies a user combination, which is a combination of multiple users U.

"User ID" is the user ID of multiple users U that make up the user combination associated with the "combination ID." "Efficiency" is information that indicates the predicted conference efficiency when multiple users U that make up the user combination associated with the "combination ID" hold an online conference. Conference efficiency is indicated, for example, within the range from 0 to 1, with higher values indicating higher conference efficiency.

The above-mentioned meeting efficiency information table is stored in the meeting efficiency information storage unit 22 for each meeting type. Meeting types are, for example, periodic meetings such as business reports, meetings such as development meetings, meetings such as business improvement meetings, and employee training meetings, but are not limited to these examples. For example, meeting types may be types based on the length of the meeting, types based on the time of day, types based on the day of the week, or types based on a combination of two or more of the meeting content, length of the meeting, time of day, and day of the week, or other types.

In this way, the meeting efficiency information storage unit 22 stores information on meeting efficiency for each combination of users U by meeting type. Note that the information stored in the meeting efficiency information storage unit 22 is not limited to the above and may include various information depending on the purpose.

3.3. Processing Unit 12
The processing unit 12 is a controller, and is realized by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) using RAM as a work area to execute various programs (examples of information processing programs) stored in a storage device inside the information processing device 1. The processing unit 12 is also a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the processing unit 12 has an acquisition unit 30, a reception unit 31, a determination unit 32, an estimation unit 33, a conference processing unit 34, and a notification unit 35, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in FIG. 3, and may be any other configuration that performs the information processing described below.

[3.3.1. Acquisition unit 30]
The acquisition unit 30 acquires various types of information. The acquisition unit 30 acquires various types of information from the storage unit 11. The acquisition unit 30 acquires various types of information from the user information storage unit 20, the meeting information storage unit 21, the meeting efficiency information storage unit 22, etc.

The acquisition unit 30 receives various information from an external information processing device via the communication unit 10. The acquisition unit 30 receives various information from the terminal device 2. For example, the acquisition unit 30 acquires information about user U from the terminal device 2, stores the acquired information about user U in the user information storage unit 20, and updates the user information table.

The acquisition unit 30 also acquires information about online conferences from the terminal device 2 and other external devices, stores the acquired information about online conferences in the conference information storage unit 21, and updates the conference information table. The acquisition unit 30 also acquires information about the efficiency of online conferences from the terminal device 2 and other external devices, stores the acquired information about online conference efficiency in the conference efficiency information storage unit 22, and updates the conference efficiency information table.

3.3.2. Reception unit 31
The reception unit 31 receives various requests from the terminal device 2. The reception unit 31 receives information indicating the various requests from the terminal device 2 via the communication unit 10.

For example, the reception unit 31 receives a conference participation request or a conference setting request from the terminal device 2. A conference participation request includes information such as a conference ID, and a conference setting request includes the user ID of a user U who will participate in the online conference, information about the date and time the conference will be held, and the user ID of the user U who will host the online conference.

The conference setting request does not necessarily have to include the user ID of the user U who will participate in the online conference. In this case, when the user U participates in the online conference, the reception unit 31 adds the user ID of the user U who participated in the online conference to the conference information table stored in the conference information storage unit 21.

[3.3.3. Judgment unit 32]
The determining unit 32 determines the meeting efficiency, which is the efficiency of the online meeting. For example, the determining unit 32 determines the meeting efficiency based on the meeting efficiency information stored in the meeting efficiency information storage unit 22.

For example, suppose the conference efficiency information table shown in Figure 6 is used, and the combination of participating users in the online conference is the combination with combination ID "G1," i.e., a combination of user U with user ID "U1," user U with user ID "U2," and user U with user ID "U3." In this case, the determination unit 32 determines that the conference efficiency is "0.8."

The meeting efficiency information is generated for each meeting type based on the meeting efficiency of past online meetings, for example. Note that the meeting efficiency information may include information on the meeting efficiency index value for each user U, instead of information on the meeting efficiency for each combination of users U. The meeting efficiency index value is a value that serves as an index of meeting efficiency. In this case, for example, the determination unit 32 may determine the meeting efficiency as the sum of the meeting efficiency index values of the users U participating in the online meeting.

The determination unit 32 can also determine the efficiency of a meeting based on the number of comments per unit time in an online meeting. For example, the determination unit 32 determines that the lower the number of comments per unit time in an online meeting, the lower the meeting efficiency.

The determination unit 32 can also determine the conference efficiency based on information indicating the efficiency of the online conference that is input by users U participating in the online conference using the terminal devices 2 during the online conference. For example, the determination unit 32 determines the conference efficiency to be the average value of the online conference efficiencies input by multiple users U participating in the online conference.

The determination unit 32 can also determine the conference efficiency as the result of weighting the online conference efficiencies entered by multiple users U participating in the online conference. The weight in the weighting is, for example, a value determined in advance for each user U, but may also be a value that increases as the number of comments per unit time by a user U increases or decreases.

The determination unit 32 can also determine the conference efficiency as the result of weighting two or more of the conference efficiency corresponding to the combination of participating users, the number of comments per unit time, and the conference efficiency input by user U. In this case, the weights used in the weighting are predetermined values, but are not limited to such an example.

The determination unit 32 determines whether the meeting efficiency determined as described above does not satisfy a predetermined condition. The predetermined condition is, for example, that the meeting efficiency is equal to or greater than a predetermined threshold, and the threshold is set for each type of meeting. For example, if the meeting efficiency is less than the threshold, the determination unit 32 determines that the meeting efficiency does not satisfy the predetermined condition.

Furthermore, if the meeting efficiency fluctuates while the online meeting is in progress, the predetermined condition may be, for example, a condition that the state in which the meeting efficiency is equal to or greater than a predetermined threshold does not occur more than a predetermined number of times, or a condition that the average value of the meeting efficiency is equal to or greater than a predetermined threshold.

[3.3.4. Estimation unit 33]
The estimation unit 33 estimates other users U who improve the meeting efficiency for each meeting type, which is a type of online meeting, as efficiency-improving users. The other users U are users U who are not participating in the online meeting for which the meeting efficiency is to be improved, and may be users U who are not participating in other online meetings but are participating in other online meetings.

For example, the estimation unit 33 can estimate an efficiency-improving user based on the meeting efficiency information stored in the meeting efficiency information storage unit 22. For example, the estimation unit 33 estimates, as an efficiency-improving user, another user U whose meeting efficiency associated with a combination of multiple users U participating in an online meeting is equal to or greater than a threshold, or another user U whose meeting efficiency associated with a combination of multiple users U participating in an online meeting is the highest.

Assume that users _U1 , _U2 , and _U3 are participating in an online conference whose efficiency is to be improved. In addition, in the conference efficiency information table, the conference efficiency associated with the combination of users _U1 , _U2 , _U3 , and _U4 is "0.7," the conference efficiency associated with the combination of users _U1 , _U2 , _U3 , and _U5 is "0.8," and the conference efficiency associated with the combination of users _U1 , _U2 , _U3 , and _U6 is "0.9." The threshold is also assumed to be "0.8." In this case, the estimation unit 33 estimates user _U4 , _U5 , or user _U5 as an efficiency improvement user.

The estimation unit 33 can also estimate, for example, predetermined other users U as efficiency-improvement users. For example, the estimation unit 33 estimates predetermined other users U as efficiency-improvement users for each conference type or combination of participating users. The estimation unit 33 can also estimate, as efficiency-improvement users, other users U whose total or weighted sum of conference efficiency index values with multiple users U participating in the online conference is equal to or greater than the above-mentioned predetermined threshold value.

3.3.5. Conference Processing Unit 34
The conference processing unit 34 receives user information, which is information about a plurality of users U, from a plurality of terminal devices 2 via the network N and the communication unit 10. The user information includes real-time information such as voice information of the users U collected in real time by a microphone and captured video information, which is information about videos of the users U captured in real time by a camera.

The conference processing unit 34 generates conference information based on the received user information of multiple users U. The conference processing unit 34 mixes the audio information included in the user information of each user U to generate audio information in the conference information. Note that the conference processing unit 34 can also select one piece of audio information from the audio information included in the user information of multiple users U and use it as audio information in the conference information.

In addition, the conference processing unit 34 can convert the user U's voice information into text information using voice recognition, and then convert the converted text information into voice information of a character designated by the user U using voice synthesis.

In addition, the speech synthesis performed by the conference processing unit 34 can adjust speaking style (volume, speed, intonation (emphasis), speech intervals (pauses), etc.) and sound quality using parameters, and the conference processing unit 34 can convert text information into audio information using speech synthesis in accordance with the speaking style and sound quality specified by the user U.

The conference processing unit 34 also generates image information from the conference information using captured video information included in the user information of each user U. For example, the conference processing unit 34 performs image processing on a body image, which is an image of the user U's body included in the captured video information of the user U, to generate video information of the user U in the conference information.

For example, the conference processing unit 34 uses a learning model or the like to identify the coordinates of each feature point from an image of the user U's body, and generates video information of the user U in the conference information by reconstructing the user U's body in three-dimensional space based on the results of the identification. The user U's body is, for example, the user U's face and neck, but may also include the user U's torso, hands, arms, legs, etc. in addition to the user U's face and neck.

The conference processing unit 34 can also generate video information of user U in the conference information, for example, by transforming a three-dimensional body model of a character designated by user U to match the body shape estimated from an image of user U's body. The character designated by user U is shown as user U's avatar in the conference information. Each user U can designate a desired character from among multiple types of characters.

The conference processing unit 34 transmits the generated conference information to multiple terminal devices 2 via the communication unit 10 and the network N, and repeats the process of playing the conference information on the multiple terminal devices 2, thereby providing online conference services to multiple users U. Note that the conference processing unit 34 changes the position and size of the video information of each user U that constitutes the conference screen for that user U, but is not limited to this example.

As shown in FIG. 3, the conference processing unit 34 has a participation processing unit 40. When the determination unit 32 determines that the conference efficiency does not satisfy a predetermined condition, the participation processing unit 40 virtually allows an efficiency improvement user estimated by the estimation unit 33 to participate in the online conference as a specific user.

Allowing a specific user to virtually participate in an online conference means including information about the specific user in the conference information as if the specific user were participating in the online conference, without using the above-mentioned real-time information about the specific user.

For example, the participation processing unit 40 virtually allows a specific user to participate in an online conference by making an avatar of the specific user appear in the online conference. The avatar of the specific user is, for example, an avatar of a character designated by the specific user.

The participation processing unit 40 stores, for example, multiple types of preset information, which is information indicating actions (body movements and speech) set in advance by specific users. The multiple types of preset information are, for example, preset information for each meeting situation, such as information indicating actions of nodding in response to comments made by participating users, information indicating actions that encourage participating users to speak, and information indicating actions that praise participating users, but are not limited to these examples.

The participation processing unit 40 can make a specific user's avatar appear in an online conference by including conversion information that converts pre-configured information into the behavior of the specific user's avatar in the conference information. The conversion information includes information such as the avatar's audio and video information.

For example, the participation processing unit 40 can determine preset information corresponding to the progress of the online conference from multiple types of preset information, and include converted information that converts the determined preset information into the behavior of a specific user's avatar in the conference information. The progress of the online conference can be determined, for example, by the number of comments per unit time, the facial expressions of participating users, the tone of their speech, etc.

The participation processing unit 40 can also make a specific user's avatar appear in the online conference so that it can be operated by some of the participating users in the online conference. For example, some of the participating users can select their desired preset information from multiple types of preset information, and the participation processing unit 40 can include conversion information in the conference information that converts the preset information selected by some of the participating users into the behavior of the specific user's avatar. Note that the participation processing unit 40 can also make a specific user appear in the online conference using the preset information as is without converting it.

The participation processing unit 40 can also include in the conference information conversion information obtained by converting the user information of some participating users into the behavior of the avatar of a specific user. The some participating users are the participating users who host the conference, but are not limited to this example. For example, they may be the participating users with the most comments, or participating users other than the participant user with the highest conference efficiency index value. In this case, the participation processing unit 40 can also make specific users appear in the online conference by using the pre-set information as is without converting it.

The participation processing unit 40 can also include in the conference information conversion information obtained by converting the user information of some participating users into the behavior of the avatar of a specific user. The "some participating users" are the participating users who host the conference, but are not limited to this example. For example, they may be the participating users with the most comments, or participating users other than the participant user with the highest conference efficiency index value.

After virtually allowing a specific user to participate in an online conference, if the conference efficiency satisfies a preset cancellation condition, the participation processing unit 40 terminates the virtual specific user's participation in the online conference by making the specific user's avatar disappear from the online conference. The preset cancellation condition is, for example, a condition in which the conference efficiency is equal to or greater than a predetermined threshold, and the threshold is set for each conference type.

Furthermore, if the meeting efficiency does not meet predetermined conditions within a predetermined period of time after virtually allowing a specific user to participate in an online meeting, the participation processing unit 40 can raise the tone of the speech from the specific user's avatar or change the facial expression of the specific user's avatar to a negative one. Negative expressions include, for example, an angry or sad expression.

[3.3.6. Notification section 35]
The notification unit 35 notifies the specific user, who has been estimated by the estimation unit 33 as an efficiency improvement user, of virtual participation information, which is information indicating participation in a virtual online conference.

For example, the notification unit 35 notifies the specific user of the virtual participation information by sending the virtual participation information to the specific user's terminal device 2 or the specific user's email address. This allows the specific user to understand that the virtual participation information was used virtually to improve the efficiency of an online conference in which they are not actually participating.

4. Processing Procedure
Next, a procedure of information processing by the processing unit 12 of the information processing device 1 according to the embodiment will be described. Fig. 7 is a flowchart showing an example of information processing by the processing unit 12 of the information processing device 1 according to the embodiment. The processing shown in Fig. 7 is repeatedly executed by the processing unit 12.

As shown in FIG. 7, the processing unit 12 of the information processing device 1 determines whether a meeting efficiency timing has arrived (step S10). The meeting efficiency timing is, for example, a timing that arrives at a predetermined cycle, but is not limited to this example. For example, the meeting efficiency timing may occur when the number of comments per unit time falls below a threshold, or may be some other timing.

If the processing unit 12 determines that the meeting efficiency timing has arrived (step S10: Yes), it determines the meeting efficiency, which is the efficiency of the online meeting (step S11). Then, the processing unit 12 determines whether the meeting efficiency determined in step S11 satisfies predetermined conditions (step S12).

If the processing unit 12 determines that the meeting efficiency does not satisfy the predetermined conditions (step S12: No), it estimates another user U who improves the meeting efficiency as an efficiency-improving user and determines the estimated efficiency-improving user as a specific user (step S13). Then, the processing unit 12 performs processing to virtually have the specific user participate in the online meeting (step S14).

When the processing of step S14 is completed, when it is determined that the meeting efficiency timing has not arrived (step S10: No), or when it is determined that the meeting efficiency satisfies the predetermined condition (step S12: Yes), the processing unit 12 terminates the processing shown in FIG. 7.

5. Hardware Configuration
The information processing device 1 according to the embodiment described above is realized by, for example, a computer 80 configured as shown in Fig. 8. Fig. 8 is a hardware configuration diagram showing an example of the computer 80 that realizes the functions of the information processing device 1 according to the embodiment. The computer 80 has a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I/F) 85, an input/output interface (I/F) 86, and a media interface (I/F) 87.

The CPU 81 operates based on programs stored in the ROM 83 or HDD 84 and controls each component. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 starts up, as well as programs that depend on the computer 80's hardware.

The HDD 84 stores programs executed by the CPU 81 and data used by such programs. The communication interface 85 receives data from other devices via the network N (see Figure 2) and sends it to the CPU 81, and transmits data generated by the CPU 81 to other devices via the network N.

The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input/output interface 86. The CPU 81 acquires data from the input devices via the input/output interface 86. The CPU 81 also outputs data generated via the input/output interface 86 to the output devices.

The media interface 87 reads programs or data stored on the recording medium 88 and provides them to the CPU 81 via the RAM 82. The CPU 81 loads the programs from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded programs. The recording medium 88 may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 80 functions as the information processing device 1 according to the embodiment, the CPU 81 of the computer 80 executes programs loaded onto the RAM 82 to realize the functions of the processing unit 12. In addition, the HDD 84 stores data in the storage unit 11. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be obtained from another device via the network N.

[6. Other]
Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

For example, the information processing device 1 described above may be implemented using multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API or network computing.

Furthermore, the above-described embodiments and variations can be combined as appropriate to the extent that the processing content is not contradictory.

7. Effects
As described above, the information processing device 1 according to the embodiment includes the determination unit 32 and the participation processing unit 40. The determination unit 32 determines the meeting efficiency, which is the efficiency of an online conference. If the meeting efficiency determined by the determination unit 32 does not satisfy a predetermined condition, the participation processing unit 40 virtually allows a specific user to participate in the online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The participation processing unit 40 also causes the avatar of a specific user to appear in the online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The participation processing unit 40 also causes avatars of specific users to appear in the online conference so that they can be operated by some of the participating users in the online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The participation processing unit 40 also includes an estimation unit 33 that estimates user U as a specific user, thereby improving meeting efficiency. This allows the information processing device 1 to further improve the efficiency of online meetings.

In addition, the estimation unit 33 estimates a user U who improves the efficiency of each type of online conference as a specific user. This allows the information processing device 1 to further improve the efficiency of online conferences.

The information processing device 1 also includes a notification unit 35 that notifies the specific user estimated by the estimation unit 33 of information indicating that the specific user will participate in a virtual online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The determination unit 32 also determines the meeting efficiency from the users participating in the online conference based on information indicating the relationship between meeting efficiency and the users participating in the online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The determination unit 32 also determines the efficiency of the meeting based on the number of comments per unit time. This allows the information processing device 1 to further improve the efficiency of online meetings.

The determination unit 32 also determines the efficiency of the online conference based on information indicating the efficiency of the online conference entered by the participating users during the online conference. This allows the information processing device 1 to further improve the efficiency of online conferences.

The above describes in detail the embodiments of the present application based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

Furthermore, the "section, module, unit" mentioned above can be read as "means" or "circuit." For example, an acquisition unit can be read as an acquisition means or an acquisition circuit.

REFERENCE SIGNS LIST 1 Information processing device 2 _{, 21} to _2n terminal devices 10 Communication unit 11 Storage unit 12 Processing unit 20 User information storage unit 21 Conference information storage unit 22 Conference efficiency information storage unit 30 Acquisition unit 31 Reception unit 32 Determination unit 33 Estimation unit 34 Conference processing unit 35 Notification unit 40 Participation processing unit 100 Information processing system

Claims (7)

a determination unit that determines conference efficiency, which is the efficiency of the online conference, based on one or more of information indicating a relationship between conference efficiency and users, information on participating users of the online conference, the number of comments per unit time in the online conference, and information indicating the efficiency of the online conference input by participating users in the online conference;
and a participation processing unit that, when the relationship between the meeting efficiency determined by the determination unit and a predetermined threshold does not satisfy a predetermined condition, virtually causes an avatar of the specific user to participate in the online conference by making the specific user appear in the online conference. The joining processing unit
The information processing device according to claim 1 , wherein an avatar of the specific user appears in the online conference so as to be operable by some of the participating users of the online conference. The joining processing unit
an estimation unit that estimates a user who improves the efficiency of the meeting as the specific user ;
The user who improves the efficiency of the meeting is
The other user is a user whose meeting efficiency associated with a combination of multiple users participating in the online conference is equal to or greater than a threshold value, a user whose meeting efficiency is the highest associated with a combination of multiple users participating in the online conference, a user who is predetermined for each type of online conference or each combination of multiple users participating in the online conference, or a user whose total or weighted sum of meeting efficiency index values with multiple users participating in the online conference is equal to or greater than a threshold value or is the highest.
3. The information processing apparatus according to claim 1 , wherein the information processing apparatus is a computer. The information processing device according to claim 3 , further comprising a notification unit that notifies the specific user estimated by the estimation unit of information indicating the specific user's virtual participation in the online conference. The predetermined condition is:
The condition is that the meeting efficiency is equal to or greater than a predetermined threshold, or the condition is that the average value of the meeting efficiency is equal to or greater than a predetermined threshold.
3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer. 1. A computer-implemented information processing method, comprising:
a determination step of determining conference efficiency, which is the efficiency of the online conference, based on one or more of information indicating a relationship between conference efficiency and users, information on participating users of the online conference, the number of comments per unit time in the online conference, and information indicating the efficiency of the online conference input by participating users in the online conference;
and a participation processing step of virtually having the specific user participate in the online conference by making an avatar of the specific user appear in the online conference if the relationship between the meeting efficiency determined by the determination step and a predetermined threshold does not satisfy a predetermined condition. a determination step of determining the meeting efficiency, which is the efficiency of the online conference, based on one or more of information indicating the relationship between the meeting efficiency and users, information on participating users of the online conference, the number of comments per unit time in the online conference, and information indicating the efficiency of the online conference input by participating users in the online conference;
and a participation processing procedure for virtually having a specific user participate in the online conference by making an avatar of the specific user appear in the online conference if the relationship between the meeting efficiency determined by the determination procedure and a predetermined threshold does not satisfy a predetermined condition.