JP7633962B2 - Virtual space video calling system and method - Google Patents

Description

The present invention relates to a virtual space video calling system and method.

Technology is known that allows multiple users to gather in a virtual space provided on a server on a network and interact through chats, games, etc. (Patent Document 1).

Patent No. 5969476

However, in recent years, with the spread of teleworking, where people work from home, there has been a lack of opportunities for communication, which has led to problems with maintaining a sense of camaraderie in the workplace and sharing information.

One idea is to gather employee avatars in a virtual office and hold video calls, but with typical video calling systems, you select the person you want to call from a list and press the call start button, regardless of the location of the avatar in the virtual office. Therefore, even if video calling is introduced in virtual offices, it is not much different from making a video call on a smartphone, and it lacks the casualness and convenience of going up to the person you want to talk to and talking to them, as is often done in real offices.

However, since the amount of data involved in a voice call is small, it is possible to automatically invite other avatars within a circular calling range set for an avatar to a voice channel and start a voice call.

However, video calls involve a larger amount of data than voice calls. Therefore, if a method were to automatically start a video call with another avatar within the calling range, it would consume a large amount of computer resources and communication bandwidth. On the other hand, it would be possible to provide a button on the screen to start a video call and start a video call when the button is pressed, but in that case, special operations would be required to start the call, which is different from starting a conversation in an actual office, making it less user-friendly and not promoting casual communication.

The object of the present invention is to provide a virtual space video calling system and method that improves usability for users.

A virtual space video calling system according to one aspect of the present invention is a virtual space video calling system in which multiple users make video calls through a virtual space, and includes a virtual space control unit that controls the virtual space provided to client terminals used by the multiple users, a video call control unit that controls video calls in the virtual space, and a call candidate management unit that manages predetermined users among the multiple users as candidates for video calls, and the call candidate management unit includes a user selection unit that displays information about the predetermined users and allows a candidate user to be selected from the predetermined users as a partner in a video call, a candidate user status confirmation unit that displays information indicating the status of the candidate users, and a movement unit that moves the avatar of the user who wishes to make a video call within a predetermined range of the avatar of the candidate user selected by the candidate user selection unit.

When the video call control unit receives a specific instruction, it can start a video call between the client terminal used by the user who wishes to make the video call and the client terminal used by the selected candidate user.

When the virtual space control unit determines that avatars in the virtual space have come into contact with each other, it transmits the predetermined instruction to the video call control unit, and the video call control unit can start a video call between the client terminals used by the users corresponding to the avatars determined by the virtual space control unit to have come into contact.

The information about the specified user may display a user status indicating the situation of the specified user, and the candidate user status confirmation unit may display an avatar status indicating the situation around the candidate user's avatar.

When the avatar of a user wishing to make a video call moves within a predetermined range, the virtual space control unit can display a predetermined effect indicating that the avatar of the user wishing to make a video call is near the avatar of the candidate user.

The virtual space has multiple virtual floors, and the virtual floor on which the avatar wishing to make a video call is located may be different from the virtual floor on which the candidate user's avatar is located.

The call candidate management unit can also display the number of viewers, which is the number of users currently viewing the avatar of a user who wishes to make a video call.

The call candidate management unit can also display the number of users viewing the avatar of a user who wishes to make a video call based on their relationship with the user who wishes to make the video call.

The user status may be created based on information obtained from a management system that manages the user's work status and information obtained from the client terminal used by the user.

The video call control unit can start a video call by displaying, within each avatar that comes into contact in the virtual space, an image acquired from a client terminal used by a user corresponding to the avatar.

1 is an overall schematic diagram of a virtual space video calling system according to an embodiment of the present invention; FIG. 1 is an explanatory diagram of the configuration of a virtual office. FIG. 1 is an explanatory diagram showing how a user who wishes to make a video call selects candidate users for the video call; FIG. 13 is an explanatory diagram showing how a video call is started when the avatar of a user desiring to make a video call moves close to the avatar of a candidate user and the two avatars come into contact. FIG. 1 is a functional diagram of a virtual space video calling system. 10 is a flowchart showing a process of collecting information from a client terminal. 13 is a flowchart of the entire process. 13 is a flowchart of a video call process. Example of follow-up information. An example of information that indicates the user's status. An example of an interest filter. 10 is a flowchart showing a virtual office display process according to the second embodiment. FIG. 13 is an explanatory diagram showing how information about a user viewing, from another floor, the avatar of a user who wishes to make a video call is notified. 13 is a flowchart showing a process of analyzing a user's situation based on information acquired from a client terminal and information acquired from another system according to the third embodiment.

In this embodiment, a virtual space video calling system is provided that allows a user wishing to make a video call to easily make a video call with another user, just like having a conversation in a real office, by simply intuitively moving their avatar towards the avatar of the person they want to talk to and bringing it into contact with the avatar.

Furthermore, in this embodiment, even if the virtual space is large or made up of multiple floors, you can easily check the appearance of the person you want to video call before moving, and you can also quickly move close to that person.

The first embodiment will be described with reference to Figures 1 to 11. Figure 1 shows an overview of the virtual space video calling system 1 of this embodiment. The virtual space video calling system 1 includes, for example, a virtual office providing server 2 and a plurality of client terminals 3.

The virtual office providing server 2 provides a virtual office 4 as a "virtual space" to each client terminal 3. The virtual office providing server 2 includes a virtual office control unit 21 as a "virtual space control unit" that controls the virtual office 4, a video call control unit 22 that controls video calls made in the virtual office, an information collection unit 23, an information provision unit 24, and a call candidate management unit 25. The virtual office providing server 2 is also connected for two-way communication with another system 5 as a "management system".

The virtual office control unit 21, which serves as a "virtual space control unit," provides each client terminal 3 with a virtual office 4 as an example of a "virtual space." Using the client terminal 3, a user can operate an avatar that corresponds to the user within the virtual office 4.

The video call control unit 22 controls video calls that users make through their avatars within the virtual office 4. As described below, in this embodiment, a video call is initiated when an instruction is received that the avatars have come into contact with each other within the virtual office 4.

In the following, a user who wishes to make a video call may be referred to as the "own user," an avatar corresponding to the "own user" as the "own avatar," avatars corresponding to candidate users for a video call as "candidate avatars," and users other than the "own user" as "other users."

The information collection unit 23 collects information from each client terminal 3. For example, the information collection unit 23 acquires from the client terminal 3 the type of operating system and application running on the client terminal 3, remaining memory capacity, CPU usage, operation of an input device such as a keyboard, and a facial image of the user. The information collection unit 23 may also acquire vital information such as body temperature, pulse rate, and blood pressure from the client terminal 3.

The information providing unit 24 delivers all or part of the information collected from each client terminal 3 to each client terminal 3.

The call candidate management unit 25 manages specific users who are candidates for video calls. The call candidate management unit 25 includes a follow list 250 as a "user selection unit", a confirmation button 251 as a "candidate user status confirmation unit", and a move button 252 as a "move unit".

As described below, the follow list 250 displays specific users who are candidates for video calls and the status of the specific users, and the user can select one desired specific user from the follow list 250.

The user can display the status of the avatar corresponding to the selected user by operating the confirmation button 251. The user can move their avatar close to the avatar of the selected user by operating the movement button 252. The user can then start a video call by moving their avatar towards the avatar of the selected user who is nearby and bringing it into contact with the avatar of the selected user.

The other systems 5 include, for example, project management, task management, workflow management, groupware, schedule management, file sharing, email management, and attendance management. Although only one other system 5 is shown in FIG. 1, the virtual office providing server 2 can be connected to multiple different other systems 5 for two-way communication.

In the following description, the virtual space video call system 1 may be abbreviated as video call system 1, and the virtual office providing server 2 may be abbreviated as server 2.

The user accesses the server 2 using the client terminal 3 and logs in to the virtual office service, causing his/her avatar GP1 to appear in the virtual office 4. As described below, the client terminal 3 has a camera in its user interface unit 33 as an image acquisition unit.

The virtual office 4 is a virtual space used by the organization to which the user belongs. The layout of the virtual office 4 can be similar to that of a real office, or it can be different from the layout of a real office. Organizations to which the user belongs include, for example, companies, government agencies, local governments, and non-profit organizations (NPOs).

In the virtual office 4, for example, furniture and fixtures such as a desk GP2 and a chair GP3, a desk nameplate GP4, a conference room GP5, and a concentration room GP6 (see FIG. 2) are drawn. While the avatar GP1 is away from a predetermined seat, a ghost avatar GP12 is displayed at that seat. The ghost avatar GP12 indicates that the user is logged in but not at the seat. For example, in FIG. 1, the avatar of user U3 is not at his/her seat because he/she has gone to another floor. At this time, the ghost avatar GP12 is displayed instead of the avatar GP1 (U3). When the avatar of another user comes into contact with the ghost avatar GP12, the avatar of the other user is moved to the location of the avatar GP1 (U3) corresponding to the ghost avatar GP12. Although not shown in the figure, the avatar used by user U3 will be described with the symbol GP1 (U3). The same applies to the following description.

For example, a case will be described where the virtual office 4 has multiple floors, such as a first floor 4 (F1) and a second floor 4 (F2). When an avatar GP1 (U3) on the first floor 4 (F1) is moving to the second floor 4 (F2), if another avatar GP1 on the first floor 4 (F1) comes into contact with a ghost avatar GP12 (U3), that avatar GP1 will move closer to the avatar GP1 (U3) on the second floor 4 (F2).

An information display section GP11 showing information about the user corresponding to the avatar is displayed in association with the avatar GP1. The information display section GP11 includes, for example, the name of the user operating the avatar GP1. In addition to the name, the information display section GP11 can also display the user's condition (status), the name of the department to which the user belongs, a number showing the activity of the user's activities, information showing the user's condition, and the like as "user status."

The user's status may include, for example, the user's own situation, such as being on the phone, in a meeting, busy, having finished work, or on vacation. The user's status may also include, for example, biological information and mental information, such as physical condition, sleep time, body temperature, heart rate, fatigue level, and happiness level.

As will be described later in FIG. 3, information indicating the user's status (information indicating the user's situation) is provided to other users using multiple information display units GP11, GP31, and GP32.

Figure 2 shows an example of a virtual office 4. The virtual office 4 is configured as a virtual two-dimensional space or a virtual three-dimensional space. The virtual office 4 may be configured as a so-called virtual 2.5-dimensional space. Figure 2 shows a relatively small virtual office 4 on one floor with four desks GP2 and 16 chairs GP3, but the virtual office 4 is not limited to this and may be a large virtual office 4 with multiple floors, each of which can accommodate tens to hundreds of people.

Conference room GP5 is an area where users hold a conference via avatars. Who is in conference room GP5 can be visually confirmed from areas other than conference room GP5. When avatar GP1 enters conference room GP5, he automatically becomes a participant in the video call. For example, when avatar GP1 (U12) enters conference room GP5 where avatar GP1 (U8) is present, a video call between the avatars is started. In this embodiment, a video call is started when avatars GP1 come into contact with each other, but in conference room GP5, a video call can be started even if avatars do not come into contact with each other. This is because conference room GP5 is an area for a conference, and it is clear that the avatar wishes to make a video call at the time of entering the room. However, a video call may also be started when avatars come into contact with each other in conference room GP5.

The concentration room GP6 is an area used by users who want to concentrate on work or thinking. Only one avatar can enter the concentration room GP6. Other avatars cannot contact an avatar in the concentration room GP6, and therefore cannot make video calls with an avatar who has entered the concentration room GP6. For example, it is possible to contact an avatar who has entered the concentration room GP6 by displaying a message on the desk nameplate GP4 of the avatar.

The avatar GP1 may have a common design for all users, or may be selectable for each part, such as hairstyle or accessories. The avatar GP1 does not have to be designed to resemble a human, and may be an animal, plant, robot, machine, figure, etc. Multiple types of avatars may be provided, such as a normal avatar GP1, a ghost avatar GP12, and a busy avatar GP1N. The busy avatar GP1N is an avatar that notifies others that the user is busy and will not accept video calls from other users. Therefore, even if another user's avatar GP1 comes into contact with the busy avatar GP1N, a video call will not be initiated.

A case where a video call is made within the virtual office 4 will be described with reference to Figures 3 and 4. A follow list 250 is displayed in the virtual office 4. The follow list 250 is information in which specific users who are candidates for video calls are registered, and is provided in list format. The display of the follow list 250 may be turned on and off by the user's operation, or may be constantly displayed.

In the follow list 250, the status of users registered as "prescribed users" is displayed in the information display section GP11. Therefore, the user can quickly grasp the status of registered users (hereinafter also referred to as registered users) just by glancing at the follow list 250. The user can select a partner for a video call from among the registered users who are in a position where a video call is possible.

The display order in the follow list 250 may be changed depending on the status of the registered users. For example, registered users who are online and available for video calls may be displayed at the top of the follow list 250. A registered user who is available for video calls is a registered user who is not in a specific situation in which it is determined that a video call is not possible, such as being away from his/her desk or on a phone call. Alternatively, a registered user who is only displayed as being online and has no other status displayed may be set as a registered user who is available for video calls.

Note that although there is no limit to the number of users that can be registered in the follow list 250, an upper limit of several to several tens of people may be set in order to conserve network and computer resources. An upper limit on the number of users that can be registered in the follow list 250 may be set according to attributes such as the user's job title or department.

Although users on the same floor may be registered in the follow list 250, it is more convenient to register users on different floors. A user in a virtual office 4 (F1) on the first floor can easily check on the other party's status or start a video call by registering a user in a virtual office 4 (F2) on another floor (e.g., the second floor).

Even if one floor of a virtual office is large, by registering other users on the same floor in the follow list 250, a user who wishes to make a video call can check the appearance of the avatar of the user with whom he or she wants to make a video call without having to scroll the screen repeatedly, and can easily start a video call without having to move his or her own avatar a long distance.

Figure 3 shows a case where user U3, whose avatar is in virtual office 4 (F1), wishes to make a video call with user U28, who is registered in follow list 250. In the following explanation, user U3 is the user who wishes to make a video call, and user U28 is the candidate user for the video call. It is assumed that user U28 is in a different virtual office 4 (F2) from the virtual office 4 (F1) where user U3's avatar is located.

As described above, the follow list 250 displays the situation (user status) of user U28. Therefore, user U3 can easily grasp the general situation of user U28 by checking the follow list 250.

However, there are cases where the information obtained from the information display unit GP11 alone does not provide a grasp of the actual situation of the call candidate user U28. For example, when other users' avatars are gathering near the avatar of user U28 and it looks like a video call is about to begin, if the avatar of user U28 is in the concentration room GP6, the status of user U28 may be "in a meeting", but the avatar of user U28 may be alone in the conference room GP5, and the situation of the user displayed in the follow list 250 may not necessarily match the situation of the avatar corresponding to the user. Therefore, unless the situation of the avatar of the call candidate user is also confirmed, it may not be possible to properly determine whether or not a video call is possible.

In response to this, it is also possible to classify the user's status in more detail, such as "alone in a conference room" or "high possibility of chatting via video call", and display this in the information display unit GP11. In this case, simply by looking at the follow list 250, it is possible to determine whether a user registered there is in a status where a video call is possible. However, since it is necessary to grasp the status of all users under the management of the call system 1 and manage their status in detail, this consumes a large amount of computer and network resources.

Therefore, in this embodiment, as shown in the upper part of FIG. 3, a "Confirm" button 251 is provided in the follow list 250. User U3 operates the confirm button 251 after selecting the person with whom he/she wishes to have a video call (here, user U28).

Then, as shown in the lower part of Figure 3, a surrounding situation window W1 showing the surrounding situation of the avatar corresponding to the video call candidate U28 is displayed on the screen of the client terminal 3 used by the user U3.

The window W1 may be displayed over the virtual office 4 (F1) in which the avatar of the user U3 is located, or the image of the virtual office 4 (F1) may be switched to the surrounding situation window W1. The surrounding situation window W1 is a display area that displays the surrounding area of the avatar corresponding to the candidate user U28 of the video call. The surrounding area of the avatar corresponding to the candidate user U28 is an area within a predetermined range from the avatar. The shape of the surrounding area may be any shape, such as a rectangle, a circle, or an ellipse. The surrounding situation window W1 can display an enlarged view of the avatar of the candidate user of the video call and its surroundings. Furthermore, at least a part of the environment of the virtual office 4 (F2), such as sound, light, messages, and vibrations, which are expressed around the avatar of the candidate user, may be reproduced in the surrounding situation window W1. For example, the situation of the candidate user U28 can be determined from the music, voice, text messages, and color of the lighting in the room that are playing in the virtual office 4 (F2) in which the candidate user U28 is located.

In addition, information display areas GP11 (omitted from FIG. 3), GP31, and GP32 that indicate the user's situation are displayed around the avatar of candidate user U28. Information indicating the user's situation is displayed in these information display areas.

In the surrounding situation window W1, a move possibility window W2 is also displayed for moving the avatar of user U3 closer to the avatar of candidate user U28. User U3, who wishes to make a video call, can move the avatar of user U3 closer to the avatar of user U28, who is a candidate for the video call, by operating the "Yes" button B1. If user U3 operates the "No" button B2, the avatar will not move. When user U3 operates the "No" button B2, the surrounding situation window W1 may be dismissed. When user U3 operates the "Yes" button B1, the view moves to the diagram shown at the top of Figure 4.

In addition, user U3 can also make his/her own avatar appear near user U28's avatar by operating the movement button 252 displayed on the follow list 250 without checking the surrounding situation of user U28's avatar.

The display periphery of window W1 may or may not match the range in which user U3's avatar appears when user U3 who wishes to make a video call operates the movement button 252.

As shown in the upper part of Figure 4, the avatar of user U3 who wishes to make a video call moves close to the avatar of user U28 who has been selected as the call partner. The avatar of user U3 appears at a location a predetermined distance ΔL away from the avatar of user U28. A predetermined effect GP13 is displayed on the avatar of user U3 to notify those around the user of the presence of the avatar of user U3.

The predetermined effect may be called a "notification effect", a "video call request effect", etc. Examples of the predetermined effect include sound, voice message, color change, change in avatar size, vibration (in cases where a vibration device is connected to the client terminal 3), and combinations of these. The effect is not limited to these, and any effect that can directly or indirectly notify user U28 of the approach of user U3 may be adopted.

As shown in the lower part of Figure 4, user U3 who wishes to make a video call moves his/her avatar towards the avatar of user U28, bringing the two avatars into contact. This opens a video call channel and starts the video call. Hereinafter, the user's code may be added to the avatar code GP1, such as avatar GP1 (U3), to distinguish which user's avatar it belongs to.

When the avatars come into contact with each other, the video call control unit 22 starts preparing for the video call. Until the video call is ready, the virtual office control unit 21 can also display an effect (not shown) indicating that the video call is being prepared on each of the avatars GP1 (U3) and GP1 (U28), who are the targets of the video call. The virtual office control unit 21 notifies not only the targets of the video call U3 and U28, but also other users in the vicinity, of the start of the video call by, for example, changing the color of the avatar, making it blink, enlarging or reducing the size, or displaying an animation.

When a video call is started, the size of each avatar GP1 (U3), GP1 (U28) participating in the video call is enlarged by a predetermined amount. The user's video VD is displayed in the enlarged avatar. That is, a video VD including the facial image of user U3 is displayed in the enlarged avatar GP1 (U3). Similarly, a video VD of user U10 is displayed in the enlarged avatar GP1 (U28).

Whether or not the user's facial image is displayed in the avatar GP1 depends on the orientation of the camera on the client terminal 3, work regulations, etc. If the user does not wish to show his or her face, the user can turn off the camera, point the camera in a direction away from the user, or display a still image or animation instead of the facial image.

To other users not participating in the video call, the avatar during the video call appears in normal size. To clearly inform other users that a video call is in progress, the color, shape, or size of the avatar during the video call may be changed. A user wishing to join the video call can join the video call by touching their avatar to either avatar GP1 (U3) or GP1 (U28) that is in the video call.

As described above, in this embodiment, a video call is initiated when avatars come into contact with each other. However, when avatars come into contact with each other, this does not necessarily mean that the outer edges of the avatars come into contact with each other, but also includes when the distance between the outer edges of the avatars becomes equal to or smaller than a predetermined distance. The avatars may partially overlap each other.

The functional and hardware configuration of the video calling system 1 will be described with reference to FIG. 5. As described in FIG. 1, the video calling system 1 includes a virtual office providing server 2 and a plurality of client terminals 3. The server 2 is connected to each client terminal 3 and other systems 5 via a communication network CN so as to enable two-way communication.

The server 2 is a computer equipped with a processor 200 and a memory 201, and functions as a virtual office providing server 2 by executing a computer program stored in the memory 201.

The server 2 includes, for example, a virtual office control unit 21, a video call control unit 22, an information collection unit 23, an information provision unit 24, a call candidate management unit 25, a database (DB in the figure) 26, and a communication unit 27. Although lines are omitted in the figure, the virtual office control unit 21, the video call control unit 22, the information collection unit 23, and the information provision unit 24 are connected to the database 26.

The database 26 includes, for example, a table for managing authentication information for each user who uses the virtual office service, a table for managing the usage history of the virtual office for each user, a table for managing the positions and display forms of various objects placed in the virtual office 4, and a table for managing user status information acquired from the information collection application 32 of each client terminal 3 (all not shown).

The communication unit 27 has the function of performing two-way communication with each client terminal 3 via the communication network CN.

The storage medium MM is a storage medium capable of retaining data for a relatively long period of time, such as a flash memory or a hard disk drive. By connecting the storage medium MM to the server 2, at least a portion of the computer program can be transferred between the storage medium MM and the memory 201.

The client terminal 3 is a computer used by a user. The client terminal 3 may be configured as, for example, a laptop personal computer, a desktop personal computer, a tablet personal computer, a mobile information terminal (including a so-called smartphone), a wearable terminal, etc.

The client terminal 3 includes, for example, a virtual office application 31, an information collection application 32, a user interface (UI in the figure) unit 33, and a communication unit 34.

The virtual office application 31 is a computer program for operating the avatar GP1 in the virtual office 4. The user interface unit 33 is a function for the user and the client terminal 3 to input and output information via an information input/output device (not shown). Examples of information input/output devices include a keyboard, pointing device, microphone, camera, monitor display, printer, and speaker. The communication unit 34 has a function for two-way communication with the server 2 via the communication network CN.

The information collection application 32 sends the information acquired from the user interface unit 33 and the information acquired from the operating system (not shown) or other applications running on the client terminal 3 to the information collection unit 23 of the server 2 either directly or after processing.

For example, an image captured by a camera can tell whether or not a user is in front of the client terminal 3. For example, a microphone signal can tell whether or not the user is talking, and the noise surrounding the user. For example, by acquiring keyboard operations, it can be known that the user is inputting some information into the client terminal 3. For example, by acquiring operations of a mouse or touch panel, it can be known that the user is operating some application such as a web browser. For example, if the client terminal 3 has a GPS (Global Positioning System), the user's current location, moving direction, moving speed, moving trajectory, etc. can be known. For example, if the client terminal 3 has a motion sensor, the user's actions can be known.

Furthermore, by acquiring the access history of other applications running on the client terminal 3, such as a web browser, it is possible to know which web pages the user has visited. For example, by acquiring information on word processing software, spreadsheet software, design support software, etc. operated from the client terminal 3, it is possible to know what software the user has been using.

Furthermore, by obtaining user information from an attendance management system, it is possible to know what time the user arrived at work and what time they left work. For example, by obtaining information from a building entrance and exit management system, it is possible to know which building the user entered, what time, and what time they left.

As described above, in this embodiment, the status of the candidate user for the video call is displayed in the follow list 250, and if the confirmation button 251 is operated, the status of the avatar used by the candidate user is displayed in window W1. The avatar status is information that indicates the status of the avatar and its surroundings, and therefore can also be called "avatar surrounding information."

Figure 6 is a flowchart showing the process in which the information collection unit 23 of the server 2 collects information from the information collection application 32 of each client terminal 3.

The information collection application 32 acquires the status of the information input/output unit (keyboard, camera, microphone, etc.) from the user interface unit 33 (S11), and acquires the status of other applications (web browser, clerical processing applications, software development applications, etc.) running on the client terminal 3 (S12). Furthermore, the information collection application 32 acquires the status of the operating system of the client terminal 3 (S13), and acquires the location information of the client terminal 3 (S14). The information collection application 32 transmits this acquired information to the information collection unit 23 of the server 2 as user status information (S15).

When the information collection unit 23 receives information from each client terminal 3 (S21), it stores the received information in the database 26 (S22) and analyzes it (S23). Here, analysis refers to a process of determining the user's situation by processing information from the information collection application 32. For example, it is a process of determining how busy the user is from the frequency with which characters are entered into the keyboard. In the embodiment described below, the situation of the user is determined from workflow, to-do list, schedule data, etc. related to the user. This determination can be achieved by utilizing so-called artificial intelligence such as deep learning and machine learning.

The information collection unit 23 stores the analysis results from step S23 in the database 26 (S24). The information collection unit 23 can also acquire information from the other system 5 in step S21. For example, the information collection unit 23 can specify the identification number of the user who operates the client terminal 3, inquire about the attendance status of that user to the attendance management system, and acquire the response.

Figure 7 is a flowchart of the overall processing of the video calling system 1. When the virtual office control unit 21 is accessed from the client terminal 3 and acquires login information from the user (information used for user authentication), it executes a login process to the virtual office service (S31). The virtual office control unit 21 executes a virtual office display process that places objects such as avatars and desks in the virtual office 4 (S32).

The information providing unit 24 refers to the interest information filter T2 (described later in FIG. 11) set for the logged-in user (S33), and displays information (user status information) set as being of interest to the user about other users around the other avatars (S34). An example of displaying the status of the user operating an avatar around the avatar was described in FIG. 3.

When the virtual office control unit 21 receives an operation from the client terminal 3 (S35), it controls the display of the virtual office 4 in response to the operation from the client terminal 3, such as changing the position or facial expression of the avatar (S36).

When the avatars come into contact with each other, the video call control unit 22 starts the video call processing (S37). An example of the video call processing will be described later. The virtual office control unit 21 executes this processing until an instruction to stop the virtual office service is given (S38).

Figure 8 is a flowchart showing the details of the video call process S37 in Figure 7. The video call control unit 22 determines whether a candidate user for a video call has been selected from among the users registered in the follow list 250 (S41).

When a candidate user for a video call is selected from among users registered in the follow list 250 (S41: YES), the video call control unit 22 determines whether the confirmation button 251 has been operated (S42).

When the confirmation button 251 is operated to check the status of the avatar corresponding to the candidate user (S42: YES), the information providing unit 24 refers to the interest information filter T2 set for the user who wishes to make a video call, and displays information set as being of interest to the candidate user for the video call around the avatar of the candidate user (S44).

The virtual office control unit 21 accepts a movement operation by a user who wishes to make a video call (S45). This movement operation includes an operation for instantly moving one's own avatar close to the candidate user's avatar (operation of the movement button 252) and an operation for moving one's own avatar with a mouse or the like.

When the move button 252 is operated in step S45, the avatar of the user who wishes to make a video call immediately moves close to the avatar of the candidate user for the video call, as described in the upper part of FIG. 4. The user who wishes to make a video call can also scroll the screen to move their own avatar toward the avatar of the candidate user for the video call without using the move button 252.

The virtual office control unit 21 determines whether the avatar of the user who wishes to make a video call has approached the avatar of the candidate user for the video call (S46). When the two avatars approach each other (S46: YES), the virtual office control unit 21 causes the avatar of the user who wishes to make a video call to display an effect GP13 for notifying the approach (S47). The effect for notifying the approach may be realized not only by an element that is recognized visually, but also by an element that is recognized auditorily or tactilely.

The virtual office control unit 21 determines whether the avatars have come into contact with each other (S48), and if the avatars have come into contact with each other (S48: YES), notifies the video call control unit 22 of that fact.

When the avatars come into contact with each other and an instruction to start a video call is given, the video call control unit 22 starts preparations for the video call, such as sharing video and audio, and starts the video call when preparations are complete (S49).

In step S49, for example, the virtual office control unit 21 displays an effect indicating preparation for a call, enlarges the size of the avatar, and sets the communication channel. The video call control unit 22 displays the video VD captured by the camera of the client terminal 3 in the avatar, and starts a voice call (S49).

The virtual office control unit 21 determines whether the contact between the avatars has been resolved, i.e., whether the avatars during the video call have separated (S50). As long as the avatars remain in contact with each other (S50: NO), the video call is maintained (S49). When the avatars separate (S50: YES), the video call ends (S51).

Even if the contact between the avatars is resolved, the video call does not end immediately, but can continue for a specified short period of time. This is to prepare for cases where, for example, the user moves the avatar by mistake.

The user can also select a user who is not registered in the follow list 250 as a candidate user for a video call (S41: NO). The user moves their avatar to the location of the avatar of the candidate user for the video call (S45, S46: YES). When the user finds the avatar of the candidate user, the user brings their avatar into contact with the avatar of the candidate user (S48: YES) and starts the video call (S49).

If the avatar of a candidate user for a video call is close to your avatar and you can check the status of the candidate user's avatar by simply scrolling the screen a little, there is little need to use the follow list 250. However, if the virtual office is large or has multiple floors, you will need to take the trouble to move your avatar to the location of the candidate user's avatar, and you will not be able to start a video call quickly. In this embodiment, you can register the candidate user in the follow list 250 and view (observe) the status of the candidate user's avatar without moving your own avatar.

Figure 9 shows an example of follow information T1. Although one follow information T1 is shown in Figure 9, follow information T1 is created for each user. The follow list 250 is created and displayed based on the follow information T1.

The follow-up information T1 includes, for example, a user ID C11, a user name C12, a department C13, a project C14, a date added to the list C15, a confirmation date and time C16, a call date and time C17, and other information C18.

The user ID C11 is information for identifying each user managed by the video call system 1. The user name C12 is the name of the user registered as a candidate for a video call. The department C13 is the department to which the registered user belongs. The project C14 is the project to which the registered user belongs. The list addition date C15 is the date on which the user identified by the user ID was registered as a candidate for a video call. The confirmation date and time is the date and time on which the status of the avatar corresponding to the registered user was confirmed (the date and time on which the confirmation button 251 was operated). The call date and time C17 is the date and time on which a video call was made with the registered user. Others C18 includes, for example, the cumulative time and the cumulative number of video calls. Note that at least some of the items of the follow information T1 shown in FIG. 9 may not be present.

A user can register a candidate user for a video call in the follow list 250 by explicitly specifying the candidate user. For example, a user can register the user in the follow list 250 by selecting the avatar of the user to be registered and operating a registration button (not shown).

Alternatively, the user can detect the user to be registered by entering the name or user ID of the user in a user search window (not shown) and register the detected user in the follow list 250. Alternatively, a user with whom the user has a video call may be automatically registered in the follow list 250. Alternatively, users who belong to the same department as the user or users who belong to the same project as the user may be automatically registered in the follow list 250.

Figure 10 is an explanatory diagram showing an example of user status information. Figure 10 shows examples of display types of user status information, methods of acquiring the information, methods of expressing the user status information, and methods of controlling the information.

Display types of user status information include, for example, "1. Business status," "2. User status (status of the user in front of the client terminal)," "3. Online status," "4. Floor status," "5. Calendar status," and "6. Communication status."

Information regarding the "business status" is obtained, for example, from the operating status of a telephone application, the operating status of an online conference application, the operating status of an email application, the operating status of a software development application, the operating status of an office processing application, the on/off state of the screen of the client terminal 3, the on/off state of the power supply of the client terminal 3, the contents posted to a chat application or an SNS (Social Networking Service), the position measurement function of the client terminal 3, etc. Depending on the contents of the business status, for example, a call mark, a meeting mark, an email mark, an in development mark, a document preparation mark, a client terminal sleeping mark, an unknown mark (when the client terminal is powered off), a moving mark, etc. are displayed around the avatar.

Information about the "user status" is obtained, for example, from the state of input to the keyboard, input sounds to the microphone, the presence or absence of an image of the user captured by the camera, the user's actions and facial expressions captured by the camera, and the user's location information. Depending on the content of the user status, for example, a sweating animation, a speech bubble animation, an away mark, a nodding mark, a current location mark (such as "head office", "factory", or "at home") is displayed around the avatar.

Information regarding the "online status" is obtained, for example, from the operation of a pre-prepared status display button (away from desk button, eating button, etc.) or from another system 5 such as an attendance management system. Depending on the content of the online status, for example, an away from desk mark, eating mark, long working hours mark, overtime warning mark, offline duration mark, etc. are displayed around the avatar.

Information regarding the "floor status" is acquired from the presence or absence of avatars entering or leaving the virtual office 4, the presence or absence of other avatars within a specified range of the operating user's avatar, etc. When another avatar for which no seat GP22 is provided enters the virtual office 4, a door ping-pong sound is made to ring from the operating user's client terminal 3. When another avatar for which a seat GP22 is provided enters the virtual office 4, the sound effect of a door opening and closing is made to ring from the operating user's client terminal 3. When another avatar enters within a specified range of the operating user's avatar, a specified effect is executed as described above, and for example, a sound effect of footsteps is made to ring from the operating user's client terminal 3. In addition to footsteps, nearby avatars may be made to blink or be larger in size.

Information about the "calendar status" is obtained, for example, from schedule management software.

Information about "communication status" is obtained, for example, from the usage status of video calls, the usage status of chat applications, etc.

In addition, in the "control" mode, for example, if the user is in a meeting or on a phone call, it is possible to prohibit the start of a video call. Alternatively, if the user is eating or traveling and another user requests a conversation, a message to that effect is sent to the client terminal 3 and displayed.

In the embodiment described below, the user's status is determined by analyzing information obtained from the workflow or to-do list.

Figure 11 is an example of an interest information filter T2 used to extract user situation information. The information providing unit 24 in this embodiment extracts specific user situation information from each user situation information based on the interest information filter T2 set for each user, and provides the extracted user situation information to the virtual office 4 in association with the avatar operated by each user. That is, in this embodiment, the display content of the user situation information associated with the avatars of other users differs for each user.

The interest information filter T2 can also be called a "predetermined policy." An interest information filter T2 is prepared for each user. The interest information filter T2 specifies the status of other users to be provided to the corresponding user.

The interest information filter T2 shown in the upper part of FIG. 11 includes, for example, a management number C21, a filter name C22, and display content C23. The filter name C22 succinctly expresses the content of the interest information filter. The display content C23 is the content of the user situation information extracted by the interest information filter.

The user status information displayed for other avatars on the same floor, the user status information displayed for other avatars in the same group, the user status information displayed for other avatars in the same subgroup, and the user status information displayed for the avatar of a user specified by the operating user may each be different.

Here, a "group" is a group to which multiple users belong, and can also be called a department. A "subgroup" is a small group that belongs to a group, and can also be called a team.

For other avatars on the same floor, for example, "1. Business status" and "5. Calendar status" are displayed. For other avatars in the same group, for example, "1. Business status", "3. Online status", "4. Floor status", and "6. Communication status" are displayed. For other avatars in the same subgroup, for example, "1. Business status", "2. User status (status of the user in front of the client terminal)", "3. Online status", "4. Floor status", "5. Calendar status", and "6. Communication status" are displayed. For avatars of other users specified by the user, for example, "1. Business status", "5. Calendar status", and "6. Communication status" are displayed.

If another avatar (another user) falls under multiple filters in interest information filter T2, a filter corresponding to a lower concept can be applied. For example, if another avatar on the same floor belongs to a group, the "within same group" filter with management number "2" is applied. If a user specifies an avatar of a specific user who belongs to a subgroup, the "specified user" filter with management number "4" is applied to the avatar of the specified user.

In addition to the above examples, a filter with less display content may be applied preferentially, or conversely, a filter with more display content may be applied preferentially.

The interest information filter T2A shown at the bottom of FIG. 11 controls the content displayed in each filter according to the avatar density in the virtual office 4. As the avatar density increases, less user situation information is placed around each avatar, making it easier for users to grasp the situation of other users.

With the virtual space video calling system 1 of this embodiment configured in this way, a user can start a video call simply by touching the avatar that represents the user to the avatar of the person they want to talk to, allowing them to easily use video calling with intuitive operations and easily communicate in the same way as they would in a real office.

Furthermore, in this embodiment, the user can easily select users with whom to make video calls from the follow list 250, improving the usability of video calls.

Furthermore, in this embodiment, the user can immediately grasp the status of the candidate user from the user status displayed in the follow list 250, so the user can easily decide whether or not to make a video call.

Furthermore, in this embodiment, the user can also check the status of the candidate user's avatar by operating the confirmation button 251. This allows the user to more appropriately decide whether or not to make a video call.

In this embodiment, the user can operate the movement button 252 to make their own avatar appear near the candidate user's avatar. Then, the user can start a video call by moving from the location where their own avatar appears toward the candidate user's avatar and making contact with it.

In this embodiment, the status of the avatar corresponding to the candidate user for the video call can be checked on the spot without moving the avatar of the user who wishes to make the video call, and after checking, the user's own avatar can be instantly moved close to the avatar of the candidate user to bring the avatars into contact with each other and start the video call. As a result, in this embodiment, in a virtual office 4 with a large single floor or a virtual office 4 with multiple floors, the availability of a video call can be appropriately determined and a video call can be easily started, improving usability for the user.

In addition, in this embodiment, the user's facial image is displayed inside the avatar, which saves computer and network resources. Furthermore, in this embodiment, the user can see the appearance of the person they are talking to through the facial image, and since the facial image does not cover the screen other than the avatar, the parties in the video call can make the video call while checking the situation inside the virtual office 4.

In this embodiment, a user who wishes to join an ongoing video call can join the video call simply by touching their avatar to any of the avatars participating in the ongoing video call.

The second embodiment will be described using Figures 12 and 13. In the following embodiments, including this one, the differences from the first embodiment will be mainly described.

In this embodiment, the number of other users viewing the user's avatar is displayed in the virtual office 4. Furthermore, in this embodiment, the number of viewing users is displayed according to the relationship between the viewing user and the user being viewed.

Figure 12 shows the display process of the virtual office. This process is executed by the call candidate management unit 25 and the virtual office control unit 21. In the following, the main body will be described as the server 2.

In the virtual office display process S37A, which is similar to step S32 in FIG. 7, objects such as avatars and furniture are arranged and displayed in the virtual office 4 (S61), and then it is determined whether any users are viewing the user's avatar (S62). The server 2 can detect which user is viewing whose avatar by acquiring the operation state of the confirmation button 251 from each client terminal 3. Viewing an avatar here means that not only the avatar being viewed, but also the surrounding area of that avatar is provided to other users.

When at least one viewing user is detected (S62: YES), the server 2 detects the relationship between the viewing user and the viewed user (S63). Examples of relationships include a mutual follow relationship in which both users are registered in the follow list 250, a relationship in which both users belong to the same department, a relationship in which both users belong to the same project, a relationship in which both users have a common follower, and other relationships (other relationships).

As shown in FIG. 13, the server 2 displays the number of users currently viewing the item for each detected relationship in the viewing status window W3 (S64). In FIG. 13, the user is notified of "mutual follow relationships: 1, same department: 1, others: 1." In the case of mutual follow relationships, the specific effect GP20 may indicate which user in the follow list 250 is viewing the item.

When the server 2 receives an instruction to change the status from the user (S65: YES), it changes the status displayed on the user's avatar (S66). The statuses include the above-mentioned ones that can be used to indicate an intention to decline a video call, such as on the phone, in a meeting, or away from the desk, and ones that can be used to indicate an intention to make a video call, such as "want to chat."

This embodiment configured in this way also achieves the same effects as the first embodiment. Furthermore, in this embodiment, the number of users currently viewing the user's avatar is displayed in the viewing status window W3, so the user can know the degree of attention being given to him or her.

Furthermore, in this embodiment, the number of users who are viewing the document is displayed in the viewing status window W3 according to their relationship with the user, improving user convenience.

Furthermore, in this embodiment, after checking the display contents of the viewing status window W3, the user can change the status displayed on his/her avatar that is directly or indirectly related to the video call. This allows the user to express his/her intention through the avatar whether or not to reject the video call in advance, or whether to accept the video call reluctantly or actively, improving user convenience.

The third embodiment will be described with reference to FIG. 14. FIG. 14 is a flowchart showing the process of analyzing the user's situation based on the information acquired from the client terminal 3 and the information acquired from the other system 5.

The process of FIG. 14 is a modified example of the process described in FIG. 6. The information collection application 32 performs a primary analysis of the information collected from the user interface unit 33 and the like before transmitting the information to the server 2 (S16). The information collection application 32 transmits the results of the primary analysis to the information collection unit 23 (S15).

Here, primary analysis means, for example, calculating the number of keystrokes per unit time, or calculating the change in the user image per unit time (such as the movement trajectory of feature points set on the user image).

The information collection unit 23 of the server 2 receives information (primary analysis results) from each client terminal 3 (S21). Furthermore, the information collection unit 23 acquires the progress status of the workflow from other systems 5 (S25) and the start status of the to-do list (S26).

The server 2 stores the information received in steps S21, S25, and S26 in the database 26 (S22), and performs a secondary analysis (S23). The secondary analysis involves determining the user's busyness from the number of keystrokes per unit time, inferring the user's actions from the movement trajectory of the user's image, and estimating the user's work status from the progress of the workflow or the to-do list. These determinations or estimations can be achieved using artificial intelligence technology. For example, an artificial intelligence model can be generated by learning a set of the user's actual status and the progress of the workflow as teacher data. If the progress of the workflow is input into the artificial intelligence model, the user's work status related to the progress of the workflow can be estimated.

This embodiment configured in this manner achieves the same effects as the first embodiment. Furthermore, in this embodiment, the load on the server 2 can be reduced by dividing the analysis of user status information between the information collection application 32 and the information collection unit 23. Furthermore, in this embodiment, the user's work status can be automatically estimated from the workflow progress status, to-do list, etc., and the status can be controlled, eliminating the need for the user to manually change the status, improving usability for the user.

Note that the operation of the avatar by the user is not limited to manual operations such as dragging. For example, the user's line of sight may be detected, and the movement of the avatar may be controlled based on the detected line of sight. Alternatively, the user's physical movements, such as the movements of the hands and feet, may be detected by a camera, and the avatar may be moved in accordance with the user's physical movements.

The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and modifications within the scope of the present invention. The above-described embodiments can be combined as appropriate.

1: Virtual space video calling system, 2: Virtual office providing server, 3: Client terminal, 4: Virtual office, 21: Virtual office control unit, 22: Video calling control unit, 23: Information gathering unit, 24: Information providing unit, 25: Call candidate management unit, GP1: Avatar

Claims (11)

A virtual space video call system in which a plurality of users make a video call through a virtual space,
a virtual space control unit that controls a virtual space provided to the client terminals used by the multiple users;
a video call control unit that controls a video call in the virtual space;
a call candidate management unit that manages a predetermined user among the plurality of users as a candidate for the video call;
The virtual space video call system includes a user selection unit, which displays information about a specified user and selects a candidate user from among the specified users to be the other party in a video call, a candidate user status confirmation unit, which displays information indicating the status of the candidate users, and a movement unit, which moves an avatar of a user who wishes to make a video call within a specified range of the avatar of the candidate user selected by the candidate user selection unit. The virtual space video call system of claim 1, wherein the video call control unit, upon receiving a specified instruction, starts a video call between a client terminal used by the user who wishes to make the video call and a client terminal used by the selected candidate user. the virtual space control unit transmits the predetermined instruction to the video call control unit when it is determined that the avatars in the virtual space have come into contact with each other;
The virtual space video call system of claim 2, wherein the video call control unit starts a video call between client terminals used by users corresponding to each avatar determined to have come into contact by the virtual space control unit. The information of the predetermined user includes a user status indicating a status of the predetermined user,
The virtual space video call system according to claim 1 , wherein the candidate user status confirmation unit displays an avatar status indicating a surrounding status of an avatar of the candidate user. The virtual space video call system of claim 1, wherein the virtual space control unit, when the avatar of the user wishing to make the video call moves within the specified range, displays a specified effect indicating that the avatar of the user wishing to make the video call is near the avatar of the candidate user. the virtual space has a plurality of virtual floors;
The virtual space video call system according to claim 1 , wherein a virtual floor on which the avatar desiring the video call is located is different from a virtual floor on which the avatars of the candidate users are located. The virtual space video call system according to claim 1 , wherein the call candidate management unit displays a number of viewers, which is the number of users currently viewing an avatar of the user who wishes to make a video call. The virtual space video call system according to claim 7 , wherein the call candidate management unit displays the number of viewers according to a relationship between a user viewing an avatar of the user who wishes to make a video call and the user who wishes to make a video call. 5. The virtual space video call system according to claim 4, wherein the user status is created based on information obtained from a management system that manages the user's work status and information obtained from a client terminal used by the user. The virtual space video call system of any one of claims 3 to 9, wherein the video call control unit starts the video call by displaying, within each avatar that has come into contact in the virtual space, an image obtained from a client terminal used by a user corresponding to each avatar. A virtual space video calling method in which client terminals used by multiple users make a video call through a virtual space, comprising:
A computer connected to the plurality of client terminals so as to be capable of two-way communication includes:
A virtual space control unit controls the virtual space provided to the client terminals used by the multiple users;
A video call control unit controls the video call in the virtual space;
managing a predetermined user among the plurality of users as a candidate for a video call;
displaying information of the predetermined user, and having a user selection unit select candidate users to be video call counterparts from among the predetermined users;
displaying information indicating the candidate user's status by a candidate user status confirmation unit;
a moving unit moving an avatar of a user who desires to make a video call within a predetermined range of an avatar of a candidate user selected by the candidate user selection unit;
determining by the virtual space control unit whether avatars in the virtual space have come into contact with each other;
The virtual space video call method includes causing the video call control unit to start a video call between client terminals used by users corresponding to the avatars determined to have come into contact by the virtual space control unit.

Images