JP7629227B2 - Video meeting evaluation system and video meeting evaluation server - Google Patents

Description

The present disclosure relates to a video meeting evaluation system and a video meeting evaluation server.

Conventionally, systems have been known for teaching knowledge or providing explanations online (see, for example, Patent Document 1).

In addition, as a method for measuring the effectiveness of such online video meetings, for example, a method of conducting a survey after the video meeting has been proposed (see, for example, Patent Document 2).

JP 2019-58625 A

The above-mentioned method of measuring effectiveness through surveys tends to be subjective and is insufficient as a way to objectively measure the effectiveness of the content of video meetings.

Another option would be to have a third party monitor the video meetings and obtain an objective evaluation, but this would be too time-consuming and laborious to be practical.

Therefore, the present invention aims to objectively evaluate video meetings, particularly their content.

According to the present invention,
an evaluation terminal for evaluating a video conference held between a first user terminal having at least a first camera unit and a first display unit and a second user terminal having at least a second camera unit and a second display unit;
A video meeting evaluation system, comprising:
the first user terminal provides the second user terminal with acquired information acquired by the first camera unit and display information displayed on the first display unit;
the second user terminal displays the provided display information in a first area of the second display unit and displays the acquired information in a second area;
the evaluation terminal acquires at least a moving image to be displayed in the second area, identifies at least a facial image included in the moving image for each predetermined frame, and calculates an evaluation value for the facial image;
A video meeting evaluation system is obtained.

According to the present disclosure, by evaluating the video footage of a video meeting, it is possible to objectively evaluate the content in particular.

FIG. 1 is a diagram showing an overall system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration example of a terminal according to the present embodiment. FIG. 2 is an example of a functional block diagram of an evaluation terminal according to the present embodiment. FIG. 2 is a functional block diagram according to the present embodiment. FIG. 2 is a functional block diagram according to the present embodiment. 4 is a diagram showing an example of a screen according to the present embodiment.

The contents of the embodiments of the present disclosure will be described below. The present disclosure has the following configuration.
[Item 1]
an evaluation terminal for evaluating a video conference held between a first user terminal having at least a first camera unit and a first display unit and a second user terminal having at least a second camera unit and a second display unit;
A video meeting evaluation system, comprising:
the first user terminal provides the second user terminal with acquired information acquired by the first camera unit and display information displayed on the first display unit;
the second user terminal displays the provided display information in a first area of the second display unit and displays the acquired information in a second area;
the evaluation terminal acquires at least a moving image to be displayed in the second area, identifies at least a facial image included in the moving image for each predetermined frame, and calculates an evaluation value for the facial image;
Video meeting evaluation system.
[Item 2]
2. The video meeting evaluation system according to claim 1,
The evaluation terminal provides graph information of the evaluation value in time series.
Video meeting evaluation system.
[Item 3]
3. The video meeting evaluation system according to claim 1,
the evaluation terminal calculates a plurality of evaluation values by evaluating the face image from a plurality of different viewpoints;
Video meeting evaluation system.
[Item 4]
A video meeting evaluation system according to any one of items 1 to 3,
The evaluation terminal calculates the evaluation value together with the audio included in the moving image.
Video meeting evaluation system.
[Item 5]
A video meeting evaluation system according to any one of items 1 to 4,
the evaluation terminal calculates the evaluation value together with an object other than the face image included in the moving image;
Video meeting evaluation system.
[Item 6]
A video meeting evaluation server that provides a video meeting to at least the first user terminal and the second user terminal and is communicably connected to a video meeting service terminal that stores a moving image acquired by at least a first camera unit of the first user terminal or a second camera unit of the second user terminal,
means for acquiring the moving image;
A means for identifying at least a face image included in the moving image for each predetermined frame;
means for calculating an evaluation value for the face image;
A video meeting evaluation server comprising:

A preferred embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

＜Basic functions＞

The emotion analysis system of this embodiment is a system that analyzes the specific emotions (feelings that arise in response to one's own or others' words or actions, such as pleasant or unpleasant, or the degree of such feelings) of a person being analyzed among multiple people in an environment where the person is holding a video meeting (hereinafter, referred to as an online session, including both one-way and two-way sessions) that differ from those of others.

Online sessions include, for example, online conferences, online classes, and online chats, in which terminals installed in multiple locations are connected to a server via a communications network such as the Internet, allowing video and images to be exchanged between multiple terminals through the server.

The video images handled in an online session include facial images and audio of the users using the terminals. The video images also include images of documents shared and viewed by multiple users. It is possible to switch between facial images and document images on the screen of each terminal and display only one of them, or to divide the display area and display facial images and document images simultaneously. It is also possible to display an image of one of multiple people on the full screen, or to split the images of some or all of the users onto smaller screens and display them.

It is possible to specify one or more of the multiple users who participate in an online session using a terminal as the subject of analysis. For example, the leader, facilitator, or manager of the online session (hereinafter collectively referred to as the organizer) specifies one of the users as the subject of analysis. The organizer of an online session may be, for example, a lecturer of an online class, a chairperson or facilitator of an online conference, or a coach of a session for coaching purposes. The organizer of an online session is usually one of the multiple users who participate in the online session, but may also be a different person who does not participate in the online session. Note that it is also possible to not specify a subject of analysis and to treat all participants as the subject of analysis.

It is also possible for the leader, facilitator or manager (hereinafter collectively referred to as the organizer) of an online session to designate a user as the subject of analysis. The organizer of an online session may be, for example, a lecturer of an online class, a chairperson or facilitator of an online conference, or a coach in a session for coaching purposes. The organizer of an online session is usually one of multiple users participating in the online session, but may also be a different person who does not participate in the online session.

As shown in FIG. 1, the video meeting evaluation system of this embodiment includes user terminals 10, 20 having at least an input unit such as a camera unit and a microphone unit, a display unit such as a display, and an output unit such as a speaker, a video meeting service terminal 30 that provides two-way video meetings to the user terminals 10, 20, and an evaluation terminal 40 that evaluates the video meeting.

<Hardware configuration example>
2 is a diagram showing an example of the hardware configuration of a computer that realizes each of the terminals 10 to 40 according to this embodiment. The computer includes at least a control unit 11, a memory 12, a storage 13, a communication unit 14, and an input/output unit 15. These are electrically connected to each other via a bus 16.

The control unit 11 is a computing device that controls the overall operation of each terminal, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing. For example, the control unit 11 is a processor such as a CPU, and executes programs stored in the storage 13 and deployed in the memory 12 to perform each information processing.

The memory 12 includes a main memory configured with a volatile storage device such as a DRAM, and an auxiliary memory configured with a non-volatile storage device such as a flash memory or a HDD. The memory 12 is used as a work area for the control unit 11, and also stores the BIOS executed when the information sharing support device 10 is started up, various setting information, etc.

Storage 13 stores various programs such as application programs. A database storing data used for each process may be constructed in storage 13. In particular, the video meeting service terminal 30 may record video images during an online session and store them in storage 13. Furthermore, the evaluation terminal 40 may acquire video images and store them together with the analysis results (evaluation results) in storage 13 managed by the evaluation terminal 40.

The communication unit 14 connects the information sharing support device 10 to a network. The communication unit 14 communicates with external devices directly or via a network access point, for example, by a wired LAN, a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), short-range or contactless communication, or other method.

The input/output unit 15 is, for example, an information input device such as a keyboard, a mouse, a touch panel, etc., and an output device such as a display.

Bus 16 is commonly connected to each of the above elements and transmits, for example, address signals, data signals, and various control signals.

In particular, the evaluation terminal according to this embodiment acquires moving images from a video meeting service terminal, identifies at least facial images contained in the moving images for each predetermined frame unit, and calculates an evaluation value for the facial images (described in more detail below).

<How to get the video>
As shown in Fig. 3, the video meeting service provided by the video meeting service terminal (hereinafter sometimes simply referred to as "the service") enables two-way communication with user terminals 10 and 20 using images and audio. This service allows a user terminal to display video images captured by the camera unit of the other user terminal on its display, and to output audio captured by the microphone unit of the other user terminal from the speaker.

The service is also configured to enable the recording of video and audio (collectively referred to as "video, etc.") by both or either of the user terminals. The recorded information Vs1, Vs2 (hereinafter referred to as "recorded information") is temporarily cached in the user terminal that initiated the recording, and is recorded either on the video meeting service terminal side, or locally only on one of the user terminals, or both. Users can view the recorded information themselves, share it with others, etc., within the scope of their use of the service.

The evaluation terminal 40 acquires the recorded information and performs analysis and evaluation as described below. The recorded information may be acquired, for example, by making a direct download request or by accessing the recorded information via a specified URL.

The evaluation terminal 40 evaluates the video acquired in the above manner through the following analysis.

The video meeting of this embodiment has a function of sharing the screens of the user terminals. That is, the first user terminal 10 provides the second user terminal 20 with the acquired information acquired by the camera unit and the display information displayed on the display unit (such as materials and images to be shared by the first user terminal).

As shown in Figure 6, the second user terminal displays the provided display information in a first area of the display unit and displays the acquired information in a second area. In this embodiment, the size of the first area is displayed larger than the size of the second area, making it possible to display the information to be shared in more detail.

The evaluation terminal 40 acquires at least a moving image to be displayed in the second area (i.e., a moving image acquired by the camera unit of the first user terminal), identifies at least a facial image contained in the moving image for each predetermined frame unit, and calculates an evaluation value for the facial image.

In this case, as shown in FIG. 6, since the second area is smaller than the first area, for example, in addition to (or instead of) the acquisition areas of "User 1 to User 3," it may be possible to analyze raw data (data with a large amount of information before it is displayed, rather than the amount of information after it is displayed) provided from the camera unit of the user terminal corresponding to the user.

Example 1
An embodiment of the present invention will be described below with reference to the drawings. Fig. 4 is a block diagram showing an example of the configuration of this embodiment. As shown in Fig. 4, the video meeting evaluation system of this embodiment includes, as its functional configuration, a video image acquisition unit 11, a biological reaction analysis unit 12, a peculiar determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16.

Each of the above-mentioned functional blocks 11 to 16 can be configured, for example, by hardware provided in the evaluation terminal 40, a DSP (Digital Signal Processor), or software. For example, when configured by software, each of the above-mentioned functional blocks 11 to 16 is actually configured with a computer's CPU, RAM, ROM, etc., and is realized by the operation of a program stored in a recording medium such as the RAM, ROM, hard disk, or semiconductor memory.

The video acquisition unit 11 acquires video from each terminal, obtained by photographing multiple people (multiple users) with a camera provided in each terminal during an online session. The video acquired from each terminal may or may not be set to be displayed on the screen of each terminal. In other words, the video acquisition unit 11 acquires video from each terminal, including video being displayed and video not being displayed on each terminal.

The biological reaction analysis unit 12 analyzes changes in biological reactions for each of multiple people based on the moving images (whether or not they are being displayed on the screen) acquired by the moving image acquisition unit 11. In this embodiment, the biological reaction analysis unit 12 separates the moving images acquired by the moving image acquisition unit 11 into a set of images (a collection of frame images) and audio, and analyzes changes in biological reactions from each.

For example, the biological reaction analysis unit 12 analyzes a change in a biological reaction related to at least one of facial expression, eye line, pulse rate, and facial movement by analyzing the user's facial image using frame images separated from the video acquired by the video acquisition unit 11. The biological reaction analysis unit 12 also analyzes a change in a biological reaction related to at least one of the user's speech content and voice quality by analyzing the sound separated from the video acquired by the video acquisition unit 11.

When a person's emotions change, this is reflected in changes in biological reactions such as facial expressions, gaze, pulse rate, facial movements, speech, and voice quality. In this embodiment, the change in the user's emotions is analyzed by analyzing the change in the user's biological reactions. In this embodiment, the emotion analyzed is, for example, the degree of comfort/discomfort. In this embodiment, the biological reaction analysis unit 12 quantifies the change in the biological reaction according to a predetermined standard, thereby calculating a biological reaction index value that reflects the content of the change in the biological reaction.

Analysis of changes in facial expressions is performed, for example, as follows. That is, for each frame image, a facial area is identified within the frame image, and the identified facial expressions are classified into multiple categories according to an image analysis model that has been trained by machine learning in advance. Then, based on the classification results, it is analyzed whether a positive or negative facial expression change has occurred between consecutive frame images, and the extent of the facial expression change, and a facial expression change index value according to the analysis results is output.

Analysis of changes in eye line is performed, for example, as follows. That is, for each frame image, the eye area is identified from within the frame image, and the direction of both eyes is analyzed to analyze where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, at the shared material being displayed, or looking outside the screen. It may also be possible to analyze whether the eye line movement is large or small, and whether the movement is frequent or infrequent. Changes in eye line are also related to the user's level of concentration. The biological response analysis unit 12 outputs an eye line change index value according to the analysis result of the eye line change.

The analysis of changes in pulse rate is performed, for example, as follows. That is, for each frame image, the facial area is identified from within the frame image. Then, using a trained image analysis model that captures the numerical values of facial color information (G in RGB), changes in the G color of the face surface are analyzed. The results are arranged along the time axis to form a waveform representing changes in color information, and the pulse rate is identified from this waveform. When a person is nervous, their pulse rate increases, and when they feel calm, their pulse rate decreases. The biological response analysis unit 12 outputs a pulse rate change index value according to the analysis results of changes in pulse rate.

The analysis of changes in facial movement is performed, for example, as follows. That is, for each frame image, the facial area is identified from within the frame image, and the facial direction is analyzed to analyze where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, the shared material being displayed, or looking outside the screen. It is also possible to analyze whether the facial movement is large or small, and whether the movement occurs frequently or infrequently. It is also possible to analyze the facial movement and the eye movement together. For example, it is possible to analyze whether the user is looking straight at the speaker being displayed, looking up or down, or looking at an angle. The biological response analysis unit 12 outputs a facial direction change index value according to the analysis result of the change in facial direction.

The analysis of the speech content is performed, for example, as follows. That is, the biological response analysis unit 12 converts the speech into a string of characters by performing a known speech recognition process on the speech for a specified period of time (for example, about 30 to 150 seconds), and performs morphological analysis on the string of characters to remove words that are unnecessary for expressing the conversation, such as particles and articles. The remaining words are then vectorized, and an analysis is performed to determine whether a positive or negative emotional change has occurred, and to what extent the emotional change has occurred, and a speech content index value corresponding to the analysis result is output.

Voice quality analysis is performed, for example, as follows. That is, the biological response analysis unit 12 identifies the acoustic characteristics of the voice by performing a known voice analysis process on the voice for a specified period of time (for example, about 30 to 150 seconds). Then, based on the acoustic characteristics, it analyzes whether a positive or negative voice quality change has occurred and to what extent the voice quality change has occurred, and outputs a voice quality change index value according to the analysis result.

The biological reaction analysis unit 12 calculates a biological reaction index value using at least one of the facial expression change index value, eye gaze change index value, pulse rate change index value, face direction change index value, speech content index value, and voice quality change index value calculated as described above. For example, the biological reaction index value is calculated by weighting the facial expression change index value, eye gaze change index value, pulse rate change index value, face direction change index value, speech content index value, and voice quality change index value.

The uniqueness determination unit 13 determines whether the change in the biological reaction analyzed for the subject of analysis is unique compared to the change in the biological reaction analyzed for other people other than the subject of analysis. In this embodiment, the uniqueness determination unit 13 determines whether the change in the biological reaction analyzed for the subject of analysis is unique compared to other people based on the biological reaction index values calculated for each of the multiple users by the biological reaction analysis unit 12.

For example, the uniqueness determination unit 13 calculates the variance of the biological reaction index values calculated for each of multiple individuals by the biological reaction analysis unit 12, and by comparing the biological reaction index value calculated for the subject of analysis with the variance, determines whether or not the changes in the biological reactions analyzed for the subject of analysis are unique compared to others.

There are three possible cases where the changes in the analyzed biological reactions of the subject are unique compared to others. The first is when no particularly significant changes in the biological reactions of others occur, but a relatively large change in the biological reaction of the subject occurs. The second is when no particularly significant changes in the biological reactions of the subject occur, but a relatively large change in the biological reaction of others occurs. The third is when a relatively large change in the biological reaction occurs in both the subject and others, but the content of the change differs between the subject and others.

The related event identification unit 14 identifies an event occurring with respect to at least one of the subject of analysis, other people, and the environment when a change in a biological reaction determined to be unique by the uniqueness determination unit 13 occurs. For example, the related event identification unit 14 identifies the behavior of the subject of analysis himself/herself from a video when a unique change in a biological reaction occurs in the subject of analysis. The related event identification unit 14 also identifies the behavior of other people from the video when a unique change in a biological reaction occurs in the subject of analysis. The related event identification unit 14 also identifies the environment from the video when a unique change in a biological reaction occurs in the subject of analysis. The environment is, for example, shared materials being displayed on the screen, or something that appears in the background of the subject of analysis, etc.

The clustering unit 15 analyzes the degree of correlation between a change in a biological reaction determined to be unique by the unique determination unit 13 (e.g., one or more combinations of eye movement, pulse rate, facial movement, speech content, and voice quality) and an event occurring when the unique change in a biological reaction occurs (an event identified by the related event identification unit 14), and if it is determined that the correlation is at or above a certain level, it clusters the person or event being analyzed based on the results of the correlation analysis.

For example, if a specific change in a biological reaction corresponds to a negative emotional change, and the event occurring when the specific change in the biological reaction occurs is also a negative event, a correlation above a certain level is detected. The clustering unit 15 clusters the subject of analysis or the event into one of multiple pre-segmented classifications according to the content of the event, the degree of negativity, the magnitude of correlation, etc.

Similarly, if a specific change in biological reaction corresponds to a positive emotional change, and the event occurring when the specific change in biological reaction occurs is also a positive event, a correlation above a certain level is detected. The clustering unit 15 clusters the subject of analysis or the event into one of multiple pre-segmented classifications according to the content of the event, the degree of positivity, the magnitude of correlation, etc.

The analysis result notification unit 16 notifies the person who designated the analysis subject (the analysis subject or the organizer of the online session) of at least one of the changes in biological reactions determined to be specific by the uniqueness determination unit 13, the events identified by the related event identification unit 14, and the classifications clustered by the clustering unit 15.

For example, the analysis result notification unit 16 notifies the analysis subject of the analysis subject's own words and actions as an event occurring when a unique change in the analysis subject's biological reaction that differs from that of others occurs (any of the three patterns described above; the same applies below). This allows the analysis subject to understand that when he or she behaves in a certain way, he or she has different emotions than others. At this time, the analysis subject may also be notified of the unique changes in the analysis subject's biological reaction identified. Furthermore, the analysis subject may also be notified of the changes in the biological reaction of the other person being compared.

For example, if the emotions felt by others in response to words or actions made by the subject without any particular awareness and with normal emotions, or words or actions made by the subject with particular emotions and awareness differ from the emotions felt by the subject himself at the time of the words or actions, the subject will be notified of the words or actions of the subject at that time. This makes it possible to discover words or actions that are well-received by others or not well-received by others despite one's own awareness.

Furthermore, the analysis result notification unit 16 notifies the organizer of the online session of events that occur when a unique change in the biological reaction of the analysis subject occurs that is different from that of others, along with the unique change in the biological reaction. This allows the organizer of the online session to know what events are influencing what emotional changes as a phenomenon unique to the specified analysis subject. Then, it becomes possible to take appropriate measures for the analysis subject depending on the content thus understood.

Furthermore, the analysis result notification unit 16 notifies the organizer of the online session of events occurring when a unique change in the biological reaction of the analysis subject occurs that is different from that of others, or the clustering results of the analysis subject. This allows the organizer of the online session to grasp the behavioral tendencies unique to the analysis subject and predict possible future behaviors and conditions, etc., depending on which classification the specified analysis subject is clustered into. It then becomes possible to take appropriate measures for the analysis subject.

In the above embodiment, a bioreaction index value is calculated by quantifying the change in bioreaction according to a predetermined standard, and based on the calculated bioreaction index value for each of a plurality of people, it is determined whether or not the change in the bioreaction analyzed for the subject of analysis is unique compared to others. However, the present invention is not limited to this example. For example, the following may be used.

That is, the biological reaction analysis unit 12 analyzes the eye movement of each of the multiple people to generate a heat map showing the direction of the eye movement. The peculiar determination unit 13 compares the heat map generated by the biological reaction analysis unit 12 for the analysis subject with the heat map generated for other people to determine whether the change in the biological reaction analyzed for the analysis subject is unique compared to the change in the biological reaction analyzed for other people.

Example 2
Hereinafter, a description will be given based on Example 2 of the present invention. Fig. 5 is a block diagram showing a configuration example according to this embodiment. As shown in Fig. 1, the video meeting evaluation system of this embodiment includes, as functional components, a moving image acquisition unit 11, a biological reaction analysis unit 12, and a reaction information presentation unit 13a.

The reaction information presenting unit 13a presents information indicating changes in biological reactions analyzed by the biological reaction analysis unit 12a, including participants not displayed on the screen. For example, the reaction information presenting unit 13a presents information indicating changes in biological reactions to the leader, facilitator, or manager of the online session (collectively referred to as the organizer below). The organizer of the online session may be, for example, a lecturer of an online class, a chairperson or facilitator of an online conference, or a coach of a session for coaching purposes. The organizer of the online session is usually one of multiple users participating in the online session, but may also be a different person who does not participate in the online session.

In this way, the organizer of an online session can keep track of participants who are not displayed on the screen in an environment where an online session is being held with multiple people.

Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can conceive of various modified or revised examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

The devices described in this specification may be realized as a single device, or may be realized by multiple devices (e.g., cloud servers) some or all of which are connected via a network. For example, the control unit 11 and storage 13 of the information sharing support device 10 may be realized by different servers connected to each other via a network.

The series of processes performed by the device described in this specification may be realized using software, hardware, or a combination of software and hardware. A computer program for realizing each function of the information sharing support device 10 according to this embodiment may be created and implemented in a PC or the like. It is also possible to provide a computer-readable recording medium on which such a computer program is stored. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The above computer program may also be distributed, for example, via a network, without using a recording medium.

In addition, the processes described in this specification using flow chart diagrams do not necessarily have to be performed in the order shown. Some processing steps may be performed in parallel. Also, additional processing steps may be employed, and some processing steps may be omitted.

In addition, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may provide other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

The present invention may also include the following configurations.
<Configuration 1>
A video meeting evaluation system that analyzes the unique emotions of a person being analyzed among multiple people in an online session, which emotions differ from those of others.
<Configuration 2>
a video acquisition unit that acquires a video obtained by photographing the plurality of people during the online session;
A biological reaction analysis unit that analyzes a change in a biological reaction of each of the plurality of people based on the moving image acquired by the moving image acquisition unit;
and a uniqueness determination unit that determines whether the changes in the biological reactions analyzed for the person being analyzed are unique compared to the changes in the biological reactions analyzed for other people other than the person being analyzed.
<Configuration 3>
The video meeting evaluation system described in configuration 2, wherein the biological reaction analysis unit analyzes changes in biological reactions related to at least one of facial expression, eye line, pulse rate, and facial movement by analyzing facial images in the video images acquired by the video image acquisition unit.
<Configuration 4>
The video meeting evaluation system according to configuration 2 or 3, wherein the biological reaction analysis unit analyzes the audio in the video image acquired by the video image acquisition unit to analyze changes in biological reactions related to at least one of the content of the speech and the voice quality.
<Configuration 5>
The biological reaction analysis unit calculates a biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard,
The video meeting evaluation system of any one of configurations 2 to 4, characterized in that the unique determination unit determines whether the change in the biological reaction analyzed for the person being analyzed is unique compared to the change in the biological reaction analyzed for other people other than the person being analyzed, based on the biological reaction index value calculated for each of the multiple people by the biological reaction analysis unit.
<Configuration 6>
The video meeting evaluation system described in configuration 5, characterized in that the unique determination unit calculates the variance of the biological reaction index values calculated for each of the multiple people by the biological reaction analysis unit, and determines whether the change in the biological reaction analyzed for the analysis subject is unique compared to the change in the biological reaction analyzed for the other person by comparing the biological reaction index value calculated for the analysis subject with the variance.
<Configuration 7>
The biological reaction analysis unit analyzes the eye movement of each of the plurality of people to generate a heat map indicating the direction of the eye movement,
The video meeting evaluation system described in configuration 3, characterized in that the unique determination unit determines whether the changes in the biological reactions analyzed for the subject of analysis are unique compared to the changes in the biological reactions analyzed for the other person by comparing the heat map generated by the biological reaction analysis unit for the subject of analysis with the heat map generated for the other person.
<Configuration 8>
The video meeting evaluation system according to any one of configurations 2 to 7, further comprising a related event identification unit that identifies an event occurring with respect to at least one of the subject of analysis, the other person, and the environment when a change in a biological reaction determined to be specific by the specificity determination unit occurs.
<Configuration 9>
The video meeting evaluation system according to configuration 8, further comprising a clustering unit that analyzes the degree of correlation between the change in biological reaction determined to be specific by the specific determination unit and the event occurring when the specific change in biological reaction occurs, and when it is determined that the correlation is at a certain level or above, clusters the analysis subjects or the event based on the analysis result of the correlation.
<Configuration 10>
The video meeting evaluation system according to configuration 8, further comprising an analysis result notification unit that notifies the subject of analysis or a host of the online session of at least one of the changes in the biological reaction determined to be specific by the specificity determination unit and the events identified by the related event identification unit.
<Configuration 11>
The video meeting evaluation system of claim 9, further comprising an analysis result notification unit that notifies the subject of analysis or the organizer of the online session of at least one of the changes in biological reactions determined to be specific by the specific determination unit, the events identified by the related event identification unit, and the classifications clustered by the clustering unit.
<Configuration 12>
A reaction analysis system that, in an environment where an online session is held with a plurality of participants, analyzes the reactions of the participants based on video images obtained by photographing the participants, regardless of whether the participants are displayed on a screen during the online session, and presents the analysis results.
<Configuration 13>
a video acquisition unit that acquires a video obtained by photographing the participants during the online session;
a biological reaction analysis unit that analyzes changes in biological reactions of the participant based on the moving image acquired by the moving image acquisition unit;
The reaction analysis system described in item 12 is characterized in that it further comprises a reaction information presentation unit that presents information indicating changes in the biological reactions analyzed by the biological reaction analysis unit, including participants who are not displayed on the screen.
<Configuration 14>
The reaction analysis system described in item 13 is characterized in that the biological reaction analysis unit analyzes changes in biological reactions related to at least one of facial expression, eye movement, pulse rate, and facial movement by analyzing facial images in the video acquired by the video acquisition unit.
<Configuration 15>
The reaction analysis system described in item 13 or item 14, characterized in that the biological reaction analysis unit analyzes changes in biological reactions related to at least one of the content of the speech and the voice quality by analyzing the audio in the video acquired by the video acquisition unit.
<Configuration 16>
The reaction analysis system described in item 13, wherein the biological reaction analysis unit analyzes where a participant who is not displayed on the screen is looking at in the shared material displayed on the screen.
<Configuration 17>
The reaction analysis system described in item 13, wherein the biological reaction analysis unit analyzes at what point during the online session a participant not displayed on the screen made a sound.
<Configuration 18>
18. The reaction analysis system according to any one of claims 13 to 17, wherein the reaction information presentation unit presents information indicating changes in the biological reaction to a host of the online session.

10, 20 User terminal 30 Video meeting service terminal 40 Evaluation terminal

Claims (5)

an evaluation terminal for evaluating a video conference held between a first user terminal having at least a first camera unit and a first display unit and a second user terminal having at least a second camera unit and a second display unit;
A video meeting evaluation system, comprising:
the first user terminal provides the second user terminal with acquired information acquired by the first camera unit and display information displayed on the first display unit;
the second user terminal displays the provided display information in a first area of the second display unit and displays the acquired information in a second area;
the evaluation terminal acquires a moving image having an amount of information smaller than the acquired information displayed at least in the second area, identifies at least a face image included in the moving image for each predetermined frame, and calculates an evaluation value for the face image.
Video meeting evaluation system. 2. The video meeting evaluation system of claim 1,
The evaluation terminal provides graph information of the evaluation value in time series.
Video meeting evaluation system. 3. The video meeting evaluation system according to claim 1, further comprising:
the evaluation terminal calculates a plurality of evaluation values by evaluating the face image from a plurality of different viewpoints;
Video meeting evaluation system. A video meeting evaluation system according to any one of claims 1 to 3,
The evaluation terminal calculates the evaluation value together with the audio included in the moving image.
Video meeting evaluation system. A video meeting evaluation system according to any one of claims 1 to 4,
the evaluation terminal calculates the evaluation value together with an object other than the face image included in the moving image;
Video meeting evaluation system.

Images