JP7790615B2 - Communication device and display method - Google Patents

Description

The present invention relates to a communication device and a display method.

2. Description of the Related Art Web conferencing devices are known that allow users to hold conferences with remote parties via a network (see, for example, Japanese Patent Application Laid-Open No. 2003-121198).

Japanese Patent Application Laid-Open No. 2018-93315

The web conferencing device allows participants to share images of the user's face, but the camera function can be turned off as needed.
In conference B, the camera function of everyone except the speaker may be turned off. In such cases, the speaker cannot see the facial expressions of other users and cannot understand their reactions to what is being said, which can hinder smooth communication.

An object of the present invention is to provide a communication device and a display method that enable smooth communication without displaying camera images.

The communication device of the present invention acquires speech or video of people from multiple other terminals, and when the number of people reaches a predetermined number or more, displays an icon generated based on the number or percentage of people exhibiting a predetermined action, facial expression, or reaction.

The display method of the present invention involves a computer acquiring speech or video of people from multiple other terminals, and when the number of people reaches a predetermined number, displaying an icon generated based on the number or percentage of people exhibiting a predetermined action, facial expression, or reaction.

According to the present invention, smooth communication can be carried out without displaying camera images.

FIG. 1 is a block diagram showing an example of the configuration of a video communication device according to an embodiment. FIG. 2 is a flowchart showing an example of a processing flow of the video communication device according to the embodiment. FIG. 3 is a diagram illustrating an image that the display control unit causes to be displayed on the display unit. FIG. 4 is a diagram for explaining a method for displaying images or icons of individual participants in sequence. FIG. 5 is a diagram for explaining a method for displaying an icon that reflects the proportion of a predetermined action or facial expression. FIG. 6 is a diagram for explaining a method for reflecting a nodding motion in a gesture icon. FIG. 7 is a diagram for explaining a method for displaying an icon that reflects the proportion of a predetermined action or facial expression. FIG. 8 is a diagram for explaining a method for reflecting the action of shaking one's head from side to side in a gesture icon.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[Embodiment]
A video communication device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of a video communication device according to an embodiment.

1, the video communication device 10 includes a camera 12, a microphone 14, an operation receiving unit 16, a communication unit 18, a display unit 20, an audio output unit 22, and a control unit (video communication control device) 24. The video communication device 10 is, for example, a personal computer (PC),
The video communication device 10 is a device such as a smartphone or tablet that has a display screen for displaying video and a camera located near the display screen and facing the same direction as the display screen. The video communication device 10 is used, for example, in a web conference, allowing participants in the web conference to communicate using video and audio.

The camera 12 captures an image of the periphery of the video communication device 10. The camera 12 is, for example,
The camera 12 is positioned so as to capture images in the direction in which the camera 12 is pointing. The camera 12 captures continuous images in time series, so-called moving images, at a predetermined frame rate. The camera 12 outputs the captured images to the image acquisition unit 30. The camera 12 may be an external camera connected to the video communication device 10.

The microphone 14 picks up sounds around the video communication device 10.
The microphone 14 picks up the voice of a person speaking into the microphone 14. The microphone 14 outputs the picked up voice to the voice acquisition unit 32. The microphone 14 may be an external microphone connected to the video communication device 10.

The operation reception unit 16 receives various operations performed by a user on the video communication device 10. The operation reception unit 16 may include, for example, a touch panel, a switch, a button, etc. The operation reception unit 16 outputs operation information related to the operation received from the user or the like to the operation control unit 34.

The communication unit 18 is connected to a communication network such as the Internet (not shown) via a wired or wireless connection. The communication unit 18 transmits and receives information to and from other external devices via the communication network.
The communication unit 18 transmits and receives information to, for example, another video communication device. The communication unit 18 transmits, for example, video captured by the camera 12 and audio picked up by the microphone 14 to, for example, another video communication device. The communication unit 18 receives, for example, video captured by the camera of, for example, another video communication device and audio picked up by the microphone. The communication unit 18 receives, for example, information restricting video captured by the camera of, for example, another video communication device from, for example, another video communication device. The communication unit 18 receives, for example, analysis results obtained by analyzing the movements or facial expressions of a person captured in video captured by, for example, another video communication device. In the following description, when it is stated that the communication unit 18 transmits video, etc. to, for example, another video communication device, this also includes the communication unit 18 transmitting video, etc. to an external server to which the other video communication device is connected and from which the other video communication device can refer to the video, etc. transmitted by the communication unit 18. Furthermore, when it is stated that the communication unit 18 receives video, etc. from another video communication device, this also includes the communication unit 18 receiving video, etc. from an external server, etc. to which the other video communication device is connected and which can refer to video, etc. transmitted by the other video communication device.

The display unit 20 displays various information. For example, the display unit 20 displays video of the user captured by the camera 12, video of other users captured by other video communication devices participating in the web conference, etc. The display unit 20 displays, for example, a predetermined icon indicating the user of the video communication device 10, or an icon indicating the user of another video communication device participating in the web conference.
The display unit 20 is a display including, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display.

The audio output unit 22 is a speaker, a headphone, or the like, which is provided in the video communication device 10 or an external speaker, a headphone, or the like connected to the video communication device 10. The audio output unit 22 outputs various types of audio. For example, the audio output unit 22 outputs audio uttered by other users that is transmitted from other video communication devices participating in the web conference.

The control unit 24 is a so-called video communication control device that controls the operation of each unit of the video communication device 10. The control unit 24 is, for example, a CPU (Central Processing Unit) or an MPU (Microprocessor Unit).
The control unit 24 executes a program stored in a storage unit (not shown) using a RAM or the like as a work area, thereby realizing various functions. Therefore, the control unit 24 executes the video display method according to the embodiment. The control unit 24 is a computer that runs the program according to the present invention. The control unit 24 is implemented, for example, by an ASIC (Application Specific Integrated Circuit).
Specific Integrated Circuit) and FPGA (Field Programmable Gate Array)
The control unit 24 may be realized by an integrated circuit such as a microcomputer. The control unit 24 may be realized by a combination of hardware and software.

The control unit 24 includes a video acquisition unit 30, an audio acquisition unit 32, an operation control unit 34, a communication control unit 36, a display control unit 38, an audio output control unit 40, an analysis unit 42, and an icon generation unit 44.
In other words, the control unit 24 realizes the functions of a video acquisition unit 30, an audio acquisition unit 32, an operation control unit 34, a communication control unit 36, a display control unit 38, an audio output control unit 40, an analysis unit 42, and an icon generation unit 44.

The video acquisition unit 30 acquires the video output by the camera 12. For example, the video acquisition unit 30 acquires the video captured by the camera 12, the video capturing an area including the face of the user of the video communication device 10.

The voice acquisition unit 32 acquires the voice output by the microphone 14. The voice acquisition unit 32 acquires, for example, the voice spoken by the user of the video communication device 10, which is acquired by the microphone 14.

The operation control unit 34 acquires operation information related to the operation accepted by the operation acceptance unit 16.

The communication control unit 36 controls the communication unit 18. The communication control unit 36 controls the communication unit 18 to transmit and receive various information to and from other external devices. The communication control unit 36 controls, for example, the communication unit 18 to transmit and receive video and audio to and from other video communication devices.

The display control unit 38 causes the display unit 20 to display various images. The display control unit 38, for example,
Videos of the user of video communication device 10 and users of other video communication devices participating in the web conference are displayed. For example, when communication unit 18 receives information restricting video display, display control unit 38 causes display unit 20 to display icons generated by icon generation unit 44 instead of the videos of the other users.

The audio output control unit 40 causes the audio output unit 22 to output various sounds.
For example, the voice output unit 22 outputs the voices of other users participating in the web conference.

The analysis unit 42 analyzes the video captured by the camera of the other video communication device and received by the communication unit 18. The analysis unit 42 analyzes the audio picked up by the microphone of the other video communication device and received by the communication unit 18. The analysis unit 42, for example, recognizes a person from the video captured by the camera of the other video communication device. For example, when the analysis unit 42 recognizes a person from the video captured by the camera of the other video communication device, the analysis unit 42 analyzes at least one of the recognized movements and facial expressions. The analysis unit 42 analyzes the information received by the communication unit 18 to obtain the percentage of people who exhibit a predetermined movement or a predetermined facial expression. The predetermined movement includes, for example, positive movements and negative movements in response to the speaker's remarks, but is not limited to these. The analysis unit 42, for example, recognizes a person from the video captured by the camera of the other video communication device.
The percentage of a predetermined action or a predetermined facial expression may be obtained from the analysis results of the video captured by the camera of the other video communication device received by the device 8.

The icon generating unit 44 generates various icons. For example, the icon generating unit 44 generates various icons.
An icon is generated according to the analysis result of the analysis unit 42. The icon generation unit 44 generates, for example, an icon that reflects the ratio obtained by the analysis unit 42. The icon generation unit 44 generates, for example, an icon that reflects a predetermined action or a predetermined expression that the analysis unit 42 has determined to be a ratio equal to or greater than a predetermined threshold. The icon generation unit 44 may also display a pre-stored icon. In this case, the icon generation unit 44 executes a function of obtaining icon data.

[Video communication device processing]
The processing of the video communication device according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the flow of processing of the video communication device according to the embodiment.

The process of Fig. 2 is started, for example, when a web conference is started. When the process of Fig. 2 is started and communication with other participants of the web conference is started by the communication unit 18, the control unit 24 acquires information indicating actions or facial expressions from the other video communication devices, which is the result of analyzing the captured video or the video captured by each participant's video communication device (step S10).
Specifically, in this embodiment, the communication control unit 36 acquires information according to the mode set in the video communication device of the other participant. In this embodiment, one of "video on mode,""video restricted mode," and "video off mode" is set in the other video communication device.

For example, when the other video communication device is set to "video on mode," the communication control unit 36 acquires the video captured by the other video communication device, which is transmitted. For example, when the other video communication device is set to "video restricted mode," the communication control unit 36 acquires information indicating the actions or facial expressions of the other participants, which is the analysis result of the video captured by the other video communication device. For example, when the other video communication device is set to "video off mode," the communication control unit 36 acquires information indicating the actions or facial expressions of the other participants, which is the analysis result of the video captured by the other video communication device.
If the attribute information is set, attribute information of other participants is acquired. The attribute information may include various information such as names and participant names.

The control unit 24 determines whether the number of participants in the web conference is equal to or greater than a predetermined number (step S12). Specifically, the communication control unit 36 makes this determination based on the number of participants in the web conference at the time of determination in step S12, but may also make this determination based on the number of participants requested to participate by the organizer of the web conference.
The predetermined number of participants is, for example, 10, but is not limited to this. The predetermined number of participants may be set to the upper limit of the number of participants in the web conference that allows the participants in the web conference to grasp the overall situation from the video and audio of other participants in the web conference on the display unit 20. If it is determined that the number of participants in the web conference is equal to or greater than the predetermined number (step S12; Yes), the process proceeds to step S14. If it is not determined that the number of participants in the web conference is equal to or greater than the predetermined number (step S12; No), the process proceeds to step S16.

If the answer to step S12 is Yes, the control unit 24 starts analyzing the behaviors or facial expressions of the other participants (step S14). Specifically, the analysis unit 42 analyzes the rate at which other video communication devices that are set to "video on mode" or "video restricted mode" simultaneously make a predetermined behavior or facial expression. For example, if three participants in a web conference
If there are zero participants and only one speaker, the speaker may automatically be in "video on mode." This is because, for example, in a lecture using a web conference, the speaker's video needs to be displayed on the video communication devices of the other participants. In the following, it is assumed that of the other 29 participants, 15 are in "video on mode," 10 are in "video restricted mode," and 4 are in "video off mode." Then, proceed to step S18.

If the determination in step S12 is No, the control unit 24 displays the video or icon of each participant (step S16).
The video acquired by the video communication device 10 is displayed on the display unit 20. FIG. 3 is a diagram for explaining the video that the display control unit 38 displays on the display unit 20. The display screen 20a shown in FIG. 3 is an example of a screen that is displayed when, for example, four people including user U1 of the video communication device 10 are holding a web conference. The same video as the display screen 20a is also displayed on the video communication devices of the other three users other than the user of the video communication device 10. The display screen 20a shown in FIG. 3 is common to all participants in the web conference.

The display screen 20a includes a first display area 50, a second display area 52, a third display area 54, and
The first display area 50 displays, for example, information about a user U1 who is a first user and is the speaker among the four users. The second display area 52 displays, for example,
For example, information about the second user among the four users is displayed in the third display area 54.
For example, the fourth display area 56 displays information about a third user out of the four users. The fourth display area 56 displays information about a fourth user out of the four users.

An image of user U1 and an icon I1 are displayed in the first display area 50. User U1 is displayed, for example, in the center of the first display area 50. Icon I1 is displayed in the upper right corner of the first display area 50. Icon I1 is an icon that indicates information about user U1. For example, icon I1 may display the initials of user U1's name, such as "YS."

An icon I2 is displayed in the center of the second display area 52. The icon I2 indicates information about the second user. For example, the icon I2 may display the initials of the second user's name, such as "RS."

An icon I3 is displayed in the center of the third display area 54. The icon I3 indicates information about the third user. For example, the icon I3 may display the initials of the third user's name, such as "KT."

An icon I4 is displayed in the center of the fourth display area 56. The icon I4 indicates information about the fourth user. For example, the icon I4 may display the initials of the fourth user's name, such as "CJ."

That is, on the display screen 20a, the first user who is the speaker selects "video on mode"
The first user selects "video restriction mode" and the second user to the fourth user select "video off mode." If the determination in step S12 is No, the "video restriction mode" does not have to be selected.

After step S14, the control unit 24 determines whether the predetermined motion or facial expression of the other participants is greater than or equal to a predetermined rate (step S18).
Based on the analysis results of step S14, it is determined whether the predetermined behavior or facial expression of the other participants is greater than or equal to a predetermined rate. The predetermined rate is preferably a rate that allows a certain degree of understanding of the reactions of the other participants to the speaker's speech, such as, for example, 10% or more of all participants, or participants set to "video on mode" or "video restricted mode" 20% or more of all participants. If it is not determined that the predetermined behavior or facial expression is greater than or equal to the predetermined rate (step S18; No),
If it is determined that the predetermined action or facial expression is greater than or equal to the predetermined ratio (step S18; Yes), the process proceeds to step S22.

If the determination in step S18 is No, the control unit 24 displays the images or icons of the individual participants in order (step S20).
4 is a diagram for explaining a method for sequentially displaying the images or icons of the individual participants.
4, the display screen 20b includes a first display area 50A, a second display area 52A, a third display area 54A, a fourth display area 56A, and a fifth display area 58A.

In the example shown in FIG. 4, the first display area 50A displays the user U
In the fourth display area 56A, an image of a user U2A who is a fourth user and an icon I1A representing the fourth user are displayed. The icon I1A represents, for example, the initials of the first user's name. In the fourth display area 56A, an image of a user U2A who is a fourth user and an icon I4A representing the fourth user are displayed.
For example, the first user and the fourth user may use the following:
It is set to "Picture On Mode".

The second display area 52A, the third display area 54A, and the fifth display area 58A are respectively:
An icon I2A indicating the second user, an icon I3A indicating the third user, and an icon I5A indicating the fifth user are displayed. For example, the icons I2A, I3A, and I5A indicate the initials of the names of the second user, the third user, and the fifth user, respectively. The second user, the third user, and the fifth user are each in "video restricted mode."
Or it is set to "Picture Off Mode".

4, the display control unit 38 displays the images or icons of the 25 participants other than the first user to the fifth user in a predetermined order or randomly in an area other than the first display area 50A in which the first user, who is the speaker, is displayed.
The size of the area in which the images or icons of the participants other than the speaker are displayed may be changed as appropriate depending on the number of participants. After step S20, the process proceeds to step S24.

If the determination in step S18 is Yes, the control unit 24 displays an icon that reflects the proportion of the predetermined action or facial expression (step S22).
displays icons that reflect the proportions of the actions or facial expressions of the participants in the web conference.
FIG. 5 is a diagram for explaining a method of displaying an icon reflecting the proportion of a predetermined action or facial expression. The display screen 20c shown in FIG. 5 is an image common to all participants of the web conference. Specifically, the display screen 20c shown in FIG. 5 shows an example of a method of displaying a positive action or proportion in response to a speaker's remark. The display screen 20c shows, for example, a case where there are 30 participants in the web conference. As shown in FIG. 5, the display screen 20c has a first display area 5
0B, a second display area 52B, a third display area 54B, a fourth display area 56B, and a first action information display area 60.

In the example shown in FIG. 5, the first display area 50B displays the user U
In the third display area 54B, an image of a user U2B who is a third user and an icon I3B representing the third user are displayed. The icon I1B represents, for example, the initials of the first user's name. In the third display area 54B, an image of a user U2B who is a third user and an icon I3B representing the third user are displayed.
indicates, for example, the initials of the third user's name.
It is set to "Picture On Mode".

The second display area 52B and the fourth display area 56B respectively display an icon I2B indicating a second user and an icon I4B indicating a fourth user. For example, the icon I
Icons I2B and I4B respectively show the initials of the names of a second user and a fourth user. The second user and the fourth user have set their devices to "video restricted mode" or "video off mode."

The first movement information display area 60 is an area where information about the proportion of movements or facial expressions of all participants participating in the web conference is displayed. Specifically, the first movement information display area 60 includes:
This is an area where information about the proportion of positive actions or facial expressions toward the speaker is displayed. The first action information display area 60 includes a gesture icon GI1 and a character icon LI1.

The gesture icon GI1 is an icon that indicates the action or facial expression of a participant participating in a web conference. Specifically, the gesture icon GI1 is an icon that indicates a positive action or facial expression of a speaker in response to a statement made by the speaker. In the example shown in FIG. 5,
The gesture icon GI1 is an icon of a nodding gesture, tilting the head as if nodding. That is, the gesture icon GI1 indicates a gesture of agreeing with what the speaker has said. Such a gesture icon GI1 is generated by the icon generation unit 44. That is, the icon generation unit 44 generates an icon of a gesture or expression when the analysis unit 42 determines that a predetermined proportion of a predetermined gesture or expression of another participant is present.

The character icon LI1 indicates the number of participants who are performing the gesture indicated by the gesture icon GI1 among all participants participating in the web conference. In the example shown in FIG. 5, the character icon LI1 indicates "12 People." That is, in the example shown in FIG. 5, the character icon LI1
indicates that 12 out of 25 other participants in the "video on mode" and "video control mode" are nodding.

In the example shown in Fig. 5, icon I2B is facing forward, which indicates that the second user is not making a positive gesture or facial expression in response to the speaker's remarks. Also, in the example shown in Fig. 5, icon I4B is tilted as if nodding, which indicates that the fourth user is making a positive gesture or facial expression in response to the speaker's remarks.

A method for reflecting a nodding motion in a gesture icon will be described with reference to FIG.
FIG. 6 is a diagram for explaining a method for reflecting a nodding motion in a gesture icon.

6 , first, the display control unit 38 displays, for example, a gesture icon GI1 tilted as if nodding in the first motion information display area 60 (step S1). Next, the display control unit 38 displays, for example, a gesture icon GI1 facing forward in the first motion information display area 60 (step S2).
By switching the display between the state of step S1 and the state of step S2 every second, the gesture icon GI1 is given a nodding motion, thereby reflecting the nodding motions of the other participants in the web conference. Note that the display control unit 38 can also reflect the nodding motions of the icon I4B shown in FIG. 5 by the method shown in FIG. 6.

Furthermore, the display screen 20c shown in FIG. 5 is not limited to a case in which all 12 people are nodding, but may also show that 12 people are showing positive emotions, such as nodding and smiling.

Fig. 7 is a diagram for explaining a method of displaying an icon reflecting a proportion indicating a predetermined action or facial expression, different from that shown in Fig. 5. The display screen 20d shown in Fig. 7 is an image common to all participants in the web conference. Specifically, the display screen 20d shown in Fig. 7 shows an example of a method of displaying positive actions or proportions and negative actions or proportions in response to a speaker's utterance. In the example shown in Fig. 7, the method of displaying positive actions or proportions is the same as the method shown in Fig. 5, and therefore a description thereof will be omitted.

7, display screen 20d includes first display area 50B, second display area 52B, third display area 54B, fourth display area 56B, first movement information display area 60, and second movement information display area 62. Display screen 20d differs from display screen 20c shown in FIG. 5 in that it includes second movement information display area 62.

The second movement information display area 62 is an area where information about the proportion of movements or facial expressions of all participants participating in the web conference is displayed. Specifically, the second movement information display area 62 includes:
This is an area where information about the proportion of negative actions or facial expressions toward the speaker is displayed. The second action information display area 62 includes a gesture icon GI2 and a character icon LI2.

The gesture icon GI2 is an icon that indicates the movement or facial expression of a participant participating in the web conference. Specifically, the gesture icon GI2 is an icon that indicates a negative movement or facial expression of a speaker in response to a statement made by the speaker. In the example shown in FIG. 7,
The gesture icon GI2 is an icon of a head shaking action. That is, the gesture icon GI2 indicates a motion of disagreeing with what the speaker has said. Such a gesture icon GI2 is generated by the icon generating unit 44.

The character icon LI2 indicates the number of participants who are performing the gesture indicated by the gesture icon GI2 among all participants participating in the web conference. In the example shown in FIG. 7, the character icon LI2 indicates "8 People." That is, in the example shown in FIG. 7, the character icon LI2
indicates that 8 out of 25 other participants in the "video on mode" and "video control mode" are shaking their heads.

A method for reflecting a head shaking action from side to side in a gesture icon will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining a method for reflecting a head shaking action from side to side in a gesture icon.

8 , first, the display control unit 38 displays, for example, a gesture icon GI2 of turning one's head to the right in the second movement information display area 62 (step S3). Next, the display control unit 38 displays, for example, a gesture icon GI2 of turning one's head to the left in the second movement information display area 62 (step S4). The display control unit 38 switches the display between the state of step S3 and the state of step S4 every second, for example, to give the gesture icon GI2 a head-shaking motion, thereby reflecting the head-shaking motions of other participants in the web conference.

After step S16, step S20, or step S22, the control unit 24 determines whether to end the process of FIG. 2 (step S24). For example, when the operation reception unit 16 receives an operation to end the process of FIG. 2 and an operation to turn off the power, the control unit 24 determines whether to end the process of FIG. 2.
If it is determined that the process is to be ended, (step S24; Y
es), the process of FIG. 2 is terminated. If it is not determined that the process is to be terminated (step S24;
No), proceed to step S10. The end of the process determined in step S24 is, for example, when
Even if the WEB conference has not ended, if some users have terminated their connection to the WEB conference, the process of Fig. 2 for that user is terminated.

As described above, in this embodiment, when video display is restricted in a web conference or the like, the icon of the user whose video display is restricted is displayed with the user's actions or facial expressions reflected in it. As a result, this embodiment makes it possible to properly grasp the reactions of other users participating in the web conference, even when they are not displaying camera images, thereby facilitating smooth communication.

Furthermore, in the present embodiment, when video display is restricted in a web conference or the like, the number of participants who reacted positively and negatively to the speaker's remarks is displayed, thereby enabling the user to better grasp the reactions of the web participants even when other users participating in the web conference are not displaying camera images.

Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

REFERENCE SIGNS LIST 10 video communication device 12 camera 14 microphone 16 operation reception unit 18 communication unit 20 display unit 22 audio output unit 24 control unit (video communication control device)
30 Video acquisition unit 32 Audio acquisition unit 34 Operation control unit 36 Communication control unit 38 Display control unit 40 Audio output control unit 42 Analysis unit 44 Icon generation unit

Claims (2)

Acquires speech or video of a person from multiple other devices,
When the number of people reaches or exceeds a predetermined number, an icon generated based on the number or ratio of people who exhibit a predetermined action, a predetermined facial expression, or a predetermined reaction is displayed.
Communication equipment. The computer
Acquires speech or video of a person from multiple other devices,
When the number of people reaches or exceeds a predetermined number, an icon generated based on the number or ratio of people who exhibit a predetermined action, a predetermined facial expression, or a predetermined reaction is displayed.
Display method.