JP7645830B2 - Interview support device, interview support method, and computer program - Google Patents

Description

The present invention relates to an interview support device, an interview support method, and a computer program.

Patent Document 1 discloses an information processing device that supports interviews so that the purpose of the interview can be easily achieved. In companies, a superior (interviewer) may interview a subordinate (interviewee) to encourage the subordinate to change his/her behavior.

JP 2021-026657 A

In order for an interviewer to encourage behavioral change in the interviewee, the interviewer needs to have interviewing skills. Interviewing skills are, for example, the ability to communicate during an interview. However, even though it is expected that interviewers have interviewing skills, there are cases where they are unable to improve their interviewing skills.

In view of the above circumstances, the present invention aims to provide an interview support device, an interview support method, and a computer program that can improve the interview skills of interviewees.

One aspect of the present invention is a method for analyzing an amount of speech of an interviewer and an amount of speech of an interviewee,
The interview support device is equipped with a feedback processing unit that provides the interviewer with at least one of advice and a warning during the interview depending on the analysis results of the amount of speech of the interviewer and the amount of speech of the interviewee.

In one aspect of the present invention, in the interview support device described above, the analysis unit analyzes the cumulative result of the speaking time length of the interviewer in a specified period as the speaking volume of the interviewer, and analyzes the cumulative result of the speaking time length of the interviewee in the specified period as the speaking volume of the interviewee.

One aspect of the present invention is the interview support device described above, in which the analysis unit derives the speaking duration of the interviewer by subtracting a predetermined first margin duration from the voice duration of the interviewer during the predetermined period, and derives the speaking duration of the interviewee by subtracting a predetermined second margin duration from the voice duration of the interviewee during the predetermined period.

In one aspect of the present invention, in the interview support device described above, the feedback processing unit derives a ratio of the amount of speech of the interviewer to the sum of the amount of speech of the interviewee and the amount of speech of the interviewee, and if the ratio is equal to or greater than a predetermined threshold, provides at least one of the advice and the warning to the interviewer.

One aspect of the present invention is an interview support method executed by the interview support device, which includes an analysis step of analyzing the amount of speech of the interviewer and the amount of speech of the interviewee, and a feedback processing step of providing at least one of advice and a warning to the interviewer during the interview depending on the analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee.

One aspect of the present invention is a computer program for causing a computer to function as the interview support device described above.

This invention makes it possible to improve the interviewer's interview skills.

FIG. 1 is a diagram illustrating an example of the configuration of an interview support system in a first embodiment. A diagram showing an example of the operation of the interview support system in the first embodiment. FIG. 11 is a diagram showing an example of image analysis regarding a supervisor's interview skills in the first embodiment. 5A to 5C are diagrams showing examples of personality type selection images in the first embodiment. FIG. 4 is a diagram showing an example of an image for an interview in the first embodiment. FIG. 11 is a diagram showing an example of an image for setting a target item in the first embodiment. A figure showing an example of an image for feedback of interview skill scores and advice in the first embodiment. FIG. 11 is a diagram showing a first example of an input image for a pre-questionnaire in the first embodiment. FIG. 11 is a diagram showing a second example of an input image for a type diagnosis questionnaire in the first embodiment. 13 is a diagram showing an example of an image for notifying a result of a personality type diagnosis in the first embodiment. FIG. FIG. 11 is a diagram showing an example of an image for managing the progress of a target item in the first embodiment. FIG. 11 is a diagram showing an example of an input image for a post-event questionnaire in the first embodiment. A figure showing a first example of the operation of the interview support device in the first embodiment. A figure showing a second example of the operation of the interview support device in the first embodiment. A figure showing an example of the configuration of an interview support system in a second embodiment. A figure showing an example of voice analysis regarding a supervisor's interview skills in the second embodiment. A figure showing an example of an image for feedback of interview skill scores and advice in the second embodiment. A diagram showing a first example of the operation of the interview support device in the second embodiment. A figure showing a second example of the operation of the interview support device in the second embodiment. A figure showing an example of the configuration of an interview support system in a third embodiment. A diagram showing a first example of the operation of the interview support device in the third embodiment. A diagram showing a second example of the operation of the interview support device in the third embodiment. A figure showing an example of the configuration of an interview support system in a fourth embodiment. FIG. 23 is a diagram showing an example of rule-based conditional branching of a scenario in the fourth embodiment. A figure showing an example of the operation of the interview support system for a training interview associated with a real interview in the fourth embodiment. FIG. 13 is a diagram showing an example of an image for an interview in the fourth embodiment. FIG. 13 is a diagram illustrating an example of machine learning of a learning model in a modified example of the fourth embodiment. FIG. 13 is a diagram showing an example of model-based scene transition of a scenario in a modified example of the fourth embodiment. A figure showing an example of the configuration of an interview support system in a fifth embodiment. FIG. 23 is a diagram showing an example of the duration of audio in the fifth embodiment. FIG. 13 is a diagram showing an example of an image for an interview in the fifth embodiment. A diagram showing an example of the operation of the interview support device in the fifth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.
First Embodiment
Fig. 1 is a diagram showing an example of the configuration of an interview support system 1a in the first embodiment. The interview support system 1a is a system that supports interviews. Interviews may be conducted one-to-one, one-to-many, or many-to-many. By supporting interviews with the interview support system 1a, it is possible to improve the interview skills of the person conducting the interview with the interviewee (interviewer).

The interview support system 1a is used, for example, for one-on-one meetings between a superior (interviewer) and a subordinate (interviewee). In this case, the interview support system 1a supports the interview, for example, from before the interview to after the interview, thereby making it possible to improve the superior's interview skills.

The interview support system 1a may be used, for example, for an interview (customer service) between a store clerk (interviewer) and a customer (interviewee). In this case, the interview support system 1a supports the interview, for example, during and after the interview, thereby making it possible to improve the interview skills (customer service skills) of the store clerk.

In the following, as an example, the interview support system is used for a one-on-one meeting between a superior and a subordinate. The superior is the person who interviews the subordinate (interviewer) and gives advice, etc. to the subordinate (mentor). The subordinate is the person who is interviewed by the superior (interviewee) and receives advice, etc. from the superior (mentor). In addition, one or more work-related goals (promises) may be agreed upon between the superior and the subordinate during the interview.

The interview support system 1a includes an interviewee device 2, an interviewer device 3, an interview support device 4a, and a communication line 5. The communication line 5 is, for example, a communication line such as the Internet or an intranet.

The interview itself may be conducted online or offline (face-to-face). If the interview itself is conducted online, the interviewee device 2 and the interviewer device 3 may be placed at a distance greater than a predetermined distance. For example, the interviewee device 2 may be placed in a first building where the subordinate is located, and the interviewer device 3 may be placed in a second building where the superior is located.

When the interview itself is conducted offline (face-to-face), the interviewee device 2 and the interviewer device 3 may be placed close to each other within a predetermined distance. For example, the interviewee device 2 and the interviewer device 3 may be placed on a desk shared by a subordinate and a superior who are conducting a face-to-face interview for a one-on-one meeting. Note that when the interview itself is conducted offline, the interview support system 1a only needs to include the interviewer device 3 used by the superior, and does not need to include the interviewee device 2 used by the subordinate.

Next, the interviewee device 2, the interviewer device 3, and the interview support device 4a will be described in detail.
[Interviewee device]
The interviewee device 2 is an information processing device such as a smartphone terminal, a tablet terminal, or a notebook computer. The interviewee device 2 is used by a subordinate (interviewee). The interviewee device 2 includes a camera 20, a microphone 21, an operation unit 22, a display unit 23, a speaker 24, a memory unit 25, a control unit 26, and a communication unit 27.

The camera 20 generates video data by capturing an image of the surroundings of the interviewee device 2. The video includes an image of the subordinate (interviewee). The camera 20 outputs the video data to the control unit 26.

The microphone 21 generates voice data by collecting sounds from around the interviewee device 2. The voice data contains the voices of the subordinates. The microphone 21 outputs the voice data to the control unit 26.

The operation unit 22 is an operation device such as a keyboard, a mouse, and a touch panel. The operation unit 22 is operated by a subordinate. The operation unit 22 outputs a signal corresponding to the operation to the control unit 26. The signal corresponding to the operation is, for example, a signal corresponding to the operation of entering a target item (promise) in an input field displayed on the display unit 23.

The display unit 23 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. When the operation unit 22 is a touch panel, the operation unit 22 and the display unit 23 may be integrated. The display unit 23 displays an image received from the communication unit 40 (relay unit) by the communication unit 27. For example, the display unit 23 displays an image (hereinafter referred to as an "interview image") used for an interview such as a one-on-one meeting. In the interview image, for example, a moving image of the superior and a moving image of the subordinate are displayed side by side. In the interview image, for example, one or more goals (promises) related to the work may be displayed in response to input operations by the superior and the subordinate.

The speaker 24 outputs the audio received from the communication unit 40 (relay unit). For example, the speaker 24 outputs the voice of a superior in a one-on-one meeting.

The memory unit 25 stores identification information of the interviewee device 2. The identification information is, for example, an IP address (Internet Protocol Address). The memory unit 25 may also store a computer program executed by the control unit 26.

The control unit 26 controls the operation of each functional unit of the interviewee device 2. For example, the control unit 26 uses the communication unit 27 to perform authentication processing for communication with the communication unit 37 via the communication unit 40. The control unit 26 causes the display unit 23 to display an image received by the communication unit 27 from the communication unit 40 through image processing. The control unit 26 causes the speaker 24 to output an audio received by the communication unit 27 from the communication unit 40 through audio processing.

The communication unit 27 communicates with the communication unit 40. The communication unit 27 communicates with the interviewer device 3 via the communication unit 40. For example, the communication unit 27 acquires video image data in which the face of the superior is captured from the communication unit 40. For example, the communication unit 27 acquires audio data in which the voice of the superior is recorded from the communication unit 40. The communication unit 27 may transmit identification information of the interviewee device 2 to the communication unit 40 in the authentication process of communication with the communication unit 40.

[Interviewer device]
The interviewer device 3 is an information processing device such as a smartphone terminal, a tablet terminal, or a notebook computer. The interviewer device 3 is used by a superior (interviewer). The interviewer device 3 includes a camera 30, a microphone 31, an operation unit 32, a display unit 33, a speaker 34, a memory unit 35, a control unit 36, and a communication unit 37.

The camera 30 generates video data by capturing an image of the surroundings of the interviewer device 3. The video includes an image of the superior (interviewer). The camera 30 outputs the video data to the control unit 36.

The microphone 31 generates voice data by collecting sounds from around the interviewer device 3. The voice data contains the voice of the superior. The microphone 21 outputs the voice data to the control unit 36.

The operation unit 32 is an operation device such as a keyboard, a mouse, and a touch panel. The operation unit 32 is operated by the superior. The operation unit 32 outputs a signal corresponding to the operation to the control unit 36. The signal corresponding to the operation is, for example, a signal corresponding to the operation of selecting a radio button displayed on the display unit 33. The signal corresponding to the operation may be, for example, an operation of entering a target item (promise) in an input field displayed on the display unit 33.

The display unit 33 is a display device such as a liquid crystal display or an organic EL display. If the operation unit 32 is a touch panel, the operation unit 32 and the display unit 33 may be integrated. The display unit 33 displays an image received by the communication unit 37. For example, the display unit 33 displays an image for an interview.

The speaker 34 outputs the audio received from the communication unit 40 (relay unit). For example, the speaker 24 outputs the audio of a subordinate in a one-on-one meeting.

The memory unit 35 stores identification information of the interviewer device 3. The memory unit 35 may also store a computer program executed by the control unit 36.

The control unit 36 controls the operation of each functional unit of the interviewer device 3. For example, the control unit 36 uses the communication unit 37 to perform authentication processing for communication with the communication unit 27 via the communication unit 40. The control unit 36 causes the display unit 33 to display an image received from the communication unit 40 by the communication unit 37 through image processing. The control unit 36 causes the speaker 34 to output an audio received from the communication unit 40 by the communication unit 37 through audio processing.

The communication unit 37 communicates with the interview support device 4a. The communication unit 37 communicates with the interviewee device 2 via the interview support device 4a. For example, the communication unit 37 acquires a video image of the subordinate's face from the communication unit 40. For example, the communication unit 37 acquires audio data in which the subordinate's voice is recorded from the communication unit 40. The communication unit 37 may transmit identification information of the interviewer device 3 to the communication unit 40.

[Interview Support Device]
The interview support device 4a is an information processing device such as a server. The interview support device 4a may be realized by cloud computing. The interview support device 4a includes a communication unit 40, a storage unit 41, and a control unit 42a. The control unit 42a includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424a, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

Some or all of the functional units of the interview support device 4a are realized as software by a processor such as a CPU (Central Processing Unit) executing a program (computer program) stored in a storage unit 41 having a non-volatile storage medium (non-transient storage medium). The program may be recorded on a computer-readable storage medium. Examples of computer-readable storage media include portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), and non-transient storage media such as hard disks built into computer systems.

Some or all of the functional units of the interview support device 4a may be realized using hardware including electronic circuits (electronic circuits or circuitry) using, for example, LSIs (Large Scale Integrated circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), or FPGAs (Field Programmable Gate Arrays).

The communication unit 40 transmits communication data such as image data and audio data generated by the control unit 42a to at least one of the communication unit 27 and the communication unit 37. The communication unit 40 transmits communication data such as image data and audio data received from the communication unit 27 to the communication unit 37. The communication unit 40 transmits communication data such as image data and audio data received from the communication unit 37 to the communication unit 27. The communication unit 40 may temporarily store the communication data in the storage unit 41.

The memory unit 41 stores identification information of the interview support device 4a. The memory unit 41 may store multiple scenarios related to interviews. The memory unit 41 may store a computer program executed by the control unit 42a. The memory unit 41 may store a mathematical model. The memory unit 41 may store a model trained using a machine learning technique (hereinafter referred to as a "trained model"). This machine learning technique is not limited to a specific technique, but is, for example, supervised learning. The memory unit 41 may store a data table.

For example, at least one of the mathematical model, the data table, and the trained model may be used to derive a score for an interview skill. For example, at least one of the mathematical model, the data table, and the trained model may be used to derive advice for improving an interview skill.

The storage unit 41 may temporarily store image data received from the communication unit 27. The storage unit 41 may also temporarily store image data received from the communication unit 37. Note that image data temporarily stored in the storage unit 41 is deleted from the storage unit 41 within a predetermined time period after it is stored.

The storage unit 41 may temporarily store the voice data received from the communication unit 27. The storage unit 41 may also temporarily store the voice data received from the communication unit 37. Note that the voice data temporarily stored in the storage unit 41 is deleted from the storage unit 41 within a predetermined time period after it is stored.

The control unit 42a controls the operation of each functional unit of the interview support device 4a. For example, the control unit 42a uses the communication unit 40 to perform authentication processing for communication with the communication unit 27. For example, the control unit 42a uses the communication unit 40 to perform authentication processing for communication with the communication unit 37.

The control unit 42a analyzes the analysis target (e.g., facial expression, gaze, nod) in the image of the boss in real time based on the result of image recognition of the image received from the communication unit 37. For example, the control unit 42a analyzes the facial expression of the boss in the face area in real time based on the recognition result of the face area of the boss in the received image. The control unit 42a derives the score of the interview skill of the boss based on the analysis result of the analysis target in the image of the boss. The control unit 42a derives advice to be provided to the boss according to the analysis result of the analysis target in the image of the boss.

The control unit 42a may analyze the analysis target in the subordinate's image in real time based on the result of image recognition for the image received from the communication unit 27. For example, the control unit 42a may analyze the subordinate's facial expression in the facial area in real time based on the recognition result of the subordinate's facial area in the received image. The control unit 42a may derive a score for the subordinate's condition based on the analysis result of the analysis target in the subordinate's image. The control unit 42a may derive advice to be provided to the superior according to the analysis result of the analysis target in the subordinate's image.

The control unit 42a generates an image for an interview. The control unit 42a uses the communication unit 40 to cause the display unit 23 and the display unit 33 to display the image for an interview.

The control unit 42a generates a feedback image to be displayed on the display unit 33. That is, the control unit 42a generates a feedback image to be provided to the superior. The control unit 42a includes in the feedback image the score of the superior's interview skills. This score may be converted, for example, into a score out of a maximum of 100 points. The control unit 42a includes in the feedback image a character string representing advice to be provided to the superior. The control unit 42a uses the communication unit 40 to cause the feedback image to be displayed on the display unit 33.

The control unit 42a may generate at least one of an image of a questionnaire before the interview (hereinafter referred to as a "pre-interview questionnaire"), an image of a questionnaire after the interview (hereinafter referred to as a "post-interview questionnaire"), and a questionnaire for diagnosing a personality type (hereinafter referred to as a "type diagnostic questionnaire"). The control unit 42a may use the communication unit 40 to cause the display unit 23 and the display unit 33 to display an image of the pre-interview questionnaire before the interview. The control unit 42a may use the communication unit 40 to cause the display unit 23 and the display unit 33 to display an image of the type diagnostic questionnaire before the actual interview. The control unit 42a may use the communication unit 40 to cause the display unit 33 to display an image of the post-interview questionnaire after the interview.

In this way, the control unit 42a supports the actual interview during the actual interview. This makes it possible to improve the supervisor's interview skills after the actual interview.

A training interview (simulation) may be conducted before the actual interview. The control unit 42a supports the training interview before the actual interview. This makes it possible to improve the supervisor's interview skills before the actual interview.

A supervisor who attends pre-production training selects the personality type (profile) of a hypothetical subordinate with whom he or she will have an interview for the training. For example, the supervisor may select the same personality type as the subordinate with whom the actual interview is scheduled.

In the training interview, the control unit 42a generates an avatar by computer graphics as a virtual subordinate of the selected personality type. The control unit 42a executes image processing and audio processing so that the avatar speaks according to a scenario. That is, the control unit 42a controls the image and audio actions of the avatar so that the avatar behaves (e.g., expresses facial expressions, speaks, etc.) according to a scenario corresponding to the selected personality type. For example, the control unit 42a controls the image and audio actions of the avatar so that the avatar's speaking speed varies according to the scenario. The control unit 42a uses the communication unit 40 to display an interview image including an avatar image on the display unit 33. The control unit 42a uses the communication unit 40 to output the voice of the avatar from the speaker 34. The control unit 42a analyzes the analysis target of the superior who reacts to the image and voice of the avatar speaking according to the scenario (e.g., the superior's facial expression, the superior's line of sight, the superior's nod).

When an avatar generated by the control unit 42a speaks according to a scenario during a training interview, the interview support system 1a only needs to include an interviewer device 3 used by the superior, and does not need to include an interviewee device 2. Also, during a training interview, instead of an avatar generated by the control unit 42a speaking according to a scenario, a human (actor) playing a virtual subordinate may speak according to a scenario.

Figure 2 is a diagram showing an example of the operation of the interview support system 1a in the first embodiment. The upper part of Figure 2 shows an example of the operation of the interview support system 1a regarding a training interview before the actual interview. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S101).

During the training interview, the control unit 42a supports the interview. For example, the scenario processing unit 421 controls the image and audio behavior of the avatar so that the avatar operates according to a scenario corresponding to the selected personality type. The scenario processing unit 421 uses the communication unit 40 to display an interview image including an image of the avatar on the display unit 33. The information output unit 422 uses the communication unit 40 to display instructions (guides) for the tutorial on the display unit 33. The information output unit 422 uses the communication unit 40 to display an input field corresponding to the scenario on the display unit 33.

During the training interview, the analysis unit 424a analyzes the analysis target of the superior (interviewee) in real time. For example, the analysis unit 424a (image analysis unit) analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior. The analysis targets are, for example, the facial expression of the superior, the line of sight of the superior, and the nod of the superior. For example, the analysis unit 424a determines whether the facial expression of the superior in the image of the superior is positive or negative based on multiple feature points defined in the area of the superior's face. Note that a combination of multiple analysis targets may be analyzed. For example, the combination of the facial expression of the superior and the line of sight of the superior may be analyzed (step S102).

During the training interview, the analysis unit 424a may analyze the analysis target in the image of the subordinate based on the scenario. The analysis unit 424a may also determine the subordinate's state of concern or empathy (emotions) based on the scenario.

After the training interview, the score processing unit 425 acquires, for example, a mathematical model, a learned model, or a data table from the memory unit 41. The score processing unit 425 derives a score for the superior's interview skills based on the analysis results of the superior's facial expressions, line of sight, nodding, etc. For example, the score processing unit 425 derives a score for the superior's interview skills by inputting the analysis results of the superior's facial expressions, etc. into the learned model. The score for the superior's interview skills is provided to the superior. The score processing unit 425 may also derive a score for the subordinate's state according to a scenario.

After the training interview, the feedback processing unit 426 acquires the formula model, the learned model, or the data table from the storage unit 41. The feedback processing unit 426 derives advice to be provided to the superior according to the analysis result of the analysis target of the superior (e.g., the number of times the superior nods). For example, the feedback processing unit 426 derives advice to be provided to the superior by inputting the analysis result of the analysis target of the superior into the learned model. The feedback processing unit 426 may derive advice to be provided to the superior by inputting the score of the superior's interview skill into the learned model. The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score of the superior's interview skill and the advice to be provided to the superior using the communication unit 40 (step S104). This allows the superior to improve his/her own interview skill before the actual interview based on the score and advice provided.

The lower part of Figure 2 shows an example of the operation of the interview support system 1a regarding an actual interview. Before the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's pre-questionnaire and the subordinate's type diagnostic questionnaire from the communication unit 27. The subordinate's pre-questionnaire is a questionnaire regarding what the subordinate wants to talk about (what he wants to consult about) with his superior during the interview (one-on-one meeting). The questionnaire processing unit 427 uses the communication unit 40 to obtain the superior's type diagnostic questionnaire from the communication unit 37 (step S201).

Before the actual interview, the diagnosis unit 428 diagnoses the personality type of the subordinate based on the subordinate's type diagnosis questionnaire. The diagnosis unit 428 diagnoses the boss's personality type based on the boss's type diagnosis questionnaire. For example, the diagnosis unit 428 diagnoses the personality type based on the results of clustering processing for multiple items in the type diagnosis questionnaire. The questionnaire processing unit 427 obtains the subordinate's advance questionnaire (step S202).

Before the actual interview, the goal processing unit 423 obtains from the memory unit 41 the progress of the goals (promises) updated by the superior and subordinate after the previous actual interview. The goal processing unit 423 uses the communication unit 40 to display the progress of the goals on the display unit 33 (step S203). This allows the superior to check the items written in the subordinate's pre-questionnaire, the subordinate's personality type, his or her own personality type, and the progress of the goals before the actual interview.

During the actual interview, the control unit 42a supports the interview. For example, the information output unit 422 uses the communication unit 40 to cause the display unit 33 to display an interview image including the personality type diagnosis result and the results of the pre-survey. The information output unit 422 also uses the communication unit 40 to cause the display unit 23 to display a similar interview image. This allows the superior and subordinate to conduct the interview while checking, for example, the personality type diagnosis result and the results of the pre-survey.

During the actual interview, the analysis unit 424a (image analysis unit) analyzes the analysis target of the superior (interviewee) in real time based on the results of image recognition of the image of the superior (interviewee). The analysis unit 424a may also analyze the analysis target of the subordinate (interviewee) in real time based on the results of image recognition of the image of the subordinate (interviewee) (step S204).

During the actual interview, the goal processing unit 423 acquires the goals (promises) entered in the input fields of the interview image displayed on the display unit 33 from the communication unit 37 using the communication unit 40. The goal processing unit 423 acquires the goals (promises) entered in the input fields of the interview image displayed on the display unit 23 from the communication unit 27 using the communication unit 40. These goals are goals that have been agreed upon between the superior and the subordinate (step S205).

After the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's post-interview questionnaire from the communication unit 27 (step S206). The score processing unit 425 derives a score for the superior's interview skills based on the analysis results of the superior's facial expression, line of sight, nodding, etc. The score processing unit 425 may derive a score for the subordinate's state of concern or empathy (feelings) based on the subordinate's post-interview questionnaire. The score processing unit 425 may derive a score for the subordinate's state based on the analysis results of the subordinate's facial expression, line of sight, nodding, etc.

After the actual interview, the feedback processing unit 426 derives advice to be provided to the superior according to the analysis result of the superior's analysis target. The feedback processing unit 426 may derive advice to be provided to the superior according to the analysis result of the superior's analysis target and the analysis result of the subordinate's analysis target. The feedback processing unit 426 causes the display unit 33 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior using the communication unit 40 (step S207). This allows the superior to improve his/her own interview skills after the actual interview based on the score and advice provided.

After the actual interview, the goal processing unit 423 obtains the progress of the goals (promises) updated by the superior from the communication unit 37 using the communication unit 40. The goal processing unit 423 obtains the progress of the goals updated by the subordinate from the communication unit 27 using the communication unit 40. The goal processing unit 423 records the progress of the updated goals in the memory unit 41 (step S208). By storing the progress (status) of these goals in the memory unit 41, they can be referred to by the superior before the next actual interview.

Next, the analysis process will be described in detail.
FIG. 3 is a diagram showing an example of image analysis regarding a superior's interview skill in the first embodiment.

[Improving communication skills of superiors during interviews]
With the objective of improving the communication ability of the superior in the interview (analysis objective), the analysis unit 424a analyzes the facial expression of the superior based on the result of image recognition of the image of the superior (interviewee). The analysis unit 424a derives the ratio of the superior's positive facial expression to the superior's negative facial expression (for example, the ratio of the length of time that the facial expression was displayed).

The score processing unit 425 derives a score for the boss's interview skills based on the analysis results (proportion of facial expressions) of the analysis target in the boss's image. The score processing unit 425 derives the score for the boss's interview skills so that the score is highest when the analysis results of the analysis target in the boss's image satisfy a specified condition. For example, the score processing unit 425 derives the highest score when the proportion of positive facial expressions satisfies the condition that the proportion is at least twice but less than three times that of negative facial expressions. The score for the boss's interview skills may be converted, for example, into a score out of a maximum of 100 points.

The feedback processor 426 provides advice to the superior according to the derived ratio. For example, if the number of positive expressions is three or more times the number of negative expressions, the feedback processor 426 includes the advice "You seem to be able to fully express trust and other feelings through your facial expressions. Please continue to use positive facial expressions when interacting with your subordinates and strive to build relationships with them." in the feedback image. For example, if the number of positive facial expressions is two or more times but less than three times the number of negative facial expressions, the feedback processor 426 includes the advice "You seem to be able to interact with your subordinates with positive facial expressions. However, you also often use negative facial expressions such as blank expressions, sadness, or anger. Try to use more positive facial expressions." in the feedback image. For example, if the number of positive facial expressions is less than two times the number of negative facial expressions, the feedback processor 426 includes the advice "You often use negative facial expressions such as sadness and anger, which may be affecting your relationships with your subordinates. Please practice using positive facial expressions." in the feedback image.

The analysis unit 424a may analyze the line of sight (eye movement) of the superior (interviewee) based on the result of image recognition of the image of the superior. The score processing unit 425 derives the score of the superior's interview skill based on the line of sight of the superior. The feedback processing unit 426 provides advice to the superior according to the line of sight of the superior. For example, if the superior's eyes are wandering (if the superior's eyes are not directed at the subordinate), the feedback processing unit 426 includes the advice "Your eyes may wander. Be careful, as the subordinate may take it that you are not open to the subordinate or that you are lying." in the feedback image. For example, if the superior is looking down, the feedback processing unit 426 includes the advice "Your eyes tend to look downward. Be careful, as the subordinate may take it that you are not in good spirits or that you are closed off." in the feedback image.

The analysis unit 424a may analyze the boss's nods (interjections) based on the results of image recognition of the boss's image (interviewee). The score processing unit 425 derives the score of the boss's interview skill based on the number of nods of the boss. The feedback processing unit 426 provides advice to the boss according to the number of nods of the boss. For example, in a 30-minute one-on-one meeting, if the number of nods of the boss is 130 or more, the feedback processing unit 426 includes the advice "There are too many interjections, which gives the subordinate the impression that the boss is not listening to what the subordinate is saying. Make sure to interject at the appropriate time" in the feedback image. For example, if the number of nods of the boss is 100 or more and less than 130, the feedback processing unit 426 includes the advice "You are showing a sufficiently receptive attitude toward your subordinate. Please continue to do so in the future" in the feedback image. For example, if the number of times the superior nods is 70 or more but less than 100, the feedback processor 426 includes in the feedback image the advice "If you use more interjections, you can show your subordinate an accepting attitude. Let's try." For example, if the number of times the superior nods is less than 70, the feedback processor 426 includes in the feedback image the advice "It seems that your subordinate does not get the impression that he is being listened to sufficiently. Let's use more interjections to show your subordinate an accepting attitude."

The analysis unit 424a analyzes the facial expression of the superior based on the result of image recognition of the superior's image. The analysis unit 424a classifies the superior's emotion based on the superior's facial expression. The analysis unit 424a classifies the emotion into, for example, anger, joy, surprise, sadness, and neutral. The analysis unit 424a analyzes the subordinate's facial expression based on the result of image recognition of the subordinate's image. The analysis unit 424a classifies the subordinate's emotion based on the subordinate's facial expression. The analysis unit 424a derives the degree of agreement (degree of empathy) between the superior's emotion and the subordinate's emotion. The score processing unit 425 derives the score of the superior's interview skill based on the degree of agreement between the superior's emotion and the subordinate's emotion. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the superior's emotion and the subordinate's emotion.

The analysis unit 424a may analyze the facial expression (state) of the subordinate based on the result of image recognition of the image of the subordinate. The score processing unit 425 derives a score of the interview skill of the superior based on the ratio of the subordinate's positive facial expression to the subordinate's negative facial expression (worry level). The feedback processing unit 426 provides advice to the superior according to the ratio of the subordinate's positive facial expression to the subordinate's negative facial expression.

[Improving the supervisor's receptiveness during interviews (improving receptive attitude or listening attitude)]
The analysis unit 424a analyzes the boss's nods (interjections) based on the results of image recognition of the boss's image, with the aim of improving the boss's receptiveness in the interview. The score processing unit 425 derives the score of the boss's interview skill based on the number of times the boss nods. The feedback processing unit 426 determines whether the number of times the boss nods is within a predetermined range. This predetermined range is determined in advance according to, for example, the length of time the one-on-one meeting will be held. The feedback processing unit 426 provides advice to the boss according to the determination result.

Next, each image to be displayed will be described in detail.
[Before the pre-production training interview]
In step S101 shown in FIG. 2, the display unit 33 displays an image for selecting a personality type based on the data transmitted from the communication unit 40.

Figure 4 is a diagram showing an example of a selection image for personality types in the first embodiment. There are "promoter" type, "supporter" type, "analyzer" type, and "controller" type, depending on the strength of self-assertion and the amount of emotional expression. These are examples of personality types.

The "promoter" type is a personality type that is assertive and expresses a lot of emotion. The "supporter" type is a personality type that is not assertive and expresses a lot of emotion. The "analyzer" type is a personality type that is not assertive and expresses a little emotion. The "controller" type is a personality type that is assertive and expresses a little emotion.

The superior, for example, selects the personality type of the subordinate with whom the actual interview is scheduled. The superior, for example, selects the personality type "promoter" by operating radio button 50.

[During a pre-show training interview]
In step S102 shown in FIG. 2, the display unit 23 displays the interview image transmitted from the communication unit 40.

FIG. 5 is a diagram showing an example of an interview image in the first embodiment. The interview image includes, for example, a moving image of the boss and a moving image of an avatar (a virtual subordinate). The interview image includes, for example, a character string representing the boss's personality type and a character string representing the avatar's personality type. The interview image includes, for example, an operation key image 60. The operation key image 60 is an operation key image associated with a goal (promise) set according to a scenario. When the operation key image 60 is pressed, the display unit 33 displays an image for managing the progress of the goal (goal management image) transmitted from the communication unit 40 in accordance with the control of the control unit 36.

When the operation key image 70 is pressed in step S102 shown in FIG. 2, the display unit 33 displays the setting image for the target item transmitted from the communication unit 40 in response to control by the control unit 36.

Figure 6 is a diagram showing an example of an image for setting a goal item in the first embodiment. The image for setting a goal item includes an input field for the goal item. When the operation key image 80 is pressed, the goal item entered in the setting image is stored in the memory unit 41. When the operation key image 90 is pressed, the display unit 33 redisplays the interview image exemplified in Figure 5 in accordance with the control of the control unit 36.

[After the pre-show training interview]
In step S<b>104 shown in FIG. 2 , the display unit 33 displays the feedback image transmitted from the communication unit 40 .

Figure 7 is a diagram showing an example of an interview skill score and advice feedback image (provided image) in the first embodiment. The feedback image shown in Figure 7 includes a score display area 100 and an advice display area 110. The display unit 33 displays the score derived by the score processing unit 425 in the score display area 100. In Figure 7, the display unit 33 displays the communication skill score and the receptive attitude (receptive ability) score in the score display area 100. The display unit 33 displays the advice derived by the feedback processing unit 426 in the advice display area 110. These make it possible to improve the supervisor's interview skills (communicative ability, receptive ability).

The display unit 33 may display a feedback image including a degree of satisfaction determined by an artificial intelligence (AI) determination by the feedback processing unit 426 based on the post-questionnaire. That is, the display unit 33 may display a feedback image including a degree of satisfaction determined using a trained model. This degree of satisfaction is, for example, the degree to which the subordinate's concerns have been resolved. The display unit 33 may display a feedback image including the progress of the superior's goals (promises), the progress of the subordinate's goals (promises), and the goal achievement degree, which is the average value of these.

[Before the actual interview]
Fig. 8 is a diagram showing a first example of an input image for a pre-questionnaire in the first embodiment. In step S201 shown in Fig. 2, the display unit 23 displays an input image for the pre-questionnaire (consultation matters) transmitted from the communication unit 40. The pre-questionnaire includes items related to matters that a subordinate would like to consult with his/her superior in a one-on-one meeting. Each item may be selected using a pull-down menu or input using a check box.

Figure 9 is a diagram showing a second example of an input image for a type diagnosis questionnaire in the first embodiment. In step S201 shown in Figure 2, the display unit 23 and the display unit 33 display an input image for the type diagnosis questionnaire (for diagnosing a personality type) transmitted from the communication unit 40. Each item may be selected using an operation key image that can be pressed.

Figure 10 is a diagram showing an example of a notification image of a personality type diagnosis result in the first embodiment. In step S202 shown in Figure 2, the display unit 23 and the display unit 33 display the notification image transmitted from the communication unit 40. This notification image includes a character string representing the personality type diagnosis result based on a type diagnosis questionnaire (for diagnosing a personality type).

In step S203 shown in FIG. 2, the display unit 33 displays the management image transmitted from the communication unit 40. That is, the display unit 33 displays a management image of the progress of the goals set in the previous interview. This allows the supervisor to check the progress of the goals set in the previous interview before the actual interview.

[During the actual interview]
In step S204 shown in FIG. 2, the display unit 23 and the display unit 33 display the interview image (for example, the interview image exemplified in FIG. 5) transmitted from the communication unit 40. The interview image includes, for example, a moving image of the superior and a moving image of the subordinate. The interview image includes, for example, a character string representing the superior's personality type and a character string representing the subordinate's personality type. The interview image may include, for example, an operation key image 60 associated with the goals (promises) set up until the previous interview. When the operation key image 60 is pressed, for example, the display unit 23 and the display unit 33 display an image for managing the progress of the goals set up until the previous interview.

Fig. 11 is a diagram showing an example of an image for managing the progress of goals in the first embodiment. The management image includes a list of set goals (promises) and each operation key image. In Fig. 11, when operation key image 120 "Match" is pressed, a "Progress Check Screen" showing the progress of the goal associated with operation key image 120 is displayed on display unit 23 and display unit 33.

When the operation key image 70 "Goal setting" is pressed in step S204 shown in FIG. 2, the display unit 33 displays the setting image of the goal items transmitted from the communication unit 40 in response to control by the control unit 36. In step S205 shown in FIG. 2, the superior (mentor) enters new goals (promises) and deadlines in the input field "Mentor's promises". When the operation key image 80 "Save" is pressed, the goals entered in the setting image are stored in the memory unit 41, for example, in association with a time. When the operation key image 90 "Back" is pressed, the display unit 33 redisplays the interview image exemplified in FIG. 5 in response to control by the control unit 36.

When the operation key image 70 "Goal setting" is pressed in step S204 shown in FIG. 2, the display unit 23 displays an image for setting the goal items transmitted from the communication unit 40 in response to control by the control unit 26. In step S205 shown in FIG. 2, the subordinate (mentor) enters a new goal item (promise) and deadline in the input field "Promise of mentor". When the operation key image 80 "Save" is pressed, the goal items entered in the setting image are stored in the storage unit 41, for example, in association with a time. When the operation key image 90 "Back" is pressed, the display unit 23 redisplays the interview image exemplified in FIG. 5 in response to control by the control unit 26.

[After the actual interview]
In step S206 shown in FIG. 2, the display unit 23 displays an input image for the post-event questionnaire.

FIG. 12 is a diagram showing an example of an input image for a post-event questionnaire in the first embodiment. The post-event questionnaire includes items related to the superior's interview skills in one-on-one meetings. Each item may be selected using a radio button or input using a slider bar. The subordinate's post-event questionnaire may include items related to key performance indicators (KPIs) (e.g., progress toward targets), for example. The subordinate's post-event questionnaire is not shared with the superior. This allows the subordinate to fill out the items in the post-event questionnaire honestly.

In step S207 shown in FIG. 2, the display unit 33 displays the feedback image transmitted from the communication unit 40. The feedback image illustrated in FIG. 7 includes a score display area 100 and an advice display area 110. This makes it possible to improve the supervisor's interview skills after the actual interview.

In step S208 shown in FIG. 2, the display unit 33 displays a management image showing the progress of the goals set in the current interview. The superior may write a message to the subordinate after the interview in the management image for each goal (promise).

In step S208 shown in FIG. 2, the display unit 23 may display an image for setting goals in response to control by the control unit 26. After the interview, the subordinate may enter new goals and deadlines in the input field "Promises for mentor." The display unit 23 may display an image for managing the progress of goals, in which a message to the subordinate after the interview is entered, in response to control by the control unit 26.

Next, an example of the operation of the interview support device 4a will be described.
13 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4a in the first embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S301).

During the training interview, the scenario processing unit 421 uses the communication unit 40 to display an interview image, including an image of an avatar that speaks according to the scenario and personality type, on the display unit 33 (step S302). The analysis unit 424a analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S303).

After the training interview, the score processing unit 425 derives a score for the interview skills of the superior based on the analysis results of the superior's facial expression, line of sight, nodding, etc. (step S304). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the image of the superior (step S305). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score for the interview skills of the superior and the advice to be provided to the superior using the communication unit 40 (step S306).

Figure 14 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4a in the first embodiment. Before the actual interview, the questionnaire processing unit 427 acquires the subordinate's pre-questionnaire and type diagnostic questionnaire, and the superior's type diagnostic questionnaire (step S401). The diagnosis unit 428 diagnoses the personality type based on the type diagnostic questionnaire (step S402). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S403).

During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis results and the results of the pre-interview questionnaire on the display unit 33 (step S404). The analysis unit 424a analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S405). The goal processing unit 423 acquires the goals (promises) entered by the superior (step S406).

After the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's post-interview questionnaire from the communication unit 27 (step S407). The score processing unit 425 derives the score of the superior's interview skill based on the analysis results of the superior's facial expression, line of sight, nod, etc. (step S408). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the image of the superior (step S409). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score of the superior's interview skill and the advice to be provided to the superior using the communication unit 40 (step S410). The goal processing unit 423 obtains the progress of the goal items (promises) updated by the superior from the communication unit 37 using the communication unit 40. The goal processing unit 423 may obtain the progress of the goal items (promises) updated by the subordinate from the communication unit 27 using the communication unit 40 (step S411). The goal processing unit 423 records the progress of the updated goals (promises) in the memory unit 41 (step S412).

As described above, the analysis unit 424a analyzes the analysis target in the image of the interviewer based on the result of image recognition on the image of the interviewer conducting an interview with the interviewee. The analysis target in the image of the interviewer is, for example, at least one of the facial expression of the interviewer, the line of sight of the interviewer, and the nod of the interviewer. The score processing unit 425 provides the interviewer with a score for the interviewing skill of the interviewer based on the analysis result of the analysis target in the image of the interviewer. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the image of the interviewer.

This makes it possible to improve the interviewing skills of the interviewer using at least one of the score and advice based on the analysis results of the interviewer's image. By improving the interviewer's interviewing skills, it is possible to promote behavioral change (growth) of the interviewee. The behavioral change of the interviewee (growth of the subordinate) may be treated as a key goal indicator (KGI). Since images taken during the interview are analyzed, and audio (content of speech) during the interview is not analyzed, it is possible to improve the interviewing skills of the interviewer while reducing the possibility of leaking sensitive and personal information within the company.

Second Embodiment
In the first embodiment, the image of the boss is the subject of analysis. In contrast, in the second embodiment, the voice of the boss is the subject of analysis, which is a difference from the first embodiment. In the second embodiment, the difference from the first embodiment will be mainly described.

Figure 15 is a diagram showing an example of the configuration of an interview support system 1b in the second embodiment. The interview support system 1b includes an interviewee device 2, an interviewer device 3, an interview support device 4b, and a communication line 5.

The interview support device 4b is an information processing device such as a server. The interview support device 4b may be realized by cloud computing. The interview support device 4b includes a communication unit 40, a memory unit 41, and a control unit 42b. The control unit 42b includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424b, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

The analysis unit 424b (voice analysis unit) analyzes the analysis target in the voice of the superior (interviewee) in real time. The analysis target is, for example, the speaking speed of the superior, the intonation of the superior's voice, the volume (volume) of the superior's voice, and the tone (condition) of the superior's voice. The analysis target may be, for example, the content of the superior's speech. The analysis target may be, for example, the number of questions asked by the superior to the subordinate. The analysis unit 424b may analyze the analysis target in the voice of the subordinate (interviewee) in real time.

Next, the analysis process will be described in detail.
FIG. 16 is a diagram showing an example of a voice analysis regarding a superior's interview skill in the second embodiment.

[Improving communication skills of superiors during interviews]
With the objective (analysis objective) of improving the communication ability of the superior in the interview, the analysis unit 424a analyzes the speaking speed of the superior (interviewee) in the speech of the superior. The score processing unit 425 derives a score of the superior's interview skill based on the superior's speaking speed. The feedback processing unit 426 judges whether the superior's speaking speed is within a predetermined range. The feedback processing unit 426 provides advice to the superior according to the judgment result.

The analysis unit 424b may further analyze the speech speed of the subordinate in the voice of the subordinate. The analysis unit 424b derives the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "You are able to speak at a pace that matches the subordinate's pace. The subordinate will feel relieved. Please continue like this" in the feedback image. For example, if the degree of agreement is 50% or more and less than 80%, the feedback processing unit 426 includes the advice "If you can speak more at a pace that matches the subordinate's pace, you will be able to build a stronger relationship of trust with the subordinate. Please do your best" in the feedback image. If the degree of match is less than 50%, the feedback processor 426 includes the advice "You may be speaking from your own perspective. Try to speak at a pace that matches your subordinate's pace" in the feedback image.

The analysis unit 424b may analyze the intonation of the voice of the superior. The analysis unit 424b classifies the emotion of the superior based on the intonation of the voice of the superior. The analysis unit 424b may analyze the intonation of the voice of the subordinate. The analysis unit 424b classifies the emotion of the subordinate based on the intonation of the voice of the subordinate. The analysis unit 424b derives the degree of agreement (degree of empathy) between the emotion of the superior and the emotion of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the degree of agreement between the emotion of the superior and the emotion of the subordinate. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the emotion of the superior and the emotion of the subordinate. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "It seems that you are sufficiently empathizing with the subordinate and are close to him/her. Please continue like this" in the feedback image. For example, if the degree of match is 50% or more and less than 80%, the feedback processing unit 426 includes in the feedback image the advice "If you can empathize more with your subordinates' feelings, the relationship of trust between you and them will be stronger. Do your best." For example, if the degree of match is less than 50%, the feedback processing unit 426 includes in the feedback image the advice "It seems that you are interacting with your subordinates without being very empathetic to them. Try to empathize with your subordinates' feelings and emotions."

The analysis unit 424b may analyze the volume of the voice of the superior. The analysis unit 424b may analyze the tone of the voice of the superior. The analysis unit 424b may analyze the volume of the voice of the subordinate. The analysis unit 424b may analyze the tone of the voice of the subordinate. The analysis unit 424b derives the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "You are able to speak in accordance with the tone of the voice of the subordinate. The subordinate will feel relieved. Please continue like this" in the feedback image. For example, if the degree of match is 50% or more but less than 80%, the feedback processing unit 426 includes in the feedback image the advice "If you can speak more in line with the tone of your subordinate's voice, you will be able to build a stronger relationship of trust with your subordinate. Try your best." For example, if the degree of match is less than 50%, the feedback processing unit 426 includes in the feedback image the advice "You may be speaking from your own perspective. Try to speak in a way that matches the volume of your subordinate's voice."

The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice. The analysis unit 424b derives the ratio of the boss's positive speech content to the boss's negative speech content. The score processing unit 425 derives the score of the boss's interview skill based on the ratio of the boss's positive speech content to the boss's negative speech content. The analysis unit 424b may analyze the contents of the subordinate's speech in the subordinate's voice. The score processing unit 425 may derive the score of the boss's interview skill based on the ratio of the subordinate's positive speech content to the subordinate's negative speech content. The score processing unit 425 provides advice to the boss according to at least one of the ratios.

The analysis unit 424b deletes the voice data after the analysis of the speech content from the storage unit 41. The analysis unit 424b may also analyze the speech content of each utterance in the voice data received by the communication unit 40 after preventing the voice data from being stored in the storage unit 41. This can reduce the possibility that the speech content of each utterance in a one-on-one meeting will be leaked.

[Improving the supervisor's receptiveness during interviews (improving receptive attitude or listening attitude)]
The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice in order to improve the boss's receptiveness in the interview. The analysis unit 424b counts the number of times the boss utters messages showing understanding or sympathy. The score processing unit 425 derives a score for the boss' interview skill based on the number of times the boss utters messages showing understanding or sympathy. The feedback processing unit 426 provides advice to the boss according to the number of times the boss utters messages showing understanding or sympathy.

[Improving supervisors' coaching skills during interviews]
The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice for the purpose of improving the boss's coaching ability in the interview. The analysis unit 424b counts the number of times a positive message is uttered by the boss. The score processing unit 425 derives a score of the boss' interview skill based on the number of times a positive message is uttered by the boss. The feedback processing unit 426 provides advice to the boss according to the number of times a positive message is uttered by the boss.

17 is a diagram showing an example of an interview skill score and advice feedback image (provided image) in the second embodiment. The feedback image shown in FIG. 17 includes a score display area 100 and an advice display area 110. The display unit 33 displays the score derived by the score processing unit 425 in the score display area 100. In FIG. 17, the display unit 33 displays the communication skill score, the receptive attitude score (receptive ability) score, and the coaching ability score in the score display area 100. The display unit 33 displays the advice derived by the feedback processing unit 426 in the advice display area 110. This makes it possible to improve the superior's interview skills (communication skill, receptive ability, coaching ability).

Next, an example of the operation of the interview support device 4b will be described.
18 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4b in the second embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S501).

During the training interview, the scenario processing unit 421 uses the communication unit 40 to display an interview image, including an image of an avatar speaking according to the scenario and personality type, on the display unit 33 (step S502). The analysis unit 424b analyzes the analysis target in the supervisor's voice in real time (step S503).

After the training interview, the score processing unit 425 derives a score for the superior's interview skills based on the results of analysis of at least one of the superior's speech rate, the superior's voice intonation, the superior's voice tone, and the superior's voice volume. Hereinafter, tone and volume are referred to as "tone, etc." (step S504). The feedback processing unit 426 derives advice to be provided to the superior according to the results of analysis of the analysis target in the superior's voice (step S504). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior, using the communication unit 40 (step S506).

Figure 19 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4b in the second embodiment. Before the actual interview, the questionnaire processing unit 427 acquires the subordinate's pre-questionnaire and type diagnostic questionnaire, and the superior's type diagnostic questionnaire (step S601). The diagnosis unit 428 diagnoses the subordinate's personality type based on the subordinate's type diagnostic questionnaire. The diagnosis unit 428 diagnoses the superior's personality type based on the superior's type diagnostic questionnaire (step S602). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S603).

During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis results and the results of the pre-interview questionnaire on the display unit 33 (step S604). The analysis unit 424b analyzes the analysis target in the boss's voice in real time (step S605). The goal processing unit 423 acquires the goals (promises) entered by the boss (step S606).

After the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's post-interview questionnaire from the communication unit 27 (step S607). The score processing unit 425 derives the score of the superior's interview skill based on the analysis results of the superior's speech speed, the superior's voice intonation, the superior's voice tone, etc. (step S608). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the superior's voice (step S609). The feedback processing unit 426 uses the communication unit 40 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior on the display unit 33 (step S610). The goal processing unit 423 uses the communication unit 40 to obtain the progress of the goal items (promises) updated by the superior from the communication unit 37 (step S611). The goal processing unit 423 records the progress of the updated goal items (promises) in the memory unit 41 (step S612).

As described above, the analysis unit 424b analyzes the analysis target in the voice of the interviewer who is conducting an interview with the interviewee. The score processing unit 425 provides the interviewer with a score for the interviewing skill of the interviewer based on the analysis result of the analysis target in the interviewer's voice. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the interviewer's voice. The analysis target in the interviewer's voice is, for example, at least one of the speaking speed of the interviewer, the intonation of the interviewer's voice, and the tone of the interviewer's voice.

This makes it possible to improve the interviewer's interviewing skills using at least one of the score and advice based on the analysis results of the interviewer's voice. By improving the interviewer's interviewing skills, it is possible to promote behavioral change (growth) in the interviewee. Even if the content of speech during the interview is excluded from the analysis in order to reduce the possibility of leaking sensitive information and personal information within the company, it is possible to improve the interviewer's interviewing skills based on the analysis results of speech speed, intonation, and volume during the interview.

The feedback processing unit 426 may derive the subordinate's issues based on the analysis results of the subordinate's speech content, for example by using a trained model. The feedback processing unit 426 may provide the superior with advice indicating specific behavioral content based on the subordinate's issues. The analysis unit 424b may determine whether the behavioral content provided to the superior has led to a behavioral change in the subordinate, based on the progress of the target items. The feedback processing unit 426 may provide the superior with a feedback image including the determination result.

Third Embodiment
In the first embodiment, the image of the boss, who is the interviewee, is the subject of analysis, but the voice of the boss is not the subject of analysis. In contrast to this, in the second embodiment, the voice of the boss is the subject of analysis, but the image of the boss is not the subject of analysis. Image-based analysis and voice-based analysis may be combined. Thus, in the third embodiment, the difference between the first and second embodiments is that the image and voice of the boss are the subjects of analysis. In the third embodiment, the differences from the first and second embodiments will be mainly described.

Figure 20 is a diagram showing an example of the configuration of an interview support system 1c in the third embodiment. The interview support system 1c includes an interviewee device 2, an interviewer device 3, an interview support device 4c, and a communication line 5.

The interview support device 4c is an information processing device such as a server. The interview support device 4c may be realized by cloud computing. The interview support device 4c includes a communication unit 40, a memory unit 41, and a control unit 42c. The control unit 42c includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424c, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

The analysis unit 424c (image and audio analysis unit) analyzes the analysis target in the image of the superior (interviewee) in real time based on the result of image recognition on the image of the superior. The analysis unit 424c may analyze the analysis target in the image of the subordinate (interviewee) in real time based on the result of image recognition on the image of the subordinate. The analysis unit 424c analyzes the analysis target in the superior's voice in real time. The analysis unit 424c may analyze the analysis target in the subordinate's voice in real time.

Next, an example of the operation of the interview support device 4c will be described.
21 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4c in the third embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S701).

During the training interview, the scenario processing unit 421 uses the communication unit 40 to display an interview image, including an image of an avatar speaking according to the scenario and personality type, on the display unit 33 (step S702). The analysis unit 424c analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S703). The analysis unit 424c analyzes the analysis target in the voice of the superior in real time (step S704).

After the training interview, the score processing unit 425 derives a score for the interview skills of the superior based on the analysis results of the superior's facial expression, line of sight, nodding, etc., and the analysis results of the superior's speech rate, the superior's voice intonation, the superior's voice tone, etc. (step S705). The feedback processing unit 426 derives advice to be provided to the superior based on the analysis results of the analysis targets in the image and voice of the superior (step S706). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score for the interview skills of the superior and the advice to be provided to the superior, using the communication unit 40 (step S707).

Figure 22 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4c in the third embodiment. Before the actual interview, the questionnaire processing unit 427 acquires the subordinate's pre-questionnaire and type diagnostic questionnaire, and the superior's type diagnostic questionnaire (step S801). The diagnosis unit 428 diagnoses the personality type based on the type diagnostic questionnaire (step S802). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S803).

During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis results and the results of the pre-interview questionnaire on the display unit 33 (step S804). The analysis unit 424c analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S805). The analysis unit 424c analyzes the analysis target in the voice of the superior in real time (step S806). The goal processing unit 423 acquires the goals (promises) entered by the superior (step S807).

After the actual interview, the questionnaire processing unit 427 acquires the subordinate's post-interview questionnaire from the communication unit 27 using the communication unit 40 (step S808). The score processing unit 425 derives the score of the superior's interview skill based on the analysis results of the superior's facial expression, line of sight, nods, etc., and the analysis results of the superior's speech speed, the superior's voice intonation, and the superior's voice tone, etc. (step S809). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the image and voice of the superior (step S810). The feedback processing unit 426 displays a feedback image including the superior's interview skill score and the advice to be provided to the superior on the display unit 33 using the communication unit 40 (step S811). The goal processing unit 423 acquires the progress of the goal items (promises) updated by the superior from the communication unit 37 using the communication unit 40 (step S812). The goal processing unit 423 records the progress of the updated goal items (promises) in the memory unit 41 (step S813).

As described above, the analysis unit 424c analyzes the analysis target in the image of the interviewer based on the result of image recognition of the image of the interviewer conducting an interview with the interviewee. The analysis unit 424c analyzes the analysis target in the voice of the interviewer conducting an interview with the interviewee. The score processing unit 425 provides the interviewer with a score for the interviewing skill of the interviewer based on the analysis result of the analysis target in the image of the interviewer and the analysis result of the analysis target in the voice of the interviewer. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the image and voice of the interviewer.

This makes it possible to efficiently improve the interviewer's interviewing skills by using at least one of the score and advice based on the analysis results of the interviewee's image and voice. By improving the interviewer's interviewing skills, it is possible to promote behavioral change (growth) in the interviewee.

Fourth Embodiment
The fourth embodiment differs from the first to third embodiments in that advice, etc. is provided to the interviewee during a training interview that corresponds to a real interview, and that a scenario of the training interview is changed during the training interview depending on the analysis results of an image, etc. of the interviewee. The fourth embodiment will be described focusing on the differences from the first to third embodiments.

Figure 23 is a diagram showing an example of the configuration of an interview support system 1d in the fourth embodiment. The interview support system 1d (interview training system) includes an interviewee device 2, an interviewer device 3, an interview support device 4d, and a communication line 5.

The interview support device 4d is an information processing device such as a server. The interview support device 4d may be realized by cloud computing. The interview support device 4d includes a communication unit 40, a memory unit 41, and a control unit 42d. The control unit 42d includes a selection acquisition unit 420, a scenario processing unit 421d, an information output unit 422, a goal processing unit 423, an analysis unit 424d, a score processing unit 425, a feedback processing unit 426d, a questionnaire processing unit 427, and a diagnosis unit 428.

The analysis unit 424d (image and audio analysis unit) executes an analysis process of the analysis target in at least one of the image and audio of the superior (interviewee), for example, at a predetermined cycle. For example, the analysis unit 424d analyzes (image analysis) the analysis target (e.g., facial expression) in the image of the superior based on the result of image recognition of the image of the superior in real time. The analysis unit 424d may analyze the analysis target in the image of the subordinate (interviewee) in real time based on the result of image recognition of the image of the subordinate. The analysis unit 424d analyzes (audio analysis) the analysis target (e.g., intonation) in the superior's audio in real time. The analysis unit 424d may analyze the analysis target in the subordinate's audio in real time.

The content of speech during training interviews does not have to be the subject of analysis. Since the content of speech (text data) during interviews is not analyzed, it is possible to improve the interviewer's interview skills while reducing the possibility of leaking sensitive and personal information within the company.

During the training interview, the scenario processing unit 421d changes the scenario of the training interview according to the analysis results of the analysis target in at least one of the image and voice of the superior. The scenario processing unit 421d controls the image and voice behavior of the avatar displayed on the display unit 33 so that the avatar behaves according to the scenario.

24 is a diagram showing an example (modification example) of a rule-based conditional branching of a scenario in the fourth embodiment. For example, the scenario processing unit 421d uses the communication unit 40 to display on the display unit 33 an interview image including an image of an avatar (virtual subordinate) that speaks according to the scenario associated with the current result (current analysis result) of the image analysis and voice analysis of the superior and the personality type of the subordinate. As an example, when the current analysis result (current analysis result) is "joy", the scenario processing unit 421d displays on the display unit 33 an interview image including a scenario associated with the analysis result "joy" and an image of an avatar that speaks according to the personality type of the selected avatar. As an example, when the current analysis result is "anger", the scenario processing unit 421d displays on the display unit 33 an interview image including a scenario associated with the analysis result "anger" and an image of an avatar that speaks according to the personality type of the selected avatar. As an example, if the current analysis result is "surprise," the scenario processing unit 421d causes the display unit 33 to display an interview image including a scenario associated with the analysis result "surprise" and an image of an avatar speaking according to the selected avatar personality type.

For example, the scenario processing unit 421d may use the communication unit 40 to display on the display unit 33 an interview image including a scenario associated with the previous and current results of the image and voice analysis of the superior and an image of an avatar speaking according to the selected avatar's personality type. As an example, when the previous analysis result (past analysis result) is "joy" and the current analysis result (current analysis result) is "surprise", the scenario processing unit 421d may display on the display unit 33 an interview image including a scenario associated with the previous analysis result "joy" and the current analysis result "surprise" and an image of an avatar speaking according to the selected avatar's personality type. As an example, when the previous analysis result is "anger" and the current analysis result is "surprise", the scenario processing unit 421d may display on the display unit 33 an interview image including a scenario associated with the previous analysis result "anger" and the current analysis result "surprise" and an image of an avatar speaking according to the selected avatar's personality type.

Figure 25 is a diagram showing an example of the operation of the interview support system 1d for a training interview associated with a real interview in the fourth embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type of the avatar selected by the supervisor based on the selection image of the personality type exemplified in Figure 4 from the communication unit 37 using the communication unit 40. The selected personality type of the avatar is, for example, the same as the personality type of the interviewee in the real interview associated with the training interview (step S901).

The questionnaire processing unit 427 uses the communication unit 40 to obtain the boss's type diagnostic questionnaire from the communication unit 37 (step S902). The diagnosis unit 428 diagnoses the boss's personality type based on the boss's type diagnostic questionnaire. For example, the diagnosis unit 428 diagnoses the personality type based on the results of clustering processing for multiple items in the type diagnostic questionnaire (step S903).

During the training interview, the analysis unit 424d analyzes the analysis target of the superior (interviewer) in real time. For example, the analysis unit 424d analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior. The analysis unit 424d outputs the analysis result of the analysis target of the superior (e.g., facial expression in the image) to the scenario processing unit 421d as the analysis result regarding the superior's interview skills.

The scenario processing unit 421d changes the training interview scenario according to the analysis results of the analysis target of at least one of the image and voice of the superior (the analysis results regarding the superior's interview skills). The scenario processing unit 421d controls the image and voice behavior of the avatar displayed on the display unit 33 so that the avatar behaves according to the scenario (step S904).

During the training interview, the feedback processing unit 426d derives advice to be provided to the superior based on the diagnosis result of the superior's personality type and the analysis result of the superior's analysis target (analysis result regarding the superior's interview skills). The feedback processing unit 426d derives the advice to be provided to the superior using, for example, a data table. In this data table, for example, the superior's personality type, the analysis result of the superior's analysis target, and the advice to be provided to the superior are associated with each other. The feedback processing unit 426d provides the derived advice to the superior (step S905). This allows the superior to improve his/her own interview skills before the actual interview based on the advice provided in real time during the training interview.

After the training interview, the feedback processing unit 426d derives advice to be provided to the superior again after the training interview, according to the analysis result of the superior's analysis target. The feedback processing unit 426d causes the display unit 33 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior, using the communication unit 40 (step S906). This enables the superior to improve his/her own interview skills before the actual interview, based on the advice provided to the superior again after the training interview.

Fig. 26 is a diagram showing an example of an interview image in the fourth embodiment. The interview image shown in Fig. 26 is displayed on the display unit 33 during a training interview that corresponds to a real interview. The interview image includes a moving image of the superior and a moving image of an avatar (a virtual subordinate).

The interview image may further include at least one of a real-time analysis result of the analysis target (e.g., facial expression) in at least one of the image and voice of the superior, advice based on the analysis result, and advice based on the personality type diagnosis result. For example, the interview image illustrated in FIG. 26 includes an analysis result display area 200, a first advice display area 201, and a second advice display area 202.

The analysis result display area 200 is an area where real-time analysis results regarding the analysis target are displayed. The first advice display area 201 is an area where advice based on the diagnosis results of the boss's personality type is displayed. The second advice display area 202 is an area where advice based on the analysis results of the analysis target of at least one of the boss's image and voice is displayed.

During the training interview, the feedback processing unit 426d displays the real-time analysis results of the supervisor's analysis target in the analysis result display area 200. The real-time analysis results may be displayed in a graph format.

During the training interview, the feedback processing unit 426d derives advice (points to consider) based on the diagnosis result of the superior's personality type based on the diagnosis result of the superior's personality type and predetermined conditions. The feedback processing unit 426d may derive advice (points to consider) based on the diagnosis result of the superior's personality type for each level of importance. The feedback processing unit 426d displays the advice based on the diagnosis result of the superior's personality type in the first advice display area 201. The first advice display area 201 may be an area in which advice based on the diagnosis result of the personality type of an avatar (virtual subordinate) is displayed. The feedback processing unit 426d may display the advice based on the diagnosis result of the avatar's personality type in the first advice display area 201.

During the training interview, the feedback processing unit 426d derives advice based on the analysis results based on the real-time analysis results of the analysis target of the superior and predetermined conditions. For example, if the ratio of the analysis result "happiness" to the overall real-time analysis results of the analysis target "facial expression" of the superior is less than a predetermined threshold, the feedback processing unit 426d may derive advice based on the analysis results, "brighten your facial expression." The feedback processing unit 426d displays the advice based on the analysis results in the second advice display area 202.

As described above, the analysis unit 424d analyzes the analysis target in at least one of the image and audio of the interviewer who is interviewing the interviewee in the training interview that is associated with the actual interview. The feedback processing unit 426d provides advice to the interviewer in the training interview depending on the analysis result of the analysis target.

As a result, advice is provided to the interviewee in accordance with the analysis results of the analysis target during the training interview, making it possible to efficiently improve the interview skills of the interviewee before the actual interview. The interviewee can train their interview skills by themselves in their own free time. The interview support system 1d analyzes at least one of the interviewee's image and voice in real time, and during the training interview associated with the actual interview, the interview support system 1d presents the analysis results and advice, etc. to the interviewee in real time, allowing the interviewer to understand areas for improvement and points to note regarding the interviewee's own facial expressions, etc., before the interview.

The scenario processing unit 421d changes the scenario during the training interview in accordance with the analysis results of the analysis subject. The scenario processing unit 421d controls the image and audio behavior of the interviewee displayed on the display unit 33 in accordance with the scenario. The scenario processing unit 421d may display the analysis results of the analysis subject on the display unit 33 during the training interview.

In this way, the interview support system 1d changes the scenario of the interview scene that needs improvement during the training interview in accordance with the analysis results, and provides the interviewee with a training environment (avatar images, etc.) that corresponds to the changed scenario during the training interview. This allows the interviewee to intensively train on the interview scene that needs improvement.

(Modification of the fourth embodiment)
The variation of the fourth embodiment differs from the fourth embodiment in that the interview support device uses a model-based scenario to train the superior's interview skills. The variation of the fourth embodiment will be described focusing on the differences from the fourth embodiment.

FIG. 27 is a diagram showing an example of machine learning of a learning model in a modified example of the fourth embodiment. The learning model is not limited to a specific machine learning model, but examples of the learning model include a model using a recurrent neural network (RNN) and a long-short-term memory model (LSTM).

In the learning stage, scenario branching data 300 is used to train the learning model. The scenario branching data is learning data related to the branching (selection) of the scenario associated with the scene (state) of the interview, i.e., learning data related to the transition (transition) of the interview scene. In the scenario branching data, the scene before the scenario branches and at least one of the image (facial expression, etc.) and voice (intonation, etc.) of the interviewee are associated with the scene after the scenario branches.

Note that the content of speech in the training interview does not have to be associated with the scenario branching data. In other words, the content of speech in the training interview does not have to be the subject of analysis. Since the content of speech (text data) during the interview is not analyzed, it is possible to improve the interview skills of the interviewee while reducing the possibility of leaking sensitive information and personal information within the company.

In the learning stage, the scenario processing unit 421d uses the scenario branching data 300 to perform learning of the learning model. For example, the scenario processing unit 421d uses the scenario branching data 300-1 to make the learning model learn that the scene "A" before the scenario branches, the analysis results of the interviewer's facial expression "neutral" and voice "neutral", and the scene "B" after the scenario branches are associated with each other. For example, the scenario processing unit 421d uses the scenario branching data 300-4 to make the learning model learn that the scene "B" before the scenario branches, the analysis results of the interviewer's facial expression "angry" and voice "angry", and the scene "E" after the scenario branches are associated with each other. Note that these analysis results do not need to be highly accurate analysis results.

Figure 28 is a diagram showing an example of a model-based scene transition of a scenario in a modified example of the fourth embodiment. In the inference stage (during the training interview) after the learning stage, the scenario processing unit 421d inputs at least one of an image and a voice of the superior during the training interview into the learning model (scenario branching model). The scenario processing unit 421d acquires the output of the learning model. The scenario processing unit 421d changes the scenario based on the output of the learning model.

For example, if the analysis result of the interviewer's facial expression "neutral" and voice "neutral" in the interview scene "A" is obtained, the scenario processing unit 421d determines that the interview scene will transition to "B" based on learning using the scenario branching data 300-1. The scenario processing unit 421d selects a scenario according to the interview scene "B" after the transition. For example, if the analysis result of the interviewer's facial expression "angry" and voice "angry" in the interview scene "B" is obtained, the scenario processing unit 421d determines that the interview scene will transition to "E" based on learning using the scenario branching data 300-4. The scenario processing unit 421d selects a scenario according to the interview scene "E" after the transition. In this way, in the modified example of the fourth embodiment, the transition of the interview scene is managed by the scenario processing unit 421d using the learning model. The scenario processing unit 421d selects the current scenario according to the current interview scene.

As described above, in the learning stage, the scenario processing unit 421d uses the scenario branching data 300 to perform learning of the learning model. In the inference stage, the scenario processing unit 421d inputs at least one of an image and a voice of the superior during the training interview into the learning model. The scenario processing unit 421d acquires the output of the learning model. The scenario processing unit 421d changes the scenario based on the output of the learning model.

As a result, interview scenes can be managed even if the rules for the scenario's conditional branching are not strictly defined, making it possible to efficiently improve interviewer interview skills before the actual interview. In addition, the branching accuracy (selection accuracy) of the scenario can be improved according to the amount of accumulated scenario branching data (amount of learning using scenario branching data).

Fifth Embodiment
The fifth embodiment differs from the first to fourth embodiments in that at least one of advice and warning is provided to the interviewer depending on the amount of speech of each of the interviewer and interviewee during the actual interview. The fifth embodiment will be described focusing on the differences from the first to fourth embodiments.

In an interview, it is often preferable for the interviewee (subordinate) to speak more than the interviewer (superior). However, the interviewer tends to speak more than the interviewee. For this reason, the interviewer himself needs to be aware during the interview whether the interviewee is speaking more than the interviewer. In such cases, the interviewer may become distracted if he becomes aware of the amount of speech he or she is making during the interview. If the interviewer becomes distracted, the quality of the interview may decline.

In the fifth embodiment, the interview support device analyzes the amount of speech of the interviewer and the amount of speech of the interviewee in real time during the actual interview. The interview support device displays advice and warnings, etc., for the interviewer on the display unit of the interviewer device during the actual interview according to the amount of speech of each of the interviewer and interviewee. This allows the interviewer to concentrate on the interview and easily understand the amount of speech of the interviewer. In other words, since the interviewer can easily notice that the interviewer has a large amount of speech (the interviewer's speaking time is long), it is possible to improve the interview skill of the interviewer (improve the interviewer's listening attitude). The interview support device may also display advice and warnings, etc., for the interviewer on the display unit of the interviewer device during the training interview according to the amount of speech of each of the interviewer and interviewee.

Figure 29 is a diagram showing an example of the configuration of an interview support system 1e. The interview support system 1e includes an interviewee device 2, an interviewer device 3, an interview support device 4e, and a communication line 5.

The interview support device 4e is an information processing device such as a server. The interview support device 4e may be realized by cloud computing. The interview support device 4e includes a communication unit 40, a storage unit 41, and a control unit 42e. The control unit 42e includes a selection acquisition unit 420, a scenario processing unit 421e, an information output unit 422, a goal processing unit 423, an analysis unit 424e, a score processing unit 425, a feedback processing unit 426e, a questionnaire processing unit 427, and a diagnosis unit 428.

The analysis unit 424e (image and audio analysis unit) has a voice estimation processing model. This voice estimation processing model is a trained model that executes a predetermined estimation process (analysis process) on voice data. The analysis unit 424e analyzes the analysis target in the voice of the superior (interviewee) in real time. The analysis unit 424e analyzes, for example, the amount of speech of the superior. The analysis unit 424e analyzes the analysis target in the voice of the subordinate (interviewee) in real time. The analysis unit 424e analyzes the amount of speech of the interviewer and the amount of speech of the interviewee.

The analysis unit 424e may analyze the analysis target in the voice of the superior in real time based on the result of image recognition of the image of the superior. For example, the analysis unit 424e may derive the amount of speech of the superior based on the length of time that the superior's mouth is moving. The analysis unit 424e may analyze the analysis target in the voice of the subordinate in real time based on the result of image recognition of the image of the subordinate. For example, the analysis unit 424e may derive the amount of speech of the subordinate based on the length of time that the subordinate's mouth is moving. Since the amount of speech is analyzed based on the image at the time of the interview (the content of the speech at the time of the interview is not analyzed), it is possible to analyze the amount of speech while reducing the possibility of leaking sensitive information and personal information within the company.

Figure 30 is a diagram showing an example of the duration of audio in the fifth embodiment. The horizontal axis in Figure 30 represents time. The time (predetermined period) from time "t0" to time "t5" illustrated in Figure 30 is, for example, a fixed cycle. The duration of this fixed cycle is, for example, determined in advance. Time "t1" represents the start time of audio 400 in the voice data of the superior. Time "t2" represents the end time of audio 400 in the voice data of the superior. Time "t3" represents the start time of audio 401 in the voice data of the subordinate. Time "t4" represents the end time of audio 401 in the voice data of the subordinate.

The analysis unit 424e derives the speaking duration of the superior by subtracting a predetermined first margin time from the speech duration of the superior from time "t0" to time "t5". Similarly, the analysis unit 424e derives the speaking duration of the subordinate by subtracting a predetermined second margin time from the speech duration of the subordinate. As an example, the speaking duration of each voice from time "t0" to time "t5" is expressed as shown in equation (1).

Speech duration = Audio duration - Margin duration … (1)

The audio time length in equation (1) can be expressed, for example, as equation (2).

Audio duration = audio end time - audio start time … (2)

The margin time length in formula (1) is expressed as formula (3), for example. Note that the margin time length of at least one of the start and end points may be 0.

Margin time length = margin time length at the beginning of the audio + margin time length at the end of the audio … (3)

The analysis unit 424e derives the boss's speaking duration by subtracting the margin time length in formula (3) from the boss's speech duration from time "t0" to time "t5". That is, the analysis unit 424e derives the boss's speaking duration by subtracting the margin time length at the start of the speech and the margin time length at the end of the speech from the speech duration of speech 400. If the boss speaks multiple times, the analysis unit 424e accumulates (sums up) the boss's speaking duration from time "t0" to time "t5". In this way, the analysis unit 424e derives the accumulated result of the boss's speaking duration.

The analysis unit 424e derives the speaking duration of the subordinate by subtracting the margin time length of formula (3) from the speech duration of the subordinate from time "t0" to time "t5". That is, the analysis unit 424e derives the speaking duration of the subordinate by subtracting the margin time length at the beginning of the speech and the margin time length at the end of the speech from the speech duration of speech 401. If the subordinate speaks multiple times, the analysis unit 424e accumulates (sums up) the speaking duration of each subordinate from time "t0" to time "t5". In this way, the analysis unit 424e derives the accumulated result of each speaking duration of the subordinate.

The feedback processing unit 426e derives the sum of the amount of speech of the superior and the amount of speech of the subordinate from time "t0" to time "t5". That is, the feedback processing unit 426e derives the sum of the speech duration of audio 400 and the speech duration of audio 401. The feedback processing unit 426e derives the ratio of the amount of speech of the superior to the sum of the amount of speech of the superior and the amount of speech of the subordinate. That is, the feedback processing unit 426e derives the ratio of the amount of speech of the superior as shown in formula (4).

Ratio of boss's speech volume = boss's speech volume / (boss's speech volume + subordinate's speech volume) … (4)

The feedback processing unit 426e derives at least one of advice and a warning when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold. The feedback processing unit 426e generates an interview image including at least one of the derived advice and warning when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold. During the actual interview, the display unit 23 displays the interview image transmitted from the communication unit 40. The derived advice and warning are provided to the superior by being displayed on the display unit 33 as part of the interview image.

31 is a diagram showing an example of an interview image in the fifth embodiment. The interview image displayed on the display unit 33 includes a third advice display area 203 when the ratio of the boss's speech volume is equal to or greater than a predetermined threshold. The third advice display area 203 is an area in which at least one of advice and warning is displayed. The advice in the third advice display area 203 is expressed by a character string of advice such as "reduce the amount of speech". The warning in the third advice display area 203 is expressed by a character string of warning such as "you speak a lot". In order to make the character string of advice and warning stand out, the font of at least one of the character strings of advice and warning may be larger than the font of other character strings in the interview image. The warning may be expressed by a predetermined icon image such as the warning image 204. The interview image may include a numerical value (e.g., "70%") representing the ratio of the boss's speech volume. The interview image may include a numerical value (e.g., "20 minutes") representing the boss's speech time in a predetermined period. Note that the output of advice and warnings is not limited to output using images. For example, the speaker 34 may output a sound corresponding to the character string of the advice or warning.

Next, an example of the operation of the interview support device 4e will be described.
FIG. 32 is a diagram (flowchart) showing an example of the operation of the interview support device 4e in the fifth embodiment. The analysis unit 424e analyzes the amount of speech of the superior (step S1001). The analysis unit 424e analyzes the amount of speech of the subordinate (step S1002). The analysis unit 424e or the feedback processing unit 426e derives the ratio of the amount of speech of the superior to the total amount of speech of the superior and the subordinate (step S1003). The feedback processing unit 426e derives advice and warning to be provided to the superior according to the ratio of the amount of speech of the superior to the total amount of speech of the superior and the subordinate. For example, when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold, the feedback processing unit 426e derives advice and warning to be provided to the superior (step S1004). The display unit 33 displays an interview image including at least one of advice and warning during the actual interview (step S1005).

As described above, the analysis unit 424e analyzes the amount of speech of the interviewer and the amount of speech of the interviewee. The feedback processing unit 426e provides at least one of advice and a warning to the interviewer during the interview depending on the analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee.

This allows the interviewer to realize that the interviewee is talking for a long time, which can improve the interviewer's interview skills (improve the interviewer's listening attitude).

The present invention can be applied to a system that supports the improvement of interview (e.g., one-on-one meetings, customer service) skills.

The above describes an embodiment of the present invention in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs that do not deviate from the gist of the present invention.

1a, 1b, 1c, 1d, 1e...interview support system, 2...interviewee device, 3...interviewer device, 4a, 4b, 4c, 4d, 4e...interview support device, 5...communication line, 20...camera, 21...microphone, 22...operation unit, 23...display unit, 24...speaker, 25...storage unit, 26...control unit, 27...communication unit, 30...camera, 31...microphone, 32...operation unit, 33...display unit, 34...speaker, 35...storage unit, 36...control unit, 37...communication unit, 40...communication unit, 41...storage unit, 42a, 42b, 42c...control unit, 50...radio button, 60...operation key image, 70...operation key image, 80...operation key image, 90...operation key image Image, 100... score display area, 110... advice display area, 120... operation key image, 200... analysis result display area, 201... first advice display area, 202... second advice display area, 203... third advice display area, 204... warning image, 300... scenario branch data, 400... voice, 401... voice, 420... selection acquisition unit, 421, 421d, 421e... scenario processing unit, 422... information output unit, 423... goal processing unit, 424a, 424b, 424c, 424d, 424e... analysis unit, 425... score processing unit, 426, 426d, 426e... feedback processing unit, 427... questionnaire processing unit, 428... diagnosis unit

Claims (4)

an analysis unit that analyzes the amount of speech of an interviewer using the interviewer device and the amount of speech of an interviewee using the interviewee device;
a feedback processing unit that provides at least one of a predetermined character string, an icon image, and a sound as at least one of an advice and a warning to a display unit of the interviewer device used by the interviewer while a predetermined interview image is being displayed according to an analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee,
the analysis unit analyzes an amount of speech of the interviewer based on a result of image recognition of the interviewer, and analyzes an amount of speech of the interviewee based on a result of image recognition of the interviewee,
the analysis unit analyzes a cumulative result of the speaking time length of the interviewer in a predetermined period as a speaking amount of the interviewer, and analyzes a cumulative result of the speaking time length of the interviewee in the predetermined period as a speaking amount of the interviewee ,
An interview support device, wherein the speaking time length of the interviewer is derived by subtracting a margin time length at the start of the voice and a margin time length at the end of the voice from the voice time length of the interviewer's voice. The interview support device of claim 1, wherein the feedback processing unit derives a ratio of the amount of speech of the interviewer to the sum of the amount of speech of the interviewee and the amount of speech of the interviewee, and provides at least one of the advice and the warning to the interviewer when the ratio is greater than or equal to a predetermined threshold. An interview support method executed by an interview support device,
an analysis step of analyzing an amount of speech of an interviewer using an interviewer device and an amount of speech of an interviewee using an interviewee device;
a feedback processing step of providing at least one of a predetermined character string, an icon image, and a sound as at least one of an advice and a warning to a display unit of the interviewer device used by the interviewer while displaying a predetermined interview image according to an analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee,
The analyzing step includes analyzing an amount of speech of the interviewer based on a result of image recognition of the interviewer, and analyzing an amount of speech of the interviewee based on a result of image recognition of the interviewee,
The analyzing step includes analyzing a cumulative result of the speaking time length of the interviewer in a predetermined period as a speaking amount of the interviewer, and analyzing a cumulative result of the speaking time length of the interviewee in the predetermined period as a speaking amount of the interviewee ,
The speaking time length of the interviewer is derived by subtracting a margin time length at the start of the voice and a margin time length at the end of the voice from the voice time length of the voice of the interviewer.
Interview support methods. A computer program for causing a computer to function as the interview support device according to claim 1 or 2 .

Images