JP7846274B2 - Job matching methods in augmented reality space - Google Patents

Description

This invention relates to a job matching method in an augmented reality space.
Regarding the method.

Traditionally, various methods have been used for evaluating personnel.

For example, Patent Document 1 discloses a method for predicting an employee's performance based on characteristic information such as aptitude test results and performance information such as evaluations by supervisors.

Japanese Patent Publication No. 2019-164738

However, while the method disclosed in Patent Document 1 can predict the performance of a specific individual based on pre-acquired information, it is difficult to identify outstanding talent from an unspecified pool of individuals.

Therefore, the present invention aims to provide a method for predicting outstanding talent from an unspecified pool of individuals.

In one embodiment of the present invention, a job matching method for evaluating personnel in an augmented reality space is provided, comprising: receiving image data and/or audio data of a personnel displayed in a captured real space; generating evaluation information for the personnel based on the received image data and/or audio data; and displaying the evaluation information together with the personnel in the captured real space.

According to this invention, it is possible to provide a method for predicting outstanding individuals from an unspecified pool of talent.

A system configuration diagram according to the first embodiment is shown. A functional configuration diagram of the server according to the first embodiment is shown. A diagram showing the functional configuration of a user terminal according to the first embodiment is shown. A diagram showing the functional configuration of the job seeker terminal according to the first embodiment is shown. An example of user data according to the first embodiment is shown. An example of job seeker data according to the first embodiment is shown. An example of a flowchart for a job matching method according to the first embodiment is shown. An example of a screen displayed on the recruiter's terminal according to the first embodiment is shown. An example of a flowchart for a job matching method according to the second embodiment is shown. An example of a screen displayed on the recruiter's terminal according to the second embodiment is shown.

<First Embodiment>
A specific example of the job matching method according to the first embodiment of the present invention will be described below with reference to the drawings. However, the present invention is not limited to these examples, and is intended to include all modifications within the meaning and scope of the claims, as indicated by the claims. In the following description, the same elements in the drawings are denoted by the same reference numerals, and redundant descriptions are omitted.

Figure 1 shows a system configuration diagram according to the first embodiment of the present invention. As shown in Figure 1, the system according to this embodiment includes a server terminal 100 and user terminals 200 and recruiter terminals 300 connected to the server terminal 100 via a network such as the Internet. For convenience of explanation, Figure 1 shows representative examples of user terminals 200 and recruiter terminals 300, but any number of user terminals and recruiter terminals can be connected to the network. Furthermore, in this embodiment, the recruiter terminal 300 is assumed to be a smart glasses device worn by the recruiter user, which displays an augmented reality space via a display. However, it is not limited to this, and may also be an information terminal such as a head-mounted display or a smartphone.

The server terminal 100 can provide a service to recruiter users wearing the recruiter terminal 200 that generates evaluation information for specific users from among the unspecified users (hereinafter referred to as "personnel") displayed in the real world, and displays this evaluation information along with the personnel.

For example, a job seeker can view evaluation information for each candidate, captured by the smart glasses' built-in camera, while walking outside, via the smart glasses' built-in display.

For example, a recruiter user can download the service application from the server terminal 100 or another platform to the recruiter terminal 300, and send and receive data related to the service with the server terminal 100 via the application. Alternatively, the recruiter user can send and receive data related to the service by accessing the web application stored on the server terminal 100 using the web browser built into the recruiter terminal 300. The following embodiments will describe, as an example, a method of providing the service via a downloaded application.

The user terminal 200 is envisioned as a terminal related to potential job seekers (described as "personnel" in this embodiment), and may be, for example, a smartphone, tablet, mobile terminal, or other information terminal, or a general-purpose computer such as a workstation or personal computer.

The user terminal 200 is envisioned as a terminal related to potential job seekers (described as "personnel" in this embodiment), and may be a wearable device that can be attached to the user's body, such as smart glasses or a head-mounted display. Alternatively, it may be a smartphone, tablet, mobile terminal, or other information terminal, or a general-purpose computer such as a workstation or personal computer.

Figure 2 shows a functional configuration diagram of the server according to the first embodiment. Note that the illustrated configuration is an example, and other configurations may also exist.

As shown in the diagram, the server terminal 100 is connected to a database (not shown) and constitutes part of the system. The server terminal 100 may be a general-purpose computer such as a workstation or personal computer, or it may be logically implemented through cloud computing.

The server terminal 100 comprises at least a communication unit 110, a storage unit 120, a control unit 130, etc., which are electrically connected to each other via a bus.

The control unit 130 is a computing device that controls the overall operation of the server terminal 100, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication. For example, the control unit 10 is a CPU (Central Processing Unit) and executes programs stored and unpacked in the memory unit 120 to perform various information processing tasks.

The control unit 130 further includes an information receiving unit 131 that receives requests or data from the user terminal 200 or the recruiter terminal 300, an evaluation information generation unit 132 that analyzes image data or voice data of personnel and generates evaluation information of those personnel based on the analyzed image data, etc., and a matching processing unit 133 that processes the matching of recruiters and personnel.

The storage unit 120 includes main memory, which is composed of a volatile storage device such as DRAM (Dynamic Random Access Memory), and auxiliary storage, which is composed of a non-volatile storage device such as flash memory or HDD (Hard Disk Drive). Memory 11 is used as a work area for the processor 10, and also stores the BIOS (Basic Input/Output System) executed when the server terminal 100 starts up, as well as various configuration information. Furthermore, a database (not shown) containing data used for each process may be constructed in the storage unit 120 or outside the server terminal 100.

The memory unit 120 further includes a user data storage unit 121 for storing data related to personnel, a recruiter data storage unit 122 for storing data related to recruiters, and a learning model storage unit 123 for storing learning models generated based on facial or appearance images and/or voice recordings obtained from users.

The communication unit 110 connects the server terminal 100 to the network. The transmitting/receiving unit 13 may also be equipped with Bluetooth® and BLE (Bluetooth Low Energy) short-range communication interfaces.

Furthermore, the data stored or functions implemented by the server terminal 100 can be stored or implemented locally by the storage unit or control unit of the job seeker terminal 300.

Figure 3 shows a functional configuration diagram of a user terminal according to the first embodiment. Note that the illustrated configuration is an example, and other configurations may also be available.

As mentioned above, the user terminal 200 can be various information terminals or general-purpose computers; however, a smartphone will be used as an example below. The user terminal 200 includes at least a communication unit 210, a display/operation unit 220, a storage unit 230, a control unit 240, etc., which are electrically connected to each other via a bus.

The control unit 240 is a computing device that controls the overall operation of the user terminal 200, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication. For example, the control unit 240 is a CPU (Central Processing Unit) and executes programs stored and unpacked in the memory unit 230 to perform various information processing tasks.

The storage unit 240 includes a main memory composed of a volatile storage device such as DRAM (Dynamic Random Access Memory), and an auxiliary storage device composed of a non-volatile storage device such as flash memory or HDD (Hard Disk Drive). The storage unit 240 is used as a work area for the control unit 240, and also stores the BIOS (Basic Input/Output System) executed when the user terminal 2 is started, as well as various setting information.

Furthermore, the storage unit 230 stores various programs, such as application programs. A database (not shown) containing data used for each process may be constructed outside the storage unit 230.

The communication unit 210 connects the user terminal 200 to the network. The transmitting/receiving unit 23 may also be equipped with Bluetooth® and BLE (Bluetooth Low Energy) short-range communication interfaces.

The display operation unit 220 is a user interface used for displaying text, images, etc., in response to input data from the control unit 240, based on instructions from the user. If the terminal is a personal computer, it consists of a display and a keyboard or mouse; if it is a smartphone or tablet, it consists of a touch panel, etc. Through the display operation unit, the user can input predetermined information by pressing keys on the keyboard, moving the cursor with a mouse, or performing taps, swipes, pinches, etc., on a touch panel.

Figure 4 shows a functional configuration diagram of the job seeker terminal according to the first embodiment. Note that the illustrated configuration is an example, and other configurations may be used.

As mentioned above, the job seeker terminal 300 can be various information terminals or general-purpose computers. Below, we will describe it using smart glasses, a glasses-like device wearable by the job seeker user, as an example, where the glasses portion acts as a display for augmented reality. The job seeker terminal 300 includes at least a communication unit 310, a display 320, a storage unit 330, a camera 350, a control unit 360, etc., which are electrically connected to each other via a bus.

The control unit 360 is a computing device that controls the overall operation of the job seeker terminal 300, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication. For example, the control unit 360 is a CPU (Central Processing Unit) and executes programs stored and unpacked in the memory unit 330 to perform various information processing tasks.

The memory unit 330 includes a main memory composed of a volatile storage device such as DRAM (Dynamic Random Access Memory), and an auxiliary memory composed of a non-volatile storage device such as flash memory or HDD (Hard Disk Drive). The memory unit 240 is used as a work area for the control unit 240, and also stores the BIOS (Basic Input/Output System) executed when the user terminal 2 is started, as well as various setting information.

Furthermore, the storage unit 330 stores various programs, such as application programs. A database (not shown) containing data used for each process may be constructed outside the storage unit 330.

The communication unit 310 connects the job seeker terminal 300 to the network. The transmitting/receiving unit 23 may also be equipped with Bluetooth® and BLE (Bluetooth Low Energy) short-range communication interfaces.

The display 320 is a user interface used to display the real-world space captured by the camera 350 (described later), and is capable of overlaying data such as text and images onto the real-world space. In the case of smart glasses, the display 320 can be, for example, an LCD or an organic EL display, and the display 320 can also function as a viewfinder for the camera 350.

Furthermore, the job seeker terminal 300 is equipped with a camera 350. The camera 26 is built into the job seeker terminal 300, captures images of the real world, and is typically positioned to align with the line of sight of the job seeker user wearing the terminal 300.

The image of the real world captured by the camera 350 is combined with data such as text and images by an image control unit (not shown), and the text and images are superimposed on the image of the real world on the display 320. The image control unit is, for example, a GPU (Graphics Processing Unit) and primarily performs calculations related to image processing. The job seeker terminal 300 is also equipped with a microphone 370. The microphone 370 is built into the job seeker terminal 300 and can collect sound from the surrounding area.

In addition, although not shown in the diagram, the job seeker terminal 300 may be equipped with a GPS (Global Positioning System) to determine its current location. The GPS receives positioning radio waves from GPS satellites and can determine latitude and longitude information, which constitutes the absolute position. It can also receive communication radio waves from multiple mobile phone base stations and determine the current location using a multi-base station positioning function. Furthermore, the job seeker terminal may be equipped with other sensors, such as an accelerometer and a geomagnetic sensor, although these are not shown in the diagram.

Figure 5 shows an example of user data stored in the server terminal 100.

The user data 1000 shown in Figure 5 stores various data related to personnel. In Figure 5, for the sake of explanation, an example of one personnel (a person identified by user ID "10001") is shown, but information on multiple personnel can be stored. Various data related to personnel may include, for example, image data associated with the personnel (hereinafter, "image data" includes moving image data), audio data, evaluation information (information related to evaluations showing the market value of the personnel in the job or転職 market), and other additional information (personnel's name, contact information, profile (hobbies, special skills, preferences, etc.), annual income, performance (achievements) in current or past workplaces, work history, career history, job or転職 activity status (currently job or転職, employed or転職 completed, etc.), and information on aptitude (vitality, interpersonal skills, teamwork, creative thinking ability, problem-solving ability, etc.)). The above additional information can be stored as user data 1000 when the server terminal 100 receives information input or selected by the personnel via the user terminal 200, or information managed by applications such as SNS. Furthermore, a learning model generated using image data such as facial images or appearance images, or audio data associated with personnel, acquired as user data 1000 and/or entered by recruiters, and output information managed as evaluation information and/or additional information, can be stored in the learning model storage unit 123. Based on this learning model, evaluation information can also be generated based on image data and/or audio data associated with new personnel.

Figure 6 shows an example of job seeker data stored on the server terminal 100.

The recruiter data 2000 shown in Figure 6 stores various data related to recruiters. In Figure 6, for the sake of explanation, an example of one recruiter (a recruiter identified by recruiter ID "20001") is shown, but information for multiple users can be stored. Various data related to recruiters can be stored, for example, basic information of the recruiter (company name, industry, date of establishment, location, number of employees, URL of the company's website, other company introduction text, etc.), and job information (desired candidate profile, working hours, number of annual holidays, treatment (salary, etc.), welfare benefits, work location, contract type (employment (regular employee, contract employee), outsourcing, etc.)). In addition, various test results and contents can also be stored, such as aptitude test scores (scores of aptitude tests for employees belonging to the recruiting company (company-wide, by department, etc.), scores for characteristics measured by aptitude tests (problem-solving ability, vitality, interpersonal skills, teamwork, management ability, stress factors, hierarchical relationships, etc.)). This information can be used for job matching based on more specific information related to the personnel, such as the additional information mentioned above.

<Processing Flow>
Referring to Figure 7, the processing flow of the matching method between personnel and job seekers executed by System 1 of this embodiment will be explained. Figure 7 is an example of a flowchart relating to the matching method according to the first embodiment of the present invention.

To use this system 1, at least the recruiter (or an employee of the recruiting company) must access the server terminal 100 using their respective web browser or application on the recruiter terminal 300 (or the terminal of each employee belonging to any department of the recruiting company). If using the service for the first time, they must enter their basic recruiter information. If they already have an account, they must log in after undergoing the prescribed authentication, such as entering their ID and password, to access the service. After this authentication, a predetermined user interface is provided on the display of the recruiter terminal 300 via a website or application, and the process proceeds to step S101 shown in Figure 7.

First, as part of step S101, the server terminal 100 receives image data and/or audio data from the recruiter terminal 300 via the communication unit 110. For example, when a recruiter user wears the recruiter terminal 300 and positions one or more candidates within their field of view via the display 320, image data captured by the recruiter terminal 300's camera 350 is transmitted to the server terminal 100. Additionally, audio data of the candidates, collected by the microphone 370 built into the recruiter terminal 300, is transmitted to the server terminal 100.

Next, in step S102, the evaluation information generation unit 132 of the control unit 130 of the server terminal 100 analyzes the received image data and/or audio data. Specifically, the evaluation information generation unit 132 extracts the image or audio features of each individual contained in the image data and/or audio data. In extracting the features, known algorithms, such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients), are used to detect feature points from the image data and/or audio data. For images, the feature quantities are calculated from the pixel values or derivatives of the feature points, and for audio, from the frequencies of the feature points.

Next, as part of the processing in step S103, the evaluation information generation unit 132 of the control unit 130 of the server terminal 100 generates personnel evaluation information based on the analyzed image data and/or audio data. Specifically, the evaluation information generation unit 132 uses the calculated feature quantities as input data, and generates evaluation information based on a learning model generated using the input data, which consists of image data or audio data associated with multiple personnel stored in the learning model storage unit 123, and the output data, which consists of evaluation information and/or additional information managed as learning data. Here, evaluation information refers to information that measures the personnel's value in the job or転職 market. For example, it could be evaluation information based on annual income, such as "personnel with an annual income of 8.5 million yen" or "personnel with an annual income of 10 million yen," or evaluation information based on performance, such as "high performer" or "low performer," based on personnel belonging to a specific department of a company. System 1 generates a learning model that takes data related to multiple sample individuals, such as image data and/or audio data of those individuals, as input data, and outputs information related to the evaluation of those individuals, such as their job title, affiliated company, annual income, performance at their affiliated company, position, and aptitude test results. Based on this learning model and the features of newly captured image data and/or audio data of those individuals, the system can infer evaluation information of those individuals, such as their expected annual income, performance, and/or position. In this way, evaluation information such as annual income, performance, and position of similar individuals can be inferred based on the height, appearance, and movements of individuals captured as image data.

Furthermore, in this step, as another example, by analyzing the facial image data and/or voice data of a candidate based on a learning model generated from previously acquired facial image data and/or voice data of other users different from the candidate, such as high performers belonging to the recruiting company, and aptitude data such as competencies (e.g., vitality, interpersonal skills, teamwork, creative thinking ability, problem-solving ability, etc.), the candidate's competencies can be estimated (for example, by evaluating each item such as vitality, interpersonal skills, teamwork, creative thinking ability, problem-solving ability, etc., on a 10-point scale). Based on the estimated competencies, the degree of similarity can be determined by comparing them with the competencies of high performers belonging to the recruiting company, thereby calculating the degree of match with the recruiting company. The evaluation information for each of the estimated competencies for the candidate can then be displayed on the recruiter terminal 300's display 320 along with the candidate, and the degree of match can also be displayed.

Next, in step S104, the evaluation information generation unit 132 processes the generated evaluation information to display it on the display 320 of the recruiter terminal 300. Specifically, the evaluation information generation unit transmits the generated evaluation information (for example, information such as "annual salary level is 8.5 million yen" as the market value of the person) to the recruiter terminal 300, and the recruiter terminal 320 processes the display of the evaluation information along with the person in the real world, who has been captured by the camera 350 and displayed on the display 320.

Figure 8 shows an example of evaluation information displayed on the recruiter terminal 300. The display 320 shows the real-world space captured by the recruiter terminal 300's camera 350, representing the recruiter user's field of view. Near each individual candidate positioned within this real-world space, their evaluation information, such as "Market Value: 10 million yen," is displayed.

As described above, according to this embodiment, job seekers can find candidates who match their desired conditions in everyday spaces where unspecified individuals come and go, such as street corners, and can approach potential candidates early on.

<Second Embodiment>
Figure 9 shows an example of a flowchart of a job matching method according to the second embodiment. This embodiment relates to a function that stores information about personnel or job postings in advance, generates personnel evaluation information based on personnel image data or voice data, extracts personnel that match the job postings, and displays them in an identifiable manner to the job posting user. The functional configurations of the server terminal 100, user terminal 200, and job posting terminal 300 are substantially the same as those described in the first embodiment, so their description is omitted.

First, as part of step S201, the server terminal 100 receives image data and/or audio data from the recruiter terminal 300 via the communication unit 110. For example, when a recruiter user wears the recruiter terminal 300 and positions one or more candidates within their field of view via the display 320, image data captured by the recruiter terminal 300's camera 350 is transmitted to the server terminal 100. Additionally, audio data of the candidates, collected by the microphone 370 built into the recruiter terminal 300, is transmitted to the server terminal 100.

Next, in step S202, the evaluation information generation unit 132 of the control unit 130 of the server terminal 100 analyzes the received image data and/or audio data. Specifically, the evaluation information generation unit 132 extracts the image or audio features of each individual contained in the image data and/or audio data. In extracting the features, known algorithms, such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients), are used to detect feature points from the image data and/or audio data, and calculate the feature quantities from the pixel values and derivative values of the feature points.

Next, as part of the processing in step S203, the evaluation information generation unit 132 of the control unit 130 of the server terminal 100 generates evaluation information for personnel based on the analyzed image data and/or audio data. Specifically, the evaluation information generation unit 132 uses the calculated feature quantities as input data, and generates evaluation information based on a learning model generated using the input data, which consists of image data or audio data associated with multiple personnel stored in the learning model storage unit 123, and the output data, which consists of evaluation information and/or additional information managed as learning data. Here, evaluation information refers to information that measures the personnel's value in the job or転職 market. For example, it can be evaluation information such as "personnel with an annual income of 8.5 million yen" or "personnel with an annual income of 10 million yen" based on annual income, or evaluation information such as "high performer" or "low performer" based on performance, using personnel belonging to a specific department of a company as a standard. System 1 generates a learning model that takes data related to multiple sample individuals, such as image data and/or audio data of those individuals, as input data, and outputs information related to the evaluation of those individuals, such as their job title, affiliated company, annual income, performance at their affiliated company, and position. Based on this learning model and the features of newly captured image data and/or audio data of those individuals, the system can infer evaluation information of those individuals, such as their expected annual income, performance, and/or position. In this way, evaluation information such as annual income, performance, and position of similar individuals can be inferred based on the height, appearance, and movements of individuals captured as image data.

Next, in step S204, the evaluation information generation unit 132 performs a matching process between personnel and employers based on the generated evaluation information and the information stored as user data 1000 or employer data 2000. For example, if the evaluation information generated in step S203 is "This person has an annual salary of 9.5 million yen or more" and "This person has an annual salary of 10 million yen or more," the evaluation information generation unit 132 refers to the employer data 2000 and extracts personnel that match the employer based on the expected annual salary (salary) included in the job information. For example, if the expected annual salary included in the job information is in the range of "9 million yen or more and 9.5 million yen or less," personnel with evaluation information in that range can be extracted. Alternatively, the evaluation information generation unit 132 can refer to the personnel's work history and job-seeking activity status included in the user data 1000, and also refer to the basic information and job information included in the employer data 2000 to extract personnel that match the employer. For example, if the basic information of the job seeker data 2000 includes the industry "IT, Software," and the additional information of the user data 1000 includes work history in the same industry, then such individuals can be extracted. Furthermore, the evaluation information generation unit 132 can refer to the job-seeking activity status of the user data 2000 and extract only individuals who are "actively seeking a new job."

Next, in step S205, the evaluation information generation unit 132 performs a process to display the results of the matching process on the display 320 of the recruiter terminal 300. Specifically, the evaluation information generation unit transmits evaluation information only for the personnel extracted as a result of the matching process (for example, information such as "annual salary level is 9.5 million yen" as the market value of the personnel) to the recruiter terminal 300. The recruiter terminal 320 then performs a process to display the evaluation information along with the personnel in the real world, which are captured by the camera 350 and displayed on the display 320.

Figure 10 shows an example of evaluation information displayed on the recruiter terminal 300. The display 320 shows the real-world space captured by the recruiter terminal 300's camera 350, representing the recruiter user's field of view. Near each individual placed within this real-world space, evaluation information for that individual is displayed, such as "9.5 million yen," "IT consultant," and "currently seeking employment." Here, only individuals matching the recruiter's profile are displayed, allowing recruiters to easily find suitable candidates. Furthermore, by using predetermined icons (e.g., the arrows shown in Figure 10), colors, and visual effects, matching individuals can be displayed in a way that makes them easily identifiable.

As described above, according to this embodiment, by using evaluation information along with information about the personnel and/or the recruiters, the accuracy of matching personnel with recruiters is improved, and recruiter users can more easily find personnel who match their desired conditions and approach potential personnel earlier.

Furthermore, as a variation, this embodiment can also be applied by acquiring user facial image data and/or audio data displayed via a video conferencing application or the like on a recruiter terminal 300 such as a PC or smartphone. Specifically, for candidate users in a job interview setting, additional information such as name, profile, annual income, performance, work history, career history, and aptitude is acquired in advance. By analyzing the candidate user's facial image data and/or audio data based on other users' facial images and/or audio data, and a learning model generated based on the additional information, evaluation information can be generated, and matching with recruiters can be performed. In this way, evaluation information can be generated not only in augmented reality space, but also based on user facial image data and/or audio data acquired through recruiter terminals such as PCs and smartphones.

The embodiments described above are merely illustrative to facilitate understanding of the present invention and are not intended to limit its scope. The present invention may be modified and improved without departing from its spirit, and it goes without saying that the present invention includes equivalents thereof.

100 Server terminals, 200 User terminals, 300 Job seeker terminals

Claims (5)

A method for evaluating candidates displayed via an application on a recruiter user's recruiter terminal, which is executed by a server terminal,
The control unit of the server terminal is:
The recruiter's terminal captures image data and/or audio data of the candidate, which is received from the recruiter's terminal.
A method for generating evaluation information to measure the talent value of the candidate in the job or career change market based on the received image data and/or audio data of the candidate, by inferring the evaluation of the candidate based on a learned model, using image data and/or audio data of multiple other candidates different from the candidate as input data, and evaluation data including any of the annual income, performance, work history, or career history of the multiple candidates as output data. The control unit,
The method according to claim 1, comprising receiving information from the recruiter's terminal regarding the candidate's name, profile, annual income, performance, work history, career, and suitability, and inferring an evaluation of the candidate based on any of the aforementioned information. The control unit,
Receiving job information from the aforementioned recruiter terminal,
The method according to claim 1, wherein a determination is made as to whether the candidate matches the conditions, based on the conditions included in the job information and the generated evaluation information. The method according to claim 1, characterized in that the aforementioned evaluation information is based on annual income. The method according to claim 1, characterized in that the evaluation information is performance-based information.