JP7280705B2 - Machine learning device, program and machine learning method - Google Patents

Description

The present invention relates to a machine learning device, program, and machine learning method.

A classification system that assigns an in-house classification to publications using a learning model is known (for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP 2018-026119 A

In the classification system of Patent Literature 1, the types of sample data to be input to the classification system are specified in advance, and the in-house classification is assigned to all of the input sample data. Therefore, when sample data includes data to be classified and data not to be classified, there are problems such as reduction in classification accuracy and reduction in utilization efficiency of system resources.

In a first aspect of the invention, a machine learning device is provided. For example, the above-described machine learning device includes (i) an evaluation regarding an evaluation target, and (ii) one or more descriptions indicating the status of the evaluation target or the reason for the evaluation. Constructing a first learning model for determining whether or not an input sentence is a sentence indicating an evaluation for an evaluation target, a condition of an evaluation target, or a reason for the evaluation, using the explanation text as teacher data. 1 model construction unit. The above-described machine learning device includes, for example, an extraction unit that extracts sentences related to an evaluation target from among one or more sentences included in text data using the first learning model constructed by the first model construction unit. Prepare.

In the machine learning device described above, the first learning model converts the input sentence to either a sentence indicating the state of the evaluation target or the reason for the evaluation, or a sentence that does not indicate the state of the evaluation target or the reason for the evaluation. may include a sentence classifier that classifies into In the machine learning device described above, the first learning model may include a plurality of sentence classifiers. In the machine learning device described above, each of the plurality of sentence classifiers may output a score indicating the likelihood that the input sentence is the sentence indicating the state of the evaluation target or the reason for the evaluation. In the machine learning device described above, the extraction unit converts the input sentence to a sentence related to the evaluation target when the total score output by each of the plurality of sentence classifiers is greater than a predetermined threshold. can be extracted as

In the machine learning device described above, the extracting unit inputs at least a part of one or more sentences included in the text data to the first learning model, and the first learning model extracts sentences indicating the state to be evaluated or the reason for the evaluation. It may have a first extraction unit that extracts the sentence determined to be the sentence related to the evaluation target. In the machine learning device described above, the extraction unit may have a condition acquisition unit that acquires information indicating a keyword or key phrase related to the evaluation target. In the machine learning device described above, the extraction unit selects, from among one or more sentences included in the text data, a sentence containing a keyword, a sentence matching a key phrase, a sentence containing words similar to the keyword, and a key phrase. It may have a second extraction unit that extracts at least one of the sentences that match the similar condition as a sentence related to the evaluation target or a sentence candidate related to the evaluation target. In the machine learning device described above, the first extraction unit may input the sentence extracted by the second extraction unit as a sentence candidate related to the evaluation target to the first learning model. In the machine learning device described above, the first extraction unit may extract a sentence determined by the first learning model as a sentence indicating the state of the evaluation target or the reason for the evaluation as a sentence related to the evaluation target.

In the machine learning device described above, the second extraction unit extracts a sentence containing two or more consecutive sentences, a sentence containing the keyword, a sentence matching the key phrase, a sentence containing a word similar to the keyword, and a key A sentence containing at least one sentence that matches the condition similar to the phrase may be extracted as a sentence candidate related to the evaluation target. The machine learning device described above may include a type information acquisition unit that acquires type information for distinguishing types of text data. In the machine learning device described above, the second extraction unit selects, from among one or more sentences included in the text data, a sentence containing a keyword, a sentence matching a key phrase, based on the type of text data indicated by the type information. , at least one of a sentence containing words similar to the keyword and a sentence similar to the key phrase matching a condition is extracted as either a sentence related to the evaluation object or a sentence candidate related to the evaluation object. You can decide.

The above-described machine learning device may include a second model building section that builds a second learning model for giving an evaluation regarding an evaluation target to an input sentence using evaluation information as teacher data. The above machine learning device may include an evaluation assigning unit that assigns an evaluation regarding an evaluation target to the sentence extracted by the extracting unit using the second learning model constructed by the second model constructing unit.

The machine learning device described above may include an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit. The above machine learning device may include a period information acquisition unit that acquires information indicating a specific period. The above-mentioned machine learning device, each of the plurality of text data, the time associated with the content of each of the plurality of text data, the time when each of the plurality of text data was recorded, or each of the plurality of text data A text data acquisition unit may be provided that acquires the text data in association with time information indicating the time when the electronic file was created or updated. In the above-described machine learning device, the extracting unit selects from among a plurality of sentences included in text data in which the time indicated by the time information associated with the text data is included in a specific period, out of the plurality of text data. , may extract sentences related to the evaluation target. In the machine learning device described above, the evaluation imparting unit may impart an evaluation regarding the evaluation target to the plurality of sentences extracted from at least part of the plurality of text data by the extracting unit. In the machine learning device described above, the index calculation unit may calculate the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.

In the machine learning device described above, the text data may include information presented by an information provider's utterances or gestures, or information perceived by the information provider. The machine learning device described above may include an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit. The above machine learning device may include a text data acquisition unit that acquires each of the plurality of text data in association with attribute information indicating the attribute of the information provider of each of the plurality of text data. In the machine learning device described above, the evaluation imparting unit uses the attribute information corresponding to the text data in which each sentence is included for each of the plurality of sentences extracted from at least a part of the plurality of text data by the extraction unit. Based on the attribute of the information provider shown, an evaluation regarding the evaluation target may be given. In the machine learning device described above, the index calculation unit may calculate the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.

In a second aspect of the invention, a machine learning method is provided. The above machine learning method includes, for example, (i) an evaluation regarding an evaluation target, and (ii) one or more descriptions indicating the status of the evaluation target or the reason for the evaluation are associated with the evaluation information. Constructing a first learning model for determining whether or not an input sentence is a sentence indicating a state of an evaluation target, an evaluation for an evaluation target, or a reason for the evaluation, using the explanation text as teacher data. 1 model building stage. In the above machine learning method, for example, using the first learning model constructed in the first model construction stage, an extraction stage of extracting sentences related to the evaluation target from among one or more sentences included in the text data. have

In a third aspect of the invention, a program is provided. A non-transitory computer-readable medium storing the above program may be provided. The program is, for example, a program for causing a computer to function as the machine learning device according to the first aspect. The program may be a program for causing a computer to execute the machine learning method according to the second aspect.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

1 schematically shows an example of a system configuration of an index estimation system 100; An example of the internal configuration of the storage unit 126 is shown schematically. An example of a data table 300 is shown schematically. An example of a data table 400 is shown schematically. An example of the internal configuration of the model construction unit 144 is shown schematically. An example of an internal configuration of an index estimation unit 166 is schematically shown. An example of the internal configuration of the machine learning type extraction unit 634 is shown schematically. An example of information processing in the evaluation target extraction unit 630 is schematically shown. An example of information processing in the evaluation target extraction unit 630 is schematically shown. An example of information processing in the evaluation target extraction unit 630 is schematically shown. An example of a data table 1100 is shown schematically. An example of the system configuration of the computer 3000 is shown schematically.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention. In addition, in the drawings, the same or similar parts may be denoted by the same reference numerals to omit redundant description.

[Outline of index estimation system 100]
FIG. 1 schematically shows an example of a system configuration of an index estimation system 100. As shown in FIG. In the present embodiment, the index estimation system 100 includes (i) information registered on various SNSs, information registered on various bulletin boards on networks, information reported on various news, and various questionnaires on business sentiment. (ii) output estimated values of economic indicators; As a result, the user of the indicator estimation system 100 can predict the state or trend of economic activity with accuracy equivalent to that of economic indicators published by public institutions such as governments and central banks, and earlier than the time of publication of the relevant economic indicators. can grasp.

Economic indicators published by public institutions such as governments and central banks are, for example, the results of surveys on economic activities published by public institutions such as governments and central banks (sometimes referred to as official survey results). listed in The above economic indicators include various indexes listed in the Bank of Japan's "Short-term Economic Survey of Enterprises," various indexes listed in the Cabinet Office's "Economy Watchers Survey," and the Ministry of Economy, Trade and Industry's "Current Production Statistics Survey." ” are exemplified. An example of an index included in the "Short-term Economic Survey of Enterprises" is the "Business Conditions Index" (sometimes referred to as the Bank of Japan Tankan). As an index described in the "Economy Watchers Survey", "Current situation judgment" is exemplified. Various "indices of mining and manufacturing" are exemplified as indices described in the "current production statistics survey." Another example of economic indicators is the economic trend index published by the Cabinet Office.

In this embodiment, the index estimation system 100 includes a communication unit 122, an input/output unit 124, a storage unit 126, a request reception unit 128, a teacher data acquisition unit 142, a model construction unit 144, and a sample data acquisition unit. 162 , a text data generator 164 and an index estimator 166 . In this embodiment, the index estimation system 100 can transmit and receive information to and from at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16 via the communication network 10.

In this embodiment, the communication network 10 transmits information between at least one of the user terminal 12 , the teacher data providing server 14 and the sample data providing server 16 and the index estimation system 100 . The communication network 10 may be a wired communication transmission line, a wireless communication transmission line, or a combination of a wireless communication transmission line and a wired communication transmission line.

Communication network 10 may include wireless communication networks, the Internet, P2P networks, leased lines, VPNs, power line communication lines, and the like. The communication system in the wireless communication network may be (i) a mobile communication system such as 3G system, LTE system, 4G system, or 5G system, (ii) Bluetooth (registered trademark), Zigbee (registered trademark), NFC (Near Field Communication), a wireless LAN system such as WiFi (registered trademark), a wireless MAN system such as WiMAX (registered trademark), a wireless data communication system such as a wireless WAN system. good.

In the present embodiment, the user terminal 12 may be an information processing terminal capable of transmitting and receiving information to and from the index estimation system 100 via the communication network 10, and details thereof are not particularly limited. The user terminal 12 can be used by a user of the index estimation system 100 as a user interface of the index estimation system 100 . Examples of the user terminal 12 include a personal computer and a mobile terminal. Examples of mobile terminals include mobile phones, smart phones, PDAs, tablets, notebook or laptop computers, wearable computers, and the like.

In this embodiment, the teacher data providing server 14 manages data of public survey results. For example, for each of a plurality of surveys, the teacher data providing server 14 stores information indicating the type of survey, information indicating the target period of the survey, and survey result data of the survey in association with each other. In response to a request from the index estimation system 100 , the teacher data providing server 14 transmits the survey result data requested by the index estimation system 100 to the index estimation system 100 .

Examples of survey types include the name of the survey, the type of survey target, the purpose of the survey, and the attributes of interviewees in the survey. Names of surveys include short-term corporate economic observation surveys, economic watchers surveys, and current production statistics surveys. Attributes of interviewees in the survey include age, gender, degree of relevance to the survey target, and the like. Examples of the degree of relevance to the survey target include the industry of the organization to which the interviewee belongs, the scale of the organization, the region in which the organization is active, the title of the interviewee in the organization, and the occupation of the interviewee. be done. The title of the person to be interviewed includes, for example, the department to which he/she belongs and the title. Other examples of the degree of relevance to the research target include prediction accuracy regarding the research target, years of experience in work or positions related to the research target, and the like.

The degree of relevance between interviewees and research subjects may be represented by continuous numerical values, or may be represented by gradual divisions. For example, the degree of relevance is set such that the higher the prediction accuracy with respect to the investigation target, the larger the numerical value representing the degree of relevance. The degree of relevance may be set such that the longer the number of years of experience in a job or position related to a survey target, the larger the numerical value representing the degree of relevance. The degree of association may be set such that the higher the rank of the position, the larger the numerical value representing the degree of association.

For example, if the subject of the survey is economic activity, "If you expect the economy to improve next quarter, and it actually does improve," or "If you expect the economy to improve next quarter, However, in reality there was no change in the economic trend," and "I expected the economy to improve next quarter, but it actually deteriorated." given a large value. The degree of relevance for ``If you expect the economy to improve next quarter and if it actually improves'' can be 1 or more, and ``If you expect the economy to improve next quarter. However, even if the degree of association is less than 1 when there is no actual change in the economic trend” and “when the economy is expected to improve next quarter, but the economy actually deteriorates” good.

In this embodiment, the sample data providing server 16 manages various data (sometimes referred to as sample data) to be analyzed by the index estimation system 100 . The sample data may be managed by creator or updater, or may be managed by creator or updater and creation date or update date. For example, if the sample data is data such as a daily business report, daily business report, etc., each time the data of the daily business report, daily business report, etc. is saved, it is handled as one sample data. If the sample data is data such as a daily business report or daily business report, it may be treated as one sample data for each person in charge who created or updated the daily business report or daily business report. If the sample data is data such as a business daily report or business daily report, each person in charge and each business day may be treated as one sample data. For example, when sample data is data posted on SNS, each post is treated as one piece of sample data.

For example, for each of a plurality of sample data, the sample data providing server 16 provides (i) information indicating the creation time or update time of the sample data, and information indicating the creator, updater, or administrator of the sample data. and (ii) the sample data are stored in association with each other. For each of the plurality of sample data, the sample data providing server 16 provides (i) at least information indicating the creation time or update time of the sample data and at least information indicating the creator, updater, or administrator of the sample data One, (ii) information indicating the type of the sample data, and (iii) the sample data may be stored in association with each other. The sample data providing server 16 transmits sample data requested by the index estimation system 100 to the index estimation system 100 in response to a request from the index estimation system 100 .

The sample data may be text data, voice data, image data, or data for any application program. The image data may be still image data or moving image data. Information included in the sample data includes information registered on various SNS, information reported in various news, information collected through various questionnaires, information obtained in negotiations with customers, daily business reports, or Examples include information written in daily business reports.

The sample data may include information presented by the information provider's utterances or gestures, or information perceived by the information provider. Gestures are exemplified by gestures, gestures, gestures, facial expressions, and the like. Examples of information related to utterances, gestures, etc. include (i) information indicating the content of utterances or gestures, and (ii) information indicating a summary or supplementary explanation by the person in charge regarding the utterances or gestures of the information provider. Examples of the supplementary explanation include the background of the above remarks or gestures, the impression of the person in charge, and the like.

For example, if the sample data is a daily business report, a daily business report, etc., the person in charge of the interview work can write information about the speeches, gestures, etc. of the interviewee in the daily business report, daily business report, etc. Create or update business daily reports, etc. The person to be interviewed in this case may be an example of an information provider. For example, if the sample data is data posted on an SNS, the poster posts the information perceived by himself/herself on the SNS. The contributor in this case may be an example of an information provider. Details of the types of sample data will be described later.

[Overview of Each Part of Index Estimation System 100]
In this embodiment, the communication unit 122 transmits and receives information to and from at least one of the user terminal 12 , the teacher data providing server 14 , and the sample data providing server 16 via the communication network 10 . For example, the communication unit 122 transmits various requests to at least one of the user terminal 12 , teacher data providing server 14 , and sample data providing server 16 . The communication unit 122 receives responses to the above requests from at least one of the user terminal 12 , the teacher data providing server 14 , and the sample data providing server 16 .

In one embodiment, the communication unit 122 requests the user terminal 12 to interact with the user of the index estimation system 100 . For example, the communication unit 122 transmits information presented to the user of the index estimation system 100 to the user terminal 12 . The communication unit 122 may transmit information of various input screens. The user terminal 12 presents the information received from the communication unit 122 to the user based on the interaction request. The information presentation mode is not particularly limited. The above information may be displayed or projected as an image, or may be output as audio. The user terminal 12 may request or prompt the user to input information. The user terminal 12 may transmit information input by the user to the communication unit 122 . Thereby, the communication unit 122 can acquire information input to the user terminal 12 by the user of the index estimation system 100 .

In another embodiment, the communication unit 122 requests the teacher data providing server 14 to transmit data related to specific survey results. As a result, the communication unit 122 can acquire various survey result data from the teacher data providing server 14 . The obtained survey result data is used, for example, as teacher data for machine learning in the model construction unit 144 .

In yet another embodiment, the communication unit 122 requests the sample data providing server 16 to transmit specific sample data. Thereby, the communication unit 122 can acquire various sample data from the sample data providing server 16 . The acquired sample data is used as input data for estimation processing in the index estimation unit 166, for example.

In the present embodiment, the input/output unit 124 receives input of information from the user of the index estimation system 100 . The input/output unit 124 presents information to the user of the index estimation system 100 . The input/output unit 124 can be used by the user of the index estimation system 100 as a user interface of the index estimation system 100 . The input/output unit 124 may have input devices such as a keyboard, pointing device, touch panel, microphone, camera, voice input system, and gesture input system. The input/output unit 124 may have an output device such as a display device, a projection device, an audio output device, or a vibration device.

In this embodiment, the storage unit 126 stores various information. The storage unit 126 may store information used in information processing of the index estimation system 100 . The storage unit 126 may store information generated by information processing of the index estimation system 100 . Details of the storage unit 126 will be described later.

In this embodiment, the request receiving unit 128 receives various requests for the index estimation system 100 . For example, the request receiving unit 128 receives a request from the user to register various settings regarding the index estimation system 100 . The request accepting unit 128 may forward the accepted request to an element suitable for processing the request.

Settings related to the indicator estimation system 100 include settings related to the types of indicators estimated by the indicator estimation system 100, various settings related to machine learning in the indicator estimation system 100, and various settings related to sample data input to the indicator estimation system 100. exemplified. Examples of the types of indicators include various indexes recorded in the Short-term Economic Survey of Enterprises, various indexes recorded in the Economy Watchers Survey, various indexes recorded in the Current Production Statistics Survey, and economic trend indexes. . Examples of settings related to machine learning include settings related to teacher data for machine learning, settings related to learning models, and the like. Examples of settings related to teacher data include the URI of data and data format. Examples of settings related to sample data include data URI, data format, and the like.

In one embodiment, the request accepting unit 128 accepts requests from users of the index estimation system 100 . The request receiving unit 128 may acquire requests from users via the user terminal 12 or the input/output unit 124 . In another embodiment, the request receiving unit 128 may receive requests from at least one of the user terminal 12 , teacher data providing server 14 and sample data providing server 16 .

In this embodiment, the teacher data acquisition unit 142 requests the teacher data providing server 14 to transmit specific survey result data. The survey result data is specified, for example, by the type of survey and the target period of the survey. As a result, the teacher data acquiring unit 142 can acquire data of specific survey results from the teacher data providing server 14 .

In one embodiment, the teacher data acquisition unit 142 stores the acquired survey result data in the storage unit 126 . In another embodiment, the teacher data acquisition unit 142 may output the acquired survey result data to the model construction unit 144 .

In the present embodiment, the model construction unit 144 constructs a learning model used in the index estimation unit 166 by using the survey result data acquired by the teacher data acquisition unit 142 as teacher data. Details of the model construction unit 144 will be described later.

In this embodiment, the sample data acquisition unit 162 requests the sample data providing server 16 to transmit specific sample data. Thereby, the communication unit 122 can acquire various sample data from the sample data providing server 16 . The sample data acquisition unit 162 obtains each of the one or more sample data at a time related to the contents of each sample data, at a time when each sample data was recorded, or at a time when an electronic file containing each sample data was created or updated. It may be acquired in association with the information indicating the time. The sample data acquisition unit 162 may acquire each of the one or more pieces of sample data in association with information indicating the attribute of the information provider of each piece of sample data.

Sample data includes, for example, (i) the type of sample data, (ii) the time when the sample data was created or updated, or the range of time It is sometimes referred to as.) and is specified by. The type of sample data is specified, for example, by the URL of the sample data providing server 16 that manages the sample data. The type of sample data may be identified by identification information for identifying the type. The type of sample data may be identified by at least one of the sample data name, creator, updater, and information provider. The type of sample data may be specified by the attribute of the information provider who provided the information indicating the contents of the sample data. Attributes of the information provider are exemplified by age, gender, degree of relevance to the evaluation target, and the like. A specific example of the attribute of the information provider may be the same as the specific example of the attribute of the interviewee described above.

The time when the sample data was created or updated may be the time when the sample data was recorded in the electronic file storing the sample data, or the time when the electronic file was created or updated. Note that instead of the time when the sample data was created or updated, a time related to the content of the sample data may be used. For example, if the sample data contains information indicating the time period related to the evaluation target, such as "Sales in December 2018 increased by 10% compared to the previous year", the time period is the sample It is used as a period related to the content of the data.

In one embodiment, the sample data acquisition unit 162 stores the acquired sample data in the storage unit 126 . The sample data acquisition unit 162 may store the acquired sample data in the storage unit 126 in association with information for distinguishing the type of the sample data (sometimes referred to as type information). . In another embodiment, the sample data acquiring section 162 may output the acquired sample data to the index estimating section 166 . In yet another embodiment, when the acquired sample data is data in a format other than text data, or when the acquired sample data includes data in a format other than text data, the sample data acquisition unit 162 , the acquired sample data may be output to the text data generator 164 .

In this embodiment, if the sample data acquired by the sample data acquisition unit 162 is data in a format other than text data, or if the sample data includes data in a format other than text data, the text data generation unit 164 generates text data from data in a format other than the text data. The text data generation unit 164 may store text data generated from specific sample data in the storage unit 126 as part of the sample data. The text data generating section 164 may output text data generated from specific sample data to the index estimating section 166 as part of the sample data.

In one embodiment, the text data generation unit 164 generates text data in which human speech included in the sample data is recorded by performing voice recognition processing on the voice data included in the sample data. In another embodiment, the text data generation unit 164 performs image recognition processing on image data included in sample data to generate text data in which characters or sign language included in the image data are recorded. .

In the present embodiment, the index estimation unit 166 uses the sample data acquired by the sample data acquisition unit 162 to output the estimated value of the type of index designated by the user. The type of index to be output is determined based on user settings or initial settings, for example.

Specifically, first, the index estimation unit 166 extracts a plurality of sample data created or updated during a period designated by the user from among the plurality of sample data acquired by the sample data acquisition unit 162 . Next, the index estimation unit 166 obtains a plurality of sentences to be analyzed by dividing each of the plurality of extracted sample data into one or more sentences. Next, the index estimating unit 166 extracts sentences that are highly likely to be related to economic activity from among the plurality of sentences to be analyzed as sentences to be evaluated.

Next, the index estimating unit 166 evaluates the degree of the state or trend of economic activity indicated by each of the sentences to be evaluated, and scores corresponding to the evaluation (referred to as evaluation scores) There is.) is given. Next, the index estimation unit 166 calculates the index by appropriately processing the evaluation score assigned to each sentence to be evaluated according to the type of index specified by the user.

According to this embodiment, the indicator estimation system 100 includes information registered in various SNSs, information reported in various news, information collected through various questionnaires, information obtained in negotiations with customers, etc. It is possible to calculate indicators related to the evaluation of the evaluation target using big data such as information written in business daily reports (sometimes referred to as information written in negotiation records), daily business reports, etc. . Details of the index estimation unit 166 will be described later.

In the present embodiment, the indicator estimation system 100 evaluates economic activity and outputs estimated values of various economic indicators as indicators indicating the state or trend of economic activity. Details of the indicator estimation system 100 was explained. However, the evaluation target of the index estimation system 100 is not limited to economic activities. In another embodiment, the indicator estimation system 100 may evaluate the popularity or name recognition of individuals, groups or corporations, products or services, or areas or landmarks.

In addition, in the present embodiment, the details of the indicator estimation system 100 will be described using, as an example, the case where the indicator estimation system 100 uses, as training data, the results of surveys on economic activities published by public institutions such as governments and central banks. be done. However, the index estimation system 100 is not limited to this embodiment. In another embodiment, the index estimation system 100 may use survey results from private research organizations as teacher data.

The indicator estimation system 100 may be an example of a machine learning device. The communication unit 122 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The input/output unit 124 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The request reception unit 128 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The model builder 144 may be an example of a first model builder and a second model builder. The sample data acquisition unit 162 may be an example of a type information acquisition unit and a text data acquisition unit. The text data generation unit 164 may be an example of a type information acquisition unit and a text data acquisition unit. The index estimation unit 166 may be an example of an extraction unit and an index calculation unit.

Business confidence may be an example of an assessment of economic activity. Economic activity may be an example of an evaluation target. A survey target may be an example of an evaluation target. Public survey results may be an example of evaluation information. Sample data may be an example of text data. An economic indicator may be an example of an indicator. A person to be interviewed may be an example of an information provider.

[Specific configuration of each part of index estimation system 100]
Each unit of index estimation system 100 may be implemented by hardware, software, or a combination of hardware and software. When at least part of the constituent elements of the index estimation system 100 are implemented by software, the constituent elements realized by the software are programs that define the operations of the constituent elements in an information processing device with a general configuration. may be implemented by invoking the

The program may be stored in a computer-readable medium such as a CD-ROM, DVD-ROM, memory, hard disk, etc., or may be stored in a storage device connected to a network. The program may be installed in a computer that constitutes at least part of the index estimation system 100 from a computer-readable medium or a storage device connected to a network. A computer may function as at least part of each unit of the index estimation system 100 by executing the program.

A program that causes a computer to function as at least part of each unit of the index estimation system 100 may include modules that define the operation of each unit of the index estimation system 100 . These programs or modules work on a data processing device, an input device, an output device, a storage device, etc. to cause the computer to function as each part of the index estimation system 100, or instruct the computer to perform information processing in each part of the index estimation system 100. or let it run.

The information processing described in the program functions as concrete means in which the software related to the program and various hardware resources of the index estimation system 100 work together when the program is read into the computer. . Then, the specific means described above realizes the calculation or processing of information according to the purpose of use of the computer in this embodiment, thereby constructing the index estimation system 100 according to the purpose of use.

The above program may be a program for causing a computer to execute a machine learning method. The above machine learning method includes, for example, (i) an evaluation regarding an evaluation target, and (ii) one or more descriptions indicating the status of the evaluation target or the reason for the evaluation are associated with the evaluation information. Constructing a first learning model for determining whether or not an input sentence is a sentence indicating a state of an evaluation target, an evaluation for an evaluation target, or a reason for the evaluation, using the explanation text as teacher data. 1 model building stage. In the above machine learning method, for example, using the first learning model constructed in the first model construction stage, an extraction stage of extracting sentences related to the evaluation target from among one or more sentences included in the text data. have

FIG. 2 schematically shows an example of the internal configuration of the storage unit 126. As shown in FIG. In this embodiment, the storage unit 126 includes a setting information storage unit 222 , a sample data storage unit 226 , a teacher data storage unit 224 and a model information storage unit 228 .

In this embodiment, the setting information storage unit 222 stores information indicating settings regarding the index estimation system 100 received by the request receiving unit 128 . Examples of the above settings include settings related to the types of indicators estimated by the indicator estimation system 100, various settings related to machine learning in the indicator estimation system 100, and various settings related to sample data input to the indicator estimation system 100. .

In this embodiment, the teacher data storage unit 224 stores various data acquired by the teacher data acquisition unit 142 as teacher data used in the model building process of the model building unit 144 . For example, for each of a plurality of pieces of teacher data, the teacher data storage unit 224 associates the data with identification information of the data, at least one of information indicating the type of the data and information indicating the target period of the data. You can store it with

The data type may be an attribute of an information provider who provided information indicating the content of the data. When the teacher data is data of various survey results, the information provider is exemplified by the interviewees in the survey. Where the data is of a particular survey result, examples of data types may be similar to the examples of survey types described above.

In this embodiment, the sample data storage unit 226 stores sample data acquired by the sample data acquisition unit 162 . The sample data storage unit 226 stores (i) identification information of the sample data, (ii) information indicating the type of the sample data, and information indicating the creation time or update time of the sample data for each of the plurality of sample data. , and at least one of information indicating the creator, updater, or administrator of the sample data, and (iii) the sample data may be stored in association with each other. The sample data storage unit 226 may store the text data generated by the text data generation unit 164 together with or instead of the sample data for each of the plurality of sample data.

In this embodiment, the model information storage unit 228 stores various information regarding the learning model constructed by the model construction unit 144 . For example, the model information storage unit 228 stores, for each of a plurality of learning models, identification information of the model, information indicating the algorithm of the model, and information indicating the parameter values of the model in association with each other. The model information storage unit 228 may store other information regarding each of the plurality of learning models.

FIG. 3 schematically shows an example of a data table 300. As shown in FIG. The data table 300 may be an example of teacher data stored in the teacher data storage unit 224 . The data table 300 may be an example of an economy watchers survey covering a specific period.

The Economy Watchers Survey obtains the cooperation of people who are in a position to observe developments closely related to the regional economy (sometimes referred to as interviewees) to accurately and quickly grasp economic trends in each region. The survey is conducted with the aim of providing basic data for assessing economic trends. Survey items in the Economy Watchers Survey include (i) judgments on the current state of the economy, (ii) reasons for the judgments on the current state, (iii) additional explanations regarding the above reasons and explanations of specific conditions, and (iv) economic conditions. Judgment on the future, (v) reasons for the judgment on the future, etc. are exemplified.

In this embodiment, the data table 300 includes information 312 indicating the field of economic activity, information 314 indicating the area to be surveyed, information 316 indicating the industry and occupation of the interviewee, and the judgment of the current state of the economy. Information 322, information 324 indicating the reason for the judgment, and information 326 indicating the additional explanation and explanation of the specific situation are stored in association with each other. Each row of the data table 300 may be an example of evaluation information. Each of the information 312 indicating the field of economic activity, the information 314 indicating the area to be investigated, and the information 316 indicating the industry and occupation of the interviewee may be examples of types of investigation. The information 322 indicating the judgment of the current state of the economy may be an example of an evaluation regarding an evaluation target. The information 324 indicating the reason for the judgment may be an example of a descriptive text indicating the reason for the evaluation. The information 326 indicating the additional explanation and the explanation of the specific situation may be an example of an explanatory text indicating the state of the evaluation target.

Note that the data structure of the data table 300 is not limited to this embodiment. In another embodiment, the data table 300 includes information 312 indicating the field of economic activity, information 314 indicating the area to be investigated, information 316 indicating the industry and occupation of the interviewee, and information indicating the reason for the judgment. 324 may not be provided. In yet another embodiment, the data table 300 includes a column for storing identification information for identifying each row of the data table 300, a column for storing information indicating the type of survey, and a column for storing information indicating the type of survey conducted. It may further comprise at least one column for storing information indicative of when the time was taken.

FIG. 4 schematically shows an example of a data table 400. As shown in FIG. The data table 400 may be an example of sample data stored in the sample data storage unit 226 . In the present embodiment, the details of the data table 400 will be described by taking as an example a case in which a salesperson of a company inputs a business daily report that records the contents of conversations, negotiations, etc. with customers as sample data. However, it should be noted that the sample data is not limited to this embodiment. In this embodiment, the above customer may be an example of a person to be interviewed.

In this embodiment, each row (sometimes called a record) of the data table 400 stores information of a single sentence. For example, if a single sample data contains multiple sentences, the data table 400 divides and stores information about the sample data into multiple records. Information relating to each of the plurality of sentences is stored in each of the plurality of records.

In this embodiment, the data table 400 associates a sample ID 412, a sentence ID 414, information 416 indicating the recording time of each sentence, information 418 indicating the content of each sentence, and information 420 indicating the type of data. store. The sample ID 412 may be any information that can identify each of the plurality of sample data, and its details are not particularly limited. The sentence ID 414 may be information that can identify each of a plurality of sentences, and its details are not particularly limited.

Information 416 indicating recording time indicates the time when each of the plurality of sentences was created or updated. Information indicating content 418 indicates the content of each of the plurality of sentences. The data type information 420 indicates the type of each of the plurality of sentences. Examples of the type of sentence include the type of sample data containing the sentence, the attribute of the information provider who provided the information indicating the content of the sentence, and the like. If the sample data is text data relating to daily business reports, daily business reports, or negotiation records, examples of information providers include customers who were interviewed.

According to the example shown in FIG. 4, information indicating attributes of the information provider is stored as the information 420 indicating the type of data. For example, the sample data acquisition unit 162 analyzes the sentences or each sentence in the business daily report, and the above attributes are given to each sentence. The above attributes may be given to each sentence by the salesperson inputting the sentence of the business daily report or the attribute corresponding to each sentence to the input/output unit 124 .

Note that the data structure of the data table 400 is not limited to this embodiment. In another embodiment, for each of a plurality of sentences, the data table 400 includes a first data table that stores a sample ID 412, a sentence ID 414, and information 418 indicating content in association with each other, and a plurality of sample data. Each may be divided into a second data table that stores the sample ID 412 and the information 416 indicating the recording time in association with each other.

FIG. 5 schematically shows an example of the internal configuration of the model building section 144. As shown in FIG. In this embodiment, the model construction unit 144 includes an extraction model construction unit 522 and an evaluation model construction unit 524 .

As described above, the model building section 144 builds various learning models used in the index estimating section 166 . As described above, the index estimating unit 166 extracts sentences that are highly likely to be related to economic activity from among the plurality of sentences to be analyzed as sentences to be evaluated. In addition, the index estimation unit 166 evaluates the degree of economic activity state or trend indicated by each sentence to be evaluated, and assigns a score corresponding to the evaluation.

In this embodiment, the extraction model construction unit 522 constructs a learning model for extracting sentences to be evaluated from among the plurality of sentences to be analyzed. Specifically, the extraction model construction unit 522 includes (i) the evaluation regarding the evaluation target, and (ii) one or more explanations indicating the state of the evaluation target or the reason for the evaluation in the evaluation information associated with the evaluation information. Building a learning model for judging whether or not an input sentence is a sentence indicating the status of an evaluation target, an evaluation for an evaluation target, or a reason for the evaluation, using one or more explanatory sentences as training data. do.

For example, the extraction model construction unit 522 uses information 326 indicating additional explanations and specific situation explanations of a plurality of records that make up the data table 300 as teacher data, so that the input sentence is (i) economic Build a learning model to determine whether it is an evaluation of an activity, (ii) the state of economic activity, or (iii) a sentence indicating the reason for the evaluation. According to the above learning model, each of the plurality of sentences input to the learning model includes a sentence similar to the sentence included in the information 326 indicating the additional explanation and the specific situation, and the additional explanation and the specific situation. sentences that are not similar to the sentences included in the information 326 indicating the description of Then, a sentence similar to the sentence included in the information 326 indicating the additional explanation and the explanation of the specific situation can be (i) the evaluation of the economic activity, (ii) the state of the economic activity, or (iii) the reason for the evaluation. is determined to be a sentence indicating

The type of learning model described above is not particularly limited. Types of learning models include neural network models (sometimes abbreviated as NN), convolutional neural networks (sometimes abbreviated as CNN), logistic regression models (sometimes abbreviated as LR), Examples include simple word embedding model (sometimes abbreviated as SWEM), long short term memory model (sometimes abbreviated as LSTM), bidirectional LTSM, and the like.

The above learning model converts the input sentence into "(i) the evaluation of economic activity, (ii) the state of economic activity, or (iii) the first sentence indicating the reason for the evaluation", or the "first A sentence classifier may be included that classifies any second sentence that is not the first sentence. The above learning model may include a sentence classifier that classifies into either "sentence indicating the state of the evaluation target or the reason for the evaluation" or "sentence not indicating the state of the evaluation target or the reason for the evaluation". good. The sentence classifier may consist of a sentence embedding generator and a classifier pair. The learning model described above may include multiple sentence classifiers.

The extraction model construction unit 522 may construct a learning model according to the type of sample data. The extraction model construction unit 522 may determine the type of learning model to be used according to the type of sample data. The extraction model construction unit 522 may determine the number of types of learning models to be used according to the types of sample data. The extraction model construction unit 522 may determine the combination of learning models to be used according to the type of sample data. Sample data may be an example of input data for a learning model. Another example of input data is data of one or more sentences included in the sample data.

The extraction model construction unit 522 constructs the above learning model using information in which each of one or more sentences included in the sample data and information indicating the type of each sentence are associated with each other as teacher data. good too. As a result, for example, a learning model is constructed that outputs a determination result according to the attribute of the information provider regarding each sentence.

In this embodiment, the evaluation model construction unit 524 constructs a learning model for assigning an evaluation score to each sentence to be evaluated. Specifically, the evaluation information is used as teacher data to build a learning model for assigning evaluation scores to input sentences.

For example, the evaluation model construction unit 524 uses, as training data, information 322 indicating the judgment of the current state of the economy and information 326 indicating additional explanations and explanations of specific situations in the plurality of records that make up the data table 300. , builds a learning model for assigning evaluation scores to input sentences.

The type of learning model described above is not particularly limited. The learning model described above may be a regression model using a convolutional neural network.

In the data table 300, when the judgment of the current state of the economy is indicated by tiered divisions, the evaluation model construction unit 524 may determine a score corresponding to each division. As a result, the evaluation score assigned to each sentence can be represented by continuous numerical values. For example, if the judgment of the current economic situation is represented by a five-level evaluation of "better", "better", "no change", "worse", and "worse", the evaluation model construction unit 524 assigns a score of 2 to the "gets better" rating. Similarly, the evaluation model construction unit 524 assigns scores of 1, 0, -1 and -2 to the evaluations of "slightly better", "no change", "slightly worse" and "worse", respectively. Give. As a result, the learning model constructed by the evaluation model constructing unit 524 assigns an arbitrary numerical value within the range of -2 to 2 as the evaluation score of the input sentence.

The extraction model construction unit 522 may be an example of a second model construction unit. The evaluation model construction unit 524 may be an example of a first model construction unit.

FIG. 6 schematically shows an example of the internal configuration of the index estimation unit 166. As shown in FIG. In this embodiment, the index estimation unit 166 includes a type determination unit 620 , an evaluation target extraction unit 630 , an evaluation unit 640 and an index generation unit 650 . In this embodiment, the evaluation target extraction unit 630 has a keyword type extraction unit 632 and a machine learning type extraction unit 634 .

The type determining unit 620 determines the type of sample data input to the index estimating unit 166 as an analysis target. For example, the type determination unit 620 refers to the storage unit 126 to obtain type information of the input sample data, and determines the type of sample data based on the type information. The type determination unit 620 may analyze the contents of the input sample data and determine the type of the sample data.

The type determination unit 620 outputs information indicating the determination result to the evaluation target extraction unit 630, for example. The type determination section 620 may output information indicating the determination result to the evaluation section 640 . In one embodiment, the type determination unit 620 outputs information indicating the type of sample data to at least one of the evaluation target extraction unit 630 and the evaluation unit 640 for each sample data. In another embodiment, the type determination section 620 outputs information indicating the type of each of one or more sentences included in the sample data to at least one of the evaluation target extraction section 630 and the evaluation section 640 .

The type determination unit 620 uses the determination result regarding the type of the input sample data to add information indicating the type of the sentence to each of one or more sentences forming a single sample data. may The type determination section 620 may store information indicating the type of each sentence in the sample data storage section 226 . For example, the type determination unit 620 stores information indicating the type of each sentence in the data table 400 .

In one embodiment, the type determination unit 620 associates a single sample data with a single type. For example, the type determination unit 620 assigns the same information as information indicating the type of the sentence to each of one or more sentences forming a single piece of sample data. For example, if the information indicating the type of the sample data indicates that the sample data is information posted on a specific SNS, the type determination unit 620, for all sentences included in the sample data, Information indicating that the sentence is information posted to a specific SNS is added.

In another embodiment, the type determination unit 620 associates single sample data with multiple types. For example, the type determination unit 620 performs the , different information is given as information indicating the types of sentences belonging to each group. For example, when the information indicating the type of sample data indicates that the sample data is information recorded in a negotiation record, daily business report, or daily business report, the type determination unit 620 obtains from the first interviewee: Information indicating the attributes of the first interviewee is added to the sentences included in the sentences regarding the obtained information, and the sentences included in the sentences about the information obtained from the second interviewee are given the second information. Add information that indicates the attributes of the person to be interviewed.

In this embodiment, the evaluation target extraction unit 630 refers to the setting information storage unit 222 and acquires setting information regarding the period to be analyzed. The evaluation target extracting unit 630 refers to the sample data storage unit 226, and extracts sample data (sometimes referred to as sample data to be analyzed) created or updated at a time that matches the period to be analyzed. get. The evaluation target extracting unit 630 extracts text data relating to each of a plurality of sample data to be analyzed as a time related to the contents of each sample data, a time when each sample data was recorded, or an electronic file containing each sample data. is acquired in association with information indicating when the was created or updated. The evaluation target extraction unit 630 may acquire text data relating to each of a plurality of sample data to be analyzed in association with information indicating the attribute of the information provider of each sample data. After that, the evaluation target extraction unit 630 uses the learning model constructed by the extraction model construction unit 522 to extract sentences related to economic activity from among the one or more sentences included in the sample data.

In this embodiment, the evaluation target extraction unit 630 uses at least the machine learning type extraction unit 634 to extract sentences related to economic activity from among one or more sentences included in the sample data. Evaluation target extraction unit 630 outputs the extracted sentence to evaluation unit 640 .

The evaluation target extraction unit 630 may use the keyword type extraction unit 632 and the machine learning type extraction unit 634 to extract sentences related to economic activity from one or more sentences included in the sample data. The evaluation target extraction unit 630 may determine how to combine the extraction processing using the keyword type extraction unit 632 and the extraction processing using the machine learning type extraction unit 634 based on the type of sample data.

In this embodiment, the keyword-type extracting unit 632 extracts sentences that match conditions related to a predetermined keyword or key phrase from one or more sentences included in the sample data. For example, the keyword-type extraction unit 632 selects, from one or more sentences included in the sample data, a sentence containing a keyword, a sentence matching a key phrase, a sentence containing a word similar to the keyword, and a sentence similar to the key phrase. At least one of the sentences matching the condition (sometimes referred to as a sentence matching a keyword or the like) is extracted as a sentence related to economic activity or a candidate for a sentence related to economic activity.

The keyword type extraction unit 632 may refer to the setting information storage unit 222 to acquire setting information indicating conditions related to keywords or key phrases. The setting information related to keywords or keyphrases includes at least one of information indicating keywords and information indicating keyphrases. The setting information related to keywords or keyphrases includes at least information indicating whether to extract sentences containing words similar to the keywords and information indicating whether to extract sentences matching conditions similar to the keyphrases. You may include one.

The keyword type extraction unit 632 may determine conditions for extraction processing based on the determination result of the type determination unit 620 . Accordingly, an appropriate condition is set according to the type of data to be input. In one embodiment, the keyword type extractor 632 may determine words similar to the keyword based on the determination result of the type determiner 620 . The keyword type extraction section 632 may determine the number of words similar to the keyword based on the determination result of the type determination section 620 . In another embodiment, the keyword type extraction unit 632 may determine a condition similar to the key phrase based on the determination result of the type determination unit 620 . The keyword type extraction section 632 may determine the number of conditions similar to the key phrase based on the determination result of the type determination section 620 .

Based on the determination result of the type determination unit 620, the keyword type extraction unit 632 extracts sentences containing keywords, sentences matching key phrases, and words similar to keywords from among one or more sentences included in the sample data. At least one of sentences and sentences matching conditions similar to the key phrase is extracted as (i) sentences related to economic activity, or (ii) candidates for sentences related to economic activity. You may decide whether For example, sentences extracted as sentences related to economic activity are output to the evaluation unit 640 . On the other hand, sentences extracted as candidate sentences related to economic activity are output to the machine learning type extraction unit 634 .

Not only the sentence that matches the keyword, etc., but also one or more sentences placed near the sentence that matches the keyword, etc., are more likely to be sentences related to economic activity than other sentences. Therefore, the keyword-type extraction unit 632 extracts sentences that include two or more consecutive sentences and includes a keyword, a sentence that matches a key phrase, a sentence that includes a word similar to the keyword, and a sentence that is similar to the key phrase. A sentence that includes at least one sentence that satisfies the condition may be extracted as a candidate for a sentence related to economic activity.

In this case, the keyword type extraction unit 632 may determine the number of sentences included in the above sentence based on the determination result of the type determination unit 620 . In one embodiment, the number of sentences may be set such that the more noise included in the sample data, the fewer the number of sentences. Here, noise refers to sentences that are not related to economic activity. This can prevent noise from being mixed into one or more sentences output to the evaluation unit 640 . In another embodiment, the number of sentences may be set such that the more noise included in the sample data, the greater the number of sentences. As a result, omission of extraction of sentences related to economic activities can be suppressed.

In this embodiment, the machine-learning extraction unit 634 uses the learning model constructed by the extraction model construction unit 522 to extract sentences related to economic activity from among one or more sentences included in the sample data. do. Specifically, the machine-learning extraction unit 634 inputs at least part of one or more sentences included in the sample data to a learning model, and the learning model outputs sentences indicating the state of economic activities or evaluations of economic activities. The sentence determined to be a sentence indicating the reason for is extracted as a sentence related to economic activity.

In one embodiment, the machine-learning extractor 634 inputs all of the sample data to be analyzed into the learning model. In another embodiment, the machine learning type extraction unit 634 inputs sentences extracted by the keyword type extraction unit 632 as candidate sentences related to economic activity to the learning model. In yet another embodiment, the machine-learning extraction unit 634 inputs sentences not extracted by the keyword-type extraction unit 632 out of the sample data to be analyzed into the learning model.

In this embodiment, the evaluation unit 640 gives an evaluation score to the sentence extracted by the evaluation target extraction unit 630 . The evaluation unit 640 may assign an evaluation score to the sentence extracted by the evaluation target extraction unit 630 using the learning model constructed by the evaluation model construction unit 524 . For example, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 to the learning model constructed by the evaluation model construction unit 524 . The evaluation unit 640 determines the output of the learning model as the evaluation score of each sentence.

The evaluation unit 640 may use the determination result of the type determination unit 620 to assign an evaluation score to the sentence extracted by the evaluation target extraction unit 630 . For example, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 and the determination result of the type determination unit 620 regarding each sentence to the learning model constructed by the evaluation model construction unit 524. . The evaluation unit 640 determines the output of the learning model as the evaluation score of each sentence.

As described above, the type of data for each sentence is determined based on, for example, the URI of the sample data that contained each sentence, the creator or updater of the sample data, the attributes of the information provider regarding the sample data, etc. be done. The evaluation unit 640 uses, for example, the attribute of the information provider as the determination result of the type determination unit 620 . As a result, the evaluation unit 640 can assign an evaluation object evaluation to each of the plurality of sentences extracted by the evaluation object extraction unit 630 based on the attribute of the information provider for each sentence.

The evaluation unit 640 assigns an evaluation score such that the deeper the information provider's knowledge of the evaluation target, the greater the influence of the information provided by the information provider on the index generated by the index generation unit 650. You can For example, when the index generation unit 650 generates an index that can substitute for the Tankan of the Bank of Japan, information posted by anonymous contributors on an SNS that can be used by an unspecified number of users is used rather than information posted by an anonymous contributor. Information provided by a third party is a better source of information. Therefore, for example, if the degree of similarity between the attributes of the interviewee of the public survey and the attributes of the information provider is greater than a predetermined standard, the evaluation unit 640 determines that the output value of the learning model is greater than the predetermined value. When it is larger, a value larger than the output value of the learning model is determined as an evaluation score, and when the output value of the learning model is smaller than a predetermined value, a value smaller than the output value of the learning model is determined as an evaluation score.

This allows, for example, the attributes of the information provider for each sentence to be taken into account in determining the evaluation score. In the Economic Watchers Survey and the Current Production Statistics Survey, which are used as teacher data, individuals, groups, or corporations with specific attributes are selected as interviewees. Therefore, when the interviewees of the survey used as training data match or resemble each other in the attributes of the information provider regarding each sentence input as the evaluation target, an evaluation score is given that takes into consideration the degree of similarity. As a result, the reliability of the index generated by the index generator 650 is improved.

The evaluation unit 640 uses the determination result of the type determination unit 620 to correct the score output by the learning model constructed by the evaluation model construction unit 524, so that the sentence extracted by the evaluation target extraction unit 630 is evaluated. A score may be given. For example, first, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 to the learning model constructed by the evaluation model construction unit 524 . The evaluation unit 640 determines the output of the learning model as a provisional value of the evaluation score of each sentence. Next, the evaluation unit 640 determines, as the evaluation score of each sentence, a value obtained by, for example, multiplying the output of the learning model by a correction coefficient according to the type of data of each sentence.

The evaluation unit 640 uses, for example, the attribute of the information provider as the determination result of the type determination unit 620 . As a result, the evaluation unit 640 can assign an evaluation object evaluation to each of the plurality of sentences extracted by the evaluation object extraction unit 630 based on the attribute of the information provider for each sentence.

For example, the accuracy of the index generated by the index generation unit 650 can be increased by setting the value of the correction coefficient such that the greater the degree of association between the information provider and the evaluation target, the greater the value of the correction coefficient. can improve. For example, the value of the correction coefficient is set such that the higher the accuracy of the past economic forecast by the information provider, the larger the value of the correction coefficient. Accuracy can be improved.

In this embodiment, the index generator 650 calculates an index indicating the state of economic activity during the analysis target period based on the evaluation scores assigned to each of the plurality of sentences extracted by the evaluation target extractor 630 . The index calculation method is appropriately determined according to the type of index. Thereby, the index estimation system 100 can output the estimated value of the index in the analysis target period.

In one embodiment, the index generation unit 650 calculates the index by summing up the evaluation scores given to each of the plurality of sentences extracted by the evaluation target extraction unit 630 . In another embodiment, the index generation unit 650 calculates the index using statistical values of evaluation scores assigned to each of the plurality of sentences extracted by the evaluation target extraction unit 630. FIG. Examples of statistical values include mean values, median values, quartiles, variances, and the like. In yet another embodiment, the index generation unit 650 substitutes at least one of the evaluation score given to each of the plurality of sentences extracted by the evaluation target extraction unit 630 and the statistical value thereof into a predetermined formula. Alternatively, the index is calculated by processing based on a predetermined algorithm. The index generation unit 650 may use the determination result of the type determination unit 620 to determine the parameters of the above formulas or algorithms. The index generator 650 may use information provider attributes to determine the parameters of the above formulas or algorithms.

The index generator 650 may normalize the calculated index. In one embodiment, the index generator 650 normalizes the index using the maximum and minimum values of the index in a period longer than the analysis target period. In another embodiment, the index generator 650 may normalize the index using the number of sentences to which evaluation scores have been assigned.

The type determination unit 620 may be an example of a type information acquisition unit. The evaluation target extraction unit 630 may be an example of a period information acquisition unit and an extraction unit. The keyword type extraction unit 632 may be an example of a condition acquisition unit and a second extraction unit. The machine learning type extractor 634 may be an example of a first extractor. The evaluation unit 640 may be an example of an evaluation granting unit. The index generator 650 may be an example of an index calculator.

FIG. 7 schematically shows an example of the internal configuration of the machine learning type extraction unit 634. As shown in FIG. In this embodiment, the machine learning type extraction unit 634 includes a learning model 720 and a determination unit 740 . In this embodiment, learning model 720 includes sentence classifier 722 , sentence classifier 724 , sentence classifier 726 , and sentence classifier 728 .

In this embodiment, each of sentence classifier 722, sentence classifier 724, sentence classifier 726, and sentence classifier 728 included in learning model 720 determines whether an input sentence is a state of economic activity or an evaluation of economic activity. A score indicating the probability of being a sentence indicating the reason for is output to determination section 740 . If the sum of the scores output by the sentence classifier 722, the sentence classifier 724, the sentence classifier 726, and the sentence classifier 728 is greater than a predetermined threshold, the determination unit 740 classifies the input sentence. are extracted as sentences related to economic activities.

In this embodiment, the sentence classifier 722 utilizes the TF-IDF model as the generator of the sentence embeddings and the LR model as the classifier. In this embodiment, the sentence classifier 724 utilizes an LSTM model as the generator of the sentence embeddings and an NN model as the classifier. In this embodiment, the sentence classifier 726 utilizes the CNN model as the generator of the sentence embeddings and the NN model as the classifier. In this embodiment, the sentence classifier 728 utilizes the SWEM model as the generator of the sentence embeddings and the LR model as the classifier.

The configuration of learning model 720 may be determined based on the determination result of type determination section 620 . For example, the type of model used for the sentence classifier that constitutes the learning model 720 is determined based on the determination result of the type determination unit 620 . A combination of models used for the sentence classifiers that constitute the learning model 720 may be determined based on the determination result of the type determination unit 620 . The number of sentence classifiers forming learning model 720 may be determined based on the determination result of type determination section 620 .

8, 9 and 10, the evaluation object extraction unit 630 uses the keyword type extraction unit 632 and the machine learning type extraction unit 634 to extract economic sentences from one or more sentences included in the sample data. An overview of information processing for extracting activity-related sentences is provided. FIG. 8 schematically shows an example of information processing in the evaluation target extraction unit 630. As shown in FIG. FIG. 9 schematically shows another example of information processing in the evaluation target extraction unit 630. As shown in FIG. FIG. 10 schematically shows still another example of information processing in the evaluation target extraction unit 630. As shown in FIG.

Information processing in the evaluation target extraction unit 630 is not limited to these embodiments. In another embodiment, the evaluation target extraction unit 630 uses only the machine learning type extraction unit 634 to extract sentences related to economic activity from one or more sentences included in the sample data.

According to the embodiment shown in FIG. 8, all of the sample data to be analyzed are first input into the keyword type extractor 632 . According to this embodiment, all sentences extracted by the keyword type extraction unit 632 are input to the machine learning type extraction unit 634 . On the other hand, sentences not extracted by the keyword type extraction unit 632 are not input to the machine learning type extraction unit 634 .

According to the embodiment shown in FIG. 9, all of the sample data to be analyzed are first input into the keyword type extractor 632 . According to this embodiment, sentences not extracted by the keyword type extraction unit 632 are input to the machine learning type extraction unit 634 . On the other hand, the sentences extracted by the keyword type extraction unit 632 are output to the evaluation unit 640 without being input to the machine learning type extraction unit 634 .

According to the embodiment shown in FIG. 10, all of the sample data to be analyzed are first input to the keyword type extractor 632 . According to this embodiment, part of the sentence extracted by the keyword-based extraction unit 632 is output to the evaluation unit 640 without being input to the machine-learning-type extraction unit 634 . On the other hand, the rest of the sentence extracted by the keyword type extraction unit 632 is input to the machine learning type extraction unit 634 .

For example, the keyword-type extraction unit 632 extracts sentences containing two or more consecutive sentences that include the keyword, sentences that match the key phrase, sentences that include words similar to the keyword, and sentences that are similar to the key phrase. Consider a case where sentences including at least one sentence that satisfies a condition are extracted as candidate sentences related to economic activity. In this case, sentences containing keywords, sentences matching keyphrases, sentences containing words similar to keywords, and sentences matching conditions similar to keyphrases may be sentences related to economic activities. Relatively high. Therefore, these sentences are output to the evaluation unit 640 without being input to the machine learning type extraction unit 634 . On the other hand, the remaining sentences extracted by the keyword type extraction unit 632 are less likely to be sentences related to economic activities. These sentences are then input to the machine learning type extraction unit 634 .

FIG. 11 schematically shows an example of a data table 1100. As shown in FIG. In this embodiment, each row of the data table 1100 may be an example of the evaluation result of the evaluation unit 640 . In this embodiment, the data table 1100 shows sample IDs 1112, sentence IDs 1114, information 1116 indicating recording times, and information 1118 indicating evaluation scores. Each of the sample ID 1112, the sentence ID 1114, and the information 1116 indicating the recording time may have the same configuration as the sample ID 412, the sentence ID 414, and the information 416 indicating the recording time. The evaluation score information 1118 indicates the evaluation score given by the evaluation unit 640 .

FIG. 12 illustrates an example computer 3000 upon which aspects of the present invention may be embodied, in whole or in part. For example, index estimation system 100 is implemented by computer 3000 .

Programs installed on the computer 3000 cause the computer 3000 to function as one or more "parts" of operations or one or more "parts" of an apparatus according to embodiments of the invention, or to and/or cause computer 3000 to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by CPU 3012 to cause computer 3000 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

Computer 3000 according to this embodiment includes CPU 3012 , RAM 3014 , graphics controller 3016 , and display device 3018 , which are interconnected by host controller 3010 . Computer 3000 also includes input/output units such as communication interface 3022 , hard disk drive 3024 , DVD-ROM drive 3026 and IC card drive, which are connected to host controller 3010 via input/output controller 3020 . The computer also includes legacy input/output units such as ROM 3030 and keyboard 3042 , which are connected to input/output controller 3020 via input/output chip 3040 .

The CPU 3012 operates according to programs stored in the ROM 3030 and RAM 3014, thereby controlling each unit. Graphics controller 3016 retrieves image data generated by CPU 3012 into a frame buffer or the like provided in RAM 3014 or itself, and causes the image data to be displayed on display device 3018 .

Communication interface 3022 communicates with other electronic devices over a network. Hard disk drive 3024 stores programs and data used by CPU 3012 within computer 3000 . DVD-ROM drive 3026 reads programs or data from DVD-ROM 3001 and provides programs or data to hard disk drive 3024 via RAM 3014 . The IC card drive reads programs and data from IC cards and/or writes programs and data to IC cards.

ROM 3030 stores therein programs that are dependent on the hardware of computer 3000, such as a boot program that is executed by computer 3000 upon activation. Input/output chip 3040 may also connect various input/output units to input/output controller 3020 via parallel ports, serial ports, keyboard ports, mouse ports, and the like.

A program is provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The program is read from a computer-readable storage medium, installed in hard disk drive 3024 , RAM 3014 , or ROM 3030 , which are also examples of computer-readable storage medium, and executed by CPU 3012 . The information processing described within these programs is read by computer 3000 to provide coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing information operations or processing according to the use of computer 3000 .

For example, when communication is performed between the computer 3000 and an external device, the CPU 3012 executes a communication program loaded into the RAM 3014 and sends communication processing to the communication interface 3022 based on the processing described in the communication program. you can command. Under the control of the CPU 3012, the communication interface 3022 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 3014, the hard disk drive 3024, the DVD-ROM 3001, or an IC card, and transmits the read transmission data. Data is transmitted to the network, or received data received from the network is written in a receive buffer area or the like provided on the recording medium.

In addition, the CPU 3012 causes the RAM 3014 to read all or necessary portions of files or databases stored in external recording media such as a hard disk drive 3024, a DVD-ROM drive 3026 (DVD-ROM 3001), an IC card, etc. Various types of processing may be performed on the data in RAM 3014 . CPU 3012 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media and subjected to information processing. CPU 3012 performs various types of operations on data read from RAM 3014, information processing, conditional decisions, conditional branching, unconditional branching, and information retrieval as specified throughout this disclosure and by instruction sequences of programs. Various types of processing may be performed, including /replace, etc., and the results written back to RAM 3014 . Also, the CPU 3012 may search for information in a file in a recording medium, a database, or the like. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 3012 selects the first attribute from among the plurality of entries. search for an entry that matches the specified condition of the attribute value of the attribute, read the attribute value of the second attribute stored in the entry, and thereby determine the first attribute that satisfies the predetermined condition An attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored in a computer readable storage medium on or near computer 3000 . In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the above program can be transferred via a network. provided to the computer 3000;

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before etc., and it should be noted that it can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for the sake of convenience, it means that it is essential to carry out in this order. not a thing

10 communication network 12 user terminal 14 teacher data providing server 16 sample data providing server 100 index estimation system 122 communication unit 124 input/output unit 126 storage unit 128 request reception unit 142 teacher data acquisition unit 144 model construction unit 162 sample data acquisition unit 164 text data generation unit 166 index estimation unit 222 setting information storage unit 224 teacher data storage unit 226 sample data storage unit 228 model information storage unit 300 data table 312 Information, 314 information, 316 information, 322 information, 324 information, 326 information, 400 data table, 412 sample ID, 414 sentence ID, 416 information, 418 information, 420 information, 522 extraction model construction unit, 524 evaluation model construction Unit 620 Type Determination Unit 630 Evaluation Target Extraction Unit 632 Keyword Extraction Unit 634 Machine Learning Extraction Unit 640 Evaluation Unit 650 Index Generation Unit 720 Learning Model 722 Sentence Classifier 724 Sentence Classifier 726 Sentence Classifier 728 Sentence Classifier 740 Determination Unit 1100 Data Table 1112 Sample ID 1114 Sentence ID 1116 Information 1118 Information 3000 Computer 3001 DVD-ROM 3010 Host Controller 3012 CPU 3014 RAM 3016 Graphic Controller 3018 Display Device 3020 Input/Output Controller 3022 Communication Interface 3024 Hard Disk Drive 3026 DVD-ROM Drive 3030 ROM 3040 Input/Output Chip 3042 Keyboard

Claims (11)

(i) an evaluation of an evaluation target, and (ii) the one or more descriptions included in the evaluation information associated with the one or more descriptions indicating the state of the evaluation target or the reason for the evaluation, as teacher data. First model construction for constructing a first learning model for determining whether or not an input sentence is a sentence indicating an evaluation for the evaluation object, a state of the evaluation object, or a reason for the evaluation. Department and
an extraction unit that extracts sentences related to the evaluation target from among one or more sentences included in text data using the first learning model constructed by the first model construction unit;
A machine learning device comprising
The text data includes information presented by an information provider's utterances or gestures, or information perceived by the information provider,
The machine learning device
a second model building unit that uses the evaluation information as training data to build a second learning model for assigning an evaluation regarding the evaluation target to an input sentence;
an evaluation assigning unit that assigns an evaluation regarding the evaluation target to the sentence extracted by the extracting unit using the second learning model constructed by the second model constructing unit;
an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit;
a text data acquisition unit that acquires each of the plurality of text data in association with attribute information indicating attributes of information providers of each of the plurality of text data;
further comprising
For each of the plurality of sentences extracted from at least a part of the plurality of text data by the extraction unit, the evaluation imparting unit provides information indicated by the attribute information corresponding to the text data in which each sentence is included. Giving an evaluation regarding the evaluation target based on the attribute of the provider,
The index calculation unit calculates the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.
Machine learning device. The extractor is
At least part of the one or more sentences included in the text data is input to the first learning model, and the first learning model determines that the sentence indicates the state of the evaluation object or the reason for the evaluation. a first extraction unit that extracts a sentence as a sentence related to the evaluation target;
a condition acquisition unit that acquires information indicating a keyword or key phrase related to the evaluation target;
Among the one or more sentences included in the text data, a sentence containing the keyword, a sentence matching the key phrase, a sentence containing a word similar to the keyword, and a condition similar to the key phrase are matched. a second extracting unit that extracts at least one sentence related to the evaluation object;
having
The machine learning device according to claim 1. The second extraction unit selects, from the one or more sentences included in the text data, a sentence containing the keyword, a sentence matching the key phrase, a sentence containing a word similar to the keyword, and the key extracting at least one sentence that matches a condition similar to the phrase as a sentence related to the evaluation target or a sentence candidate related to the evaluation target;
The first extraction unit is
inputting the sentence extracted by the second extraction unit as a sentence candidate related to the evaluation object into the first learning model;
extracting a sentence determined by the first learning model to be a sentence indicating the state of the evaluation target or the reason for the evaluation as a sentence related to the evaluation target;
The machine learning device according to claim 2. The extractor is
At least part of the one or more sentences included in the text data is input to the first learning model, and the first learning model determines that the sentence indicates the state of the evaluation object or the reason for the evaluation. a first extraction unit that extracts a sentence as a sentence related to the evaluation target;
a condition acquisition unit that acquires information indicating a keyword or key phrase related to the evaluation target;
Among the one or more sentences included in the text data, a sentence containing the keyword, a sentence matching the key phrase, a sentence containing a word similar to the keyword, and a condition similar to the key phrase are matched. a second extracting unit that extracts at least one of the sentences as a sentence related to the evaluation target or a sentence candidate related to the evaluation target;
has
The first extraction unit is
inputting the sentence extracted by the second extraction unit as a sentence candidate related to the evaluation object into the first learning model;
extracting a sentence determined by the first learning model to be a sentence indicating the state of the evaluation target or the reason for the evaluation as a sentence related to the evaluation target;
The machine learning device according to claim 1. The second extraction unit extracts a sentence containing two or more consecutive sentences, a sentence containing the keyword, a sentence matching the key phrase, a sentence containing a word similar to the keyword, and a sentence containing the key phrase. Extracting a sentence containing at least one sentence that matches a similar condition as a sentence candidate related to the evaluation object;
The machine learning device according to claim 3 or 4 . further comprising a type information acquisition unit that acquires type information for distinguishing types of the text data,
Based on the type of the text data indicated by the type information, the second extracting unit matches the sentence including the keyword and the key phrase from among the one or more sentences included in the text data. At least one of a sentence, a sentence containing a word similar to the keyword, and a sentence matching a condition similar to the key phrase is selected from either a sentence related to the evaluation object or a sentence candidate related to the evaluation object. to decide whether to extract as,
The machine learning device according to any one of claims 3 to 5 . The first learning model converts the input sentence into either a sentence indicating the state of the evaluation target or the reason for the evaluation, or a sentence that does not indicate the state of the evaluation target or the reason for the evaluation. including a sentence classifier to classify,
The machine learning device according to any one of claims 1 to 6 . an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit;
a period information acquisition unit that acquires information indicating the specific period;
When each of the plurality of text data is related to the content of each of the plurality of text data, when each of the plurality of text data was recorded, or when an electronic file containing each of the plurality of text data is a text data acquisition unit that acquires text data in association with time information indicating when it was created or updated;
further comprising
The extracting unit selects, from among the plurality of text data, a plurality of sentences included in the text data whose time indicated by the time information associated with the text data is included in the specific period, the evaluation target. extract sentences related to
The evaluation giving unit gives a plurality of sentences extracted from at least a part of the plurality of text data by the extracting unit an evaluation regarding the evaluation target,
The index calculation unit calculates the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.
The machine learning device according to any one of claims 1 to 7 . The information presented by the information provider's utterances or gestures, or the information perceived by the information provider is information described in a daily business report or a daily business report.
The machine learning device according to any one of claims 1 to 8. A program for causing a computer to function as the machine learning device according to any one of claims 1 to 9 . A computer calculates (i) an evaluation regarding an evaluation target, and (ii) the one or more descriptions included in the evaluation information associated with the one or more descriptions indicating the status of the evaluation target or the reason for the evaluation. constructing a first learning model for determining whether or not an input sentence is a sentence indicating the state of the evaluation target, the evaluation for the evaluation target, or the reason for the evaluation, using the training data as training data; 1 model building stage;
an extracting step in which the computer extracts sentences related to the evaluation target from among one or more sentences included in text data using the first learning model constructed in the first model constructing step;
A machine learning method comprising
The text data includes information presented by an information provider's utterances or gestures, or information perceived by the information provider,
The machine learning method comprises:
a second model construction step in which the computer constructs a second learning model for assigning an evaluation regarding the evaluation object to an input sentence using the evaluation information as teacher data;
an evaluation step in which the computer uses the second learning model built in the second model building step to give an evaluation regarding the evaluation target to the sentence extracted in the extraction step;
an index calculation step in which the computer calculates an index indicating the state or trend of the evaluation object in a specific period based on the evaluation in the evaluation giving step;
a text data acquisition step in which the computer acquires each of the plurality of text data in association with attribute information indicating the attributes of the information provider of each of the plurality of text data;
further having
In the evaluation step, each of the plurality of sentences extracted from at least a part of the plurality of text data in the extraction step is indicated by the attribute information corresponding to the text data in which each sentence was included. Based on the attributes of the information provider, giving an evaluation regarding the evaluation target,
The index calculation step includes calculating the index based on the evaluation given to each of the plurality of sentences in the evaluation giving step,
machine learning method.

Images