JP2020129232A - Machine learning device, program, and machine learning method - Google Patents

Abstract

To provide a machine learning device, a program, and a machine learning method in which classification accuracy does not decrease even when data to be classified and data to not be classified are mixed in sample data.SOLUTION: A machine learning device comprises: a model construction unit including an evaluation model construction unit that constructs a first learning model, wherein the first learning model determines whether an input sentence is a sentence indicating the evaluation of the evaluation target, the state of the evaluation target, or the reason for the evaluation using one or more explanations, which is included in the evaluation information associated with one or more explanations indicating the evaluation of the evaluation target, and the state of the evaluation target or the reason for the evaluation, as a teacher data; and an index estimation unit including an extraction unit that extracts, from one or more sentences included in text data, the sentence related to the evaluation target using the first learning model constructed by the evaluation model construction unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a machine learning device, a program, and a machine learning method.

There is known a classification system that assigns an in-house classification to a gazette using a learning model (for example, Patent Document 1).
[Prior Art Document]
[Patent Document]
[Patent Document 1] Japanese Unexamined Patent Application Publication No. 2018-0261119

In the classification system of Patent Document 1, the type of sample data input to the classification system is specified in advance, and the in-house classification is assigned to all the input sample data. Therefore, when the data to be classified and the data not to be classified are mixed in the sample data, there are problems such as a decrease in classification accuracy and a decrease in utilization efficiency of system resources.

In a first aspect of the invention, a machine learning device is provided. The machine learning device described above includes, for example, (i) an evaluation regarding an evaluation target, and (ii) one or more evaluation information included in the evaluation information associated with one or more explanatory texts indicating the state of the evaluation target or the reason for the evaluation. A first learning model for constructing a first learning model for determining whether or not the input sentence is a sentence indicating an evaluation of an evaluation target, a state of the evaluation target, or a reason for the evaluation, using the explanatory text as teacher data. 1 model building section is provided. The machine learning device described above includes, for example, an extraction unit that extracts a sentence related to an evaluation target from one or more sentences included in the text data using the first learning model constructed by the first model construction unit. Prepare

In the above machine learning device, the first learning model is either a sentence indicating the state of the evaluation target or the reason for the evaluation, or a sentence that is not a state indicating the evaluation target or the reason for the evaluation. It may include a text classifier for classifying into. In the above machine learning device, the first learning model may include a plurality of sentence classifiers. In the above machine learning device, each of the plurality of sentence classifiers may output a score indicating the likelihood that the input sentence is a sentence indicating the state of the evaluation target or the reason for the evaluation. In the above machine learning device, the extraction unit, when the total value of the scores output by each of the plurality of sentence classifiers is larger than a predetermined threshold value, the input sentence, the sentence related to the evaluation target. May be extracted as

In the above machine learning device, the extraction unit inputs at least a part of one or more sentences included in the text data to the first learning model, and the first learning model indicates a state of an evaluation target or a reason for the evaluation. It may have a 1st extraction part which extracts a sentence determined to be as a sentence relevant to an evaluation object. In the above machine learning device, the extraction unit may include a condition acquisition unit that acquires information indicating a keyword or key phrase related to the evaluation target. In the above machine learning device, the extraction unit extracts a sentence including a keyword, a sentence matching a key phrase, a sentence including a word similar to the keyword, and a key phrase from one or more sentences included in the text data. A second extraction unit that extracts at least one of the sentences that match the similar conditions as a sentence related to the evaluation target or a candidate of a sentence related to the evaluation target may be included. In the above machine learning device, the first extraction unit may input a sentence extracted by the second extraction unit as a candidate for a sentence related to the evaluation target into the first learning model. In the above machine learning device, the first extraction unit may extract a sentence that the first learning model has determined to be a sentence indicating a state of an evaluation target or a reason for the evaluation, as a sentence related to the evaluation target.

In the above machine learning device, the second extraction unit is a sentence including two or more continuous sentences, including a sentence including a keyword, a sentence matching a key phrase, a sentence including a word similar to the keyword, and a key. A sentence including at least one sentence that matches a condition similar to a phrase may be extracted as a candidate for a sentence related to an evaluation target. The machine learning device may include a type information acquisition unit that acquires type information for distinguishing the types of text data. In the above machine learning device, the second extraction unit may, based on the type of text data indicated by the type information, a sentence including a keyword or a sentence that matches a key phrase from one or more sentences included in the text data. At least one of a sentence including a word similar to a keyword and a sentence matching a condition similar to a key phrase is to be extracted as a sentence related to an evaluation target or a candidate of a sentence related to an evaluation target. You may decide.

The machine learning device described above may include a second model building unit that builds a second learning model for giving an evaluation regarding an evaluation target to an input sentence by using the evaluation information as teacher data. The machine learning device may include an evaluation assigning unit that assigns an evaluation regarding an evaluation target to the sentence extracted by the extracting unit using the second learning model constructed by the second model constructing unit.

The above machine learning device may include an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit. The machine learning device may include a period information acquisition unit that acquires information indicating a specific period. The machine learning device includes each of the plurality of text data, a time associated with each content of the plurality of text data, a time when each of the plurality of text data was recorded, or a plurality of each of the plurality of text data. A text data acquisition unit may be provided that is associated with time information indicating the time when the electronic file was created or updated. In the above machine learning device, the extraction unit selects, from among a plurality of text data, a plurality of sentences included in the text data in which a time period indicated by time information associated with the text data is included in a specific period. , Sentences relevant to the evaluation target may be extracted. In the above machine learning device, the evaluation giving unit may give the evaluation regarding the evaluation target to the plurality of sentences extracted from at least a part of the plurality of text data by the extraction unit. In the above machine learning device, the index calculation unit may calculate the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.

In the above machine learning device, the text data may include information presented by a statement or gesture of the information provider, or information perceived by the information provider. The above machine learning device may include an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period based on the evaluation by the evaluation giving unit. The machine learning device may include a text data acquisition unit that acquires each of the plurality of text data in association with attribute information indicating an attribute of each information provider of the plurality of text data. In the above machine learning device, the evaluation assigning unit uses the attribute information corresponding to the text data containing each sentence for each of the plurality of sentences extracted from at least a part of the plurality of text data by the extracting unit. The evaluation regarding the evaluation target may be given based on the attribute of the information provider shown. In the above machine learning device, the index calculation unit may calculate the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit.

In a second aspect of the invention, a machine learning method is provided. The machine learning method described above includes, for example, (i) an evaluation regarding an evaluation target, and (ii) one or more evaluation information included in the evaluation information associated with one or more explanatory texts indicating the state of the evaluation target or the reason for the evaluation. Constructing a first learning model for determining whether or not the input sentence is a sentence indicating the state of the evaluation target, the evaluation of the evaluation target, or the reason for the evaluation, using the explanatory text as the teacher data. It has one model building stage. The machine learning method is, for example, an extraction step of extracting a sentence related to the evaluation target from one or more sentences included in the text data using the first learning model constructed in the first model construction stage. Have.

In the third aspect of the present invention, a program is provided. A non-transitory computer-readable medium storing the above program may be provided. The above program is, for example, a program for causing a computer to function as the machine learning device according to the first aspect. The above program may be a program for causing a computer to execute the machine learning method according to the second aspect.

Note that the above summary of the invention does not enumerate all necessary features of the present invention. Further, a sub-combination of these feature groups can also be an invention.

1 schematically shows an example of a system configuration of an index estimation system 100. 1 schematically shows an example of the internal configuration of the storage unit 126. An example of the data table 300 is schematically shown. An example of the data table 400 is schematically shown. 1 schematically shows an example of the internal configuration of the model construction unit 144. An example of the internal configuration of the index estimation unit 166 is schematically shown. An example of the internal configuration of the machine learning type extraction unit 634 is schematically shown. An example of information processing in the evaluation target extraction unit 630 is schematically shown. An example of information processing in the evaluation target extraction unit 630 is schematically shown. An example of information processing in the evaluation target extraction unit 630 is schematically shown. 1 schematically shows an example of a data table 1100. 1 schematically shows an example of a system configuration of a computer 3000.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all of the combinations of features described in the embodiments are essential to the solving means of the invention. In the drawings, the same or similar parts may be denoted by the same reference numerals, and redundant description may be omitted.

[Outline of Index Estimation System 100]
FIG. 1 schematically shows an example of the system configuration of the index estimation system 100. In the present embodiment, the index estimation system 100 includes (i) information registered in various SNSs, information registered in various bulletin boards on the network, information reported in various news, and various questionnaires regarding business sentiment. The information collected by the above, the information obtained through negotiations with customers, etc., the information described in the daily business report or the daily business report, and the like are acquired, and (ii) the estimated value of the economic index is output. As a result, the user of the index estimation system 100 can determine the state or trend of economic activity at the same accuracy as the economic index published by a public institution such as the government or central bank, and at an earlier time than the publication time of the economic index. You can figure it out.

Economic indicators published by governments, central banks, and other public institutions are, for example, survey results (sometimes referred to as public survey results) on economic activities published by governments, central banks, and other public institutions. It is described in. As the above-mentioned economic indicators, the various indices listed in the “Corporate Short-term Economic Observation Survey” by the Bank of Japan, the various indices listed in the “Economic Watcher Survey” by the Cabinet Office, and the “Statistics Survey on Production Dynamics” by the Ministry of Economy, Trade and Industry The various indexes described in “” are exemplified. An example of the index described in the “Short-term corporate economic survey survey” is the “Business Conditions Index” (sometimes referred to as the BOJ Tankan). As the index described in the "Business Watcher Survey," "current situation judgment" is exemplified. As the index described in the “Statistical Survey on Production Dynamics”, various “mining and industrial indexes” are exemplified. Another example of economic indicators is the Business Trend Index published by the Cabinet Office.

In the present embodiment, the index estimation system 100 includes a communication unit 122, an input/output unit 124, a storage unit 126, a request reception unit 128, a teacher data acquisition unit 142, a model construction unit 144, and a sample data acquisition unit. 162, a text data generation unit 164, and an index estimation unit 166. In the present embodiment, the index estimation system 100 can send and receive information to and from at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16 via the communication network 10.

In the present embodiment, the communication network 10 transfers information between the index estimation system 100 and at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16. The communication network 10 may be a wired communication transmission path, a wireless communication transmission path, or a combination of a wireless communication transmission path and a wired communication transmission path.

The communication network 10 may include a wireless communication network, the Internet, a P2P network, a dedicated line, a VPN, a power line communication line, or the like. The communication system in the wireless communication network may be a mobile communication system such as (i) 3G system, LTE system, 4G system, 5G system, and (ii) Bluetooth (registered trademark), Zigbee (registered trademark), NFC. (Near Field Communication), a short-distance wireless communication method, a wireless LAN method such as WiFi (registered trademark), a wireless MAN method such as WiMAX (registered trademark), a wireless data communication method such as a wireless WAN method. Good.

In the present embodiment, the user terminal 12 may be any information processing terminal that can send and receive information to and from the index estimation system 100 via the communication network 10, and the details thereof are not particularly limited. The user terminal 12 can be used by a user of the index estimation system 100 as a user interface of the index estimation system 100. Examples of the user terminal 12 include a personal computer and a mobile terminal. Examples of the mobile terminal include a mobile phone, a smartphone, a PDA, a tablet, a notebook computer or a laptop computer, and a wearable computer.

In the present embodiment, the teacher data providing server 14 manages the data of official survey results. For example, the teacher data providing server 14 stores, for each of the plurality of surveys, information indicating the type of the survey, information indicating the target period of the survey, and the survey result data of the survey in association with each other. The teacher data providing server 14 transmits the data of the survey result requested by the index estimation system 100 to the index estimation system 100 in response to the request from the index estimation system 100.

Examples of the type of survey include the name of the survey, the type of survey target, the purpose of the survey, and the attributes of the interviewees in the survey. Examples of the name of the survey include corporate short-term economic observation survey, business watcher survey, and production dynamic statistics survey. Examples of attributes of the interviewee in the survey include age, sex, and degree of association with the survey target. Examples of the degree of relevance to the survey target include the industry of the organization to which the interviewee belongs, the size of the organization, the area in which the organization operates, the title of the interviewee in the organization, the occupation of the interviewee, etc. To be done. The titles of the interviewees include departments, positions, and the like. Other examples of the degree of association with the survey target include the prediction accuracy of the survey target, the number of years of experience in the business or post related to the survey target, and the like.

The degree of association between the interview target person and the survey target may be represented by a continuous numerical value or may be represented by a stepwise division. For example, the degree of association is set such that the better the prediction accuracy with respect to the survey target, the larger the numerical value representing the degree of association. The degree of association may be set such that the longer the number of years of experience in the business or post related to the survey target, the larger the numerical value representing the degree of association. The degree of association may be set such that the higher the rank of the post, the larger the numerical value representing the degree of association.

For example, if the survey target is economic activity, "We expect that the next quarter's economy will be better than the present, and if the economy actually becomes better," "We expect the next quarter's economy to be better than the present However, as a numerical value showing the degree of relation, in the order of ``when there is no actual change in economic trends'' and ``when the economy in the next quarter was expected to be better than the present, but the economy actually deteriorated'' A large value is given. "If we expect the next quarter's economy to be better than it is now, and in fact it's been better," the relevant condition may be 1 or more, and "we expect the next quarter's economy to be better than now. However, even if the degree of relation is less than 1, "if there is no actual change in the economic trend" and "if the economy in the next quarter is expected to be better than the present, but the economy actually deteriorates" Good.

In the present embodiment, the sample data providing server 16 manages various data (sometimes referred to as sample data) to be analyzed by the index estimation system 100. The sample data may be managed for each creator or updater, or for each creator or updater and creation date or update date. For example, when the sample data is data such as a business daily report and a business daily report, each time the data such as the business daily report and the business daily report is saved, it is treated as one sample data. When the sample data is data such as a daily business report and a daily business report, each person in charge who created or updated the daily business report and the daily business report may be treated as one sample data. When the sample data is data such as a daily business report and a daily business report, it may be treated as one sample data for each person in charge and each business day. For example, if the sample data is data posted on the SNS, it is treated as one sample data for each posting.

For example, the sample data providing server 16 (i) information indicating the creation time or update time of the sample data, and information indicating the creator, updater, or administrator of the sample data for each of the plurality of sample data. And (ii) the sample data are stored in association with each other. The sample data providing server 16 at least, for each of the plurality of sample data, (i) at least information indicating the creation time or update time of the sample data, and information indicating the creator, updater, or administrator of the sample data. One side, (ii) information indicating the type of the sample data, and (iii) the sample data may be stored in association with each other. The sample data providing server 16 transmits the sample data requested by the index estimation system 100 to the index estimation system 100 in response to the request from the index estimation system 100.

The sample data may be text data, audio data, image data, or data for any application program. The image data may be still image data or moving image data. Information included in the sample data includes information registered in various SNS, information reported in various news, information collected by various questionnaires, information obtained through negotiations with customers, daily business reports, or The information described in the daily work report is exemplified.

The sample data may include information presented by a statement or gesture of the information provider, or information perceived by the information provider. Gestures include gestures, gestures, gestures, facial expressions, and the like. Examples of the information regarding the utterance, gesture, and the like include (i) information indicating the content or gesture of the utterance, and (ii) information indicating the summary or supplementary explanation by the person in charge of the utterance or gesture of the information provider. As a supplementary explanation, the background of the above-mentioned statement or gesture, the impression of the person in charge, etc. are exemplified.

For example, when the sample data is data such as a daily business report and a daily business report, the person in charge of the hearing work writes the information about the statements, gestures, etc. of the interviewee into the daily business report, the daily business report, and the like. Create or update daily work reports. The interview target person in this case may be an example of an information provider. For example, when the sample data is data posted on the SNS, the poster posts the information perceived by the user on the SNS. The poster in this case may be an example of an information provider. Details of the types of sample data will be described later.

[Outline of each part of the index estimation system 100]
In the present embodiment, the communication unit 122 exchanges information with at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16 via the communication network 10. For example, the communication unit 122 transmits various requests to at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16. The communication unit 122 receives a response to the above request from at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16.

In one embodiment, the communication unit 122 requests the user terminal 12 to interact with the user of the index estimation system 100. For example, the communication unit 122 transmits the information presented to the user of the index estimation system 100 to the user terminal 12. The communication unit 122 may transmit information of various input screens. The user terminal 12 presents the information received from the communication unit 122 to the user based on the above-mentioned request regarding the interaction. The mode of presenting information is not particularly limited. The above information may be displayed or projected as an image, and may be output as sound. The user terminal 12 may request or urge the user to enter information. The user terminal 12 may transmit the information input by the user to the communication unit 122. Accordingly, the communication unit 122 can acquire the information input to the user terminal 12 by the user of the index estimation system 100.

In another embodiment, the communication unit 122 requests the teacher data providing server 14 to send data regarding a specific survey result. Thereby, the communication unit 122 can acquire various survey result data from the teacher data providing server 14. The acquired survey result data is used as, for example, machine learning teacher data in the model construction unit 144.

In still another embodiment, the communication unit 122 requests the sample data providing server 16 to send specific sample data. Accordingly, the communication unit 122 can acquire various sample data from the sample data providing server 16. The acquired sample data is used as input data for estimation processing in the index estimation unit 166, for example.

In the present embodiment, the input/output unit 124 receives input of information from the user of the index estimation system 100. The input/output unit 124 presents information to the user of the index estimation system 100. The input/output unit 124 can be used by the user of the index estimation system 100 as a user interface of the index estimation system 100. The input/output unit 124 may include an input device such as a keyboard, a pointing device, a touch panel, a microphone, a camera, a voice input system, and a gesture input system. The input/output unit 124 may include an output device such as a display device, a projection device, a sound output device, and a vibration device.

In this embodiment, the storage unit 126 stores various kinds of information. The storage unit 126 may store information used in information processing of the index estimation system 100. The storage unit 126 may store information generated by the information processing of the index estimation system 100. Details of the storage unit 126 will be described later.

In the present embodiment, the request receiving unit 128 receives various requests to the index estimation system 100. For example, the request receiving unit 128 receives a request from the user for registering various settings related to the index estimation system 100. The request receiving unit 128 may transfer the received request to an element suitable for processing the request.

Settings related to the index estimation system 100 include settings related to the types of indexes estimated by the index estimation system 100, various settings related to machine learning in the index estimation system 100, and various settings related to sample data input to the index estimation system 100. It is illustrated. Examples of the types of indicators include various indices listed in the Corporate Short-term Economic Observation Survey, various indices listed in the Business Watcher Survey, various indices listed in the Production Statistics Survey, and Business Trend Index. .. Examples of settings related to machine learning include settings related to machine learning teacher data and settings related to learning models. Examples of settings relating to teacher data include a data URI and a data format. Examples of settings related to sample data include a data URI and a data format.

In one embodiment, the request receiving unit 128 receives a request from the user of the index estimation system 100. The request reception unit 128 may acquire a request from the user via the user terminal 12 or the input/output unit 124. In another embodiment, the request receiving unit 128 may receive a request from at least one of the user terminal 12, the teacher data providing server 14, and the sample data providing server 16.

In the present embodiment, the teacher data acquisition unit 142 requests the teacher data providing server 14 to transmit data of a specific survey result. The data of the survey result is specified by, for example, the type of the survey and the target period of the survey. Thereby, the teacher data acquisition unit 142 can acquire the data of the specific survey result from the teacher data providing server 14.

In one embodiment, the teacher data acquisition unit 142 stores the acquired survey result data in the storage unit 126. In another embodiment, the teacher data acquisition unit 142 may output the acquired survey result data to the model construction unit 144.

In the present embodiment, the model construction unit 144 constructs a learning model used in the index estimation unit 166 by using the survey result data acquired by the teacher data acquisition unit 142 as the teacher data. Details of the model construction unit 144 will be described later.

In the present embodiment, the sample data acquisition unit 162 requests the sample data providing server 16 to transmit specific sample data. Accordingly, the communication unit 122 can acquire various sample data from the sample data providing server 16. The sample data acquisition unit 162 determines that each of the one or more sample data is associated with the content of each sample data, each sample data is recorded, or an electronic file including each sample data is created or updated. It may be acquired in association with the information indicating the time. The sample data acquisition unit 162 may acquire each of the one or more sample data in association with the information indicating the attribute of the information provider of each sample data.

The sample data includes, for example, (i) the type of the sample data, (ii) the time when the sample data is created or updated, or the range related to the time (the above time or the range related to the time is May be referred to as). The type of sample data is specified by, for example, the URL of the sample data providing server 16 that manages the sample data. The type of sample data may be specified by the identification information for identifying the type. The type of sample data may be specified by at least one of the name of the sample data, the creator, the updater, and the information provider. The type of sample data may be specified by the attribute of the information provider who has provided the information indicating the content of the sample data. Examples of the attribute of the information provider include age, sex, and degree of association with the evaluation target. The specific example of the attribute of the information provider may be the same as the specific example of the attribute of the interview target person described above.

The time when the sample data is created or updated may be the time when the sample data is recorded in the electronic file that stores the sample data, or the time when the electronic file is created or updated. Instead of the time when the sample data is created or updated, the time related to the content of the sample data may be used. For example, when the sample data includes information indicating the period regarding the evaluation target, such as “Sales in December 2018 increased by 10% compared to the previous year”, the relevant period is the sample. It is used as a time related to the content of data.

In one embodiment, the sample data acquisition unit 162 stores the acquired sample data in the storage unit 126. The sample data acquisition unit 162 may store the acquired sample data in the storage unit 126 in association with information for distinguishing the type of the sample data (may be referred to as type information). .. In another embodiment, the sample data acquisition unit 162 may output the acquired sample data to the index estimation unit 166. In still another embodiment, when the acquired sample data is data in a format other than text data, or when the acquired sample data includes data in a format other than text data, the sample data acquisition unit 162 The acquired sample data may be output to the text data generation unit 164.

In the present embodiment, when the sample data acquired by the sample data acquisition unit 162 is data in a format other than text data, or when the sample data includes data in a format other than text data, the text data generation unit 164 Generates text data from data in a format other than the text data. The text data generation unit 164 may store the text data generated from the specific sample data in the storage unit 126 as a part of the sample data. The text data generation unit 164 may output the text data generated from the specific sample data to the index estimation unit 166 as a part of the sample data.

In one embodiment, the text data generation unit 164 generates the text data in which the human voice included in the voice data is recorded by performing the voice recognition process on the voice data included in the sample data. In another embodiment, the text data generation unit 164 performs text recognition processing on the image data included in the sample data to generate text data in which characters or sign language included in the image data are recorded. ..

In this embodiment, the index estimation unit 166 uses the sample data acquired by the sample data acquisition unit 162 to output the estimated value of the index of the type designated by the user. The type of the index to be output is determined based on, for example, the setting by the user or the initial setting.

Specifically, first, the index estimation unit 166 extracts a plurality of sample data created or updated in the period designated by the user from the plurality of sample data acquired by the sample data acquisition unit 162. Next, the index estimation unit 166 obtains a plurality of sentences to be analyzed by dividing each of the extracted plurality of sample data into one or more sentences. Next, the index estimation unit 166 extracts a sentence having a high possibility of being associated with economic activity as a sentence to be evaluated from the plurality of sentences to be analyzed.

Next, the index estimating unit 166 evaluates, for each of the sentences to be evaluated, the degree of the state or trend of the economic activity indicated by the sentence, and a score corresponding to the evaluation (in the case of an evaluation score). There is). Next, the index estimation unit 166 calculates the index by appropriately processing the evaluation score given to each sentence to be evaluated according to the type of the index specified by the user.

According to the present embodiment, the index estimation system 100 uses the information registered in various SNSs, the information reported in various news, the information collected by various questionnaires, and the information obtained in negotiations with customers and the like. It is possible to calculate the index related to the evaluation of the evaluation target by using big data such as the information described in the business daily report or the business daily report (sometimes referred to as the information described in the negotiation record). .. Details of the index estimation unit 166 will be described later.

In the present embodiment, details of the index estimation system 100 will be described as an example in which the index estimation system 100 targets an economic activity as an evaluation target and outputs estimated values of various economic indexes as an index indicating a state or trend of the economic activity. Was explained. However, the evaluation target of the index estimation system 100 is not limited to economic activity. In another embodiment, the index estimation system 100 may evaluate the popularity or popularity of an individual, a group or a corporation, a product or a service, or a region or a landmark.

Further, in the present embodiment, the details of the index estimation system 100 will be described by taking as an example the case where the index estimation system 100 uses, as teacher data, a survey result regarding economic activities published by a public institution such as a government or a central bank. To be done. However, the index estimation system 100 is not limited to this embodiment. In another embodiment, the index estimation system 100 may use a survey result by a private research institution as teacher data.

The index estimation system 100 may be an example of a machine learning device. The communication unit 122 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The input/output unit 124 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The request reception unit 128 may be an example of a condition acquisition unit, a type information acquisition unit, a period information acquisition unit, and a text data acquisition unit. The model building unit 144 may be an example of a first model building unit and a second model building unit. The sample data acquisition unit 162 may be an example of a type information acquisition unit and a text data acquisition unit. The text data generation unit 164 may be an example of a type information acquisition unit and a text data acquisition unit. The index estimation unit 166 may be an example of an extraction unit or an index calculation unit.

Business confidence may be an example of an evaluation regarding economic activity. Economic activity may be an example of an evaluation target. The survey target may be an example of an evaluation target. The official survey result may be an example of evaluation information. The sample data may be an example of text data. The economic index may be an example of the index. The interview target person may be an example of an information provider.

[Specific Configuration of Each Part of Index Estimation System 100]
Each unit of the index estimation system 100 may be implemented by hardware, software, or a combination of hardware and software. When at least a part of the constituent elements of the index estimation system 100 is realized by software, the constituent element realized by the software is a program that defines an operation related to the constituent element in an information processing device having a general configuration. May be implemented by activating.

The program may be stored in a computer-readable medium such as a CD-ROM, a DVD-ROM, a memory, or a hard disk, or may be stored in a storage device connected to the network. The program may be installed in a computer forming at least a part of the index estimation system 100 from a computer-readable medium or a storage device connected to a network. By executing the program, the computer may function as at least a part of each unit of the index estimation system 100.

The program that causes the computer to function as at least a part of each unit of the index estimation system 100 may include a module that defines the operation of each unit of the index estimation system 100. These programs or modules work on a data processing device, an input device, an output device, a storage device, or the like to cause a computer to function as each unit of the index estimation system 100, or to cause a computer to perform an information processing method in each unit of the index estimation system 100. Let it run.

The information processing described in the program functions as a specific means in which the software related to the program and various hardware resources of the index estimation system 100 cooperate with each other when the program is read by the computer. .. Then, the above-mentioned specific means realizes calculation or processing of information according to the purpose of use of the computer in the present embodiment, whereby the index estimation system 100 according to the purpose of use is constructed.

The above program may be a program for causing a computer to execute a machine learning method. The machine learning method described above includes, for example, (i) an evaluation regarding an evaluation target, and (ii) one or more evaluation information included in the evaluation information associated with one or more explanatory texts indicating the state of the evaluation target or the reason for the evaluation. Constructing a first learning model for determining whether or not the input sentence is a sentence indicating the state of the evaluation target, the evaluation of the evaluation target, or the reason for the evaluation, using the explanatory text as the teacher data. It has one model building stage. The machine learning method is, for example, an extraction step of extracting a sentence related to the evaluation target from one or more sentences included in the text data using the first learning model constructed in the first model construction stage. Have.

FIG. 2 schematically shows an example of the internal configuration of the storage unit 126. In the present embodiment, the storage unit 126 includes a setting information storage unit 222, a sample data storage unit 226, a teacher data storage unit 224, and a model information storage unit 228.

In the present embodiment, the setting information storage unit 222 stores the information indicating the setting related to the index estimation system 100 accepted by the request accepting unit 128. Examples of the above settings include settings related to the type of index estimated by the index estimation system 100, various settings related to machine learning in the index estimation system 100, and various settings related to sample data input to the index estimation system 100. ..

In the present embodiment, the teacher data storage unit 224 stores various data acquired by the teacher data acquisition unit 142 as teacher data used in the model building process of the model building unit 144. For example, the teacher data storage unit 224 associates, for each of the plurality of teacher data, identification information of the data, at least one of information indicating the type of the data and information indicating the target period of the data, and the data. It may be attached and stored.

The data type may be an attribute of the information provider who has provided the information indicating the content of the data. When the teacher data is data of various survey results, the information provider is exemplified by the interview target person in the survey. When the above-mentioned data is data of a specific survey result, the specific example of the data type may be the same as the specific example of the survey type described above.

In the present embodiment, the sample data storage unit 226 stores the sample data acquired by the sample data acquisition unit 162. The sample data storage unit 226, for each of the plurality of sample data, (i) identification information of the sample data, (ii) information indicating the type of the sample data, and information indicating the creation time or update time of the sample data. , And at least one piece of information indicating the creator, updater, or administrator of the sample data and (iii) the sample data may be stored in association with each other. The sample data storage unit 226 may store the text data generated by the text data generation unit 164 for each of the plurality of sample data together with or instead of the sample data.

In the present embodiment, the model information storage unit 228 stores various kinds of information regarding the learning model constructed by the model construction unit 144. For example, the model information storage unit 228 stores, for each of the plurality of learning models, identification information of the model, information indicating the algorithm of the model, and information indicating the value of the parameter of the model in association with each other. The model information storage unit 228 may store, for each of the plurality of learning models, other information regarding the model.

FIG. 3 schematically shows an example of the data table 300. The data table 300 may be an example of teacher data stored in the teacher data storage unit 224. The data table 300 may be an example of a business watcher survey that covers a specific period.

The Economic Watcher Survey, with the cooperation of those who are in a position to observe deep movements related to the local economy (sometimes referred to as interviewees), can accurately and quickly grasp the economic trends in each region. However, it will be carried out with the purpose of using it as a basic material for determining economic trends. Survey items in the economic watcher survey include (i) judgment of the current economic situation, (ii) reason for judgment of the current situation, (iii) additional explanation on the above reason, and explanation of concrete situation, (iv) economic situation Examples include the judgment regarding the future and (v) the reason for the judgment regarding the future.

In the present embodiment, the data table 300 indicates information 312 indicating the field of economic activity, information 314 indicating the area to be surveyed, information 316 indicating the type and occupation of the interviewee, and the current economic situation judgment. The information 322, the information 324 indicating the reason for the determination, and the information 326 indicating the additional explanation and the explanation of the specific situation are stored in association with each other. Each row of the data table 300 may be an example of evaluation information. Each of the information 312 indicating the field of economic activity, the information 314 indicating the area to be surveyed, and the information 316 indicating the type of business and occupation of the interviewee may be an example of the type of survey. The information 322 indicating the current economic situation determination may be an example of an evaluation regarding an evaluation target. The information 324 indicating the reason for the determination may be an example of an explanatory note indicating the reason for the evaluation. The information 326 indicating the additional explanation and the explanation of the specific situation may be an example of the explanation sentence indicating the state of the evaluation target.

The data structure of the data table 300 is not limited to this embodiment. In another embodiment, the data table 300 includes information 312 indicating the field of economic activity, information 314 indicating the area to be surveyed, information 316 indicating the type and type of job of the interviewee, and information indicating the reason for the judgment. At least one of 324 may not be provided. In yet another embodiment, the data table 300 includes a column for storing identification information for identifying each row of the data table 300, a column for storing information indicating the type of survey, and a survey. It may further comprise at least one of the columns for storing information indicating the time of day.

FIG. 4 schematically shows an example of the data table 400. The data table 400 may be an example of the sample data stored in the sample data storage unit 226. In the present embodiment, the details of the data table 400 will be described by taking as an example the case where a sales person in charge of a company inputs a daily business report recording the contents of conversations with customers, negotiations, etc. as sample data. However, it should be noted that the sample data is not limited to this embodiment. In the present embodiment, the customer may be an example of a hearing target person.

In the present embodiment, each row (may be referred to as a record) of the data table 400 stores information of a single sentence. For example, when a single sample data includes a plurality of sentences, the data table 400 stores the information about the sample data by dividing it into a plurality of records. In each of the plurality of records, information regarding each of the plurality of sentences is stored.

In the present embodiment, the data table 400 associates a sample ID 412, a sentence ID 414, information 416 indicating the recording time of each sentence, information 418 indicating the content of each sentence, and information 420 indicating the type of data. To store. The sample ID 412 may be any information that can identify each of the plurality of sample data, and the details thereof are not particularly limited. The sentence ID 414 may be any information as long as it can identify each of a plurality of sentences, and the details thereof are not particularly limited.

The information 416 indicating the recording time indicates the time when each of the plurality of sentences was created or updated. The information 418 indicating the content indicates the content of each of the plurality of sentences. The information 420 indicating the type of data indicates each type of a plurality of sentences. Examples of the type of sentence include the type of sample data in which the sentence was included, the attribute of the information provider who provided the information indicating the content of the sentence, and the like. When the sample data is text data related to a daily business report, a daily business report, or a negotiation record, the information provider is exemplified by a customer or the like who is a hearing target.

According to the example shown in FIG. 4, as the information 420 indicating the type of data, information indicating the attribute of the information provider is stored. The above-mentioned attribute is given to each sentence, for example, when the sample data acquisition unit 162 analyzes the sentence of the daily business report or each sentence. The above-mentioned attribute may be given to each sentence by the sales staff inputting the sentence of the daily business report or the attribute corresponding to each sentence to the input/output unit 124.

The data structure of the data table 400 is not limited to this embodiment. In another embodiment, for each of the plurality of sentences, the data table 400 includes a first data table that stores the sample ID 412, the sentence ID 414, and the information 418 indicating the content in association with each other, and the plurality of sample data. Each of them may be divided into a second data table that stores the sample ID 412 and the information 416 indicating the recording time in association with each other.

FIG. 5 schematically shows an example of the internal configuration of the model construction unit 144. In the present embodiment, the model construction unit 144 includes an extraction model construction unit 522 and an evaluation model construction unit 524.

As described above, the model construction unit 144 constructs various learning models used in the index estimation unit 166. As described above, the index estimation unit 166 extracts a sentence having a high possibility of being associated with economic activity as a sentence to be evaluated from a plurality of sentences to be analyzed. In addition, the index estimation unit 166 evaluates, for each sentence to be evaluated, the degree of the state or trend of the economic activity indicated by the sentence, and gives a score corresponding to the evaluation.

In the present embodiment, the extraction model construction unit 522 constructs a learning model for extracting a sentence to be evaluated from a plurality of sentences to be analyzed. Specifically, the extraction model construction unit 522 includes (i) the evaluation regarding the evaluation target, and (ii) the evaluation information associated with one or more explanatory sentences indicating the state of the evaluation target or the reason for the evaluation. Construct a learning model for determining whether or not the input sentence is a sentence indicating the state of the evaluation target, the evaluation of the evaluation target, or the reason for the evaluation, by using one or more explanatory sentences that are used as teacher data. To do.

For example, the extraction model construction unit 522 uses the information 326 indicating the additional description and the specific situation of the plurality of records constituting the data table 300 as teacher data, and the input sentence is (i) economic. Build a learning model to determine whether it is an evaluation of an activity, (ii) a state of economic activity, or (iii) a statement showing the reason for the evaluation. According to the above learning model, each of the plurality of sentences input to the learning model is similar to the sentence included in the information 326 indicating the additional explanation and the specific situation, and the additional explanation and the specific situation. The sentence is classified as a sentence that is not similar to the sentence included in the information 326 indicating the description. Then, a sentence similar to the sentence included in the information 326 indicating the additional explanation and the explanation of the specific situation indicates (i) evaluation of economic activity, (ii) state of economic activity, or (iii) reason for the evaluation. It is determined that the sentence is the one shown.

The type of learning model described above is not particularly limited. The types of learning models include a neural network model (sometimes abbreviated as NN), a convolutional neural network (sometimes abbreviated as CNN), a logistic regression model (sometimes abbreviated as LR), Examples include a simple word embedding model (sometimes abbreviated as SWEM), a long short term memory model (sometimes abbreviated as LSTM), and a Bidirectional LTSM.

The above learning model converts the input sentence into “(i) evaluation of economic activity, (ii) state of economic activity, or (iii) first sentence indicating reason for the evaluation” or “first A sentence classifier that classifies the sentence into any of the "second sentences other than the first sentence" may be included. The above learning model may include a sentence classifier that classifies into either “statement of evaluation target or sentence indicating evaluation reason” or “statement not of evaluation target state or evaluation reason”. Good. The sentence classifier may include a sentence embedding generator and a classifier pair. The learning model described above may include a plurality of sentence classifiers.

The extraction model construction unit 522 may construct a learning model according to the type of sample data. The extraction model construction unit 522 may determine the type of learning model to be used according to the type of sample data. The extraction model construction unit 522 may determine the number of types of learning models to be used according to the type of sample data. The extraction model construction unit 522 may determine a combination of learning models to be used according to the type of sample data. The sample data may be an example of input data of the learning model. Another example of the input data is data of one or more sentences included in the sample data.

The extraction model construction unit 522 constructs the above learning model by using, as teacher data, information in which one or more sentences included in the sample data are associated with information indicating the type of each sentence. Good. Thereby, for example, a learning model that outputs the determination result according to the attribute of the information provider regarding each sentence is constructed.

In the present embodiment, the evaluation model construction unit 524 constructs a learning model for giving an evaluation score to each of the sentences to be evaluated. Specifically, the evaluation information is used as teacher data to construct a learning model for giving an evaluation score to the input sentence.

For example, the evaluation model construction unit 524 uses, as teacher data, the information 322 indicating the current business condition judgment of a plurality of records that form the data table 300 and the information 326 indicating the additional explanation and the concrete situation. , Build a learning model for giving an evaluation score to the input sentence.

The type of learning model described above is not particularly limited. The learning model described above may be a regression model using a convolutional neural network.

In the data table 300, when the judgment of the current state of the economy is shown by stepwise division, the evaluation model construction unit 524 may determine the score corresponding to each division. Thereby, the evaluation score given to each sentence can be expressed by a continuous numerical value. For example, if the current state of the economy is represented by a five-level evaluation of “becoming better”, “becoming slightly better”, “not changing”, “becoming slightly worse” and “being worse”, the evaluation model building unit 524 gives a score of 2 to the evaluation of “improving”. Similarly, the evaluation model construction unit 524 assigns scores of 1, 0, -1, and -2 to the evaluations of "slightly improved", "unchanged", "slightly worse", and "worse", respectively. Give. As a result, the learning model constructed by the evaluation model constructing unit 524 gives an arbitrary numerical value within the range of −2 to 2 as the evaluation score of the input sentence.

The extraction model construction unit 522 may be an example of the second model construction unit. The evaluation model construction unit 524 may be an example of the first model construction unit.

FIG. 6 schematically shows an example of the internal configuration of the index estimation unit 166. In the present embodiment, the index estimation unit 166 includes a type determination unit 620, an evaluation target extraction unit 630, an evaluation unit 640, and an index generation unit 650. In the present embodiment, the evaluation target extraction unit 630 includes a keyword type extraction unit 632 and a machine learning type extraction unit 634.

The type determination unit 620 determines the type of sample data input to the index estimation unit 166 as an analysis target. For example, the type determination unit 620 refers to the storage unit 126, acquires the type information of the input sample data, and determines the type of sample data based on the type information. The type determination unit 620 may analyze the content of the input sample data and determine the type of the sample data.

The type determination unit 620 outputs information indicating the determination result to, for example, the evaluation target extraction unit 630. The type determination unit 620 may output information indicating the determination result to the evaluation unit 640. In one embodiment, the type determination unit 620 outputs, for each sample data, information indicating the type of the sample data to at least one of the evaluation target extraction unit 630 and the evaluation unit 640. In another embodiment, the type determination unit 620 outputs, for each of one or more sentences included in the sample data, information indicating the type of the sentence to at least one of the evaluation target extraction unit 630 and the evaluation unit 640.

The type determination unit 620 uses the determination result regarding the type of the input sample data, and adds information indicating the type of the sentence to each of one or a plurality of sentences forming the single sample data. May be. The type determination unit 620 may store information indicating the type of each sentence in the sample data storage unit 226. For example, the type determination unit 620 stores information indicating the type of each sentence in the data table 400.

In one embodiment, the type determination unit 620 associates a single sample data with a single type. For example, the type determination unit 620 gives the same information to each of one or a plurality of sentences that form a single sample data as information indicating the type of the sentence. For example, when the information indicating the type of sample data indicates that the sample data is information posted to a specific SNS, the type determination unit 620 determines that all the sentences included in the sample data include Information indicating that the sentence is information posted to a specific SNS is added.

In another embodiment, the type determination unit 620 associates a single sample data with a plurality of types. For example, the type determination unit 620 determines, for each of one or more sentences belonging to the first group and one or more sentences belonging to the second group, among a plurality of sentences forming a single sample data. , Different information is added as information indicating the type of sentence belonging to each group. For example, when the information indicating the type of sample data indicates that the sample data is information recorded in a negotiation record, a daily business report, or a daily business report, the type determination unit 620 obtains from the first interview target person. The sentence included in the sentence related to the obtained information is given the information indicating the attribute of the first interviewee, and the sentence included in the sentence related to the information obtained from the second interviewee is set to the second sentence. Information indicating the attributes of the interviewee is added.

In the present embodiment, the evaluation target extraction unit 630 refers to the setting information storage unit 222 and acquires the setting information regarding the period to be analyzed. The evaluation target extraction unit 630 refers to the sample data storage unit 226 to extract the sample data created or updated at a time that matches the period to be analyzed (sometimes referred to as sample data to be analyzed). get. The evaluation target extraction unit 630 outputs the text data relating to each of the plurality of sample data to be analyzed, the time associated with the content of each sample data, the time each sample data was recorded, or an electronic file containing each sample data. Is acquired in association with the information indicating the time when was created or updated. The evaluation target extraction unit 630 may acquire the text data related to each of the plurality of sample data to be analyzed in association with the information indicating the attribute of the information provider of each sample data. After that, the evaluation target extraction unit 630 uses the learning model constructed by the extraction model construction unit 522 to extract a sentence related to economic activity from one or more sentences included in the sample data.

In the present embodiment, the evaluation target extraction unit 630 uses at least the machine learning type extraction unit 634 to extract a sentence related to economic activity from one or more sentences included in the sample data. The evaluation target extraction unit 630 outputs the extracted sentence to the evaluation unit 640.

The evaluation target extraction unit 630 may use the keyword type extraction unit 632 and the machine learning type extraction unit 634 to extract a sentence related to economic activity from one or more sentences included in the sample data. The evaluation target extraction unit 630 may determine how to combine the extraction process using the keyword type extraction unit 632 and the extraction process using the machine learning type extraction unit 634 based on the type of sample data.

In the present embodiment, the keyword type extraction unit 632 extracts, from one or more sentences included in the sample data, a sentence that matches a predetermined keyword or key phrase-related condition. For example, the keyword type extraction unit 632 is similar to the key phrase, the sentence including the keyword, the sentence matching the key phrase, the sentence including the word similar to the keyword, and the key phrase from the one or more sentences included in the sample data. At least one of the sentences that match the condition (may be referred to as a sentence that matches the keyword, etc.) is extracted as a sentence related to economic activity or a candidate for a sentence related to economic activity.

The keyword type extraction unit 632 may refer to the setting information storage unit 222 to acquire the setting information indicating the condition regarding the keyword or the key phrase. The setting information regarding the keyword or the key phrase includes at least one of the information indicating the keyword and the information indicating the key phrase. The setting information regarding the keyword or the key phrase is at least information indicating whether to extract a sentence including a word similar to the keyword and information indicating whether to extract a sentence matching a condition similar to the key phrase. One may be included.

The keyword type extraction unit 632 may determine the extraction processing condition based on the determination result of the type determination unit 620. As a result, appropriate conditions are set according to the type of input data. In one embodiment, the keyword type extraction unit 632 may determine a word similar to the keyword based on the determination result of the type determination unit 620. The keyword type extraction unit 632 may determine the number of words similar to the keyword based on the determination result of the type determination unit 620. In another embodiment, the keyword type extraction unit 632 may determine the condition similar to the key phrase based on the determination result of the type determination unit 620. The keyword type extraction unit 632 may determine the number of conditions similar to the key phrase based on the determination result of the type determination unit 620.

The keyword type extraction unit 632 includes a sentence including a keyword, a sentence that matches a key phrase, and a word similar to the keyword from one or more sentences included in the sample data based on the determination result of the type determination unit 620. At least one of the sentence and a sentence that matches the condition similar to the key phrase is extracted as (i) a sentence related to economic activity or (ii) extracted as a candidate for a sentence related to economic activity. May be determined. For example, a sentence extracted as a sentence related to economic activity is output to the evaluation unit 640. On the other hand, the sentence extracted as the candidate of the sentence related to the economic activity is output to the machine learning type extraction unit 634.

It is highly possible that not only the sentence that matches the keyword or the like but also one or more sentences arranged near the sentence that matches the keyword or the like are sentences related to economic activity, as compared with the other sentences. Therefore, the keyword-type extraction unit 632 is a sentence including two or more consecutive sentences, including a sentence including a keyword, a sentence matching a key phrase, a sentence including a word similar to the keyword, and a sentence similar to the key phrase. A sentence including at least one of the sentences that match the condition may be extracted as a candidate for a sentence related to economic activity.

In this case, the keyword type extraction unit 632 may determine the number of sentences included in the above sentence based on the determination result of the type determination unit 620. In one embodiment, the number of sentences may be set such that the more noise included in the sample data, the smaller the number of sentences. Here, noise means a sentence that is not related to economic activity. As a result, it is possible to prevent noise from being mixed into one or more sentences output to the evaluation unit 640. In another embodiment, the number of sentences may be set such that the more noise included in the sample data, the greater the number of sentences is. Thereby, omission of extraction of sentences related to economic activity can be suppressed.

In the present embodiment, the machine learning type extraction unit 634 uses the learning model constructed by the extraction model construction unit 522 to extract a sentence related to economic activity from one or more sentences included in the sample data. To do. Specifically, the machine learning type extraction unit 634 inputs at least a part of one or more sentences included in the sample data into the learning model, and the learning model evaluates the sentence indicating the state of economic activity or the economic activity. The sentence that is determined to be the reason for is extracted as a sentence related to economic activity.

In one embodiment, the machine learning type extraction unit 634 inputs all of the sample data to be analyzed into the learning model. In another embodiment, the machine learning type extraction unit 634 inputs a sentence extracted by the keyword type extraction unit 632 as a candidate of a sentence related to economic activity into the learning model. In still another embodiment, the machine learning type extraction unit 634 inputs, into the learning model, sentences that are not extracted by the keyword type extraction unit 632 in the sample data to be analyzed.

In the present embodiment, the evaluation unit 640 gives an evaluation score to the sentence extracted by the evaluation target extraction unit 630. The evaluation unit 640 may give an evaluation score to the sentence extracted by the evaluation target extraction unit 630 using the learning model constructed by the evaluation model construction unit 524. For example, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 into the learning model constructed by the evaluation model construction unit 524. The evaluation unit 640 determines the output of the learning model as the evaluation score of each sentence.

The evaluation unit 640 may use the determination result of the type determination unit 620 to add an evaluation score to the sentence extracted by the evaluation target extraction unit 630. For example, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 and the determination result of the type determination unit 620 regarding each sentence into the learning model constructed by the evaluation model construction unit 524. .. The evaluation unit 640 determines the output of the learning model as the evaluation score of each sentence.

As described above, the data type of each sentence is determined based on, for example, the URI of the sample data including each sentence, the creator or updater of the sample data, the attribute of the information provider regarding the sample data, and the like. To be done. The evaluation unit 640 uses, for example, the attribute of the information provider as the determination result of the type determination unit 620. As a result, the evaluation unit 640 can give an evaluation regarding the evaluation target to each of the plurality of sentences extracted by the evaluation target extraction unit 630 based on the attribute of the information provider regarding each sentence.

The evaluation unit 640 assigns an evaluation score such that the more knowledge the information provider has about the evaluation target, the greater the influence that the information provided by the information provider has on the index generated by the index generation unit 650. You can do it. For example, when the index generation unit 650 generates an index that can be a substitute for the Bank of Japan Tankan, the management of the company is better than the information posted by anonymous posters on the SNS that can be used by an unspecified number of users. The information provided by the person is more suitable as a source of information. Therefore, for example, when the degree of similarity between the attribute of the hearing target of the public survey and the attribute of the information provider is larger than a predetermined standard, the evaluation unit 640 determines that the output value of the learning model is greater than the predetermined value. When it is larger, a value larger than the output value of the learning model is determined as the evaluation score, and when the output value of the learning model is smaller than the predetermined value, a value smaller than the output value of the learning model is determined as the evaluation score.

Thereby, in the determination of the evaluation score, for example, the attribute of the information provider regarding each sentence can be considered. In the economic watcher survey, production dynamic statistics survey, etc. used as teacher data, individuals, groups, or corporations having specific attributes are selected as interviewees. Therefore, when the interview target of the survey used as teacher data matches or is similar to the attribute of the information provider regarding each sentence input as the evaluation target, an evaluation score considering the degree of similarity is given. By doing so, the reliability of the index generated by the index generation unit 650 is improved.

The evaluation unit 640 uses the determination result of the type determination unit 620 to correct the score output by the learning model constructed by the evaluation model construction unit 524, so that the sentence extracted by the evaluation target extraction unit 630 is evaluated. You may give a score. For example, first, the evaluation unit 640 inputs each of the one or more sentences extracted by the evaluation target extraction unit 630 into the learning model constructed by the evaluation model construction unit 524. The evaluation unit 640 determines the output of the learning model as a provisional value of the evaluation score of each sentence. Next, the evaluation unit 640 determines, for example, a value obtained by multiplying the output of the learning model by a correction coefficient according to the type of data of each sentence as the evaluation score of each sentence.

The evaluation unit 640 uses, for example, the attribute of the information provider as the determination result of the type determination unit 620. As a result, the evaluation unit 640 can give an evaluation regarding the evaluation target to each of the plurality of sentences extracted by the evaluation target extraction unit 630 based on the attribute of the information provider regarding each sentence.

For example, the accuracy of the index generated by the index generation unit 650 is set by setting the value of the correction coefficient such that the value of the correction coefficient increases as the degree of association between the information provider and the evaluation target increases. Can improve. For example, by setting the value of the correction coefficient such that the higher the accuracy of the past business forecast by the information provider is, the larger the value of the correction coefficient is, the value of the economic index generated by the index generation unit 650 is calculated. The accuracy can be improved.

In the present embodiment, the index generation unit 650 calculates an index indicating the state of economic activity in the analysis target period, based on the evaluation score given to each of the plurality of sentences extracted by the evaluation target extraction unit 630. The index calculation method is appropriately determined according to the type of index. Thereby, the index estimation system 100 can output the estimated value of the index in the analysis target period.

In one embodiment, the index generation unit 650 calculates the index by totaling the evaluation scores given to each of the plurality of sentences extracted by the evaluation target extraction unit 630. In another embodiment, the index generation unit 650 calculates the index by using the statistical value of the evaluation score given to each of the plurality of sentences extracted by the evaluation target extraction unit 630. Examples of statistical values include average value, median value, quartile, variance, and the like. In still another embodiment, the index generation unit 650 substitutes at least one of the evaluation score and its statistical value given to each of the plurality of sentences extracted by the evaluation target extraction unit 630, into a predetermined mathematical expression, Alternatively, the index is calculated by processing based on a predetermined algorithm. The index generation unit 650 may use the determination result of the type determination unit 620 to determine the parameters of the above formula or algorithm. The index generation unit 650 may determine the parameters of the above formula or algorithm using the attribute of the information provider.

The index generation unit 650 may normalize the calculated index. In one embodiment, the index generation unit 650 normalizes the index by using the maximum value and the minimum value of the index in a period longer than the analysis target period. In another embodiment, the index generation unit 650 may normalize the index by using the number of sentences to which the evaluation score is attached.

The type determination unit 620 may be an example of a type information acquisition unit. The evaluation target extraction unit 630 may be an example of a period information acquisition unit and an extraction unit. The keyword type extraction unit 632 may be an example of the condition acquisition unit and the second extraction unit. The machine learning type extraction unit 634 may be an example of the first extraction unit. The evaluation unit 640 may be an example of the evaluation giving unit. The index generation unit 650 may be an example of the index calculation unit.

FIG. 7 schematically shows an example of the internal configuration of the machine learning type extraction unit 634. In the present embodiment, the machine learning type extraction unit 634 includes a learning model 720 and a determination unit 740. In the present embodiment, the learning model 720 includes a sentence classifier 722, a sentence classifier 724, a sentence classifier 726, and a sentence classifier 728.

In the present embodiment, each of the sentence classifier 722, the sentence classifier 724, the sentence classifier 726, and the sentence classifier 728 included in the learning model 720 evaluates the input sentence regarding the state of economic activity or economic activity. The score indicating the likelihood of being a sentence indicating the reason is output to the determination unit 740. The determination unit 740 receives the input sentence when the total value of the scores output by each of the sentence classifier 722, the sentence classifier 724, the sentence classifier 726, and the sentence classifier 728 is larger than a predetermined threshold value. Is extracted as a sentence related to economic activity.

In the present embodiment, the sentence classifier 722 uses the TF-IDF model as the sentence embedding generator and the LR model as the classifier. In the present embodiment, the text classifier 724 uses the LSTM model as the sentence embedding generator and the NN model as the classifier. In the present embodiment, the sentence classifier 726 uses the CNN model as the sentence embedding generator and the NN model as the classifier. In the present embodiment, the sentence classifier 728 uses the SWEM model as the sentence embedding generator and the LR model as the classifier.

The configuration of the learning model 720 may be determined based on the determination result of the type determination unit 620. For example, the type of model used for the sentence classifier that constitutes the learning model 720 is determined based on the determination result of the type determination unit 620. The combination of models used for the sentence classifier configuring the learning model 720 may be determined based on the determination result of the type determination unit 620. The number of sentence classifiers forming the learning model 720 may be determined based on the determination result of the type determination unit 620.

Using FIG. 8, FIG. 9, and FIG. 10, the evaluation target extraction unit 630 uses the keyword type extraction unit 632 and the machine learning type extraction unit 634 to select an economy from one or more sentences included in the sample data. An outline of information processing for extracting sentences related to activities is explained. FIG. 8 schematically shows an example of information processing in the evaluation target extraction unit 630. FIG. 9 schematically shows another example of information processing in the evaluation target extraction unit 630. FIG. 10 schematically shows still another example of information processing in the evaluation target extraction unit 630.

The information processing in the evaluation target extraction unit 630 is not limited to these embodiments. In another embodiment, the evaluation target extraction unit 630 uses only the machine learning type extraction unit 634 to extract a sentence related to economic activity from one or more sentences included in the sample data.

According to the embodiment shown in FIG. 8, all the sample data to be analyzed are first input to the keyword type extraction unit 632. According to this embodiment, all the sentences extracted by the keyword type extraction unit 632 are input to the machine learning type extraction unit 634. On the other hand, sentences not extracted by the keyword type extraction unit 632 are not input to the machine learning type extraction unit 634.

According to the embodiment shown in FIG. 9, all the sample data to be analyzed are first input to the keyword type extraction unit 632. According to the present embodiment, sentences that have not been extracted by the keyword type extraction unit 632 are input to the machine learning type extraction unit 634. On the other hand, the sentence extracted by the keyword type extraction unit 632 is output to the evaluation unit 640 without being input to the machine learning type extraction unit 634.

According to the embodiment shown in FIG. 10, all the sample data to be analyzed are first input to the keyword type extraction unit 632. According to this embodiment, a part of the sentence extracted by the keyword type extraction unit 632 is output to the evaluation unit 640 without being input to the machine learning type extraction unit 634. On the other hand, the rest of the sentence extracted by the keyword type extraction unit 632 is input to the machine learning type extraction unit 634.

For example, the keyword-type extraction unit 632 is a sentence including two or more consecutive sentences, and includes a sentence including a keyword, a sentence matching a key phrase, a sentence including a word similar to the keyword, and a sentence similar to the key phrase. Consider a case where a sentence including at least one of the sentences that match the condition is extracted as a candidate for a sentence related to economic activity. In this case, a sentence including a keyword, a sentence matching a key phrase, a sentence including a word similar to a keyword, and a sentence matching a condition similar to a key phrase may be a sentence related to economic activity. Relatively high. Therefore, these sentences are output to the evaluation unit 640 without being input to the machine learning type extraction unit 634. On the other hand, the remaining sentences extracted by the keyword type extraction unit 632 are relatively unlikely to be sentences related to economic activities. Therefore, these sentences are input to the machine learning type extraction unit 634.

FIG. 11 schematically shows an example of the data table 1100. In the present embodiment, each row of the data table 1100 may be an example of the evaluation result of the evaluation unit 640. In the present embodiment, the data table 1100 shows a sample ID 1112, a sentence ID 1114, information 1116 indicating a recording time, and information 1118 indicating an evaluation score. Each of the sample ID 1112, the sentence ID 1114, and the information 1116 indicating the recording time may have the same configuration as the sample ID 412, the sentence ID 414, and the information 416 indicating the recording time. The information 1118 indicating the evaluation score indicates the evaluation score given by the evaluation unit 640.

FIG. 12 illustrates an example computer 3000 in which aspects of the invention may be embodied in whole or in part. For example, the index estimation system 100 is realized by the computer 3000.

The program installed in the computer 3000 causes the computer 3000 to function as an operation associated with an apparatus according to an embodiment of the present invention or one or more “units” of the apparatus, or the operation or the one or more “parts”. Parts” and/or cause computer 3000 to perform processes or stages of processes according to embodiments of the present invention. Such programs may be executed by CPU 3012 to cause computer 3000 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

The computer 3000 according to the present embodiment includes a CPU 3012, a RAM 3014, a graphic controller 3016, and a display device 3018, which are interconnected by a host controller 3010. The computer 3000 also includes input/output units such as a communication interface 3022, a hard disk drive 3024, a DVD-ROM drive 3026, and an IC card drive, which are connected to the host controller 3010 via the input/output controller 3020. The computer also includes legacy input/output units such as ROM 3030 and keyboard 3042, which are connected to input/output controller 3020 via input/output chip 3040.

The CPU 3012 operates according to programs stored in the ROM 3030 and the RAM 3014, and thereby controls each unit. The graphic controller 3016 obtains image data generated by the CPU 3012 in a frame buffer or the like provided in the RAM 3014 or itself, and causes the image data to be displayed on the display device 3018.

The communication interface 3022 communicates with other electronic devices via the network. The hard disk drive 3024 stores programs and data used by the CPU 3012 in the computer 3000. The DVD-ROM drive 3026 reads a program or data from the DVD-ROM 3001 and provides the hard disk drive 3024 with the program or data via the RAM 3014. The IC card drive reads programs and data from the IC card and/or writes programs and data to the IC card.

The ROM 3030 stores therein a boot program executed by the computer 3000 at the time of activation, and/or a program dependent on the hardware of the computer 3000. The input/output chip 3040 may also connect various input/output units to the input/output controller 3020 via parallel ports, serial ports, keyboard ports, mouse ports, and the like.

The program is provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The program is read from the computer-readable storage medium, installed in the hard disk drive 3024, the RAM 3014, or the ROM 3030, which is also an example of the computer-readable storage medium, and executed by the CPU 3012. The information processing described in these programs is read by the computer 3000 and brings about the cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing the operation or processing of information according to the use of the computer 3000.

For example, when communication is performed between the computer 3000 and an external device, the CPU 3012 executes the communication program loaded in the RAM 3014, and performs the communication process on the communication interface 3022 based on the process described in the communication program. You may order. The communication interface 3022 reads the transmission data stored in the transmission buffer area provided in the recording medium such as the RAM 3014, the hard disk drive 3024, the DVD-ROM 3001, or the IC card under the control of the CPU 3012, and the read transmission is performed. The data is transmitted to the network, or the received data received from the network is written in a reception buffer area or the like provided on the recording medium.

Further, the CPU 3012 causes the RAM 3014 to read all or necessary portions of files or databases stored in an external recording medium such as a hard disk drive 3024, a DVD-ROM drive 3026 (DVD-ROM 3001), and an IC card. Various types of processing may be performed on the data in RAM 3014. The CPU 3012 may then write back the processed data to an external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored on the recording medium and processed. The CPU 3012 may retrieve various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval from the data read from the RAM 3014, which are described in various places in this disclosure and are specified by the instruction sequence of the program. Various types of processing may be performed, including /replacement, etc., and the result is written back to RAM 3014. Further, the CPU 3012 may search for information in files, databases, etc. in the recording medium. For example, when a plurality of entries each having the attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 3012 selects the first entry from the plurality of entries. Search the entry whose attribute value of the attribute of the specified attribute matches the specified condition, read the attribute value of the second attribute stored in the entry, and set it as the first attribute that satisfies the predetermined condition. The attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored on a computer-readable storage medium on or near computer 3000. Further, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the above program can be executed via the network. Provide to the computer 3000.

Although the present invention has been described above using the embodiment, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is apparent to those skilled in the art that various changes or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

The execution order of each process such as operation, procedure, step, and step in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is "preceding" or "prior to". It should be noted that the output of the previous process can be realized in any order unless the output of the previous process is used in the subsequent process. The operation flow in the claims, the description, and the drawings is described by using “first,” “next,” and the like for the sake of convenience, but it is essential to carry out in this order. Not a thing.

10 communication network, 12 user terminal, 14 teacher data providing server, 16 sample data providing server, 100 index estimation system, 122 communication unit, 124 input/output unit, 126 storage unit, 128 request reception unit, 142 teacher data acquisition unit, 144 Model building unit, 162 sample data acquisition unit, 164 text data generation unit, 166 index estimation unit, 222 setting information storage unit, 224 teacher data storage unit, 226 sample data storage unit, 228 model information storage unit, 300 data table, 312 Information, 314 information, 316 information, 322 information, 324 information, 326 information, 400 data table, 412 sample ID, 414 sentence ID, 416 information, 418 information, 420 information, 522 extraction model construction unit, 524 evaluation model construction Unit, 620 type determination unit, 630 evaluation target extraction unit, 632 keyword type extraction unit, 634 machine learning type extraction unit, 640 evaluation unit, 650 index generation unit, 720 learning model, 722 sentence classifier, 724 sentence classifier, 726 Text classifier, 728 Text classifier, 740 Judgment unit, 1100 data table, 1112 sample ID, 1114 sentence ID, 1116 information, 1118 information, 3000 computer, 3001 DVD-ROM, 3010 host controller, 3012 CPU, 3014 RAM, 3016 Graphic controller, 3018 display device, 3020 input/output controller, 3022 communication interface, 3024 hard disk drive, 3026 DVD-ROM drive, 3030 ROM, 3040 input/output chip, 3042 keyboard

Claims (12)

(I) evaluation regarding the evaluation target, and (ii) the one or more explanation sentences included in the evaluation information associated with one or more explanation sentences indicating the state of the evaluation subject or the reason for the evaluation, as teacher data. A first model construction for constructing a first learning model for determining whether or not the input sentence is a sentence indicating an evaluation of the evaluation target, a state of the evaluation target, or a reason for the evaluation Department,
An extraction unit that extracts a sentence related to the evaluation target from one or more sentences included in the text data using the first learning model constructed by the first model constructing unit;
A machine learning device. The first learning model changes the input sentence into either a sentence indicating the state of the evaluation target or the reason for the evaluation, or a sentence that is not the state of the evaluation target or a sentence indicating the reason for the evaluation. Including a sentence classifier to classify,
The machine learning device according to claim 1. The extraction unit is
At least a part of the one or more sentences included in the text data is input to the first learning model, and it is determined that the first learning model is a sentence indicating the state of the evaluation target or the reason for the evaluation. A first extraction unit that extracts a sentence as a sentence related to the evaluation target;
Have
The machine learning device according to claim 1 or 2. The extraction unit is
A condition acquisition unit for acquiring information indicating a keyword or key phrase related to the evaluation target,
From the one or more sentences included in the text data, a sentence including the keyword, a sentence matching the key phrase, a sentence including a word similar to the keyword, and a condition similar to the key phrase. A second extraction unit that extracts at least one of the sentences to be performed as a sentence related to the evaluation target or a candidate for a sentence related to the evaluation target,
Have
The machine learning device according to claim 3. The first extraction unit,
The sentence extracted by the second extracting unit as a candidate of the sentence related to the evaluation target is input to the first learning model,
A sentence determined to be a sentence indicating the state of the evaluation target or the reason for the evaluation is extracted as a sentence related to the evaluation target.
The machine learning device according to claim 4. The second extraction unit is a sentence including two or more consecutive sentences, including a sentence including the keyword, a sentence matching the key phrase, a sentence including a word similar to the keyword, and the key phrase. Extracting a sentence including at least one of the sentences that match a similar condition as a candidate of a sentence related to the evaluation target;
The machine learning device according to claim 4 or 5. Further comprising a type information acquisition unit for acquiring type information for distinguishing the type of the text data,
The second extraction unit matches a sentence including the keyword and the key phrase from the one or more sentences included in the text data based on the type of the text data indicated by the type information. At least one of a sentence, a sentence including a word similar to the keyword, and a sentence matching a condition similar to the key phrase is one of a sentence related to the evaluation target and a candidate of a sentence related to the evaluation target. Decide whether to extract as,
The machine learning device according to any one of claims 4 to 6. A second model constructing unit that constructs a second learning model for giving an evaluation regarding the evaluation target to the input sentence by using the evaluation information as teacher data;
An evaluation assigning unit that assigns an evaluation regarding the evaluation target to the sentence extracted by the extracting unit using the second learning model constructed by the second model constructing unit;
Further comprising,
The machine learning device according to any one of claims 1 to 7. Based on the evaluation by the evaluation giving unit, an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period,
A period information acquisition unit that acquires information indicating the specific period,
Each of the plurality of text data, a time relating to the content of each of the plurality of text data, a time when each of the plurality of text data was recorded, or an electronic file including each of the plurality of text data, A text data acquisition unit that is acquired in association with time information indicating the time of creation or update,
Further equipped with,
Of the plurality of text data, the extraction unit selects the evaluation target from among a plurality of sentences included in the text data in which the time indicated by the time information associated with the text data is included in the specific period. Extract sentences related to
The evaluation giving unit gives a plurality of sentences extracted from at least a part of the plurality of text data by the extracting unit, and gives an evaluation regarding the evaluation target,
The index calculation unit calculates the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit,
The machine learning device according to claim 8. The text data includes information presented by a statement or gesture of the information provider, or information perceived by the information provider,
The machine learning device,
Based on the evaluation by the evaluation giving unit, an index calculation unit that calculates an index indicating the state or trend of the evaluation target in a specific period,
A text data acquisition unit that acquires each of the plurality of text data in association with attribute information indicating an attribute of each information provider of the plurality of text data,
Further equipped with,
The evaluation giving unit, for each of the plurality of sentences extracted from at least a part of the plurality of text data by the extraction unit, information indicated by the attribute information corresponding to the text data including each sentence. Based on the attributes of the provider, give an evaluation about the evaluation target,
The index calculation unit calculates the index based on the evaluation given to each of the plurality of sentences by the evaluation giving unit,
The machine learning device according to claim 8. A program for causing a computer to function as the machine learning device according to any one of claims 1 to 10. (I) evaluation regarding the evaluation target, and (ii) the one or more explanation sentences included in the evaluation information associated with one or more explanation sentences indicating the state of the evaluation subject or the reason for the evaluation, as teacher data. First model construction for constructing a first learning model for determining whether or not the input sentence is a sentence indicating the state of the evaluation target, the evaluation for the evaluation target, or the reason for the evaluation Stages,
An extraction step of extracting a sentence related to the evaluation target from one or more sentences included in the text data using the first learning model constructed in the first model constructing stage;
A machine learning method.