JP7485706B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

In an online job search service, it may be necessary to add questions related to a company posted on a Q&A (Question and Answer) site (referred to as a "question site") to a company information detail page that introduces the company.

JP 2005-332271 A

However, conventional technology has room for improvement in terms of efficiently providing information to customers (referred to as "users"). For example, because questions posted on question sites are diverse, questions that are not related to employment are often posted, even if they are in the job category. With conventional technology, it is difficult to efficiently filter out such questions that are not related to employment.

The present application has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that enable efficient provision of information to users.

In order to solve the above-mentioned problems and achieve the object, the information processing device according to the present invention is characterized by having a generation unit that extracts useful strings and non-useful strings that are useful in a service other than the service to which the posted information is provided from strings included in posted information related to a specific target, and generates string information that associates target information indicating the target with the useful strings and non-useful strings.

The information processing method according to the present invention is an information processing method executed by an information processing device, and is characterized by including a generation step of extracting useful strings and non-useful strings that are useful in a service other than the service to which the posted information is provided from strings included in posted information related to a specific target, and generating string information that associates target information indicating the target with the useful strings and the non-useful strings.

The information processing program according to the present invention is characterized in that it causes a computer to execute a generation procedure for extracting useful strings and non-useful strings that are useful in a service other than the service to which the posted information is provided from strings included in posted information related to a specific target, and generating string information that associates target information indicating the target with the useful strings and non-useful strings.

The present invention allows information to be provided to users efficiently.

FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the information processing device according to the embodiment. FIG. 3 is a diagram showing a specific example 1 of the selection process according to the embodiment. FIG. 4 is a diagram showing a second specific example of the selection process according to the embodiment. FIG. 5 is a diagram showing a specific example of the extraction process according to the embodiment. FIG. 6 is a flowchart showing an example of the flow of information processing according to the embodiment. FIG. 7 is a diagram illustrating an example of a hardware configuration.

Below, the information processing device, information processing method, and information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. In addition, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[Embodiment]
The configuration of the information processing system 100 according to the embodiment, the configuration of the information processing device 10, a specific example of information processing, and the flow of information processing will be described below in this order, and finally, the effects of the embodiment will be described.

1. Configuration of information processing system 100
The processing of the information processing system 100 according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing a configuration example of the information processing system 100 according to the embodiment. Below, the configuration example of the information processing system 100, the processing of the information processing system 100, and the effects of the information processing system 100 will be described in that order.

(1-1. Configuration example of information processing system 100)
1 includes an information processing device 10. The information processing system 100 may include a plurality of information processing devices 10. The information processing system 100 involves data input to the information processing device 10, such as a question site posting (referred to as "posted information") 20, and data output by the information processing device 10, such as a Q&A list (referred to as "display information") 31 displayed on a job hunting support site display screen 30.

(1-1-1. Information processing device 10)
The information processing device 10 is a device (computer) used by an administrator of a job hunting support site to collect question site posts 20 and create a job hunting support site display screen 30. The information processing device 10 accepts operations by the administrator of the job hunting support site. The information processing device 10 is realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. The example of FIG. 1 shows a case where the information processing device 10 is realized by a desktop PC.

(1-1-2. Q&A site post 20)
The message 20 posted to the question site is text data posted on the question site by users of the site, and includes a question and a response to the question. Here, the message 20 posted to the question site may be data consisting of only a question. The message 20 posted to the question site may also be data consisting of only a posted question and a response that has been evaluated as the optimal response (best answer) among the responses to the question. Furthermore, the message 20 posted to the question site may include image data, video data, or audio data in addition to text data.

(1-1-3. Job hunting support site display screen 30)
The job hunting support site display screen 30 is a web page created for each company (or "business" as appropriate) registered on the job hunting support site, and includes, in addition to a Q&A list 31, information regarding company overview, job openings, annual income/salary, etc.

(1-2. Processing of Information Processing System 100)
In the information processing system 100, first, an operator of the information processing device 10 executes an input process for a message 20 posted to a question site (see FIG. 1 (1)). Next, the information processing device 10 executes an information process for generating a Q&A list 31 from the message 20 posted to the question site (see FIG. 1 (2)). Finally, the information processing device 10 executes an output process for a job hunting support site display screen 30 including the Q&A list 31 (see FIG. 1 (3)). Below, the processing of the information processing system 100 will be described in detail in the order of input processing, information processing, and output processing.

(1-2-1. Input Processing)
1(1), the information processing device 10 receives an input of a message 20 posted to a question site from an operator and acquires input data. At this time, the information processing device 10 may receive an input of the message 20 posted to a question site transmitted from a terminal (not shown) via the communication unit 11 of the information processing device 10.

(1-2-2. Information Processing)
1B, the information processing device 10 generates a Q&A list 31 from the questions posted on the question site 20. The information processing will be described in detail below in the order of the posted information classification process, the character string information generation process, and the posted information selection process.

(1-2-2-1. Posted Information Classification Processing)
As shown in FIG. 1 (2-1), the information processing device 10 performs a process of classifying the inputted question site posted message 20 using a classification model (appropriately, "learning model") 14d, which is a machine learning model (posted information classification process). At this time, the information processing device 10 performs a process of classifying the inputted question site posted message 20 using a classification model 14d such as a deep neural network (DNN) that has been trained using annotation data (appropriately, "learning data") in which collected questions are labeled with whether or not the questions are useful information related to employment. As shown in the example of FIG. 1, when the question site posted message 20 is inputted, the information processing device 10 outputs a classification list 14b-1 indicating the usefulness judgment result for each question as a classification result.

Explaining using the example of FIG. 1, the information processing device 10 classifies question No. 1 posted on the question site post 20, "Is Company Z a foreign-owned company?", as a question related to employment based on the company name "Company Z" and the keyword "foreign-owned company." The information processing device 10 also classifies question No. 2 posted on the question site post 20, "What are the annual salaries at Company Y and Company R?", as a question related to employment based on the company names "Company Y" and "Company R" and the keyword "annual salary."

On the other hand, the information processing device 10 classifies question No. 3 in the question site posting 20, "Is Company L's app convenient?", as a question not related to employment because it contains the company name "Company L" but is a question about a product of "Company L", an app development company. Also, the information processing device 10 classifies question No. 4 in the question site posting 20, "Can I wear Company U's clothes to the interview?", as a question not related to employment because it contains the company name "Company U" but is a question about a product of clothing sales company "Company U".

The information processing device 10 outputs a classification list 14b-1 that lists the employment-related usefulness judgments for question No. 1 "○", question No. 2 "○", question No. 3 "×", question No. 4 "×", etc., as a classification result based on the above classification model 14d.

(1-2-2-2. Character String Information Generation Process)
As shown in FIG. 1 (2-2), the information processing device 10 performs a process of generating a keyword dictionary (suitably referred to as "character string information") 14b-2 used for keyword matching (described later) (character string information generation process). Here, the keyword dictionary 14b-2 is a list of keywords used to determine whether information is useful for a specific service (e.g., a support site for employment, company information, finance, etc.) created for each company. The keywords listed in the keyword dictionary 14b-2 include keywords related to company names (hereinafter, "company name keywords"), as well as useful keywords (suitably referred to as "useful character strings") that are useful when displaying a question on the above service, and non-useful keywords (suitably referred to as "non-useful character strings") that are not useful when displaying a question on the above service.

Here, we will explain how to list company name keywords. The information processing device 10 acquires the official name of a company and the company ID from a company name database, which is a catalog of company names. For example, the information processing device 10 acquires the official name of a company "Y Co., Ltd.", "Y Co., Ltd.", and the company ID "23," and generates a keyword dictionary 14b-2 that links "Y Co., Ltd." and "23." Next, the information processing device 10 acquires information held by an external resource such as an Internet encyclopedia ("external resource information" as appropriate), and adds variations in the spelling of the company name to the above keyword dictionary 14b-2. For example, the information processing device 10 associates "Y Co., Ltd." with being spelled as "Y Co., Ltd.", "Y," "Y Japan," etc., based on information such as the address, date of establishment, name of the president, and number of employees as the above external resource information.

Next, we will explain how to list useful keywords and non-useful keywords. Here, useful keywords are character strings extracted for each company, and are character strings that indicate that questions containing the character string are likely to be useful for a specified service. For example, a useful keyword included in an employment-related question from a workwear sales company, "Company W," is "job openings." On the other hand, non-useful keywords are character strings extracted for each company, and are character strings that indicate that questions containing the character string are likely not useful for a specified service. For example, a non-useful keyword included in a non-employment-related question from a software development company, "Company M," is "update."

When creating a list of useful and unuseful keywords related to employment, the information processing device 10 uses the classification model 14d learned from annotation data related to employment to temporarily label collected unlabeled questions that include specific company names. For example, when creating a keyword dictionary 14b-2 related to "Company Y," the information processing device 10 accepts input of an unlabeled question that includes the company name "Company Y," and uses the classification model 14d related to employment to output temporary label data that is a question that has been assigned a temporary label of employment-related or non-employment-related.

Then, the information processing device 10 extracts, as useful keywords, words that are not often included in the positive examples of the annotation data and the negative examples of the temporary label data, and are relatively frequently included in the positive examples of the temporary label data. On the other hand, the information processing device 10 extracts, as non-useful keywords, words that are not often included in the negative examples of the annotation data and the positive examples of the temporary label data, and are relatively frequently included in the negative examples of the temporary label data. At this time, the information processing device 10 extracts useful keywords and non-useful keywords based on the frequency of appearance of the words and the amount of mutual information. The information processing device 10 may also determine the part of speech of the character string included in the question sentence, and extract only a specific part of speech (e.g., nouns).

Finally, the information processing device 10 generates a keyword dictionary 14b-2 that links the extracted useful and unuseful keywords with company names and company IDs. At this time, the information processing device 10 may extract useful and unuseful keywords that are common to a company category based on the keyword dictionary 14b-2 generated for each company, or may extract general-purpose useful and unuseful keywords that are common to all companies.

(1-2-2-3. Posted Information Selection Process)
1 (2-3), the information processing device 10 uses the keyword dictionary 14b-2 to perform keyword matching of the question text posted to the question site 20 (posted information selection process). At this time, the information processing device 10 outputs a selection list 14b-3, which is a keyword matching result (selection result) that links the question text with a company ID based on the company name keywords written in the keyword dictionary 14b-2.

In addition, the information processing device 10 deletes questions from the selection list 14b-3 based on the classification list 14b-1, which is the classification result. For example, the information processing device 10 deletes questions No. 3 and No. 4, which have a usefulness judgment of "x" in the classification list 14b-1, from among questions No. 1 to No. 4 listed in the selection list 14b-3.

Furthermore, the information processing device 10 changes the link between the question sentence and the company ID in the selection list 14b-3 based on the useful keywords and unuseful keywords listed in the keyword dictionary 14b-2. For example, if there is a question sentence that is linked to a company ID among the question sentences that do not contain a useful keyword, the information processing device 10 removes the link from the company ID. On the other hand, if there is a question sentence that is linked to a company ID among the question sentences that contain a unuseful keyword, the information processing device 10 removes the link from the company ID. At this time, the information processing device 10 can execute the selection process using either the useful keyword or the unuseful keyword. In other words, if there is a question sentence that contains both a useful keyword and an unuseful keyword, the information processing device 10 may give priority to linking the useful keyword, or may give priority to removing the unuseful keyword from the link.

By the above-mentioned process, the information processing device 10 outputs the selection list 14b-3, which is the selection result created for each company, as a Q&A list 31, which is display information to be displayed on the job hunting support site display screen 30. For example, the information processing device 10 outputs the question "How much is the annual salary at Company Y?" and the answer as the Q&A list 31 to be displayed on the job hunting support site display screen 30 for "Company Y."

The information processing device 10 can also output, as display information, information about companies to which similar questions have been posted, based on a selection list 14b-3, which is a selection result created for each company. For example, the information processing device 10 can output "Company R" as a company other than "Company Y" to which the question "How much is the annual salary?" has been posted.

(1-2-3. Output processing)
As shown in Fig. 1 (3), the information processing device 10 outputs a job hunting support site display screen 30 including a Q&A list 31 (see Fig. 1 (3)). At this time, the information processing device 10 may transmit the Q&A list 31, which is the output display information, to a terminal (not shown) via the communication unit 11.

(1-3. Effects of Information Processing System 100)
The effects of the information processing system 100 will be described in detail after specifically explaining the problems with the technology that does not use useful keywords and unuseful keywords.

(1-3-1. Issues)
First, a problem with the classification process using a technology that does not use useful keywords and non-useful keywords, that is, a classification model 14d trained for each predetermined service, will be described. For example, consider classifying a question "If I obtain a qualification such as XX provided by Company M, will it be advantageous to get a job at Company Y?" using the classification model 14d related to employment. In the classification process using the classification model 14d, it reacts to the words "qualification" and "employment" and determines that the question is related to employment. However, in the above classification process, it reacts to the company names "Company M" and "Company Y" and determines that the question is related to employment at "Company M" and "Company Y". Here, since the qualification name "XX" is a qualification provided by "Company M", it is not appropriate to classify it as a question related to employment at "Company M". In the above example, it is preferable that a question containing the qualification name "XX" and the word "qualification" is not linked to "Company M" but is linked to "Company Y". On the other hand, in order to solve the above problem, it is difficult to create a classification model 14d related to employment for each company such as "Company M" and "Company Y" and use it in the classification process.

As described above, different companies have different standards for whether a question is relevant to employment, so it is difficult to create a Q&A list 31 to be displayed on a job hunting support site using only the classification process of the general-purpose classification model 14d. Therefore, in the information processing system 100 according to the embodiment, in addition to the classification process of the classification model 14d, a keyword dictionary 14b-2 containing useful and unuseful keywords for each company is created, and the keyword dictionary 14b-2 is used to select the Q&A list 31, thereby resolving the above problem.

(1-3-2. Overview)
In the information processing system 100, the information processing device 10 acquires the question site posted text 20 posted on the question site, and selects the Q&A list 31 to be displayed on the job hunting support site from the question site posted text 20 for each company registered on the job hunting support site based on at least one of useful keywords that are useful when displaying the Q&A list 31 on the job hunting support site and unuseful keywords that are not useful, and displays the Q&A list 31. At this time, the information processing device 10 classifies the acquired question site posted text 20 using a classification model 14d that has learned the question site posted text 20 and its usefulness on the job hunting support site, and selects a question to be displayed on the Q&A list 31 from the question site posted text 20 classified as useful. Furthermore, the information processing device 10 extracts useful keywords that are useful on the job hunting support site and unuseful keywords that are not useful from among words included in the question site posted text 20 related to the company, and generates a keyword dictionary 14b-2 in which a company ID indicating the company is associated with the useful keywords and unuseful keywords.

(1-3-3. Effects)
Therefore, the information processing system 100 can efficiently provide information to users. That is, the information processing system 100 can efficiently create the Q&A list 31 to be viewed by users of the job hunting support site. Furthermore, the information processing system 100 can automatically generate the keyword dictionary 14b-2 including useful keywords and non-useful keywords for creating the above-mentioned Q&A list 31, so that the Q&A list 31 can be created more efficiently and effectively.

2. Configuration of information processing device 10
The configuration of the information processing device 10 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the information processing device 10 according to the embodiment. As shown in Fig. 2, the information processing device 10 has a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15.

(2-1. Communication Unit 11)
The communication unit 11 is realized by, for example, a network interface card (NIC) etc. The communication unit 11 is connected to a predetermined communication network by wire or wirelessly, and transmits and receives information to and from various devices.

(2-2. Input unit 12)
The input unit 12 is realized by, for example, a keyboard, a mouse, etc. The input unit 12 accepts various operations from an administrator or the like of the information processing device 10. For example, the input unit 12 accepts input of a question site post 20 from an administrator or the like of the information processing device 10.

(2-3. Output unit 13)
The output unit 13 is realized by, for example, a liquid crystal display, etc. The output unit 13 displays various information. For example, the output unit 13 displays a job hunting support site display screen 30 generated by the control unit 15 of the information processing device 10.

(2-4. Storage unit 14)
The storage unit 14 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in Fig. 2, the storage unit 14 according to the embodiment has a collected information storage unit 14a, a processing result storage unit 14b, a learning data storage unit 14c, and a learning model 14d. The storage unit 14 stores various pieces of information referenced when the control unit 15 operates and various pieces of information acquired when the control unit 15 operates.

(2-4-1. Collected information storage unit 14a)
The collected information storage unit 14a stores collected information acquired by the acquisition unit 15a of the information processing device 10. For example, the collected information storage unit 14a stores posted information which is the question site posted message 20 acquired from a question site server, business information which is information on a business acquired from a business name database, external resource information acquired from an external resource such as an Internet encyclopedia, and the like.

(2-4-2. Processing result storage unit 14b)
The processing result storage unit 14b stores the processing results output by the classification unit 15b, the generation unit 15c, and the selection unit 15d of the information processing device 10. For example, the processing result storage unit 14b stores a classification list 14b-1 output by the classification unit 15b, which is the classification result, a keyword dictionary (character string information) 14b-2 output by the generation unit 15c, which is the generation result, and a selection list 14b-3 output by the selection unit 15d, which is the selection result. The processing result storage unit 14b may also store a Q&A list (display information) 31 displayed by the display unit 15e.

(2-4-3. Learning Data Storage Unit 14c)
The learning data storage unit 14c stores learning data for learning the machine learning model. For example, the learning data storage unit 14c stores labeled annotation data such as "text posted on a Q&A site, usefulness judgment regarding employment" as learning data "input data, correct answer information" to be input to the machine learning model 14d.

(2-4-4. Learning model 14d)
The learning model 14d is a word-based linear classification model that is trained to output a useful judgment regarding employment in response to an input of a question site post 20. For example, the learning model 14d is a trained model generated using annotation data regarding employment stored in the learning data storage unit 14c.

(2-5. Control Unit 15)
The control unit 15 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing device 10 using a RAM as a working area. The control unit 15 is also realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 2, the control unit 15 has an acquisition unit 15a, a classification unit 15b, a generation unit 15c, a selection unit 15d, a display unit 15e, and a learning unit 15f, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 15 is not limited to the configuration shown in FIG. 2, and may be other configurations as long as they perform the information processing described below. Also, the connection relationships between the processing units in the control unit 15 are not limited to the connection relationships shown in FIG. 2, and may be other connection relationships.

(2-5-1. Acquisition unit 15a)
The acquisition unit 15a acquires posted information posted on the question site. For example, the acquisition unit 15a acquires posted messages 20 on the question site, which are posted information, from a server that manages the question site. The acquisition unit 15a also acquires company information, which is company information, from a company name database. The acquisition unit 15a also acquires external resource information from external resources such as Internet encyclopedias.

The acquisition unit 15a collects and stores the acquired post information, business information, and external resource information in the information storage unit 14a.

(2-5-2. Classification unit 15b)
The classification unit 15b classifies the acquired posted information using the machine learning model 14d that has learned about the posted information and its usefulness in a predetermined service. For example, the classification unit 15b classifies the acquired question site posted message 20 into either "employment-related" or "non-employment-related" using the classification model 14d that has learned about the question site posted message 20 and its usefulness in a job-hunting support site using annotation data.

The classification unit 15b acquires the question site posts 20 to be classified from the collected information storage unit 14a. Meanwhile, the classification unit 15b stores in the processing result storage unit 14b a classification list 14b-1 in which the usefulness judgment is indicated for each question, which is the classification result.

(2-5-3. Generation unit 15c)
The generation unit 15c extracts useful strings (useful keywords) that are useful in a service other than the service to which the posted information is provided, and non-useful strings (non-useful keywords) that are not useful, from among strings included in the posted information related to a specific target, based on learning data used for training the machine learning model 14d and the classification result output by using the machine learning model 14d from the posted information related to the target, and generates string information in which target information indicating the target is associated with the useful strings and the non-useful strings. For example, the generation unit 15c generates, as the string information, information used to select posted information to be placed and displayed in content related to the target when the content is displayed.

To explain this with a specific example, the generation unit 15c extracts useful keywords that are useful on a job hunting support site and unuseful keywords that are not useful from among the words contained in the questions site posts 20 related to "Company Y" based on the annotation data used to train the employment-related classification model 14d and the classification list 14b-1 output from the classification model 14d from the questions site posts 20 related to the company "Company Y." The generation unit 15c then generates a selection list 14b-3 that associates the company ID "23" indicating "Company Y" with the useful keywords and unuseful keywords.

Explaining the process of extracting useful keywords and non-useful keywords, the generation unit 15c calculates a judgment value including the frequency of occurrence or mutual information of a string, extracts as useful strings strings whose judgment values are less than a predetermined threshold in positive examples of the learning data, whose judgment values are less than the predetermined threshold in negative examples of the classification results, and whose judgment values are equal to or greater than the predetermined threshold in positive examples of the classification results, and generates string information. That is, the generation unit 15c extracts words that are rare in questions indicating employment-related matters at all companies and that are significantly more common in questions indicating employment-related matters at a specific company, as useful keywords that are employment-related keywords that indicate the characteristics of each company.

Meanwhile, the generation unit 15c calculates a judgment value including the frequency of occurrence or mutual information of a string, extracts strings whose judgment value is less than a predetermined threshold in negative examples of the learning data, whose judgment value is less than the predetermined threshold in positive examples of the classification results, and whose judgment value is equal to or greater than the predetermined threshold in negative examples of the classification results, as non-useful strings, and generates string information. That is, the generation unit 15c extracts words that are rare in questions indicating non-employment-related topics at all companies and that are significantly more common in questions indicating non-employment-related topics at a specific company, as non-useful keywords that are non-employment-related keywords that indicate the characteristics of each company.

The generation unit 15c acquires annotation data, which is learning data, from the learning data storage unit 14c. The generation unit 15c also acquires questions site posts 20, which are posted information related to a specific target, from the collected information storage unit 14a. Meanwhile, the generation unit 15c stores keyword dictionary 14b-2, which is character string information of the generated result, in the processing result storage unit 14b.

(2-5-4. Selection unit 15d)
The selection unit 15d selects display information in which a predetermined target is associated with the posted information, based on useful character strings (useful keywords) that are useful when displaying display information related to the posted information in a predetermined service, and/or non-useful character strings (non-useful keywords) that are not useful. For example, the selection unit 15d selects a Q&A list 31, which is display information, for each business entity (company) registered on the job hunting support site, based on at least one of useful keywords and non-useful keywords in the job hunting support site.

The selection unit 15d selects display information from the posted information classified as useful. For example, the selection unit 15d selects a Q&A list 31 for each company from the posts 20 to the question site classified by the classification unit 15b as "employment-related," which is useful for a job-hunting support site. At this time, the selection unit 15d creates a selection list 14b-3 for each company by linking the posts 20 to the question site that include a company name with the company ID, deletes the posts 20 to the question site that are classified by the classification unit 15b as "non-employment-related" from the selection list 14b-3, and selects a Q&A list 31 for each company.

Furthermore, the selection unit 15d selects posted information including useful keywords as display information, and does not select posted information including unuseful keywords as display information. For example, if there is a question sentence that is linked to a company ID among the questions that do not include useful keywords in the selection list 14b-3 for each company created based on the company name, the selection unit 15d removes the question sentence from the link with the company ID. On the other hand, if there is a question sentence that is linked to a company ID among the questions that include unuseful keywords in the selection list 14b-3 for each company created based on the company name, the selection unit 15d removes the question sentence from the link with the company ID. At this time, the selection unit 15d can execute the selection process using either the useful keywords or the unuseful keywords. That is, if there is a question sentence that includes both useful keywords and unuseful keywords, the selection unit 15d may preferentially link the useful keywords, or may preferentially remove the unuseful keywords from the link.

The selection unit 15d acquires the posted information, ie, the question site posts 20, from the collected information storage unit 14a. The selection unit 15d also acquires the classification list 14b-1, which is the classification result, from the processing result storage unit 14b. On the other hand, the selection unit 15d stores the final selection list 14b-3 for each company, which is the selection result, i.e., the Q&A list 31 for each company, in the processing result storage unit 14b.

(2-5-5. Display unit 15e)
The display unit 15e displays the display information. For example, the display unit 15e displays a Q&A list 31 for each company on the job hunting support site display screen 30. Furthermore, the display unit 15e displays the names of companies appearing in common questions as display information on the job hunting support site display screen 30. The display unit 15e acquires the display information such as the Q&A list 31 for each company from the processing result storage unit 14b. Furthermore, the display unit 15e may transmit the display information such as the Q&A list 31 for each company to a business operator terminal or a database (not shown).

(2-5-6. Learning unit 15f)
The learning unit 15f uses annotation data in which a label is added to the collected question text indicating whether the question text is useful information related to employment, to train the machine learning model 14d so as to output a classification result indicating whether the input question site post 20 is "employment-related" or "non-employment-related". At this time, the learning unit 13f may train the machine learning model 14d by backpropagation or the like. The learning unit 15f may also train multiple machine learning models 14d.

[3. Specific examples of information processing]
Next, a specific example of the information processing according to the embodiment will be described. A specific example of the selection process using useful keywords, the selection process using unuseful keywords, and the extraction process of useful keywords and unuseful keywords of the information processing device 10 will be described below.

(3-1. Selection process using useful keywords)
A selection process using useful keywords according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing a specific example 1 of the selection process according to the embodiment. Below, an example of useful keywords included in a question text related to employment from a workwear sales company "Company W" will be described.

As shown in FIG. 3, when "job vacancies" and the like are registered as useful keywords for "Company W" (see FIG. 3 (1)), the information processing device 10 selects questions containing "job vacancies" as employment-related questions for "Company W" (see FIG. 3 (2)). In other words, the information processing device 10 determines that the keyword "job vacancies" is highly likely to be an employment-related question in questions that include the company name of "Company W". As shown in the example of FIG. 3, questions that contain useful keywords indicate a high possibility of being useful for a specified service (e.g., a job hunting support site).

(3-2. Selection process using non-useful keywords)
A selection process using non-useful keywords according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing a specific example 2 of the selection process according to the embodiment. Below, an example of non-useful keywords included in a question text not related to employment from a software development company "Company M" will be described.

As shown in FIG. 4, when "update", "spreadsheet software E", etc. are registered as non-useful keywords for "Company M" (see FIG. 4 (1)), the information processing device 10 selects questions containing "update" and "spreadsheet software E" as non-employment-related questions for "Company M" (see FIG. 4 (2)). That is, the information processing device 10 determines that keywords such as "update", which is related to the services of "Company M", and "spreadsheet software E", which is a product of "Company M", are likely to be non-employment-related questions in questions that include the company name of "Company M". As shown in the example of FIG. 4, questions that contain non-useful keywords are likely to be unuseful for a specified service (e.g., a job hunting support site).

(3-3. Extraction process of useful and non-useful keywords)
The extraction process of useful keywords and non-useful keywords according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing a specific example of the extraction process according to the embodiment. Here, a positive example of annotation data is a question sentence labeled as "employment-related" for all companies, a negative example of annotation data is a question sentence labeled as "non-employment-related" for all companies, a positive example of temporary label data is a question sentence temporarily labeled as "employment-related" for a specific company, and a negative example of temporary label data is a question sentence temporarily labeled as "non-employment-related" for a specific company.

As shown in FIG. 5, the information processing device 10 extracts as useful keywords words that are rare in positive examples of annotation data, rare in negative examples of temporary label data, and common in positive examples of temporary label data. In other words, the information processing device 10 extracts words that are rare in questions related to employment at all companies, and that are noticeably common in questions related to employment at a specific company, as useful keywords that are employment-related keywords that indicate the characteristics of each company.

On the other hand, the information processing device 10 extracts as non-useful keywords words that are rare in negative examples of the annotation data, rare in positive examples of the temporary label data, and common in negative examples of the temporary label data. In other words, the information processing device 10 extracts words that are rare in questions indicating non-employment-related issues across all companies, and that are significantly common in questions indicating non-employment-related issues at a specific company, as non-useful keywords that are non-employment-related keywords that indicate the characteristics of each company.

[4. Information processing flow]
The procedure of information processing of the information processing device 10 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the flow of information processing according to the embodiment. Note that the following steps S101 to S105 may be executed in a different order. Also, among the following steps S101 to S105, some processing may be omitted.

(4-1. Posted Information Acquisition Process)
First, the acquisition unit 15a of the information processing device 10 executes a posted information acquisition process (step S101). For example, the acquisition unit 15a acquires a posted message 20 from a question site server.

(4-2. Posted Information Classification Processing)
Second, the classification unit 15b of the information processing device 10 executes a posted information classification process (step S102). For example, the classification unit 15b classifies the question site posted message 20 into employment-related or non-employment-related messages using the employment-related classification model 14d.

(4-3. Character String Information Generation Process)
Third, the generating unit 15c of the information processing device 10 executes a character string information generating process (step S103). For example, the generating unit 15c generates a keyword dictionary 14b-2 in which useful keywords and non-useful keywords are linked for each company from the annotation data and the question site posts 20 for each company, using the classification model 14d related to employment.

(4-4. Posted Information Selection Process)
Fourth, the selection unit 15d of the information processing device 10 executes a posted information selection process (step S104). For example, the selection unit 15d creates a selection list 14b-3 for each company based on the keyword dictionary 14b-2, and generates a Q&A list 31 to be displayed on the job hunting support site.

(4-5. Posted Information Display Processing)
Fifth, the display unit 15e of the information processing device 10 executes the posted information display process (step S105), and ends the process. For example, the display unit 15e displays the job hunting support site display screen 30 including the Q&A list 31 for each company.

5. Effects of the embodiment
Finally, effects of the embodiment will be described below: Effects 1 to 7 corresponding to the processing according to the embodiment will be described below.

(5-1. Effect 1)
In the process according to the embodiment described above, posted information posted on a question site is acquired, display information in which a predetermined target is associated with the posted information is selected, and the display information is displayed. Therefore, in this process, information can be efficiently provided to users.

(5-2. Effect 2)
In the process according to the embodiment described above, the acquired posted information is classified using a machine learning model that has learned about the posted information and its usefulness in a predetermined service, and display information is selected from the posted information classified as useful. Therefore, in this process, information can be efficiently provided to users using a classification model in a predetermined service.

(5-3. Effect 3)
In the process according to the embodiment described above, posted information including useful character strings is selected as display information, and posted information including non-useful character strings is not selected as display information. Therefore, in this process, it is possible to efficiently provide effective information to users.

(5-4. Effect 4)
In the process according to the embodiment described above, display information that associates businesses registered on the job hunting support site with posted information is selected based on useful character strings that are useful when displaying display information on the job hunting support site and/or non-useful character strings that are not useful. Therefore, this process can efficiently provide information to users of the job hunting support site.

(5-5. Effect 5)
In the process according to the embodiment described above, useful strings that are useful in a service other than the service to which the posted information is provided and non-useful strings that are not useful are extracted from strings included in the posted information related to a specific target based on the learning data used to train the machine learning model and the classification results output from the posted information related to a specific target using the machine learning model, and string information is generated that associates the useful strings and non-useful strings with target information indicating the specific target. Therefore, in this process, by effectively generating string information for each specific target, it is possible to efficiently provide information to users.

(5-6. Effect 6)
In the process according to the embodiment described above, a judgment value including the frequency of occurrence or mutual information of a character string is calculated, and character strings whose judgment value is less than a predetermined threshold in positive examples of the learning data, whose judgment value is less than the predetermined threshold in negative examples of the classification result, and whose judgment value is equal to or greater than the predetermined threshold in positive examples of the classification result are extracted as useful character strings, and character string information is generated. Therefore, in this process, by generating character string information including useful character strings effectively for each predetermined target, information can be efficiently provided to users.

(5-7. Effect 7)
In the process according to the present embodiment described above, a judgment value including the frequency of occurrence or mutual information of a character string is calculated, and character strings for which the judgment value is less than a predetermined threshold in negative examples of the learning data, the judgment value is less than the predetermined threshold in positive examples of the classification result, and the judgment value is equal to or greater than the predetermined threshold in negative examples of the classification result are extracted as non-useful character strings, and the character string information is generated. Therefore, in this process, by generating character string information including non-useful character strings effectively for each predetermined target, it is possible to efficiently provide information to users.

[Hardware configuration]
The information processing device 10 according to the embodiment described above is realized, for example, by a computer 1000 having a configuration as shown in Fig. 7. The information processing device 10 will be described below as an example. Fig. 7 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each component. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000, etc.

HDD 1400 stores programs executed by CPU 1100 and data used by such programs. Communication interface 1500 receives data from other devices via a specified communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the information processing device 10 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 15. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a specified communication network.

〔others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the scope of the gist of the above-described embodiments.

Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

For example, the information processing device 10 described above may be realized by multiple server computers, and depending on the functions, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

The above-described embodiments and variations can be combined as appropriate to the extent that they do not cause inconsistencies in the processing content.

The above-mentioned "section, module, unit" can be read as "means" or "circuit." For example, a control unit can be read as a control means or a control circuit.

REFERENCE SIGNS LIST 10 Information processing device 11 Communication unit 12 Input unit 13 Output unit 14 Memory unit 14a Collected information memory unit 14b Processing result memory unit 14c Learning data memory unit 14d Learning model 15 Control unit 15a Acquisition unit 15b Classification unit 15c Generation unit 15d Selection unit 15e Display unit 15f Learning unit 20 Question site post (posted information)
30 Job hunting support site display screen 31 Q&A list (display information)
100 Information Processing System

Claims (8)

a classification unit that classifies the acquired posted information using a machine learning model that has learned about posted information related to a predetermined target and usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation unit that extracts useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for learning the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generates string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Equipped with
the generation unit calculates a judgment value including an appearance frequency of a character string, and generates, as the useful character string, a character string in which the judgment value is less than a predetermined threshold in a positive example of the learning data, the judgment value is less than the predetermined threshold in a negative example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a positive example of the classification result.
23. An information processing apparatus comprising: a classification unit that classifies the acquired posted information using a machine learning model that has learned about posted information related to a predetermined target and usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation unit that extracts useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for learning the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generates string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Equipped with
the generation unit calculates a judgment value including an appearance frequency of a character string, and generates, as the non-useful character string, a character string for which the judgment value is less than a predetermined threshold in a negative example of the learning data, the judgment value is less than the predetermined threshold in a positive example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a negative example of the classification result.
23. An information processing apparatus comprising: the generation unit extracts useful character strings that are useful on the job hunting support site and non-useful character strings that are not useful from character strings included in posted information that is posted on a question site and is related to businesses registered on a job hunting support site, and generates the useful character strings and the non-useful character strings associated with the businesses.
3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer. the generation unit generates, as the character string information, information used for selecting posted information to be arranged and displayed within content related to a target when the content is displayed;
4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer. An information processing method executed by an information processing device,
a classification step of classifying the acquired posted information using a machine learning model that has learned about posted information related to a predetermined subject and the usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation step of extracting useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for learning the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generating string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Including,
the generating step calculates a judgment value including an appearance frequency of a character string, and generates, as the useful character string, a character string in which the judgment value is less than a predetermined threshold in a positive example of the learning data, the judgment value is less than the predetermined threshold in a negative example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a positive example of the classification result;
23. An information processing method comprising: An information processing method executed by an information processing device,
a classification step of classifying the acquired posted information using a machine learning model that has learned about posted information related to a predetermined subject and the usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation step of extracting useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for learning the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generating string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Including,
the generating step calculates a judgment value including an appearance frequency of a character string, and generates, as the non-useful character string, a character string for which the judgment value is less than a predetermined threshold in a negative example of the learning data, the judgment value is less than a predetermined threshold in a positive example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a negative example of the classification result;
23. An information processing method comprising: a classification step of classifying the acquired posted information using a machine learning model that has learned about posted information related to a predetermined subject and usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation step of extracting useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for training the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generating string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Run the following on your computer :
the generation step includes calculating a judgment value including an appearance frequency of a character string, and generating, as the useful character string, a character string in which the judgment value is less than a predetermined threshold in a positive example of the learning data, the judgment value is less than the predetermined threshold in a negative example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a positive example of the classification result;
2. An information processing program comprising: a classification step of classifying the acquired posted information using a machine learning model that has learned about posted information related to a predetermined subject and usefulness of the posted information on a second website different from a first website to which the posted information is provided;
a generation step of extracting useful strings that are useful on the second website and non-useful strings that are not useful from strings included in the posted information based on learning data used for training the machine learning model and a classification result output from the posted information related to the target using the machine learning model, and generating string information in which target information indicating a string that identifies the target is associated with the useful strings and the non-useful strings;
Run the following on your computer:
the generation step includes calculating a judgment value including an appearance frequency of a character string, and generating, as the non-useful character string, a character string for which the judgment value is less than a predetermined threshold in a negative example of the learning data, the judgment value is less than a predetermined threshold in a positive example of the classification result, and the judgment value is equal to or greater than the predetermined threshold in a negative example of the classification result;
23. An information processing program comprising:

Images