JP2016177477A - Information provision device, information provision method, and information provision program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support to properly produce a task.SOLUTION: An image provision apparatus according to the present application, includes a reception section, a type classification section, a type extraction section, and a provision section. The reception section receives a reply to a requested task and a determination reason of the replay from a contractor. The type classification section classifies the determination reason received by the reception section into a type based on the determination reason and replay. The type extraction section extracts a determination reason belonging to a prescribed type on the basis of the type classified by the type classification section. The provision section provides the determination reason extracted by the type extraction section.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information providing apparatus, an information providing method, and an information providing program.

  2. Description of the Related Art In recent years, crowdsourcing that entrusts business to an unspecified number of people is known as an aspect of business consignment form. As such a technique related to crowdsourcing, a technique for protecting the profits of an orderer (hereinafter referred to as a requester) and a contractor (hereinafter referred to as a worker) is known. For example, in such a technique, a deliverable for a commissioned work (hereinafter referred to as a task) entrusted to a worker by the requester is checked, and the deliverable is delivered to the requester only when the check is passed.

JP 2014-6581 A JP 2004-295607 A

  However, the above-described conventional technology cannot always generate an appropriate task. For example, in the above prior art, the requester creates a task text. Here, the task text may be interpreted differently depending on the worker. For this reason, it is difficult for the requester to create a task text for obtaining a desired product from a worker. For this reason, the above-described conventional technology cannot always generate an appropriate task.

  The present application has been made in view of the above, and an object thereof is to provide an information providing apparatus, an information providing method, and an information providing program capable of supporting generation of an appropriate task.

  The information providing apparatus according to the present application classifies the acceptance reason received by the accepting part into a type based on the judgment reason and the answer, and a reception part that accepts an answer to the requested task and the judgment reason for the answer from the contractor. A type classification unit, a type extraction unit that extracts a determination reason belonging to a predetermined type based on the types classified by the type classification unit, and a provision unit that provides a determination reason extracted by the type extraction unit It is characterized by having.

  According to one aspect of the embodiment, there is an effect that generation of an appropriate task can be supported.

FIG. 1 is an explanatory diagram illustrating an example of provision processing by the provision system according to the embodiment. FIG. 2 is a diagram illustrating an example of a task. FIG. 3 is a diagram illustrating an example of a task with a determination reason. FIG. 4 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. FIG. 5 is a diagram illustrating an example of a task information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of the determination reason information storage unit according to the embodiment. FIG. 7 is a diagram illustrating an example of a type. FIG. 8 is an explanatory diagram for explaining the flow of processing by the information providing apparatus. FIG. 9 is a sequence showing a providing process procedure by the providing system. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the information providing apparatus.

  Hereinafter, a mode for carrying out an information providing apparatus, an information providing method, and an information providing program according to the present application (hereinafter referred to as “embodiment”) will be described in detail with reference to the drawings. In addition, the information providing apparatus, the information providing method, and the information providing program according to the present application are not limited by this embodiment. Moreover, in the following embodiment, the same code | symbol is attached | subjected to the same site | part and the overlapping description is abbreviate | omitted.

[1. Embodiment)
[1-1. Provision Processing According to Embodiment]
First, an example of a providing process according to the embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating an example of provision processing by the provision system 1 according to the embodiment. In the example of FIG. 1, a requester Re (corresponding to an example of an orderer) requests a worker Wa to Wc (corresponding to an example of a contractor) by an information providing apparatus 100 managed by an intermediary (corresponding to an example of a requested task). ) Is provided to provide information useful for the generation of.

  As shown in FIG. 1, the providing system 1 includes an ordering terminal 10, order receiving terminals 50 </ b> A to 50 </ b> C, and an information providing apparatus 100. The ordering terminal 10, the order receiving terminals 50 </ b> A to 50 </ b> C, and the information providing apparatus 100 are connected to a network so that they can communicate with each other by wire or wirelessly. Hereinafter, when the devices of the order receiving terminals 50A to 50C are collectively referred to without distinction, they may be described as “order receiving terminal 50”. In the following, when workers Wa to Wc are collectively referred to without distinction, they may be described as “worker W”.

  The ordering terminal 10 and the order receiving terminal 50 are information processing apparatuses such as a PC (Personal Computer), a tablet-type terminal, a smartphone, a mobile phone, and a PDA (Personal Digital Assistant). The information providing apparatus 100 is a server apparatus that mediates task delegation between the ordering terminal 10 and the order receiving terminal 50.

  The requester Re creates a task. Specifically, the requester Re creates a question sentence as a task. Here, for example, when the question sentence created by the requester Re is an ambiguous expression, there is a problem that the request intention of the requester Re is not transmitted to the worker W. In this case, since the worker W creates a product of the task with an interpretation different from the intention of the requester Re, there is a possibility of creating a product different from the product desired by the requester Re. Therefore, the information providing apparatus 100 executes a providing process for providing the requester Re with information useful for improving the text of the question text of the task created by the requester Re.

  First, the requester Re creates a task for inputting an answer to the question sentence. FIG. 2 is a diagram illustrating an example of a task. In the example of FIG. 2, the question sentence Q1 of the task T1 is “Is this coffee?”. In the example of FIG. 2, the object related to the question sentence Q1 of the task T1 is the image P1 on which coffee is depicted. Further, as shown in FIG. 2, the task T1 is provided with an answer input field AF1 for inputting an answer to the question sentence Q1. In the example of FIG. 2, “Yes”, “No”, or “I don't know” is entered in the answer input field AF1.

  Then, the requester Re operates the ordering terminal 10 to transmit the generated task T1 to the information providing apparatus 100 (step S1). As a result, the information providing apparatus 100 receives the task T1 generated by the requester Re from the ordering terminal 10. Then, the information providing apparatus 100 stores the received task T1.

  Subsequently, the information providing apparatus 100 generates a task T11 with a reason for determination by adding a column for asking the reason for determining the answer to the question sentence Q1 to the task T1. FIG. 3 is a diagram illustrating an example of a task with a determination reason. As illustrated in FIG. 3, the determination reason input fields ReF <b> 1 to ReF <b> 2 for inputting the determination reason for the answer to the question sentence Q <b> 1 are provided in the task T <b> 11 with a determination reason. In the determination reason input field ReF1, if the answer to the question sentence Q1 is “Yes”, the reason why the answer is “Yes” is entered. On the other hand, if the answer to the question sentence Q1 is “No”, the reason why the answer is “No” is entered in the determination reason input field ReF2. When the answer is “I don't know”, the determination reason is input to both the determination reason input field ReF1 and the determination reason input field ReF2. For example, the reason why the answer can be “yes” is entered in the determination reason input field ReF1. On the other hand, the reason why the answer can be “no” is entered in the determination reason input field ReF2.

  Then, the information providing apparatus 100 presents the task T11 with a reason for determination to the workers Wa to Wc (step S2). For example, the information providing apparatus 100 transmits a task T11 with a reason for determination to the order receiving terminals 50A to 50C of the workers Wa to Wc registered as crowd ordering contractors. As a result, the order receiving terminals 50A to 50C receive the task T11 with a reason for determination from the information providing apparatus 100.

  Subsequently, the workers Wa to Wc operate the order receiving terminals 50A to 50C to input an answer to the question sentence Q1 of the task T11 in the answer input field AF1. Further, the workers Wa to Wc input the determination reason for the answer input in the answer input field AF1 in the determination reason input fields ReF1 to ReF2. Then, the order receiving terminals 50A to 50C transmit the answer input to the answer input field AF1 by the workers Wa to Wc and the determination reason input to the determination reason input fields ReF1 to ReF2 to the information providing apparatus 100. Thereby, the information provision apparatus 100 receives the answer with respect to the question sentence Q1, and the judgment reason of an answer from the workers Wa-Wc (step S3).

  Then, the information providing apparatus 100 classifies the accepted determination reasons into types (step S4). Specifically, the information providing apparatus 100 classifies the determination reasons accepted in the type based on the determination reason and the answer. For example, the information providing apparatus 100 classifies the determination reason into six types of PP type, NP type, PNP type, PN type, NN type, and PNN type based on the determination reason and the positive expression and negative expression of the answer. Such a type will be described in detail below.

  First, the reason for determination will be described. The determination reason may be formed by, for example, “reason” and “result”. As an example, in the case of the determination reason “coffee is written because the product is written with coffee”, “coffee is written on the product” corresponds to the reason. On the other hand, “It is coffee” corresponds to a consequence. That is, when the determination reason is formed by “reason” and “consequence”, “reason” corresponds to the determination reason and “consequence” corresponds to the answer. In addition, the reason for the determination of “coffee because coffee is written on the product” is based on “whether or not coffee is written on the product”. Note that the judgment criteria are self-evident when the “reason” and the “result” are the same. For example, a judgment reason such as “coffee because coffee” is an obvious judgment criterion.

  Next, various types will be described. The PP (Positive Positive) type is a type of determination reason in which the reason and the result are positive expressions. For example, in the case of a determination reason “coffee is written on the product, it is coffee”, the reason is an affirmative expression “coffee is written on the product”. Also, the consequence is an affirmative expression “I am coffee”. Therefore, the determination reason “coffee is written because the product is coffee” is classified as a PP type because both the reason and the result are positive expressions.

  The NP (Negative Positive) type is a determination reason type in which the reason is a negative expression and the result is a positive expression. For example, in the case of a judgment reason “There is no evidence that it is not coffee”, the reason is a negative expression “There is no evidence that it is not coffee”. Also, the consequence is an affirmative expression “I am coffee”. Therefore, the reason for the determination that “it is coffee because there is no evidence that it is not coffee” is classified into the NP type because the reason is a negative expression and the result is a positive expression.

  The PNP (Positive Negative Positive) type describes two reasons, and is a type of judgment reason formed by a positive expression reason, a negative expression reason, and a result of the positive expression. For example, in the case of the judgment reason “The product is written as coffee and there is no evidence that it is not coffee”, one reason is an affirmative expression that “the product is written as coffee” and the other reason Is a negative expression that there is no evidence that it is not coffee. Also, the consequence is an affirmative expression “I am coffee”. Therefore, the reason for the judgment that “the product is written as coffee and there is no evidence that it is not coffee” is formed by the reason for the positive expression, the reason for the negative expression, and the result of the positive expression. And PNP type.

  The PN (Positive Negative) type is a determination reason type in which the reason is a positive expression and the result is a negative expression. For example, in the case of a judgment reason “curry is not coffee”, the reason is an affirmative expression “curry”. The consequence is a negative expression "not coffee". Therefore, the reason for the determination that “the curry is not coffee” is classified as the PN type because the reason is a positive expression and the result is a negative expression.

  The NN (Negative Negative) type is a determination reason type in which the reason and the result are negative expressions. For example, in the case of a determination reason “no coffee beans because it does not contain coffee beans”, the reason is a negative expression “no coffee beans are contained”. The consequence is a negative expression "not coffee". Therefore, the reason for the determination that “it is not coffee because it does not contain coffee beans” is classified into the NN type because the reason and the result are negative expressions.

  The PNN (Positive Negative Negative) type describes two reasons, and is a judgment reason type formed by a positive expression reason, a negative expression reason, and a negative expression result. For example, in the case of the judgment reason “it is not black liquid but not coffee”, one reason is a negative expression “not black liquid” and the other reason is “transparent liquid” Is an affirmative expression. The consequence is a negative expression "not coffee". Therefore, the reason for the determination “not coffee because it is not a black liquid but a transparent liquid” is formed by the reason for the positive expression, the reason for the negative expression, and the result of the negative expression. Classified into types.

  Note that the reason for determination that the sentence of reason is inappropriate is classified as a long type. For example, the determination reason described in the determination reason input field ReF2 of “No” even though the answer input in the answer input field AF1 is “Yes” is classified into a long type. In another example, a determination reason that is completely different from the answer to the question sentence Q1 is classified into a long type. For example, even if you answered “No” to the question “Is this coffee?”, The reason for the decision “Yes, it can be yes, because what changes the taste of food is a seasoning.” , Classified into the long type. In another example, a sentence that is not simply a reason such as an impression or a question sentence is classified into a long type. For example, the determination reason such as “No” or “Is this curry?” Is classified into a long type.

  Subsequently, the information providing apparatus 100 extracts a determination reason belonging to a predetermined type based on the classified type (step S5). For example, the information providing apparatus 100 extracts a determination reason belonging to a type other than the PN type and the NP type. That is, the information providing apparatus 100 extracts a determination reason belonging to the PP type, the NN type, the PNP type, and the PNN type. This is because a judgment reason in which a judgment criterion is described in a positive expression is considered to be a useful judgment reason because a relatively useful judgment criterion is often described. In the NN type, since the even number is the PP type, it is considered that the reason for determination belonging to the NN type is also a useful determination reason. Thereby, the information provision apparatus 100 can extract the useful judgment reason among the judgment reasons received from the workers Wa to Wc.

  In the previous example, the information providing apparatus 100 has a PP type determination reason “Coffee is written on the product because it is coffee” as a useful determination reason, and an NN type determination reason “no coffee beans are contained. So it ’s not coffee. ”, PNP type judgment reason“ The product is written as coffee and there is no evidence that it is not coffee. ”And PNN type judgment reason“ It ’s not a black liquid but a transparent liquid. It is not coffee because there is. " On the other hand, the information providing apparatus 100 does not extract the NP type determination reason “Coffee because there is no evidence that it is not coffee” and the PN type determination reason “No curry because it is curry”. For example, it is considered that the criterion of “whether it is curry” or not for determining whether it is coffee is not appropriate because there are many non-coffee items.

  Subsequently, the information providing apparatus 100 classifies the extracted determination reasons into clusters (step S6). Specifically, the information providing apparatus 100 classifies the extracted determination reasons into clusters having the same determination reason and determination criteria. For example, when the number of extracted determination reasons is greater than a predetermined value, the information providing apparatus 100 classifies the extracted determination reasons into clusters that have a common determination criterion with other determination reasons using a hierarchical clustering technique.

  Thereafter, the information providing apparatus 100 extracts useful judgment reasons so that various judgment criteria can be covered with a small number of judgment reasons. Specifically, the information providing apparatus 100 extracts a representative determination reason from each classified cluster (step S7). For example, the information providing apparatus 100 extracts a representative determination reason from each cluster based on the distance function.

  The information providing apparatus 100 provides the requester with the extracted determination reason (step S8). Thereby, the information provision apparatus 100 can provide the requester Re with a useful determination reason carefully selected among the determination reasons received from the workers Wa to Wc.

  As described above, the information providing apparatus 100 according to the embodiment receives an answer to a requested task and a determination reason for the answer from a worker. In addition, the information providing apparatus 100 classifies the determination reasons received in a type based on the determination reason and the answer. Further, the information providing apparatus 100 extracts a determination reason belonging to a predetermined type based on the classified type. The information providing apparatus 100 provides the requester with the extracted determination reason.

  Further, the information providing apparatus 100 according to the embodiment classifies the extracted determination reasons into clusters having the same determination reason and determination criteria. In addition, the information providing apparatus 100 extracts a representative determination reason from the classified clusters. Further, the information providing apparatus 100 provides the extracted determination reason.

  As a result, the information providing apparatus 100 can provide information useful for improving the text of the task, and therefore can support generation of an appropriate task. For example, the information providing apparatus 100 can extract a determination reason based on a useful determination criterion from among the determination reasons for an answer to a task question sentence and provide it to the requester. Can help. For this reason, the information providing apparatus 100 can cause the requester to generate a high-quality task, and can reduce time and effort for the requester to consider the description of the question.

  Moreover, since the information provision apparatus 100 can extract the judgment reason based on an appropriate judgment criterion from the judgment reasons received from the worker, it can provide the requester with a judgment reason useful for improving the question text. . For example, even when the information providing apparatus 100 collects a large number of various determination reasons from the worker by crowdsourcing, the information providing apparatus 100 can carefully select the determination reasons useful for improving the question sentence and provide them to the requester.

  Further, since the information providing apparatus 100 can extract a small number of judgment reasons that cover various judgment criteria from among the judgment reasons for answers to the task question text, it is possible to provide judgment reasons useful for improving the question text. Can be carefully selected and provided to the requester. For this reason, the information providing apparatus 100 can improve the question text with reference to the number of judgment reasons that are easy for the requester to handle, and therefore, it is possible to reduce the trouble for the requester to consider the description of the question.

  Moreover, since the information provision apparatus 100 can improve the quality of a task, it can provide the business consignment form attractive to a worker. For example, the information providing apparatus 100 can support the generation of a question sentence in which the requester's request intention is described with high accuracy. Therefore, the worker creates a deliverable by interpreting it differently from the requester's intention. Can be prevented from being rejected. Further, the information providing apparatus 100 can prevent a situation in which a high-quality worker creates an undesired product of a requester due to an ambiguous description of a task question sentence and is suspected as a spammer.

  In FIG. 1, an example in which the providing system 1 includes one ordering terminal 10, three order receiving terminals 50 </ b> A to 50 </ b> C, and one information providing apparatus 100 is illustrated. May include a plurality of ordering terminals 10, a plurality of order receiving terminals 50 </ b> A to 50 </ b> C, and a plurality of information providing apparatuses 100.

  In addition, in FIG. 1, an example in which a question is received from three workers Wa to Wc has been shown for the sake of simplicity. However, the reason is not limited to three workers Wa to Wc, and a useful determination reason is extracted. Accept reason of judgment from enough workers.

[1-2. Configuration of Information Providing Device According to Embodiment]
Next, the configuration of the information providing apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 4 is a diagram illustrating a configuration example of the information providing apparatus 100 according to the embodiment. As illustrated in FIG. 4, the information providing apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The information providing apparatus 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator who uses the information providing apparatus 100, and a display unit (for example, a liquid crystal display) that displays various types of information. ).

(About the communication unit 110)
The communication unit 110 is realized by a NIC or the like, for example. The communication unit 110 is connected to a network by wire or wirelessly, and transmits / receives information to / from the ordering terminal 10 and the order receiving terminal 50 via the network.

(About the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 includes a task information storage unit 121 and a determination reason information storage unit 122.

(About the task information storage unit 121)
The task information storage unit 121 stores information related to tasks. Specifically, the task information storage unit 121 stores a task question sentence and information on an object for each task. Here, FIG. 5 shows an example of the task information storage unit 121 according to the embodiment. As illustrated in FIG. 5, the task information storage unit 121 includes items such as “task ID”, “question sentence”, and “target object”.

  “Task ID” indicates identification information for identifying a task. “Question sentence” indicates a question sentence of a task. For example, the “question sentence” stores a description of the question sentence created by the requester. The “object” indicates an object related to a task question sentence. For example, the “object” stores an image depicting the subject of the task question sentence.

  That is, FIG. 5 shows an example in which the question sentence of the task T1 identified by the task ID “T1” is “is this coffee?”. In addition, an example in which the object of the task T1 is the image P1 is shown.

(About determination reason information storage unit 122)
The determination reason information storage unit 122 stores information related to a determination reason that is a reason for determining an answer to a task question sentence. Specifically, the determination reason information storage unit 122 stores, for each task, an answer to the task question sentence and a reason for determining the answer. Here, FIG. 6 illustrates an example of the determination reason information storage unit 122 according to the embodiment. As illustrated in FIG. 6, the determination reason information storage unit 122 includes items such as “task ID”, “reason ID”, “answer”, “determination reason”, “type”, and “cluster”.

  “Task ID” indicates identification information for identifying a task. “Reason ID” indicates identification information for identifying a reason for determination. “Answer” indicates an answer to a task question sentence. Specifically, “answer” stores an answer input by the worker in the answer input field of the task. For example, “answer” stores answers such as “yes”, “no”, and “don't know” input by the worker. The “determination reason” indicates the reason why the answer to the task question sentence is determined. Specifically, “reason for determination” stores the reason input to the task determination reason input field by the worker. For example, the reason for determining “yes” is stored in the “reason for determination” when the answer is “yes”. On the other hand, when the answer is “No”, the reason for determining “No” is stored in “Reason for determination”. When the answer is “I don't know”, the “reason for determination” stores the reason why the answer can be “yes” and the reason why the answer can be “no”. “Type” indicates a type of a predetermined type to which the determination reason belongs. Specifically, the “type” stores a type based on the determination reason and the answer. For example, the “type” stores a type to which the determination reason belongs among six types such as PP type, NP type, PNP type, PN type, NN type, and PNN type. The “type” stores a long type when the reason for determination is inappropriate. The “cluster” stores the cluster to which the determination reason belongs. For example, in “cluster”, a cluster having the same judgment criterion as another judgment reason is stored.

  That is, FIG. 6 illustrates an example in which the answer of the reason ID “Re11” to the question text of the task T1 identified by the task ID “T1” is “Yes”. In addition, the reason for the determination of the answer “Yes” for the reason ID “Re11” is an example in which “the item is coffee because it is written as coffee”. Further, the reason for the reason ID “Re11” indicates an example belonging to the type “PP type”. In addition, the determination reason of the reason ID “Re11” is an example classified as “cluster A”.

(About the control unit 130)
The control unit 130 uses, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like to store various programs (corresponding to an example of the information providing program) stored in a storage device inside the information providing apparatus 100 in the RAM. This is realized by being executed as a work area. The control unit 130 is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As shown in FIG. 4, the control unit 130 includes a reception unit 131, a transmission unit 132, a reception unit 133, a type classification unit 134, a type extraction unit 135, a cluster classification unit 136, and a cluster extraction unit 137. And a providing unit 138 to realize or execute the functions and operations of information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 4, and may be another configuration as long as it performs a providing process described later. In addition, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship illustrated in FIG. 4, and may be another connection relationship.

(About the receiver 131)
The receiving unit 131 receives information regarding tasks from the ordering terminal 10. Specifically, the receiving unit 131 receives a task question sentence, an object related to the question sentence, and the like from the ordering terminal 10. Here, the task question sentence is created by, for example, a requester. Further, the object related to the task question text is, for example, an image related to the question text. As an example, the receiving unit 131 receives from the ordering terminal 10 a question sentence “Is this coffee?” And an image depicting coffee that is an object related to the question sentence. In addition, when receiving information about a task from the ordering terminal 10, the receiving unit 131 stores the received information about the task in the task information storage unit 121. For example, the receiving unit 131 assigns a task ID for identifying a task, and stores the question sentence and the target object in the task information storage unit 121 in association with the task ID.

(About transmitter 132)
The transmission unit 132 transmits information on the task to the contractor. Specifically, the transmission unit 132 first generates a task with a reason for determination by adding a column for asking the reason for determining the answer to the question sentence to the task received by the receiving unit 131. For example, as a task with a reason for determination, the transmission unit 132 inputs a reason for inputting a reason for answering “Yes” to the task created by the requester and a reason for making the answer “No”. Generate a task with a reason input field. Then, the transmission unit 132 transmits the generated task with a determination reason to the worker. For example, the transmission unit 132 transmits a task with a determination reason to a worker registered as a crowdsourcing contractor.

(About the reception unit 133)
The accepting unit 133 accepts a delivery from the order receiving terminal 50. Specifically, the accepting unit 133 accepts an answer to the requested task and a determination reason for the answer from the worker. For example, the accepting unit 133 accepts the answer input in the answer input field of the task with the determination reason transmitted by the transmitting unit 132 and the determination reason of the answer input in the determination reason input field.

(About the type classification unit 134)
The type classification unit 134 classifies the determination reasons received by the reception unit 133 into types based on the determination reason and the answer. Then, the type classification unit 134 stores the types classified into the “type” items corresponding to the determination reason among the “type” items in the determination reason information storage unit 122. For example, the type classification unit 134 classifies the determination reasons into six types of PP type, NP type, PNP type, PN type, NN type, and PNN type based on the determination reason and the positive expression and negative expression of the answer. This point will be described with reference to FIG.

  FIG. 7 is an explanatory diagram for explaining the classification to which the determination reason belongs. The reason for the determination “yes” for the answer to the question “Is this tea?” In Example 1 shown in FIG. 7 is “Powdered green tea. So this is tea.” The reason for this judgment is that it is written as “powdered green tea”, and “tea” corresponds to the consequence, so it belongs to the PP type. The reason for the determination of “yes” in response to the question “is this a carbonated beverage?” In Example 2 is “because it is described as cider”. The reason for this judgment is that it is described as “Cider”, and the answer “Yes” (= soda drink) corresponds to the consequence, so it belongs to the PP type.

  The answer to the question “Is this an instant food?” In Example 3 is “Yes” because the answer is “Yes, because it is not a full-fledged food, such as making tea leaves in a pot.” It is. The reason for this decision is that it is not a full-fledged way of making food such as making tea leaves in a pot, and the answer “Yes” (= instant food) is a consequence, so NP Belongs to type.

  The reason for the determination “No” to the question “Is this coffee?” In Example 4 is “Because dressing is not coffee”. Although this reason seems to be NN type at first glance, it is not NN type. This determination reason belongs to the PN type because “dressing” corresponds to the reason and “not coffee” corresponds to the result. The reason for the determination “No” in response to the question “is this tea?” In Example 5 is “coffee”. This determination reason belongs to the PN type because “coffee” corresponds to the reason and “not tea” corresponds to the result.

  The reason for answering “No” to the question sentence “Is this green tea?” In Example 6 is “Because milk tea does not contain green tea ingredients”. The reason for this judgment is that the milk tea does not contain green tea ingredients, and the answer “No” (= not green tea) corresponds to the outcome, and therefore belongs to the NN type. The reason for the determination “No” in response to the question sentence “Is this a seasoning?” Is “The seasoning is what seasons the dish.” The reason for this determination is NN type although there is no negative expression. In this example, since the question sentence is “Is this a seasoning?”, The result is “No” in the answer (= not a seasoning). In addition, when the reasoning “seasoning is what makes a dish taste” is expressed by a logical expression, it becomes “condiment” → “what makes a dish taste”. Since this kinematic pair changes from “flavoring to cooking” to “seasoning”, the “NN” type is determined based on the criterion of “flavoring cooking”. As described above, when the necessary sentence related to the target “is this target?” In the question sentence is included in the sentence of the reason, the NN type is used.

  The answer to the question “Is this coffee?” In Example 8 is “No” because the answer is “This is an herbal tea and coffee is not included in the ingredients, so it is not coffee”. The reasons for this decision include three facts: “This is an herbal tea”, “Coffee is not included in the ingredients”, and “Not coffee”. For this reason, "This is herb tea" is the first reason, "Coffee is not included in the ingredients" is the second reason, and "Not coffee" is the result. Here, the first reason is an affirmative expression, and the second reason and consequence is a negative expression, so the PNN type. The answer to the question in Example 9 “Is this an instant food?” Is “No” because the reason is “Instant food is easy and easy to use, but it is used after roasting beans, so no It can be. The reason for this decision is roughly divided into two sentences: “Instant foods can be used easily and easily” and “Because this is used after roasting beans, it can be no.” Can do. Since the former sentence is the same as in the case of Example 7, it is NN type. The latter sentence is a typical PN type. Since the sentence of the reason for judgment is a combination of these two sentences, it becomes a PNN type. In this way, when the reason sentence is divided into a plurality of parts, the type of the sentence after the division is determined, and the combination of them is set as the reason type.

  Here, the type classification unit 134 classifies the determination reason into a type by a document classification method using machine learning. For example, the type classification unit 134 may classify the reason for determination using BoW (Bag of Words) as a document classification method using machine learning. However, the classification method using BoW cannot be used in word-based BoW expression when the reason for determination is partial character string information, and it is not obvious in what units the feature should be defined. . Therefore, the type classification unit 134 considers a linear identification model that explicitly uses all the partial character strings as features in order to prevent the feature from being missed, and classifies the reason for determination by a classification method using the maximum partial character strings.

Specifically, the type classification unit 134 classifies the determination reason into a type based on the maximum partial character string having the maximum length among the partial character strings having the equivalence relation included in the determination reason. Here, partial character strings having an equivalence relationship will be described. For example, document set d _1, d 2 of the decision _reason, when a · · · d _n, and T connecting put $ special symbols that do not exist in the document between the documents. The number of occurrences of the partial character string q in T is defined as occ (q). Further, the appearance location of the partial character string q in T is defined as P (q) = p ₁ , p ₂ ,... P _{occc (q)} ]. _Further, defined as _{P (q) -c = [p} 1 -c, p 2 -c, ··· p occ (q) -c]. Here, when α and β are partial character strings including an empty character, the two partial character strings q ₁ and q ₂ are q ₁ = αq ₂ β and P (q ₁ ) − | α | = When P (q ₂ ) is satisfied, q ₁ and q ₂ are defined to have an equivalence relationship. Among the partial character strings q having such an equivalence relationship, the one having the maximum length | q | is the maximum partial character string. Then, the type classification unit 134 classifies the reason for determination using, for example, the maximum partial character string as a feature. Thereby, the type classification | category part 134 classify | categorizes the reason for determination with a high precision by a type.

(About the type extraction unit 135)
The type extraction unit 135 extracts a useful determination reason among the determination reasons received from the worker. Specifically, the type extraction unit 135 extracts a determination reason belonging to a predetermined type based on the type classified by the type classification unit 134. For example, the type classifying unit 135 determines, as the predetermined type, a type in which the determination reason is an affirmative expression and a reply is a negative expression, and a determination that belongs to a type other than a type in which the determination reason is a negative expression and the answer is a positive expression Extract the reason. That is, the type classification unit 135 extracts a reason for determination belonging to PP type, NN type, PNP type, and PNN type, which are types other than the PN type and the NP type. The reason for the judgment belonging to PP type, NN type, PNP type or PNN type is that the judgment criteria are described in affirmative expression, so that relatively useful judgment criteria are often described, and useful judgments are made. This is because it is considered to be the reason. On the other hand, the determination reason belonging to the NP type or the PN type is not an extraction target because there are few cases where relatively useful determination criteria are described. As an example, the type extraction unit 135 refers to the “type” stored in the determination reason information storage unit 122 and extracts the determination reasons belonging to the PP type, the NN type, the PNP type, and the PNN type.

(About the cluster classification unit 136)
The cluster classification unit 136 classifies the determination reasons extracted by the type extraction unit 135 into clusters having the same determination reasons and determination criteria. For example, the cluster classification unit 136 classifies the determination reason into clusters when the number of determination reasons extracted by the type extraction unit 135 is greater than a predetermined value.

  As one aspect, the cluster classification unit 136 repeats the hierarchical clustering method when the generalization of the determination reason extracted by the type extraction unit 135 is difficult or inappropriate, thereby allowing other determination reasons and determination criteria. Classify the reason for judgment into a common cluster. That is, determination reasons based on the same determination criteria are classified into the same cluster. In the example of FIG. 6, the cluster classification unit 136 determines the reason for the reason ID “Re11” “the coffee is written because the product is coffee” and the reason for the reason ID “Re13” Since it is written as “Coffee and there is no evidence that it is not coffee, it is coffee”, the judgment criterion is “Coffee is written on the product”, so it is classified as “Cluster A”.

  For example, the cluster classification unit 136 classifies the determination reason by condensed clustering as a hierarchical clustering method. As an example, the cluster classification unit 136 generates a cluster from each determination reason extracted by the type extraction unit 135, and sequentially merges two clusters having the smallest result of the distance function between the clusters, thereby determining the determination reason. Are classified into clusters.

  In another example, the cluster classification unit 136 classifies the determination reason by branching clustering as a hierarchical clustering method. As an example, the cluster classification unit 136 generates one cluster from the determination reason extracted by the type extraction unit 135, and sequentially divides the generated cluster based on a distance function between the clusters, thereby determining the determination reason as a cluster. Classify into:

  Here, the cluster classification unit 136 executes the condensation type clustering or the branching type clustering using, for example, tf-idf or cosine similarity as the distance function. As an example, the cluster classification unit 136 performs clustering using the cosine similarity calculated for only the top k cases. Note that “k” is a value obtained by experiments or the like and having a good cluster reason.

(About the cluster extraction unit 137)
The cluster extraction unit 137 extracts a useful determination reason among the determination reasons extracted by the type extraction unit 135. Specifically, the cluster extraction unit 137 extracts a representative determination reason from the clusters classified by the cluster classification unit 136. For example, the cluster extraction unit 137 converts the representative determination reason into a type other than the type in which the determination reason is a positive expression and the answer is a negative expression, and the determination reason is a negative expression and the answer is a positive expression. Extract based on the probability of belonging. That is, the cluster extraction unit 137 extracts based on the probability that the determination reason belongs to a PP type, NN type, PNP type, or PNN type that is a type other than the PN type and the NP type. As an example, the cluster extraction unit 137 extracts a determination reason having the highest probability of belonging to the PP type, the NN type, the PNP type, and the PNN type. Thereby, the cluster extraction unit 137 can extract the useful determination reason with higher accuracy than the case of extracting the determination reason randomly.

(About the providing unit 138)
The providing unit 138 provides information useful for improving the text of the question sentence of the task created by the requester. Specifically, the providing unit 138 provides the determination reason extracted by the cluster extracting unit 137 to the requester who is the orderer of the task. For example, when the number of determination reasons extracted by the type extraction unit 135 is greater than a predetermined value, the providing unit 138 provides the determination reason extracted by the cluster extraction unit 137 and extracted by the type extraction unit 135. If the number of determination reasons is less than the predetermined value, the determination reason extracted by the type extracting unit 135 is provided. As an example, the providing unit 138 provides the requester with a list of determination reasons arranged in descending order of probability belonging to the PP type, the NN type, the PNP type, and the PNN type.

  Here, the overall flow of the reception process, the type classification process, and the cluster classification process will be described with reference to FIG. FIG. 8 is an explanatory diagram for explaining the flow of processing by the information providing apparatus 100. As shown in FIG. 8, the information providing apparatus 100 first inputs a determination reason as step 1. Specifically, the information providing apparatus 100 receives an answer to the requested task and a reason for determining the answer from the worker.

  Next, the information providing apparatus 100 extracts an appropriate determination reason as step 2. Specifically, the information providing apparatus 100 first classifies the accepted determination reason into a type based on the determination reason and the answer. Subsequently, the information providing apparatus 100 extracts a determination reason belonging to a predetermined type based on the classified type.

  Finally, the information providing apparatus 100 lists the reasons for determination as step 3. Specifically, the information providing apparatus 100 first classifies the extracted determination reasons into clusters. Subsequently, the information providing apparatus 100 extracts a representative determination reason from the classified clusters. Then, the information providing apparatus 100 lists the extracted determination reasons and provides them to the requester.

[1-3. Provision Processing Procedure According to Embodiment]
Next, a processing procedure by the providing system 1 according to the embodiment will be described with reference to FIG. FIG. 9 is a sequence showing a providing process procedure by the providing system 1 according to the embodiment.

  As shown in FIG. 9, the ordering terminal 10 transmits a task to the information providing apparatus 100 (step S101). For example, the ordering terminal 10 transmits a question sentence and an image related to the question sentence to the information providing apparatus 100. As a result, the information providing apparatus 100 receives a task from the ordering terminal 10. Then, the information providing apparatus 100 stores the received task question sentence and the image in the task information storage unit 121 in association with each other.

  Subsequently, the information providing apparatus 100 presents a task with a determination reason (step S102). For example, the information providing apparatus 100 transmits a task with a determination reason in which an answer determination reason input column is added to the task received from the ordering terminal 10 to the order receiving terminal 50A and the order receiving terminal 50B (hereinafter referred to as the order receiving terminal 50).

  After that, the information providing apparatus 100 receives an answer to the task question and the reason for judging the answer from the order receiving terminal 50 (step S103). For example, the worker operates the order receiving terminal 50, inputs an answer in the answer input field, and inputs the determination reason in the determination reason input field. Then, the order receiving terminal 50 transmits the answer input in the answer input field by the worker and the determination reason input in the determination reason input field to the information providing apparatus 100. Thereby, the information providing apparatus 100 receives from the order receiving terminal 50 the answer input to the answer input field by the requester and the determination reason input to the determination reason input field.

  Then, the information providing apparatus 100 classifies the determination reasons into types (step S104). For example, the information providing apparatus 100 classifies the accepted determination reason into a type based on the determination reason and the answer.

  Subsequently, the information providing apparatus 100 extracts a determination reason belonging to a predetermined type based on the classified type (step S105). For example, the information providing apparatus 100 determines, as the predetermined type, a determination that belongs to a type other than the type in which the determination reason is an affirmative expression and the answer is a negative expression, and the determination reason is a negative expression and the answer is an affirmative expression. Extract the reason.

  Thereafter, the information providing apparatus 100 classifies the extracted determination reasons into clusters (step S106). For example, the information providing apparatus 100 classifies the extracted determination reasons into clusters having the same determination criteria as other determination reasons by the hierarchical clustering method.

  Subsequently, the information providing apparatus 100 extracts a representative determination reason from the classified clusters (step S107). For example, the information providing apparatus 100 extracts a typical determination reason based on a probability that the determination reason belongs to a PP type, an NN type, a PNP type, or a PNN type, which is a type other than the PN type and the NP type.

Then, the information providing apparatus 100 lists the extracted determination reasons and provides them to the requester (step S108). For example, when the number of determination reasons extracted by the type extraction unit 135 is greater than a predetermined value, the information providing apparatus 100 provides the determination reason extracted by the cluster extraction unit 137 and is extracted by the type extraction unit 135. If the number of determination reasons is less than a predetermined value, the determination reason extracted by the type extracting unit 135 is provided.

[1-4. Effects of the embodiment
As described above, the information providing apparatus 100 according to the embodiment includes the reception unit 133, the type classification unit 134, the type extraction unit 135, and the provision unit 138. The accepting unit 133 accepts an answer to the requested task and a reason for judging the answer from the contractor. The type classification unit 134 classifies the determination reasons received by the reception unit 133 into types based on the determination reason and the answer. The type extraction unit 135 extracts a determination reason belonging to a predetermined type based on the types classified by the type classification unit 134. The providing unit 138 provides the determination reason extracted by the type extracting unit 135.

  Further, in the information providing apparatus 100 according to the embodiment, the type extracting unit 135, as the predetermined type, has a type of reason for determination that is an affirmative expression and a type of answer that is a negative expression and a type of reason for a negative expression that is positive for a reply. The reason for determination belonging to a type other than the type that is the expression is extracted.

  In the information providing apparatus 100 according to the embodiment, the type classification unit 134 classifies the determination reason into a type by a document classification method using machine learning.

  In the information providing apparatus 100 according to the embodiment, the type classification unit 134 classifies the determination reason into a type based on the maximum partial character string having the maximum length among the partial character strings having the equivalence relation included in the determination reason. To do.

  The information providing apparatus 100 according to the embodiment further includes a cluster classification unit 136 and a cluster extraction unit 137. The cluster classification unit 136 classifies the determination reasons extracted by the type extraction unit 135 into clusters having the same determination reasons and determination criteria. The cluster extraction unit 137 extracts a representative determination reason from the clusters classified by the cluster classification unit 136. The providing unit 138 provides the determination reason extracted by the cluster extracting unit 137.

  In the information providing apparatus 100 according to the embodiment, the cluster classification unit 136 classifies the determination reasons into clusters when the number of determination reasons extracted by the type extraction unit 135 is larger than a predetermined value. The cluster extraction unit 137 extracts a representative determination reason from the clusters classified by the cluster classification unit 136. The providing unit 138 provides the determination reason extracted by the cluster extraction unit 137 when the number of determination reasons extracted by the type extraction unit 135 is greater than a predetermined value, and the determination reason extracted by the type extraction unit 135 If the number is less than the predetermined value, the determination reason extracted by the type extracting unit 135 is provided.

  Further, in the information providing apparatus 100 according to the embodiment, the cluster classification unit 136 generates a cluster from each determination reason extracted by the type extraction unit 135, and obtains two clusters having the smallest result of the distance function between the clusters. By merging in order, the reasons for judgment are classified into clusters.

  In the information providing apparatus 100 according to the embodiment, the cluster classification unit 136 generates one cluster from the determination reason extracted by the type extraction unit 135, and sequentially divides the generated cluster based on the distance function between the clusters. By doing so, the reason for determination is classified into clusters.

  Further, in the information providing apparatus 100 according to the embodiment, the cluster extraction unit 137 determines a typical determination reason as a type in which the determination reason is an affirmative expression and a reply is a negative expression, and a determination reason is a negative expression. Is extracted based on the probability of belonging to a type other than the type that is an affirmative expression.

  As a result, the information providing apparatus 100 can provide information useful for improving the text of the task, and therefore can support generation of an appropriate task. For example, the information providing apparatus 100 can extract a determination reason based on a useful determination criterion from among the determination reasons for an answer to a task question sentence and provide it to the requester. Can help. For this reason, the information providing apparatus 100 can cause the requester to generate a high-quality task, and can reduce time and effort for the requester to consider the description of the question.

  Moreover, since the information provision apparatus 100 can extract the judgment reason based on an appropriate judgment criterion from the judgment reasons received from the worker, it can provide the requester with a judgment reason useful for improving the question text. . For example, even when the information providing apparatus 100 collects a large number of various determination reasons from the worker by crowdsourcing, the information providing apparatus 100 can carefully select the determination reasons useful for improving the question sentence and provide them to the requester.

  Further, since the information providing apparatus 100 can extract a small number of judgment reasons that cover various judgment criteria from among the judgment reasons for answers to the task question text, it is possible to provide judgment reasons useful for improving the question text. Can be carefully selected and provided to the requester. For this reason, the information providing apparatus 100 can improve the question text with reference to the number of judgment reasons that are easy for the requester to handle, and therefore, it is possible to reduce the trouble for the requester to consider the description of the question.

  Moreover, since the information provision apparatus 100 can improve the quality of a task, it can provide the business consignment form attractive to a worker. For example, the information providing apparatus 100 can support the generation of a question sentence in which the requester's request intention is described with high accuracy. Therefore, the worker creates a deliverable by interpreting it differently from the requester's intention. Can be prevented from being rejected. Further, the information providing apparatus 100 can prevent a situation in which a high-quality worker creates an undesired product of a requester due to an ambiguous description of a task question sentence and is suspected as a spammer.

[2. (Modification)
The information providing apparatus 100 according to the above-described embodiment may be implemented in various different forms other than the above-described embodiment. Therefore, in the following, another embodiment of the information providing apparatus 100 will be described.

[2-1. Cluster classification)
In the above embodiment, the information providing apparatus 100 has been described with an example in which the reason for determination extracted by the type extracting unit 135 is classified into clusters. Here, the information providing apparatus 100 may classify the determination reasons received by the receiving unit 133 into clusters, not limited to the determination reasons extracted by the type extracting unit 135.

  Specifically, first, the accepting unit 133 of the information providing apparatus 100 accepts an answer to the requested task and a reason for judging the answer from the worker. Subsequently, the cluster classification unit 136 classifies the determination reason received by the reception unit 133 into clusters. Thereafter, the cluster extraction unit 137 extracts a representative determination reason from the clusters classified by the cluster classification unit 136. Then, the providing unit 138 provides the determination reason extracted by the cluster extracting unit 137.

  As a result, the information providing apparatus 100 can classify the determination reasons received by the receiving unit 133 into clusters as well as the determination reasons extracted by the type extracting unit 135. Even when the number of judgment reasons is too small, representative judgment reasons can be extracted from the cluster and provided to the requester. In this case, for example, the information providing apparatus 100 may not include the type classification unit 134 and the type extraction unit 135.

[2-2. (Type classification)
In the above embodiment, an example in which the information providing apparatus 100 provides the determination reason extracted by the cluster extraction unit 137 has been described. Here, the information providing apparatus 100 may provide the determination reason extracted by the type extraction unit 135 as well as the determination reason extracted by the cluster extraction unit 137.

  Specifically, first, the accepting unit 133 of the information providing apparatus 100 accepts an answer to the requested task and a reason for judging the answer from the worker. Subsequently, the type classification unit 134 of the information providing apparatus 100 classifies the determination reasons received by the reception unit 133 into types based on the determination reason and the answer. After that, the type extraction unit 135 extracts a determination reason belonging to a predetermined type based on the types classified by the type classification unit 134. Then, the providing unit 138 provides the determination reason extracted by the type extracting unit 135.

  Accordingly, the information providing apparatus 100 can provide the determination reason extracted by the type extraction unit 135 as well as the determination reason extracted by the cluster extraction unit 137, and thus the determination extracted by the cluster extraction unit 137. Even if the number of reasons is too small, the requester can be provided with a useful reason for the decision. In this case, for example, the information providing apparatus 100 may not include the cluster classification unit 136 and the cluster extraction unit 137.

[2-3. mapping〕
In the above embodiment, the information providing apparatus 100 has been described with an example in which the reason for determination extracted by the type extracting unit 135 is classified into clusters. Here, the information providing apparatus 100 may classify the determination reason into clusters after mapping the determination reason (instance) of the lower element to the determination reason (class) of the upper element.

  Specifically, the cluster classification unit 136 of the information providing apparatus 100 classifies the determination reason into clusters after converting the determination reason of the lower concept into the determination reason of the higher concept among the determination reasons. For example, the answer “Yes” for the answer to the question “Is this coffee?” Is “Coffee” because it is written as “Coffee”, and “Coffee” corresponds to the higher concept, so it is the reason for the higher concept. In addition, the answer to the question “Is this coffee?” For the answer “Yes” is “Coffee because it is a coffee can”, and “Coffee” is a superordinate concept. On the other hand, the answer to the question “Is this coffee?” Is “Yes” because of the reason “Georgia (registered trademark) coffee”, “Georgia (registered trademark)” is the brand name of coffee and subordinate to “coffee” Since it falls under the concept, it becomes the reason for judging the subordinate concept. Here, there are not relatively many properties that specify coffee, but if you include instances that are subordinate concepts, they will increase and the number of clusters will increase. Therefore, in this case, the cluster classification unit 136 converts “Georgia (registered trademark)” of the determination reason “coffee because Georgia (registered trademark)” into a higher concept “coffee” and classifies the cluster into clusters. Thereby, the cluster classification | category part 136 can suppress the number of clusters.

  In another example, the answer to the question “Is this coffee?” “No” is the reason for judging “No coffee because it is food”, because “Food” falls under the superordinate concept. . On the other hand, the answer to the question “Is this coffee?” Is “No” because the curry is a subordinate concept of “Food”. Become. In this case, the cluster classification unit 136 converts “curry” of the determination reason “not curry because it is curry” into “superior” “food” and classifies it into clusters. Here, the reason why the answer is “No” is a reference to a property that the coffee does not have, so there are many, and the number of clusters increases. Therefore, in this case, the cluster classification unit 136 converts the “curry” for the determination reason “not curry because it is curry” into the “food” of the higher concept and classifies it into clusters. Thereby, the cluster classification | category part 136 can suppress the number of clusters.

  As described above, the information providing apparatus 100 classifies the determination reasons into clusters after converting the determination reasons of the lower concept into the determination reasons of the higher concept among the determination reasons. As a result, the information providing apparatus 100 can unify the reason for determining the lower concept into the reason for determining the upper concept, and thus can suppress the number of clusters. For this reason, since the information providing apparatus 100 extracts representative determination reasons from the suppressed number of clusters, the information providing device 100 can provide a number of determination reasons that are easy to handle for the requester.

[2-4. (We give weight to the reasons for positive responses)
In the above embodiment, the information providing apparatus 100 has been described with an example in which the reason for determination extracted by the type extracting unit 135 is classified into clusters. Here, the information providing apparatus 100 may perform the clustering by weighting the determination reason for the positive expression answer than the determination reason for the negative expression answer.

  Specifically, the cluster classification unit 136 of the information providing apparatus 100 sets the determination reason related to the answer of the positive expression among the determination reasons more weighted than the determination reason related to the answer of the negative expression and makes it the center of the cluster. This is because the determination reason for the positive expression answer is often a useful determination reason because a direct reason is often written as the determination reason compared to the determination reason for the negative expression answer. For this reason, the cluster classification unit 136 provides a useful determination reason by making the determination reason related to the answer of the positive expression more weighted than the determination reason related to the answer of the negative expression and making it the center of the cluster.

  In this way, the information providing apparatus 100 places the determination reason for the positive expression answer among the determination reasons more weighted than the determination reason for the negative expression answer, and sets the determination reason as the center of the cluster. As a result, the information providing apparatus 100 can place importance on the determination reason related to an affirmative expression answer, which is a useful determination reason, and can provide the requester with a useful determination reason with high accuracy.

[2-5. Extraction of reason for determination based on tf-idf value]
In the above embodiment, the information providing apparatus 100 has been described by taking an example in which a representative determination reason is extracted from the cluster. Here, the information providing apparatus 100 may extract a representative determination reason based on the tf-idf value of the determination reason.

  Specifically, the cluster extraction unit 137 of the information providing apparatus 100 selects a representative determination reason from the clusters selected based on the tf-idf value of the determination reason among the clusters classified by the cluster classification unit 136. Extract from For example, the cluster extraction unit 137 preferentially selects a cluster having a high determination reason tf-idf value from the clusters classified by the cluster classification unit 136, and extracts a representative determination reason from the selected clusters. . This is because a cluster having a high tf-idf value and a few few words strongly influences the same meaning and is considered to be a useful determination reason.

  The tf-idf value is calculated based on the product of two indices, tf (Term Frequency) indicating the appearance frequency of words and idf (Inverse Document Frequency) indicating the inverse document frequency. Here, idf works as a kind of general word filter. Specifically, idf serves to increase the importance of words that appear only in a specific document by lowering the importance of general words that appear in many documents. For this reason, the higher the tf-idf value, the higher the possibility that it is a more useful determination reason. Therefore, the cluster extraction unit 137 preferentially selects a cluster having a high determination reason tf-idf value from the clusters classified by the cluster classification unit 136, and extracts a representative determination reason from the selected clusters. Thus, it is possible to extract a useful determination reason with high accuracy.

  As described above, the information providing apparatus 100 extracts a representative determination reason from the clusters selected based on the tf-idf value of the determination reason among the clusters classified by the cluster classification unit 136. As a result, the information providing apparatus 100 can extract the reason for determination useful for the requester with high accuracy, and can provide the reason for determination useful for the requester with high accuracy.

[2-6. Bayon clustering]
In the above embodiment, an example in which the information providing apparatus 100 extracts a representative determination reason from the clusters classified by the cluster classification unit 136 has been described. Here, the information providing apparatus 100 may extract a representative determination reason using Bayon clustering.

  For example, in the Bayon clustering, the information providing apparatus 100 first stores all data in one cluster. Thereafter, the information providing apparatus 100 performs clustering by repeatedly dividing the data set into two. As an example, the information providing apparatus 100 first selects one cluster to be divided from a plurality of clusters. Here, the information providing apparatus 100 selects, for example, a cluster having the worst grouping as a cluster to be divided. Subsequently, the information providing apparatus 100 randomly selects two elements from the cluster, and generates two clusters storing the respective elements. Thereafter, the information providing apparatus 100 calculates the similarity between all the elements in the original cluster and the elements selected at random. Subsequently, the information providing apparatus 100 adds an element to the cluster with the higher calculated similarity. Here, the information providing apparatus 100 moves elements between clusters until there are no more movable elements, and refines the division result.

  And the information provision apparatus 100 utilizes the bias | inclination of the weight of Bayon clustering. For example, the cluster extraction unit 137 of the information providing apparatus 100 generates a ranking by adding the two weights of the heaviest dimension in the Bayon clustering centroid. Then, the cluster extraction unit 137 preferentially extracts the reason for determination having a high ranking ranking.

  In this way, the information providing apparatus 100 extracts the representative determination reason based on the ranking by adding the two weights of the heaviest dimension in the Bayon clustering centroid. As a result, the information providing apparatus 100 can extract the reason for determination useful for the requester with high accuracy, and can provide the reason for determination useful for the requester with high accuracy.

[2-7. Weight verbs)
In the above embodiment, the information providing apparatus 100 has been described with an example in which the reason for determination extracted by the type extracting unit 135 is classified into clusters. Here, the information providing apparatus 100 may weight the verb when vectorizing the determination reason.

  For example, the type classification unit 134 of the information providing apparatus 100 vectorizes the features of the verb among the features of the reason for determination by weighting the features of the other parts of speech. This is because verbs are often more useful criteria than other parts of speech. Thereby, since the information provision apparatus 100 can improve the quality of clustering, a useful judgment reason can be extracted with a higher precision.

[2-8. (Extract reason for judgment based on distance from cluster center of gravity)
In the above embodiment, the information providing apparatus 100 has a typical determination reason other than the type in which the determination reason is an affirmative expression and the answer is a negative expression, and the type in which the determination reason is a negative expression and the answer is an affirmative expression. An example of extraction based on the probability of belonging to the above type has been described. Here, the information providing apparatus 100 may extract a representative determination reason based on the distance from the center of gravity of the cluster.

  Specifically, when the cluster extraction unit 137 of the information providing apparatus 100 extracts a representative determination reason from the clusters classified by the cluster classification unit 136, the cluster extraction unit 137 is representative based on the distance from the cluster centroid. The reason for the decision. For example, the cluster extraction unit 137 preferentially extracts a determination reason close to the center of the cluster from among the determination reasons that are elements included in the cluster.

  As described above, the information providing apparatus 100 extracts a representative determination reason based on the distance from the center of gravity of the cluster. Thereby, the information provision apparatus 100 can extract a useful judgment reason with high precision compared with the case where the reason for judgment is extracted at random.

[2-9. Stop word)
In the above embodiment, the information providing apparatus 100 has been described with an example in which the reason for determination extracted by the type extracting unit 135 is classified into clusters. Here, the information providing apparatus 100 may perform clustering in consideration of a stop word or the like.

  Specifically, the cluster classification unit 136 of the information providing apparatus 100 reads general words (for example, “ha”, “no”, “is”, “mas”, etc., which are functional words such as particles and auxiliary verbs). Except for this, clustering of reason for judgment is performed. As a result, the information providing apparatus 100 can classify judgment reasons that are common to uncommon words as judgment criteria into the same cluster, and thus can extract useful judgment reasons with high accuracy.

[2-10. (Applicable)
In the above-described embodiment, an example has been described in which the information providing apparatus 100 extracts and provides a determination reason useful for improving a task question sentence in crowdsourcing. Here, the information providing apparatus 100 may extract and provide useful information in various applications without being limited to the reason for determination useful for improving a task question sentence. For example, the information providing apparatus 100 may extract and provide information useful for business improvement from information such as inquiries and complaints collected at a call center or the like. As a result, the information providing apparatus 100 can extract and provide useful information from information obtained in various fields, so that it is possible to improve operations in various fields.

[2-11. (Examine the extracted reasons for judgment)
In the above embodiment, an example in which the information providing apparatus 100 provides the determination reason extracted by the type extracting unit 135 or the cluster extracting unit 136 has been described. Here, the information providing apparatus 100 may provide the information after thoroughly examining the determination reason extracted by the type extracting unit 135 or the cluster extracting unit 136.

  Specifically, the information providing apparatus 100 presents the determination reason extracted by the type extracting unit 135 or the cluster extracting unit 136 to the worker W again. For example, the information providing apparatus 100 presents to the worker W a question sentence asking whether or not the determination reason extracted by the type extracting unit 135 or the cluster extracting unit 136 is appropriate as a reason for answering the question sentence. As an example, the information providing apparatus 100 presents such a question sentence to another worker different from the worker who provided the determination reason.

  Then, the worker W operates the order receiving terminal 50 to transmit to the information providing apparatus 100 an answer as to whether or not the reason for determination is appropriate as the reason for answering the question text. Thereby, the information providing apparatus 100 receives from the order receiving terminal 50 an answer as to whether or not the reason for determination is appropriate as the reason for the answer to the question sentence. Then, the information providing apparatus 100 provides the requester Re with the determination reason that is answered as being appropriate as the reason for the answer to the question sentence.

  As described above, the information providing apparatus 100 provides the information providing apparatus 100 after carefully examining the reason for determination extracted by the type extracting unit 135 or the cluster extracting unit 136. Thereby, since the information provision apparatus 100 can improve the determination reason and system to provide, it can provide a higher quality determination reason.

  Note that the information providing apparatus 100 further presents a question sentence asking whether or not the reason for the determination that is appropriate as the reason for the answer to the question sentence is appropriate to other workers, and repeatedly conducted the examination. May be provided to the requester Re. Further, the information providing apparatus 100 may perform a scrutiny of the cluster by presenting to the worker W a question sentence asking whether or not the cluster classified by the cluster classification unit 136 is well organized.

[2-12. Effect of modification)
As described above, the information providing apparatus 100 according to the modification includes the receiving unit 133, the cluster classification unit 136, the cluster extraction unit 137, and the providing unit 138. The accepting unit 133 accepts an answer to the requested job and a determination reason for the answer from the contractor. The cluster classification unit 136 classifies the determination reason received by the reception unit 133 into clusters having the same determination reason and determination criterion. The cluster extraction unit 137 extracts a representative determination reason from the clusters classified by the cluster classification unit 136. The providing unit 138 provides the determination reason extracted by the cluster extracting unit 137.

  As a result, the information providing apparatus 100 according to the modification can classify the determination reasons received by the receiving unit 133 into clusters as well as the determination reasons extracted by the type extracting unit 135. Even if the number of judgment reasons extracted by (2) is too small, representative judgment reasons can be extracted from the cluster and provided to the requester.

  Further, in the information providing apparatus 100 according to the modification, the cluster classification unit 136 classifies the determination reason into clusters after converting the determination reason of the lower concept into the determination reason of the higher concept among the determination reasons.

  Thereby, the information providing apparatus 100 according to the modified example can unify the reason for determining the lower concept into the reason for determining the upper concept, and thus can suppress the number of clusters. For this reason, since the information providing apparatus 100 extracts representative determination reasons from the suppressed number of clusters, the information providing device 100 can provide a number of determination reasons that are easy to handle for the requester.

  Further, in the information providing apparatus 100 according to the modified example, the cluster classification unit 136 sets the determination reason for the positive expression answer among the determination reasons to be more weighted than the determination reason for the negative expression answer, and sets it as the center of the cluster. .

  As a result, the information providing apparatus 100 according to the modified example can place importance on the determination reason related to an affirmative expression answer that is a useful determination reason, and therefore can provide the requester with a useful determination reason with high accuracy. .

  In the information providing apparatus 100 according to the modification, the cluster extraction unit 137 selects a representative determination reason based on the tf-idf value of the determination reason among the clusters classified by the cluster classification unit 136. Extract from

  As a result, the information providing apparatus 100 according to the modification can extract the determination reason useful for the requester with high accuracy, and can provide the requester with useful determination reason with high accuracy.

[3. Others]
In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  For example, the task information storage unit 121 illustrated in FIG. 4 may be stored in a storage server or the like without the information providing apparatus 100. In this case, the information providing apparatus 100 acquires information about the task by accessing the storage server.

  Further, the information providing apparatus 100 may be a classification apparatus that performs only the type classification process by the type classification unit 134 and the cluster classification process by the cluster classification unit 136 without performing the type extraction process, the cluster extraction process, and the provision process. Good. In this case, the classification device does not include the type extraction unit 135, the cluster extraction unit 137, and the provision unit 138. Then, the extraction apparatus having the type extracting unit 135, the cluster extracting unit 137, and the providing unit 138 performs the extraction of the determination reason and provides the extracted determination reason.

  Further, the information providing apparatus 100 according to the embodiment described above is realized by a computer 1000 having a configuration as shown in FIG. 10, for example. Hereinafter, the information providing apparatus 100 will be described as an example. FIG. 10 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the information providing apparatus 100. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the communication network 50 and sends the data to the CPU 1100, and transmits the data generated by the CPU 1100 to other devices via the communication network 50.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. It is.

  For example, when the computer 1000 functions as the information providing apparatus 100 according to the embodiment, the CPU 1100 of the computer 1000 implements the function of the control unit 130 by executing a program loaded on the RAM 1200. The HDD 1400 stores data in the storage unit 120. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them. However, as another example, these programs may be acquired from other devices via the communication network 50.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, including the aspects described in the disclosure section of the invention, based on the knowledge of those skilled in the art, It is possible to implement the present invention in other forms with improvements.

  Further, the information providing apparatus 100 described above may be realized by a plurality of server computers, and depending on functions, an external platform or the like may be realized by calling an API (Application Programming Interface) or network computing. Can be changed flexibly.

  Further, the “means” described in the claims can be read as “section (module, unit)” or “circuit”. For example, the receiving unit can be read as receiving means or a receiving circuit.

DESCRIPTION OF SYMBOLS 1 Provision system 10 Order terminal 50 Order receiving terminal 100 Information provision apparatus 121 Task information storage part 131 Reception part 132 Transmission part 133 Reception part 134 Type classification part 135 Type extraction part 136 Cluster classification part 137 Cluster extraction part 138 Provision part

Claims (23)

A reception unit that receives an answer to the requested job and a reason for judging the answer from the contractor;
A type classification unit that classifies the determination reasons received by the reception unit into types based on the determination reasons and answers;
A type extracting unit for extracting a reason for determination belonging to a predetermined type based on the type classified by the type classifying unit;
An information providing apparatus comprising: a providing unit that provides a determination reason extracted by the type extracting unit. The type extracting unit
As the predetermined type, a type in which a determination reason is a positive expression and a reply is a negative expression, and a determination reason that belongs to a type other than a type in which the determination reason is a negative expression and a response is a positive expression are extracted. The information providing apparatus according to claim 1. The type classification unit is
The information providing apparatus according to claim 1, wherein the reason for determination is classified into the type by a document classification method using machine learning. The type classification unit is
The said reason for judgment is classified into the said type based on the maximum partial character string with the longest length among the partial character strings with the equivalence relation contained in the said reason for judgment. Information providing device described in one. A cluster classification unit that classifies the determination reasons extracted by the type extraction unit into clusters having the same determination reason and determination criteria;
A cluster extraction unit for extracting a representative determination reason from the clusters classified by the cluster classification unit;
The providing unit includes:
The information providing apparatus according to claim 1, wherein the determination reason extracted by the cluster extraction unit is provided. The cluster classification unit includes:
If the number of determination reasons extracted by the type extraction unit is greater than a predetermined value, classify the determination reasons into the clusters,
The cluster extraction unit includes:
Extracting a representative judgment reason from the clusters classified by the cluster classification unit,
The providing unit includes:
When the number of determination reasons extracted by the type extraction unit is greater than a predetermined value, the determination reason extracted by the cluster extraction unit is provided, and the number of determination reasons extracted by the type extraction unit is a predetermined value The information providing apparatus according to claim 5, wherein if there are fewer, the reason for determination extracted by the type extracting unit is provided. The cluster classification unit includes:
A cluster is generated from each determination reason extracted by the type extraction unit, and the two clusters having the smallest distance function result between the clusters are sequentially merged to classify the determination reason into the cluster. The information providing apparatus according to claim 5 or 6. The cluster classification unit includes:
Generating one cluster from the determination reason extracted by the type extracting unit, and classifying the determination reason into the cluster by sequentially dividing the generated cluster based on a distance function between the clusters. The information provision apparatus as described in any one of Claims 5-7. The cluster extraction unit includes:
The representative determination reason is extracted based on a type in which the determination reason is an affirmative expression and a reply is a negative expression, and a probability that the determination reason belongs to a type other than a type in which the determination reason is a negative expression and the answer is a positive expression. The information providing apparatus according to claim 5, wherein the information providing apparatus is an information providing apparatus. The cluster classification unit includes:
The information provision according to any one of claims 5 to 9, further comprising: classifying the determination reason into the cluster after converting the determination reason of the lower concept into the determination reason of the higher concept among the determination reasons. apparatus. The cluster classification unit includes:
11. The judgment reason related to an affirmative expression answer among the judgment reasons is weighted more than the judgment reason related to a negative expression answer, and is set to the center of the cluster. Information provision device. The cluster extraction unit includes:
The representative determination reason is extracted from clusters selected based on the tf-idf value of the determination reason among the clusters classified by the cluster classification unit. The information providing apparatus according to any one of the above. A reception unit that receives an answer to the requested job and a reason for judging the answer from the contractor;
A cluster classification unit that classifies the determination reason received by the reception unit into clusters having the same determination reason and determination criterion;
A cluster extraction unit for extracting a representative determination reason from the clusters classified by the cluster classification unit;
An information providing apparatus comprising: a providing unit that provides a determination reason extracted by the cluster extracting unit. The cluster classification unit includes:
A cluster is generated from each determination reason received by the reception unit, and the two reasons having the smallest distance function result between the clusters are sequentially merged to classify the determination reason into the cluster. The information providing apparatus according to claim 13. The cluster classification unit includes:
The cluster is generated according to a determination reason received by the reception unit, and the generated cluster is sequentially divided based on a distance function between the clusters, thereby classifying the determination reason into the clusters. Item 15. The information providing device according to Item 13 or 14. The cluster extraction unit includes:
The representative determination reason is extracted based on a type in which the determination reason is an affirmative expression and a reply is a negative expression, and a probability that the determination reason belongs to a type other than a type in which the determination reason is a negative expression and the answer is a positive expression. The information providing device according to any one of claims 13 to 15, wherein The cluster classification unit includes:
The information providing method according to any one of claims 13 to 16, wherein the reason for determining a lower concept among the reasons for determining is converted into a reason for determining a higher concept, and then the reason for determining is classified into the cluster. apparatus. The cluster classification unit includes:
18. The determination reason related to an affirmative expression answer among the determination reasons is weighted more than the determination reason related to a negative expression answer, and is set to the center of the cluster. Information provision device. The cluster extraction unit includes:
The representative determination reason is extracted from clusters selected based on the tf-idf value of the determination reason among the clusters classified by the cluster classification unit. The information providing apparatus according to any one of the above. An information providing method executed by the information providing apparatus,
A reception process for receiving an answer to the requested work and a reason for judging the answer from the contractor;
A type classification step for classifying the determination reasons received by the reception step into types based on the determination reasons and answers;
A type extraction step for extracting a determination reason belonging to a predetermined type based on the type classified by the type classification step;
A providing step of providing a reason for determination extracted by the type extracting step. An acceptance procedure for accepting an answer to the requested work and a reason for judging the answer from the contractor;
A type classification procedure for classifying the determination reason received by the reception procedure into a type based on the determination reason and the answer;
A type extraction procedure for extracting a reason for determination belonging to a predetermined type based on the type classified by the type classification procedure;
An information providing program that causes a computer to execute a providing procedure that provides a reason for determination extracted by the type extracting procedure. An information providing method executed by the information providing apparatus,
A reception process for receiving an answer to the requested work and a reason for judging the answer from the contractor;
A cluster classification step of classifying the determination reason received by the reception step into clusters having a common determination criterion and determination criterion;
A cluster extraction step of extracting a representative determination reason from the clusters classified by the cluster classification step;
And a providing step of providing a reason for determination extracted by the cluster extracting step. An acceptance procedure for accepting an answer to the requested work and a reason for judging the answer from the contractor;
A cluster classification procedure for classifying the judgment reason accepted by the acceptance procedure into clusters having the same judgment reason and judgment criteria;
A cluster extraction procedure for extracting a representative determination reason from the clusters classified by the cluster classification procedure;
An information providing program that causes a computer to execute a providing procedure that provides a reason for determination extracted by the cluster extracting procedure.

Images