JP7038499B2 - Classification system, control method of classification system, and program - Google Patents

Description

The present invention relates to a technique for classifying publications such as patent publications (including application publication publications and patent publication publications).

A classification number is given to the patent gazette. For example, a patent gazette is issued after being granted an International Patent Classification (IPC). In Patent Document 1, when the user designates the international patent classification, the user grasps the international patent classification by displaying the title and the abstract of the invention of the patent gazette to which the international patent classification to be designated is given. Techniques for facilitating are disclosed.

While the International Patent Classification is a classification commonly used in the world, the user cannot arbitrarily specify the classification method and the like. In order to facilitate patent search even in R & D departments and technical departments that are unfamiliar with patent search, patent gazettes of their own company and other companies are given their own classification (called in-house classification).

JP-A-2007-233910

In the work of assigning an in-house classification, the person in charge of assigning the classification visually confirms, for example, a newly issued patent gazette, and manually assigns the classification. Such work causes a load. Classification may be given by mistake. In addition, feedback that the classification is incorrect may be transmitted from the development department to the person in charge of classification, but it is not always reflected in the subsequent classification work.

In addition, when an in-house classification is given, a qualitative evaluation may be given to the target publication together, and such an evaluation may be given by mistake. And, such a qualitative evaluation is not always performed at the same time as the internal classification, but may be performed independently.

The classification system according to the present invention is based on teacher data showing the relationship between the input data relating to at least one publication and the output data indicating at least one classification given to at least one publication corresponding to the input data. , The model construction unit that constructs the learning model, the processing target data input unit that inputs the processing target data related to at least one publication as the input data to the training model, and the learning according to the input of the processing target data. According to the result output from the model, the providing unit that provides information on the processing target data to the user terminal, the receiving unit that receives the teacher information that is the source of the teacher data from the user terminal, and the teacher information. It also has a teacher data input unit that inputs the teacher data to the model construction unit.

According to the present invention, the work of assigning an in-house classification can be efficiently performed. In addition, the feedback of the classification result can be effectively utilized. Further, according to the present invention, the work of giving a qualitative evaluation can be efficiently performed.

It is a figure explaining the outline of the classification work. It is a figure which shows an example of the structure including the classification system. It is a figure which shows an example of the structure of a learning model. It is a figure explaining the teacher data. It is a figure which shows an example of the flowchart of the process performed by a classification system. It is a figure which shows an example of the UI screen in a user terminal. It is a figure which shows the example of the evaluation classification data stored in the classified DB. It is a figure explaining the value used in the conversion table. It is a figure explaining another example of a teacher data. It is a figure which shows the other example of the UI screen in a user terminal. It is a figure explaining another example of a teacher data. It is a figure which shows the other example of the UI screen in a user terminal. It is a figure which shows the other example of the UI screen in a user terminal.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The configurations described in the following embodiments are merely examples, and the present invention is not limited to the configurations shown.

<Usage form of internal classification>
Prior to the description of the embodiment, first, the current usage mode in which the in-house classification is used will be described. FIG. 1 is a diagram showing an example of utilization form of in-house classification. In-house classification is used in workflows such as patent search and patent information watching.

In the first scene W1, the person in charge of investigation sets a predetermined search formula and delivery destination. For example, a search condition used in an SDI (Selective Dissemination of Information) search and its delivery destination are set. The SDI search is to perform a search using preset search conditions for newly issued publications. The search results are automatically delivered to the set delivery destination. Since it is not necessary to classify all new publications, in the first scene W1, the new publications are roughly narrowed down by using a predetermined search condition such as an international search classification. Here, the "publication" includes various publications such as patents, utility models, designs, trademark publications, and patent publication publications. It may include not only publications issued by the Japan Patent Office but also technical documents and papers issued on a regular basis. Any object may be used as long as it is in the form of assigning a classification to some document.

In the second scene W2, the person in charge of classification assigns an in-house classification to new publications. The in-house classification can be freely set in the user company or the like. For example, it is possible to classify by development department or product. Hierarchical classification such as major classification, middle classification, and minor classification is also possible. In-house classification is generally used for sorting new publications. Of course, the sorting of new publications is an example, and in-house classification can be used in various forms. New publications with in-house classification will be sent to the relevant department. Multiple in-house classifications may be assigned. For example, two in-house classifications, classification C1 and classification C2, may be assigned to a certain publication P1.

In the third scene W3, the person in charge of each department confirms the sent new publication. However, if the classification is incorrect, the person in charge of classification will be notified that the classification is incorrect. In addition, the person in charge of each department confirms the new publication and gives an evaluation (weighting) to each publication. For example, a user company or the like can independently set various evaluations (weights) such as a value indicating the degree of relevance to the company's technology and a value indicating the degree of possibility of infringement.

In the fourth scene W4, the investigator watches the progress of the publication according to the evaluation (weighting) given by each department, and discusses with the department in charge as necessary.

The above is the explanation of the workflow of internal classification that is generally performed.

<Embodiment 1>
In this embodiment, for example, a mode in which the classification work of the second scene W2 of the workflow shown in FIG. 1 is processed by the classification system described later will be described. The classification system described in this embodiment does not have to be used in the workflow system as shown in FIG. 1, and can be used in any form.

<Structure>
FIG. 2 is a diagram showing an example of the configuration of the classification system 200 according to the embodiment. The classification system 200 includes a model building unit 201, a learning model 202, a teacher data input unit 203, a processing target data input unit 204, an output result providing unit 205, a providing destination database (DB) 206, a teacher information receiving unit 207, and a teacher information integration. It has a unit 208, a teacher information reflection unit 209, and a classified DB 210. FIG. 1 is merely an example of a configuration and may include other configurations. Moreover, not all of the configurations shown in FIG. 1 are essential requirements. The classification system 200 may be configured to be communicable with the publication DB 250 via a network.

The classification system 200 can be realized as an information processing device. The information processing device may have a CPU, a memory, an HDD, and a network interface. In each part shown in FIG. 2, the program stored in the HDD is temporarily read into the memory, and the CPU may function as each part shown in FIG. 2 by executing the program read into the memory. .. Further, at least a part of each part shown in FIG. 2 may be realized by a plurality of information processing devices connected to each other through various networks. Further, at least a part of each part shown in FIG. 2 (for example, the model building part 201 and the learning model 202) may be realized by distributed processing by a plurality of information processing devices.

The classification system 200 is configured to enable communication with a terminal used by the user (hereinafter referred to as a user terminal 261), and provides search results to the user terminal 261 or is described later from the user terminal 261. Teacher information may be received through the network.

The user terminal 261 may be any kind of terminal such as a personal computer, a tablet, and a mobile terminal. The classification system 200 manages the login of users, and may provide the user terminal 261 with search conditions, search results, and the like unique to the logged-in user.

The classification system 200 of the present embodiment may be configured to be different for each user company. This is because if the learning model 202 constructed by the company A based on the teacher data customized for the company A is used by the company B, which is different from the company A, the desired result of the company B cannot be obtained. Further, it is contrary to the intention of the company A to let the company A and the rival company C use the learning model 202 customized for the company A, and it is not appropriate from the viewpoint of information protection. Therefore, in the classification system 200 of the present embodiment, the learning model 202 is customized for each user company. The user company is not limited to the company, but may be any organization, group, or the like.

The model building unit 201 builds the learning model 202. In addition, in this specification, "construction" is used as a concept including generating a new model, changing or updating a generated model, and replacing an existing model with a newly generated model. And. For example, the model building unit 201 may generate a new model when there is no learning model built. If there is a learning model that has already been built, the model building unit 201 may newly build a learning model by changing or updating the configuration of a part of the learning model. The model building unit 201 may generate a new learning model and replace the already built learning model with the generated learning model.

The model building unit 201 may build a neural network as shown in FIG. 3, for example, as the learning model 202. The neural network includes an input layer 310, an intermediate layer 320, and an output layer 330. Each layer is composed of multiple nodes. FIG. 3 shows an example in which the input layer 310 includes nodes 311, 312, 313, the intermediate layer 320 includes nodes 321, 322, 323, and the output layer 330 includes nodes 331, 332, 333. .. The number of nodes included in each layer is not limited to the illustrated example, and is generally composed of a large number of nodes. The intermediate layer 320 is not limited to one layer, and may be composed of a plurality of layers. Weights are set for each node, and the value obtained by multiplying the input data by the weight is transmitted to the subsequent nodes.

In supervised learning, for example, input data and output data corresponding to the input data are prepared as teacher data. The model building unit 201 repeats the process of adjusting the weight set in each node so that the input data becomes equal to the output data. In this way, the model building unit 201 builds the learning model 202. After that, when the data to be processed is input to the learning model 202, the learning model 202 outputs the result according to the constructed model.

In the present embodiment, the model building unit 201 may build the learning model 202 by executing a known machine learning process. For example, the model building unit 201 may use a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Other methods may be used. The learning model 202 may be constructed by SVM (Support Vector Machine) instead of the neural network. In this embodiment, the learning model 202 itself may be of any kind.

The teacher data input unit 203 inputs the teacher data to the model construction unit 201. In the initial stage, the teacher data input unit 203 may input data obtained from the classified DB 210 and the publication DB 250 as teacher data.

The publication DB 250 is, for example, a DB in which all available published publications are stored. As described above, the publication DB 250 includes various publications such as patents, utility models, designs, trademark publications, patent publication publications, etc., and is issued not only by the JPO but also on a regular basis. The technical literature and the technical papers to be published may be included. As described above, the publication DB 250 does not have to be a single DB, but may be a set of a plurality of DBs. The publication DB 250 may include not only the publications in Japan but also the DBs of publications in other countries.

The classified DB 210 stores, for example, data associated with the identification information of the publication (for example, application number, publication number, etc.) and the in-house classification given to the publication. For example, the publication P10 stores data such that classification C1 and classification C2 are assigned.

The teacher data input unit 203 may input teacher data as shown in FIG. 4, for example, as initial teacher data. The teacher data may be data showing the relationship between the input data relating to at least one publication and the output data indicating at least one classification given to at least one publication corresponding to the input data. More specifically, the output data may be data indicating the degree of approximation of each of the plurality of classifications that can be assigned as an in-house classification. FIG. 4 is a diagram illustrating an example of teacher data. In the example of FIG. 4, it is assumed that a plurality of classifications that can be assigned as internal classifications are classification C1, classification C2, and classification C3. In the teacher data shown in FIG. 4, text data of the summary sentence of Gazette P1 is prepared as input data, and as output data corresponding to this input data, a value 1 indicating the degree of approximation of classification C1 to Gazette P1 and a value 1 indicating the degree of approximation to Gazette P1 are provided. An example is shown in which data including a value 0 indicating the degree of approximation of the classification C2 and a value 1 indicating the degree of approximation of the classification C3 with respect to the publication P1 are prepared. In this embodiment, it is assumed that the value 1 indicates the closest value. That is, the publication P1 prepares teacher data indicating that the data should be classified into the classification C1 and the classification C3. On the other hand, in the present embodiment, the value 0 indicates a value that is the least approximate. That is, the publication P1 prepares teacher data indicating that the classification should not be classified into the classification C2.

In this way, the model building unit 201 uses the teacher data as shown in FIG. 4 to build the learning model 202. For example, the model building unit 201 may change the weight of each node of the neural network constituting the learning model 202. When the publication P1 is input to the learning model 202 constructed in this way as the processing target data, data indicating the degree of approximation of each of the classifications C1 and C2 can be obtained as output. When the teacher data of FIG. 4 is used as the initial teacher data, when the publication P1 is input to the learning model 202 as the processing target data, the degree of approximation of each of the classification C1 and the classification C3 is as close to 1 as possible. can get. When another publication P11 is input to the learning model 202 as the processing target data, the values indicating the degree of approximation of the classification C1, the classification C2, and the classification C3 are output respectively.

FIG. 4 shows an example in which text data of a summary sentence is used as input data, but the present invention is not limited to this. It may be text data in the scope of claims or full-text text data. In addition, the user may input an arbitrary text document. Further, as the input data, bibliographic data may be input or image data may be input. Further, some of these data may be used, or a plurality of types of data may be used.

The teacher data input unit 203 may acquire necessary data corresponding to the input data (for example, text data of the abstract of the publication) from the publication DB 250 as the initial teacher data, and the necessary data corresponding to the output data (for example). For example, data indicating the classification given to a specific publication) may be acquired from the classified DB 210. When the necessary data corresponding to the input data is stored in the classified DB 210 (for example, in the example of FIG. 4, when the summary sentence is also stored in the classified DB 210), the teacher data input unit 203 is referred to as a publication. Teacher data obtained without reference to DB 250 may be used.

The teacher data may include additional information indicating which part of the publication (eg, bibliographic information, abstracts, claims, drawings, etc.) was used for learning. The processing target data input unit 204 may refer to the additional information and use the corresponding portion in the publication input as the processing target data as the input data.

The processing target data input unit 204 acquires new information from the publication DB 250. The processing target data input unit 204 may acquire new information at a predetermined timing. For example, new information may be acquired each time the publication DB 250 is updated. New information may be acquired at predetermined period intervals from the time when the previous new information was acquired, such as weekly or monthly. The new arrival information may be a publication newly issued from the time when the processing target data input unit 204 acquires the previous new arrival information. The processing target data input unit 204 inputs the publication specified by the new arrival information into the learning model 202 as the processing target data. The processing target data input unit 204 may use the publication whose target is narrowed down by a predetermined search condition from the new information as the processing target data.

Further, the processing target data input unit 204 may input any publication other than the publication specified by the new arrival information into the learning model 202 as the processing target data. For example, when the types of in-house classification are increased, the already classified publications may be input to the learning model 202 as the processing target data. Further, when the conditions for narrowing down new information are changed due to the expansion of the technical field or the like, the published publications that have not been processed so far may be input to the learning model 202 as the processing target data.

The output result providing unit 205 inputs identification information (for example, application number, publication number, patent number, etc.) for identifying the processing target data according to the result output by the learning model 202 when the processing target data is input to the learning model 202. It may be provided to the user terminal 261.

The output result providing unit 205 specifies the classification to be given to the publication of the data to be processed based on the result output by the learning model 202. For example, the output result providing unit 205 identifies that the classification in which the value indicating the degree of approximation of each classification output by the learning model 202 exceeds a predetermined threshold value is the classification given to the publication of the data to be processed. good. The output result providing unit 205 assigns the specified classification to the publication of the data to be processed. The output result providing unit 205 may specify a user corresponding to the assigned classification and provide identification information for identifying the data to be processed to the user terminal 261 of the specified user.

The provider DB 206 stores data indicating the user of the provider who provides the information of the publication to which the predetermined classification is given. For example, the user to whom the publication to which the classification C1 is given may be provided may be data indicating that the user belongs to the development department D1. There may be a plurality of users to be provided. That is, if a plurality of users belong to the development department D1, it may be provided to the plurality of users. In addition, the provider may span a plurality of departments.

The content provided may be bibliographic information such as identification information regarding the gazette, and may include some information regarding the content such as a summary. The output result providing unit 205 notifies the registered e-mail address of the providing destination user or the like that the output result has been extracted. When the user terminal 261 receives this notification, the output result providing unit 205 may present the output result to the accessing user terminal 261.

Further, the output result providing unit 205 outputs to the classified DB 210 data in which the identification information of the publication input as the processing target data in the learning model 202 and the classification indicated by the result output from the learning model 202 are associated with each other. , Update the classified DB 210. In the present embodiment, the classified DB 210 is first updated using the classification indicated by the result output from the learning model 202. After that, a mode of modifying the data stored in the classified DB 210 based on the teacher information from the user will be described. However, it is not limited to this example.

The teacher information receiving unit 207 receives the teacher information transmitted from the user terminal 261 in response to the output result being provided from the output result providing unit 205. The teacher information may be information specified by the user whether the result output by the output result providing unit 205 (that is, the content output by the learning model 202) is correct (appropriate) or incorrect (inappropriate). .. For example, it is assumed that the publication P21 provided to the members of the development department D1 is a publication not related to the development department D1. In such a case, the member of the development department D1 notifies the information to the effect that the publication is not related to the own department as teacher information. The teacher information receiving unit 207 receives such teacher information. Further, if a member of the development department D1 determines that the publication P21 is related to the development department D2 of another department rather than its own department, a teacher who specifies that the publication P21 is newly assigned a classification C2. Information may be received.

When the teacher information receiving unit 207 receives a plurality of teacher information, the teacher information integration unit 208 integrates the teacher information. There are two modes in which the teacher information integration unit 208 integrates.

One is a case where a plurality of classifications are output from the learning model 202 for one publication. That is, it is a case where teacher information is sent from each of the user terminals of a plurality of classification destinations. For example, it is assumed that the publication P3 is input to the learning model 202 as the data to be processed, and the learning model 202 outputs the corresponding results for all of the classifications C1, C2, and C3. The teacher information receiving unit 207 received information as teacher information from the members of the development department D1 corresponding to the classification C1 and the members of the development department D2 corresponding to the classification C2 that the publication is not related to the own department. Suppose. Then, the teacher information integration unit 208 integrates the teacher information received in this way. That is, the teacher information integration unit 208 integrates a plurality of teacher information, and the publication P3 integrates the teacher information having the content that it is correct to output the classification C3. Publication P3 may further include teacher information stating that it is incorrect to output classification C1 and classification C2. The integrated teacher information is transmitted to the teacher information reflection unit 209.

Another case is the case where a plurality of users are registered in the provision destination of one classification. That is, it is a case where teacher information is received from a plurality of users in the same department. In such cases, there are various ways to adopt teacher information. For example, the teacher information integration unit 208 may output only the teacher information that has been evaluated in common by all users to the teacher information reflection unit 209. In other words, the teacher information integration unit 208 does not have to adopt the gazette as teacher information when some users make different evaluations. According to such a configuration, the teacher data in which the opinions of all the users are in agreement is reflected, so that a more robust learning model 202 can be constructed.

The teacher information integration unit 208 may adopt teacher information by majority vote when some users make different evaluations. When a large number of users make evaluations, it is expected that the evaluations of all users may not completely match. If the amount of teacher information that can be adopted is small, the number of teacher data will not be sufficient. As a result, the evolution of the learning model 202 may not progress and sufficient results may not be obtained. Therefore, if some users make different evaluations, the teacher information integration unit 208 may adopt the teacher information by majority vote.

The teacher information integration unit 208 may change the weight for each user. The proficiency level is different for each user. Therefore, the weight is heavier for users with high proficiency and lighter for users with low proficiency. Then, the result of weighting and averaging the values evaluated by each user (that is, teacher information) may be adopted as the teacher information to be output to the teacher information reflection unit 209. For example, even if multiple users with low proficiency output teacher information that the classification is inappropriate, if the user with high proficiency does not judge that the classification is inappropriate, the teacher information The integration unit 208 does not have to output the teacher information that the classification is inappropriate to the teacher information reflection unit 209.

The teacher information integration unit 208 may be configured to be able to receive instructions from a user who is a leader. If some of the plurality of users have different evaluations, it may be determined whether or not to adopt the information as teacher information according to an instruction from the reader. Specifically, the teacher information integration unit 208 determines whether or not teacher information different from the others is included. When different teacher information is included, a UI screen for inquiring which teacher information is to be adopted is provided to the user terminal registered as a reader. Upon receiving the designation of the teacher information to be adopted from the user terminal, the teacher information integration unit 208 adopts the designated teacher information.

The teacher information reflection unit 209 reflects the teacher information output from the teacher information integration unit 208. The teacher information reflection unit 209 updates the classified DB 210 based on the teacher information. For example, when the publication P3 is input to the learning model 202 as the processing target data as in the above example, the degree of approximation of the classification C1 output from the learning model 202 is 0.6, and the degree of approximation of the classification C2 is 0. 7. The case where the degree of approximation of the classification C3 is 0.8 will be described as an example. Upon receiving this result, the output result providing unit 205 may determine that the classification having a degree of approximation exceeding a predetermined threshold value (for example, 0.5) is an appropriate classification. As a result, the output result providing unit 205 specifies that the classifications corresponding to the publication P3 are the classification C1, the classification C2, and the classification C3. Then, the information regarding the publication P3 is output to the user terminal 261 corresponding to each classification. Further, the output result providing unit 205 stores the classified data in which the publication P3 is associated with the classifications C1, C2, and C3 in the classified DB 210. At this time, it is assumed that the integrated teacher information is information indicating that the classification C3 is appropriate and the classifications C1 and C2 are inappropriate for the classification with respect to the publication P3. Then, the teacher information reflection unit 209 changes the classification data regarding the publication P3 already stored in the classified DB 210. That is, the classification data related to the publication P3 already stored is changed to the classification data in which only the publication P3 and the classification C3 are associated with each other.

Further, the teacher information reflection unit 209 outputs teacher data based on the integrated teacher information to the teacher data input unit 203. That is, the teacher data is output to the teacher data input unit 203 using the publication P3 as input data and the output data of the approximation value 0 of the classification C1, the approximation value 0 of the classification C2, and the approximation value 1 of the classification C3. do. In response to this, the teacher data input unit 203 inputs the teacher data to the model construction unit 201 and reconstructs the learning model 202. By such processing, the subsequent processing target data is processed using the learning model that reflects the teacher data.

FIG. 5 is a flowchart showing an example of the processing according to the present embodiment. The process shown in FIG. 5 is executed in the classification system 200 when the user logs in to the classification system 200 using the user terminal 261 and constructs a learning model using the teacher data.

In step S501, the teacher data input unit 203 acquires teacher data. When constructing a new learning model, the teacher data input unit 203 refers to the classified DB 210 and acquires the classified data that has been manually classified by the classifier or the like in the past as teacher data. You can do it.

In step S502, the teacher data input unit 203 inputs the acquired teacher data to the model construction unit 201. The model building unit 201 builds a learning model 202 using the input teacher data. It should be noted that the initial teacher data may be arbitrary teacher data and is not limited to the example described. Further, in the state where the learning model has already been constructed, the processes of steps S501 and S502 may be omitted.

In step S503, the processing target data input unit 204 acquires the processing target data from the publication DB 250. For example, the processing target data input unit 204 acquires new information regarding a publication issued during a predetermined period. The processing target data input unit 204 may acquire new arrival information according to various conditions such as a distribution frequency set by the user terminal 261. The processing target data input unit 204 inputs the acquired new publication as the processing target data into the learning model 202. The learning model 202 outputs a result based on the input processing target data.

In step S504, the output result providing unit 205 provides the information regarding the processing target data to the user terminal according to the output result. From the learning model 202, data indicating the degree of approximation of each classification corresponding to the publication of the input data is output. The output result providing unit 205 identifies a classification having a degree of approximation exceeding a predetermined threshold value. The output result providing unit 205 refers to the providing destination DB 206 and identifies the providing destination user corresponding to the specified classification. Then, the result is provided to the user terminal of the specified user. For example, the output result providing unit 205 may provide the user terminal with a list of identification information that identifies the data to be processed.

In step S505, the teacher information receiving unit 207 receives the teacher information from the user. For example, publication data with classifications related to the development department to which the user belongs and a list of publications are sent to the user terminal 261. The user confirms the gazette using the user terminal 261 and specifies that the gazette determined to be unrelated to his / her own development department is an improperly classified gazette.

FIG. 6 is a diagram showing an example of a UI screen 600 that specifies that a publication that the user has determined is not related to his / her own development department is a publication that has been inappropriately classified. The UI screen 600 is displayed on the user terminal 261. On the UI screen 600, a list of new publications for the development department D1 to which the user belongs is displayed. The user can check the check boxes displayed side by side in the list to specify that the publication is inappropriately classified, or specify an appropriate classification destination by a pull-down menu. Although FIG. 6 shows an example of a list screen, check boxes and pull-down menus may be displayed on the detail screen for displaying the details of each publication. When it is specified that the publication is inappropriately classified via such a UI screen 600, the information is transmitted as teacher information. Further, if the user can determine an appropriate classification destination, an appropriate classification destination may be specified by a pull-down menu, and in this case, the appropriate classification destination may also be transmitted as teacher information.

In step S506, the teacher information integration unit 208 integrates a plurality of teacher information received by the teacher information receiving unit 207. For example, when a plurality of teacher information is received by the teacher information receiving unit 207 with respect to a specific publication, the teacher information integrating unit 208 integrates the teacher information.

In step S507, the teacher information reflection unit 209 reflects the teacher information in the classified DB 210. For example, in the classified DB 210, it is assumed that the classification data in which the classifications associated with the publication P3 are classification C1, classification C2, and classification C3 are stored. At this time, when the teacher information indicating that the classification C1 and the classification C2 are incorrect is obtained, the teacher information reflection unit 209 updates the classification data associated with the publication P3 to the classification data in which only C3 is used.

In step S508, the teacher information reflection unit 209 generates teacher data based on the teacher information. In the above example, the teacher data is generated using the publication P3 as input data and the approximation value 0 of the classification C1, the approximation value 0 of the classification C2, and the approximation value 1 of the classification C3 as output data. The generated teacher data is output to the teacher data input unit 203.

In step S509, the teacher data input unit 203 outputs the teacher data to the model construction unit 201. The model building unit 201 builds the learning model 202 based on the teacher data.

In step S510, the processing target data input unit 204 determines whether or not there is the next processing target data. For example, it is determined whether it is the timing to acquire the next new publication. If there is data to be processed next, the process proceeds to step S503, and the process is repeated thereafter.

Although not shown, the process shown in FIG. 5 may be completed when the instruction to stop the classification process is received from the user terminal 261.

As described above, according to the present embodiment, the learning model 202 can execute the process of assigning the in-house classification, so that the classification can be efficiently assigned. Further, by using the result output by the learning model 202 as teacher data, the learning model 202 can be evolved and the feedback of the classification result can be efficiently utilized.

<Embodiment 2>
In the first embodiment, when information about a specific publication is input as data to be processed, a mode in which the learning model outputs the degree of approximation of each classification to the specific publication has been described. As explained in the section "Usage of in-house classification", each development department evaluates each classified gazette. In the second embodiment, a mode in which the evaluation is used as a parameter for adjusting the degree of approximation will be described. In the example of the first embodiment, an example in which a value 1 indicating that the classification is appropriate and a value 0 indicating whether the classification is appropriate is used as the teacher data has been described. In the second embodiment, a mode in which the value of the output data used as the teacher data is used as an arbitrary value between 0 and 1 is described according to the evaluation.

Further, in the first embodiment, the form in which the value indicating the degree of approximation output from the learning model 202 is used in the scene of determining whether or not the classification is correct has been described. In the second embodiment, a mode of changing the order of the list of information provided to the user terminal according to the value indicating the degree of approximation will also be described.

The configuration of the classification system in the second embodiment can be the same as that described in the first embodiment.

In the second embodiment, the classified DB 210 stores evaluationd classification data in which at least one classification and the evaluation of the classification are associated with each publication. FIG. 7 is a diagram showing an example of evaluationd classification data stored in the classified DB 210. For example, in Gazette P31, two classifications C1 and C2 are associated. Further, the classification C1 of the publication P31 is associated with the evaluation of high weighting, and the classification C2 of the publication P31 is associated with the evaluation of low weighting. The evaluations such as large weighting and small weighting are evaluations given by each development department. For example, it is an evaluation showing the degree of relevance to the company's technology. The higher the evaluation, that is, the higher the weight, the higher the degree of relevance to the in-house technology, indicating that the publication is noteworthy.

In the second embodiment, the teacher data input unit 203 uses the evaluationd classification data as the teacher data. The teacher data input unit 203 scores the evaluated classification data as shown in FIG. 7.

FIG. 8 is a diagram for explaining an example of scoring. FIG. 8 is a diagram showing an example of a conversion table used when converting to a score. In the conversion table, weighting parameters for each evaluation are set. For example, when the evaluation attached to the classification C1 of the publication P31 is "high weighting", a score of a value 1 is given to the classification C1 of the publication P31. When the evaluation attached to the classification C2 of the publication P31 is "small weighting", a score of a value of 0.6 is given to the classification C2 of the publication P31.

FIG. 9 is a diagram showing an example of teacher data in the second embodiment. As described in the first embodiment, the text data of the abstract is used as the input data for each publication. On the other hand, unlike the first embodiment, as the output data, a value scored based on the evaluation is used as a value indicating the degree of approximation. FIG. 9 shows an example in which the publications P31 and P32 shown in FIG. 7 are used as teacher data. Classification C1 and classification C2 are attached to Gazette P31, and classification C3 is not given. Therefore, in the teacher data of FIG. 9, a value 0 is set as the degree of approximation of the classification C3 with respect to the publication P31. Further, since the weighting attached to the classification C1 of the publication P31 is "large weighting", a value 1 is set as the approximation degree of the classification C1 with respect to the publication P31 in the teacher data of FIG. Further, since the weighting attached to the classification C2 of the publication P31 is "small weighting", the degree of approximation of the classification C3 to the publication P31 is set to a value of 0.6 in the teacher data of FIG. ..

When the teacher data input unit 203 inputs such teacher data to the model construction unit 201, the model construction unit 201 constructs the learning model 202 according to the teacher data. When the processing target data input unit 204 inputs the processing target data to the constructed learning model 202, the learning model 202 outputs a predetermined output value (value indicating the degree of approximation) as output data.

The output result providing unit 205 provides the result output from the learning model 202 to the user terminal 261. For example, as described in the first embodiment, the output result providing unit 205 identifies the classification having an approximation degree exceeding a predetermined threshold value as the classification of the publication input as the processing target data. The output result providing unit 205 may provide the user terminal 261 with a list of publications having a degree of approximation exceeding a predetermined threshold and whose order is changed according to the degree of approximation. For example, it is assumed that a list of publications classified as classification C1 is provided to a user in development department D1. Further, it is assumed that the new publication group is input to the learning model 202 as the data to be processed. In this case, the output result providing unit 205 extracts the publication whose degree of approximation with respect to the classification C1 exceeds a predetermined threshold value, which is output from the learning model 202. Then, from the extracted gazettes, a list in which the order of the gazettes is changed in descending order of the degree of approximation with respect to the classification C1 is generated. The output result providing unit 205 may provide the list generated in this way to the user terminal 261.

FIG. 10 is a diagram showing an example of a UI screen for displaying the list provided by the output result providing unit 205 on the user terminal 261. In the second embodiment, it is possible to give a weight to each publication. The publication to which the weight is given in this way is received by the teacher information receiving unit 207 as teacher information. Here, although the list box is used for giving the weight, it may be an inbox for directly inputting a value, or it may be a control for other screen input.

The teacher information integration unit 208 may integrate a plurality of teacher information as described in the first embodiment. For example, a majority vote may be adopted, weighting may be adopted, or user determination may be adopted.

As described in the first embodiment, the teacher information reflection unit 209 generates teacher data reflecting the teacher information and outputs it to the teacher data input unit 203. For example, using a conversion table as shown in FIG. 8, teacher data reflecting the weighting is generated. In this embodiment, an example is shown in which a conversion table common to all classifications is used, but the conversion table is not limited to this. A different conversion table may be used for each classification. The teacher data input unit 203 inputs the teacher data to the model construction unit 201, and the model construction unit 201 constructs the learning model 202.

As described above, in the present embodiment, since the user's evaluation can be included in the element of the teacher data, the learning model 202 that outputs a more appropriate classification can be constructed.

In the second embodiment, in order to simplify the explanation, an example in which only the major classification is used has been described, but there may be a middle classification in the major classification, and a configuration in which a further minor classification is provided in the middle classification is also possible. good. In such a case, the parameters may be further subdivided.

In the second embodiment, the example in which the order of the publications is changed in descending order of the degree of approximation and displayed on the user terminal 261 has been described, but the same display mode may be adopted in the first embodiment. That is, even when evaluation (weighting) is not given to the classification and information on evaluation (weighting) is not used as teacher information, the order is changed according to the degree of approximation output by the learning model 202. A list of publications may be provided to user terminal 261.

<Embodiment 3>
In the first embodiment, when information about a specific publication is input as data to be processed, a mode in which the learning model outputs the degree of approximation of each classification to the specific publication has been described. In the third embodiment, in addition to the degree of approximation of the classification, when information about a specific publication is input as data to be processed, the learning model outputs the degree of risk for the specific publication. Here, the degree of risk means the conflict of the target object such as the target company, the target business, the target product, and the target part, and more specifically, whether or not the target object conflicts with a specific publication. It is the degree to show. In the third embodiment, first, in addition to the approximation degree of classification described in the first embodiment, the learning model 202 is made to learn the degree of risk, the processing target data is input to the learning model, and the degree of approximation and the degree of risk of classification are output. Explain an example.

The configuration of the classification system in the third embodiment can be the same as that described in the first embodiment.

In the third embodiment, the classified DB 210 stores at least one classification for each publication, and the degree of approximation and the degree of risk of the classification are associated with each other.

In the third embodiment, the teacher data input unit 203 uses the gazette stored in the classified DB 210, the classification given to this gazette, the degree of approximation thereof, and the degree of risk as the teacher data.

FIG. 11 is a diagram showing an example of teacher data in the third embodiment. As described in the first embodiment, the text data of the abstract is used as the input data for each publication. FIG. 11 shows an example in which Gazette P1 and Gazette P2 are used as teacher data. In the publication P1, the degree of approximation 1 is given to the classification C1, the degree of approximation 0 is given to the classification C2, the degree of approximation 1 is given to the classification C3, and the degree of risk 0.8 is given. Further, in the publication P2, the degree of approximation 1 is given to the classification C1, the degree of approximation 0 is given to the classification C2, the degree of approximation 0 is given to the classification C3, and the degree of danger 1 is given. Here, in this example, one gazette is given one degree of risk, but one gazette may be given one degree of risk.

When the teacher data input unit 203 inputs such teacher data to the model construction unit 201, the model construction unit 201 constructs the learning model 202 according to the teacher data. When the processing target data input unit 204 inputs the processing target data to the constructed learning model 202, the learning model 202 outputs a predetermined output value (classification approximation degree and risk degree) as output data.

The output result providing unit 205 provides the result output from the learning model 202 to the user terminal 261. For example, as described in the first embodiment, the output result providing unit 205 identifies the classification having an approximation degree exceeding a predetermined threshold value as the classification of the publication input as the processing target data. The output result providing unit 205 may provide the user terminal 261 with a list of publications having a degree of approximation exceeding a predetermined threshold and whose order is changed according to the degree of approximation, including the degree of risk. For example, it is assumed that a list of publications classified as classification C1 is provided to a user in development department D1. Further, it is assumed that the new publication group is input to the learning model 202 as the data to be processed. In this case, the output result providing unit 205 extracts the publication whose degree of approximation with respect to the classification C1 exceeds a predetermined threshold value, which is output from the learning model 202. Then, from the extracted gazettes, a list in which the order of the gazettes is changed in descending order of the degree of approximation with respect to the classification C1 is generated. The output result providing unit 205 may provide the list generated in this way to the user terminal 261. Here, since the list also includes the degree of danger, it is also possible to generate and use a list in which the order of the publications is changed in descending order of the value of the degree of danger. Further, the publication presented to the user does not have a degree of approximation exceeding a predetermined threshold value, and instead, a publication having a degree of risk exceeding a predetermined threshold value may be used.

FIG. 12 is a diagram showing an example of a UI screen for displaying the list provided by the output result providing unit 205 on the user terminal 261. In the third embodiment, each publication is given a classification and a degree of risk and displayed. Here, the degree of approximation can also be displayed in this displayed list. Then, the user can modify any of the classification, its approximation, and the risk, and the target publication, the modified classification, its approximation, and the risk are teacher information as teacher information. It is received by the receiving unit 207.

The teacher information integration unit 208 may integrate a plurality of teacher information as described in the first embodiment. For example, a majority vote may be adopted, weighting may be adopted, or user determination may be adopted.

As described above, in the present embodiment, since the user's evaluation can be included in the element of the teacher data, it is possible to construct a learning model 202 that outputs a more appropriate classification and risk level.

In the third embodiment, the learning model 202, which is one learning model, has been described with the configuration in which the publication, the classification, the degree of approximation and the degree of risk are learned as teacher data, but because of the learning model for classification and the degree of risk. The learning model can be configured separately, and the learning model for this degree of risk may be further subdivided into classifications. Specifically, when there is a classification of C1 and C2, there is only one learning model for classification, and a learning model for risk is prepared for C1 and C2. In the case of such a configuration, the classification, the degree of approximation, and the degree of risk set in advance by the user are read, first, the target publication and the classification and the degree of approximation are learned by the learning model for classification, and then the classification is performed. The published publication is trained using the publication and the risk level in a learning model for the risk level corresponding to the classification. When information about a specific publication is input as data to be processed, the learning model for classification first outputs an appropriate classification, and the learning model for the degree of risk corresponding to the output classification is used for processing. The risk level of the target publication is output. The degree of danger here can be said to be the degree of danger in the target classification.

Further, in order to output the degree of risk in the above-mentioned target classification, a learning model for the degree of risk corresponding to the classification was used in addition to the learning model for classification, but not only the degree of risk in the target classification but also the learning model was used. It may be configured to output the risk level of the target department and the risk level of the target device, which is realized by constructing a learning model for the risk level of the target department and a learning model for the risk level of the target device, respectively. be able to. In this case, the input teacher data may train the gazette data and the risk level of this gazette data to the learning model for the risk level of the corresponding target department and the learning model for the risk level of the corresponding target device. You will need it. As a result, for example, the user discovers a plurality of publications that may conflict with respect to the product A, assigns a quantified value to each of the publications, and trains the learning model for the risk of the product A to be new. When the publication of is input to this learning model as processing target data, the learning model outputs the degree of risk for the target publication. That is, the degree of risk that the product A conflicts with the target publication is output. In addition, although the description has been made on the premise that the classification is given by the learning model in the embodiment, the configuration for giving the degree of risk by the learning model described above does not necessarily include the operation of giving the classification. Only the operation of outputting the risk level of the target publication by the learning model is included in the scope of the present invention. On the contrary, it may be configured to obtain the degree of risk by a learning model and then give a classification.

Further, although the degree of danger is explained in the third embodiment, the degree of danger may be set by the development department instead of the weighting in W3 of FIG.

In the above-described embodiment, the classification of the publications relating to patents has been mainly described as an example, but the present invention is not limited to this. It can be applied to the classification process of any kind of publication.

<Embodiment 4>
In the first embodiment, when information about a specific publication is input as data to be processed, a mode in which the learning model outputs the degree of approximation of each of a plurality of in-house classifications that can be given to the specific publication has been described. In the fourth embodiment, when the information regarding the application is input as the processing target data, the learning model is IPC, CPC (Cooperative Patent Classification), ECLA (European Classification), ICO (In Computer Only), FI (File Index), and the like. We will explain the classification by multiple domestic and foreign patent offices and international organizations that can be granted to the application such as F-term, and the form to output the degree of closeness.

The configuration of the classification system in the fourth embodiment can be the same as that described in the first embodiment.

The model construction unit 201 of the classification system 200 according to the present embodiment creates a learning model based on the teacher data showing the relationship between the plurality of input data related to the application and the output data indicating at least one classification given to the application. To construct. The process executed by the model building unit 201 is the same as that described in the first embodiment. Here, the application may be a patent application, a utility model registration application, a design registration application, or a trademark registration application, and the plurality of input data relating to the application are data relating to documents and properties submitted at the time of application.

The processing target data input unit 204 inputs a plurality of processing target data related to the application into the learning model as a plurality of input data. More specifically, when the application is a patent application or a utility model registration application, the plurality of processed data shall be at least the application, the specification, the claims or the scope of the utility model registration request, the drawings and the abstract. Any may be included. In addition, when the application is a design registration application, the plurality of data to be processed includes at least one of the application and the article related to the design described in the application, the description of the article related to the design, the description of the design, and the drawing. good. Here, the drawing may include a front view, left and right side views, a rear view, a perspective view, and the like. Further, if the application is a trademark registration application, the plurality of processed data may include at least one of the application and the designated goods, designated services and trademarks described in the application. Here, the trademark may include characters, figures, colors only, sounds, holograms, and the like.

Similar to the classification system 200 according to the first embodiment, the classification system 200 according to the present embodiment provides information on the plurality of processing target data according to the result output from the learning model in response to the input of the plurality of processing target data. A providing unit (output result providing unit 205) provided to the terminal 261, a receiving unit (teacher information receiving unit 207) that receives teacher information that is the source of teacher data from the user terminal 261, and teacher data according to the teacher information. Has a teacher data input unit 203, which inputs the information to the model construction unit 201.

Further, the providing unit of the classification system 200 according to the present embodiment is a processing target that serves as a basis for determining that at least one classification determined to be assigned to the application is assigned to the plurality of processed data. The data is provided to the user terminal 261.

FIG. 13 is a diagram showing another example of the UI screen in the user terminal. By displaying the UI screen shown in the figure on the user terminal 261, the user selects a plurality of input data related to the application, and the plurality of classifications that can be given to the application provided by the classification system 200 and the grant basis. And can be confirmed.

The user inputs data related to the application in the item of input data 301. For example, if the application is a patent application, data relating to the application, specification, claims, drawings and abstract may be entered in the input data 301 field. When data is input to the items of the input data 301, the items included in the input data 301 are listed on the item specification screen 302. The user specifies a summary described in the abstract and a part of a plurality of processed data to be input to the learning model by designating all or a part of the claims by the item specification screen 302. Can be specified. Similarly, the paragraph designation screen 303 lists the paragraphs of the specification included in the input data 301. The user can specify a paragraph of the specification by the paragraph specification screen 303 and specify a part of a plurality of processing target data to be input to the learning model. When the user finishes specifying the processing target data, the user presses the candidate extraction button 304, inputs the specified plurality of processing target data into the learning model, and outputs the output data indicating at least one classification given to the application. obtain. The output data includes a value indicating the degree of approximation of each of the plurality of classifications that can be given to the application corresponding to the input data.

The classification candidate 305 on the UI screen shows a plurality of classifications that can be given to the application and the degree of approximation of each of the plurality of classifications. In this example, the plurality of classifications that can be given to the application are the IPCs "G06F 15/60" and "G06F 17/00", and the degree of approximation is "approximate value 1" and "approximate value 1" for "G06F 15/60". About "G06F 17/00", it is an "approximate value 0.8". In this example, a plurality of classifications that can be given to the application are displayed sorted in descending order of approximation, but the order of display is arbitrary.

When one of the plurality of categories displayed in the classification candidate 305 is selected by the user, the item that is the basis for granting the classification is displayed in the item-based grant basis description location 306, and the grant basis is displayed. The paragraph of the specification that has become is displayed in the paragraph-based grant basis description location 307. Further, when one of the items displayed in the item-based grant basis description location 306 or the paragraph displayed in the paragraph-based grant basis description location 307 is selected, the item or paragraph becomes the grant basis description location 308. The full text is displayed. By double-clicking any one of the plurality of classifications displayed in the classification candidate 305, the explanation of the classification can be displayed. In this way, the user can confirm the validity of the classification by confirming the description that is the basis for granting the classification automatically assigned by the learning model.

In the classification system 200 according to the present embodiment, when a plurality of items or a plurality of paragraphs are input to the learning model separately, the item or paragraph whose classification degree of approximation indicated by the output data of the learning model is equal to or more than a predetermined value is selected. , May be determined as the basis for grant. Further, the classification system 200 inputs the input data in which a plurality of items and a plurality of paragraphs are input to the training model and a specific item or a specific paragraph deleted from the plurality of items and the plurality of paragraphs in the training model. Depending on the case, the basis for assigning the classification may be determined based on how the output data changes. For example, if the degree of approximation drops by a threshold value or more when a specific item or a specific paragraph is deleted, the specific item or the specific paragraph may be used as the basis for grant. Here, an example in which the particle size of the grant basis is set as a paragraph has been described, but a sentence unit, a phrase unit, and a word unit can be used as the grant basis by the same processing. Then, it may be configured to highlight the portion that is the basis for granting. Examples of highlights are paragraphs, sentences, phrases, words only, changing the background color of the text compared to other parts, changing the text color compared to other parts, and underlining the text. You can draw it, make it bold, or increase the size of the font to highlight it. In addition, as an application example of FIG. 13, a paragraph serving as a grant basis may be displayed in the grant basis description portion 308, and a function of further highlighting sentences, phrases, and words of the grant basis may be provided.

The user selects the classification that is considered to best represent the characteristics of the application from the plurality of classifications displayed in the classification candidate 305, and presses the first determination button 309 to classify the classification as the first classification. Can be granted to the application as. Further, the user selects a classification that is considered to well represent the characteristics of the application from the plurality of classifications displayed in the classification candidate 305, and presses the classification determination button 310 to apply for the classification. Can be given to.

When the processing target data that is the basis for determining that a certain classification is given to the application is included in the predetermined processing target data among a plurality of processing target data, the providing department sets the classification as the first classification to the user terminal. May be provided. Here, the predetermined data to be processed is data that is considered to well represent the characteristics of the application. For example, when the application is a patent application, a summary or a higher-level claim that is considered to well represent the characteristics of the invention. It is a description of the section.

Further, the providing unit may provide the user terminal with the one having the highest degree of approximation among the plurality of classifications that can be given to the application as the first classification. Further, in the classification system 200, when the description portion provided as the grant ground is a description portion that well expresses the characteristics of the application, the classification given based on the grant ground may be the first classification.

The processing target data input unit 204 of the classification system 200 inputs text data and image data related to the application into the training model as a plurality of input data, and the model construction unit 201 is assigned the text data and the image data to the application. A learning model may be constructed based on the teacher data showing the relationship with the output data showing at least one classification. Here, if the application is a patent application, the text data may include the application, the specification, the claims and the abstract, and the image data may include the drawings and the chemical formulas, formulas and tables inserted in the specification. May include. In this case, the training model is a single model, and text data and image data are trained as a set of data. The text data to be input may be limited to the paragraph of the specification explaining the drawing, or may be obtained by performing OCR (Optical Character Recognition) processing on the drawing.

The processing target data input unit 204 of the classification system 200 inputs text data related to the application to the first sub-learning model included in the learning model as a part of the plurality of input data, and inputs image data related to the application to a plurality of inputs. As part of the data, input to the second sub-learning model included in the training model, the model building unit 201 indicates the relationship between the text data and the output data indicating at least one classification given to the application. Based on, the first sub-learning model is constructed, and the second sub-learning model is constructed based on the teacher data showing the relationship between the image data and the output data indicating at least one classification given to the application. May be good. In this case, the learning model includes a first sub-learning model and a second sub-learning model, text data is input to the first sub-learning model, and image data is input to the second sub-learning model. In this way, the sub-learning model may be constructed separately for each type of data, and the output data of the sub-learning model may be integrated to form the output data indicating the classification given to the application.

Classifications by domestic and foreign patent offices and international organizations may be revised, including consolidation. Even in such a case, the classification system 200 according to the present embodiment may be used. When there is a revision of the classification, either rebuild the learning model based on the new classification, or leave the learning model as it was built based on the old classification and perform conversion of the old and new classifications on the output data of the learning model. Good as it is.

Further, in this example, an example in which the IPC is displayed in the classification candidate 305 has been described, but the classification candidate 305 may display domestic and foreign patent classifications such as CPC, ECLA, ICO, FI, or F-term.

When the input data is data related to a design registration application, the classification candidate 305 may display domestic and foreign design classifications such as design classification and D-term. Similarly, when the input data is data related to a trademark registration application, the classification candidate 305 may display classifications related to domestic and foreign trademarks such as graphic classification of graphic trademarks, similar group codes of designated goods and services, and nice classifications. .. In this case, the learning model may include the trademark as input data and the graphic classification or the like as input data, or may include the designated goods and services as input data and the similar group code or nice classification as output data. There may be.

The learning model may be constructed by teacher data using designated goods and services as input data and appropriate designated goods and services as output data. Here, the appropriate designated goods and services refer to the registered designated goods and services. Further, the learning model may be constructed by teacher data using the trademark as input data and the designated goods and services as output data. This provides a learning model that proposes designated goods and services to be designated when filing a similar trademark application. Further, the learning model may be constructed by teacher data using the trademark as input data and the title as output data.

The matters described in the fourth embodiment can be appropriately applied to the first, second and third embodiments, for example, in addition to the in-house classification in the first, second or third embodiment, as an example 1. The configuration is such that the publication classification of the fourth embodiment is given, and the in-house classification and the publication classification can be given to the in-house application. The gazette classification given here can be used as the gazette classification given by the applicant at the time of filing, and when the gazette classification given by the JPO at the time of publication of the application is published and incorporated as data, in the fourth embodiment. The granted gazette classification may be overwritten with the gazette classification granted by the JPO. Further, as Example 2, the configuration may be such that the grounds for granting the in-house classification are displayed in the respective embodiments 1, 2 and 3 as in the configuration in which the grounds for granting the publication classification are displayed in the fourth embodiment.

On the contrary, the matters described in the first, second or third embodiment can be appropriately applied to the fourth embodiment, and for example, the teacher information received from the plurality of user terminals described in the first embodiment is integrated. (This function has a sub-function that determines teacher information by majority vote when teacher information from multiple user terminals is different, a sub-function that determines teacher information by adding weight to each user, and teacher information based on instructions from the reader. A sub-function for determining the above may be added) can also be applied to the fourth embodiment.

Each part for realizing the function of the above-described embodiment can be implemented by hardware or software, for example. When implemented by software, the program code that controls the hardware may be executed by various processors such as CPU and MPU. Hardware such as a circuit for realizing the function of the program code may be provided. Part of the program code may be implemented in hardware, and the rest may be executed by various processors.

200 Classification system 201 Model construction unit 202 Learning model 203 Teacher data input unit 204 Processing target data input unit 205 Output result provision unit 205
206 Destination DB
207 Teacher Information Receiving Department 208 Teacher Information Integration Department 209 Teacher Information Reflecting Department 210 Classified DB
250 Gazette DB
261 User terminal

Claims (22)

Input data relating to at least one gazette, and output data indicating the degree of risk given to the gazette corresponding to the input data, which is a degree indicating whether or not the object conflicts with the gazette. A model building unit that builds a learning model based on teacher data showing the relationship with
A processing target data input unit that inputs processing target data related to at least one publication to the learning model as the input data, and a processing target data input unit.
A providing unit that provides information on the processing target data to the user terminal according to the result output from the learning model in response to the input of the processing target data.
A receiving unit that receives teacher information that is the source of the teacher data from the user terminal, and
A classification system having a teacher data input unit that inputs the teacher data according to the teacher information to the model construction unit. Input data relating to at least one gazette, output data indicating at least one classification given to at least one gazette corresponding to the input data, and output indicating the degree of risk given to the gazette corresponding to the input data. A model building unit that builds a learning model based on teacher data that shows the relationship with the data,
A processing target data input unit that inputs processing target data related to at least one publication to the learning model as the input data, and a processing target data input unit.
A providing unit that provides information on the processing target data to the user terminal according to the result output from the learning model in response to the input of the processing target data.
A receiving unit that receives teacher information that is the source of the teacher data from the user terminal, and
A classification system having a teacher data input unit that inputs the teacher data according to the teacher information to the model construction unit. According to the result output from the learning model, the providing unit provides information on the processing target data to which a risk is given to each of the classifications given to one of the publications included in the input data. To provide to
The classification system according to claim 1 or 2 . The learning model includes a first learning model and a second learning model different from the first learning model.
The model building unit
The first learning model is based on the teacher data showing the relationship between the input data relating to at least one publication and the output data indicating at least one classification given to at least one classification corresponding to the input data. Build and
The second learning model is constructed based on the teacher data showing the relationship between the input data relating to at least one publication and the output data indicating the degree of danger given to the publication corresponding to the input data.
The processing target data input unit inputs the processing target data relating to at least one publication to the first learning model and the second learning model as the input data.
The providing unit provides information on the processing target data to the user terminal according to the results output from the first learning model and the second learning model in response to the input of the processing target data.
The classification system according to any one of claims 1 to 3 . The model building unit is based on teacher data showing the relationship between the input data relating to at least one classified gazette and the output data indicating the degree of risk given to the classified gazette corresponding to the input data. Build the second learning model,
The classification system according to claim 4 . The model building unit has input data relating to at least one publication corresponding to at least one of the target department or the target device, and the publication corresponding to at least one of the target department or the target device corresponding to the input data. The second learning model is constructed based on the output data indicating the degree of danger given to the data and the teacher data indicating the relationship between the two.
The classification system according to claim 4 . The teacher information transmitted from the user terminal includes information indicating that a specific classification of a specific publication is not appropriate.
The model building unit builds a learning model that does not assign the specific classification to the specific publication based on the teacher data according to the teacher information.
The classification system according to any one of claims 1 to 6 . The output data to which the relationship is indicated by the teacher data includes a value indicating the degree of approximation of each of the plurality of classifications that can be given to at least one of the publications corresponding to the input data.
The classification system according to any one of claims 1 to 7 . The classification system according to claim 8 , wherein the value indicating the degree of approximation includes a value indicating that they are close to each other or a value indicating that they are not close to each other. The classification system according to claim 9 , wherein the value indicating the degree of approximation further includes an intermediate value between a value indicating approximation and a value indicating non-approximation. The receiving unit receives the teacher information including the evaluation, and receives the teacher information.
The value indicating the degree of approximation is determined according to the evaluation.
The classification system according to claim 9 or 10 . The processing target data input unit inputs the processing target data related to a plurality of publications into the learning model.
The provision unit provides to the user terminal a list of publications in which the order of a plurality of publications is changed in descending order of values indicating approximation, according to any one of claims 9 to 11 . The described classification system. The classification system according to claim 12, wherein the plurality of publications are new publications issued in a predetermined period. The providing part
Get the information that identifies the user corresponding to each category,
Based on the acquired information that identifies the user, information regarding the processing target data is provided to the user terminal of the user corresponding to the classification output from the learning model.
The classification system according to any one of claims 9 to 13 . The providing part
When the degree of approximation output from the learning model in response to the input of the processing target data exceeds a predetermined threshold value, the processing target data is transferred to the user terminal of the user corresponding to the classification having the approximation degree exceeding the threshold value. Offer to,
When the degree of approximation is equal to or less than a predetermined threshold value, the data to be processed is not provided to the user terminal of the user corresponding to the classification having the degree of approximation equal to or less than the threshold value.
The classification system according to claim 14 . When the degree of approximation output from the learning model exceeds the threshold value for a plurality of classifications in response to the input of the processing target data, the providing unit sends the processing target data to the user terminal of the user corresponding to each classification. Provide information about
The receiving unit receives teacher information from each user terminal and receives teacher information.
It also has an integration unit that integrates teacher information received from multiple user terminals.
The classification system according to claim 14 or 15 . When the teacher information of some user terminals is different from the teacher information of other user terminals, the integrated unit determines the teacher information to be adopted by a majority vote.
The classification system according to claim 16 . The integrated part is
Acquires the weight information set for each user,
When the teacher information of some user terminals is different from the teacher information of other user terminals, the teacher information is determined by adding the weight of each user.
The classification system according to claim 16 . The integrated part is
The reader can be identified among the users,
When the teacher information of some user terminals is different from the teacher information of other user terminals, the instruction from the reader is received and the teacher information is determined based on the received instruction.
The classification system according to claim 16 . Input data relating to at least one gazette, and output data indicating the degree of risk given to the gazette corresponding to the input data, which is a degree indicating whether or not the object conflicts with the gazette. It is a control method of a classification system having a model building unit that builds a learning model based on teacher data showing the relationship with.
A processing target data input step in which processing target data relating to at least one publication is input to the learning model as the input data,
A provision step of providing information on the processing target data to the user terminal according to the result output from the learning model in response to the input of the processing target data, and a provision step.
A reception step for receiving teacher information that is the source of the teacher data from the user terminal, and
A teacher data input step in which the teacher data corresponding to the teacher information is input to the model construction unit, and
A method of controlling a classification system. Input data relating to at least one gazette, output data indicating at least one classification given to at least one gazette corresponding to the input data, and output indicating the degree of risk given to the gazette corresponding to the input data. It is a control method of a classification system having a model building unit that builds a learning model based on teacher data showing the relationship with the data.
A processing target data input step in which processing target data relating to at least one publication is input to the learning model as the input data,
A provision step of providing information on the processing target data to the user terminal according to the result output from the learning model in response to the input of the processing target data, and a provision step.
A reception step for receiving teacher information that is the source of the teacher data from the user terminal, and
A teacher data input step in which the teacher data corresponding to the teacher information is input to the model construction unit, and
A method of controlling a classification system. A program for making a computer function as each part of the classification system according to any one of claims 1 to 19 .

Images