KR20150015424A - Method for online evaluating patents - Google Patents

Abstract

Provided by an embodiment disclosed in the present invention is a method to receive the evaluation result of a sample patent in advance from an expert to build a patent evaluation engine in a server. The method includes the steps in which: the server firstly maps candidates of relevant evaluation elements onto one or more predefined patent evaluation items and provides the result to the computer of an expert; the server receives an evaluation result for each of the evaluation items of the sample patent from the expert computer; the server calculates a correlation between each evaluation item and each evaluation element based on the evaluation result for each evaluation item by the expert; each evaluation item is mapped again onto each evaluation element based on the calculated correlation; and an evaluation engine is built by studying the evaluation result by the expert and using the evaluation elements mapped on the evaluation items.

Description

{METHOD FOR ONLINE EVALUATION PATENTS}

This specification relates to a method for receiving a patent evaluation online by an expert in advance in order to construct a patent evaluation engine.

In recent years, there have been many efforts and achievements in intellectual property rights for technology protection in Korea compared to other developed countries. In this process, it is true that the intellectual property rights holders have a lot of intellectual property rights, and the cost and effort to maintain and maintain the intellectual property rights are one of the burden.

In addition, it is also a difficult task to identify technologies that have been so rapidly changed that they have been registered, but that are no longer required to be maintained, and which, among other things, have to be invested heavily.

Therefore, intellectual property owners are assessing their intellectual property rights by rating their own intellectual property rights, or requesting commercial or nonprofit organizations to evaluate patented technologies.

On the other hand, the results of the patent evaluation include the maintenance of the patent, the provision of the utilization strategy of the patent, the support of the research plan, the evaluation of the right, the economic and the environment, (Strategic alliance), allocation of R & D planning resources, assessment of technology-secured loans for financial institution loans, support for direct and indirect technological development projects of government, evaluation for selection of project (assignment), intangible asset value We evaluate the valuation, technical, economic, and social aspects of the valuation, evaluation of the intangible asset's technology value for the customer's technology based on clear and objective data, present value of the inventor, compensation for the inventor (for depreciation) , Technology evaluation such as intellectual property right for technology transaction (technology transfer, M & A), valuation of intellectual property rights for technology guarantee setting, investment attraction etc. Janghi wide.

However, since the time and cost of the high-level human resources required for data analysis, which is the process of reporting to the intellectual property owner after the technology evaluation of the registered patent, is significant, most of the work is manually performed, It is necessary.

1 is an exemplary diagram showing the necessity of introducing a patent evaluation system.

As can be seen from FIG. 1 (a), if there are a large number of patents, for example, hundreds or thousands of patents, it will take a considerable amount of time and money to refer the patent evaluation to a specialist.

However, as shown in FIG. 1 (b), when a large number of patents are firstly filtered by a system, only a small amount of patents, for example, dozens of patents, can be evaluated by an expert, And reduce costs.

Accordingly, one embodiment disclosed herein is intended to provide a method for receiving an evaluation of a sample patent on-line from an expert as a precondition for constructing a patent evaluation engine for patent evaluation. It is also an object of the present disclosure to provide an evaluation engine in a system for patent evaluation.

In order to solve the above-described problems, a method according to an embodiment of the present invention provides a method of receiving an evaluation result of a sample patent in advance from an expert in order to build a patent evaluation engine in a server. The method comprising the steps of: mapping one or more predefined patent evaluation items and candidates of relevant evaluation elements to a server of an expert; Receiving, by the server, an evaluation result for each sample item from the expert computer; A step of the server calculating a degree of correlation between each evaluation item and each evaluation element based on the evaluation results of the experts by the evaluation items; Remapping each evaluation item and each evaluation element based on the calculated correlation; And constructing an evaluation engine by learning an evaluation result of an expert using an evaluation element mapped to the evaluation item.

According to another aspect of the present invention, there is provided a server for receiving an evaluation result of a sample patent from an expert in order to construct a patent evaluation engine. The server first maps one or more predefined patent evaluation items to candidates of related evaluation elements and provides the server to the computer of the expert and receives the evaluation result for each sample item from the expert computer on the sample patent Receiving unit; Calculating a degree of correlation between each of the evaluation items and each of the evaluation factors based on the evaluation results of the experts by the evaluation items, re-mapping each evaluation item and each of the evaluation factors based on the calculated degree of correlation, And a controller for building an evaluation engine by learning the evaluation results of the experts using the evaluation elements mapped to the items.

The evaluation element may be information extracted from at least one of bibliographic information, progress information, specification, and claims of the registered patent.

The evaluation element may include information extracted by processing the specification of the registered patent and claims in a natural language.

The evaluation items may include one or more of rightness, technicality, and usability.

The method includes receiving information about a particular patent from a user device; Receiving an evaluation request for the specific patent from the user device; And outputting the evaluation result calculated using the established evaluation engine for a specific patent identified using the information.

The outputting of the evaluation result may include outputting the evaluation result calculated in advance when the evaluation result of the specific patent confirmed using the information is calculated in advance, In response to the user's request, immediately calculate and output the evaluation result.

In the step of outputting the evaluation result, the evaluation result may be immediately calculated and output in response to the request of the user for a specific patent identified using the information.

The evaluation result by the expert is performed for each technical field, and the evaluation engine is constructed for each technical field. In the step of outputting the evaluation result, an evaluation engine of a technical field corresponding to the technical field of the specific patent can be used have.

The evaluation engine can be constructed based on the evaluation results of the experts having good correlation based on the calculated correlation, based on the results of evaluation by a plurality of experts per technical field.

According to one embodiment presented herein, a patent can be automatically evaluated by a system and the evaluation result can be presented. Further, according to the evaluation engine according to the embodiment disclosed in the present specification, the patent can be quantified and more objectively evaluated.

1 is an exemplary diagram showing the necessity of introducing a patent evaluation system.
2 is a diagram illustrating an overall configuration of a patent evaluation system according to an embodiment of the present invention.
FIG. 3 is an exemplary view showing one or more servers 100 shown in FIG. 2 in detail.
FIG. 4 is an exemplary view showing in detail the configuration of the domestic / foreign patent evaluation servers 110 and 130 in FIG.
5 is a flowchart illustrating a method of constructing an evaluation engine by learning an expert evaluation result according to an embodiment of the present invention.
6 is a flowchart illustrating a method of providing a patent evaluation service using an evaluation engine according to an embodiment of the present invention.
FIG. 7 shows the physical configurations of the evaluation servers 110 and 130 and the service servers 120 and 140, respectively, presented in an embodiment of the present invention.
8 to 10 are exemplary views showing a web page screen.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In the present application, the term "comprising" or "comprising" or the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

Further, the suffix "module" and "part" for components used in the present specification are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Furthermore, terms including ordinals such as first, second, etc. used in this specification can be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

2 is a diagram illustrating an overall configuration of a patent evaluation system according to an embodiment of the present invention.

2, the patent evaluation system according to an embodiment of the present invention includes at least one server 100 and at least one database (hereinafter referred to as 'DB') 190 do. The one or more servers 100 may be remotely managed by the management apparatus 500 shown in FIG.

The one or more servers 100 may be connected to a wired / wireless network to provide evaluation results services and other services to the user device 600. Specifically, upon receiving a request for an evaluation service for a specific patent from a user device, the one or more servers 100 may provide an evaluation result for the specific patent.

FIG. 3 is an exemplary view showing one or more servers 100 shown in FIG. 2 in detail.

3, one or more servers 100 may include a domestic patent evaluation server 110, a domestic patent service server 120, an overseas (e.g., US) patent evaluation server 130, ) Patent service server 140 as shown in FIG. In FIG. 3, the domestic patent evaluation server 110 and the foreign (for example, US) patent evaluation server 130 are shown as being physically separated from each other, but they can be physically integrated into one server. Also, although the domestic patent service server 120 and the foreign patent service server 140 are physically separated from each other, they may be physically integrated into one server. Further, the illustrated servers 110, 120, 130, 140 may be physically integrated into one server.

3, the above-mentioned one or more databases 190 include patent information DBs 191 and 192, evaluation element (or evaluation index) DBs 193 and 194, similar patent DBs 195 and 196, , And an evaluation result DB 197, 198. Although each DB is shown separately for domestic patent evaluation and US patent evaluation, it can be merged. For example, the Korean patent information DB 191 and the foreign patent information DB 192 of a foreign country (for example, the United States of America) can be merged into one, and the Korean evaluation factor (or evaluation index) , Or the United States) evaluation index (or evaluation index) DB 194 can be merged into one. Alternatively, the DBs may all be merged into one and separated by fields.

Such DBs may be generated based on those received from the external DB provider. For this reception, the server 100 may include a data collection unit 150 for receiving a raw DB of Korean or foreign (for example, the United States) from an external DB provider. The data collecting unit 150 physically includes a network interface (NIC). In addition, the data collection unit 150 may logically be a program including an application programming interface (API). The data collector 150 may process the raw DB received from the external DB provider and store the raw DB in one or more DBs 190 connected to the server 100 in the patent information DBs 191 and 192.

The domestic / foreign patent evaluation servers 110 and 130 include specification processing units 111 and 131, natural language processing units 112 and 132, key word processing units 113 and 133, similar patent processing units 114 and 134, (Or evaluation indexes) 115 and 135, and evaluation engine sections 116 and 136. [

The specification processing units 111 and 131 extract and parse (or convert) each information from the at least one DB 190 (e.g., patent information DBs 191 and 192). For example, the specification processing units 111 and 131 may extract one or more pieces of information such as patent specifications, bibliographic information, progress information, claims, drawings, and the like, and store them in the respective fields of the evaluation element (or evaluation index) DB.

The natural language processing units 112 and 132 perform natural language processing on the extracted patent specification and text included in the claims. Here, natural language processing refers not to a special programming language for computers but to computer processing of natural language used in general paintings. For example, the natural language processing units 112 and 132 may perform sentence analysis, syntax analysis, compound word processing, and the like. In addition, the natural language processing units 112 and 132 may perform a semantic process.

The key word processing units 113 and 133 extract key words from each patent based on the result of the natural language processing. In order to extract the keyword, techniques such as VSM (Vector Space Model) and LSA (Latent Semantic Analysis) can be used. Here, the core of the patent specification is a word representing the subject of the patent specification. For example, in the case of the present patent specification, " patent evaluation " As such, it may be advantageous to extract as many keywords as possible to represent the subject matter of the individual patent specification, but it is preferable to select the appropriate number because it may increase the inaccuracies if the number is increased.

Based on the extracted key word, the similar patent processing units 114 and 134 can search for patents most similar to each patent, and store the search results in the similar patent DBs 195 and 196. In general, similar patent families are known to belong to the same subclass in the International Patent Classification (IPC), but one embodiment presented herein can search for similar patents in other subclasses beyond the same subclass . Thus, in order to improve accuracy when searching for similar patents in other subclasses, it is most important that key words are extracted correctly. In particular, as described above, if the number of key words is simply increased, an incorrect key word is extracted, thereby making it possible to make an error that a completely different patent is searched by a similar patent in another sub-class. Therefore, one embodiment of the present invention calculates a suitable number of key words based on the simulation result according to the number of key words, and searches for similar patents with the calculated number of key words.

The evaluation element (or evaluation index) processing unit 115 or 116 extracts the value of the evaluation element (or the evaluation index) from at least one of the patent specification, bibliographic information, progress information, claim range, And stores it in the evaluation element DB 194.

The evaluation factors may be different for domestic patent evaluation and foreign patent evaluation. For example, the evaluation factors for domestic patent evaluation are summarized as follows.

Evaluation Element Name Explanation One Independent claim length Number of words in independent claim 2 Claims Number of claims 3 Number of claims Number of categories (objects or methods) of independent claims 4 Independent port number Number of independent claims 5 Domestic family patent number Number of domestic family patents (family based on split application, same basic priority) 6 Overseas family patent number Family patents of overseas countries 7 Annual registration number Number of registration year 8 First, Whether there was a priority claim 9 Eligibility Date The period until the filing date 10 Number of comments submitted Number of submissions 11 Decision of Refusal The number of the charges of the judgment of refusal decision 12 Total number of citations Total number of citations 13 Joint filing Number of joint applicants 14 Number of licensees Number of licensees 15 Number of active judges Referee's billing count 16 First ref Referee's billing count 17 Early public availability Whether there was a public disclosure 18 Information Provided Number of times information is provided 19 Number of appeals Number of appeals 20 Third party review request Whether the applicant or a third party has filed a claim for review 21 Number of invalid referees The number of judgments 22 Number of judges who confirm passive scope of right The number of judgments 23 Conducted Jesus Number of embodiments 24 Number of drawings Number of drawings 25 The number of words in the description of the invention Number of words in the detailed description part of the specification 26 Number of IPC Number of IPC classification codes 27 Rights holder change Number of rights holder change 28 Litigation information If there is a lawsuit, the number of times 29 Prior Art Literature Number of prior art documents presented during the review process

On the other hand, evaluation factors for overseas patents, for example, US patent evaluation, are summarized in the following table.

Evaluation factor Explanation One Independent claim length Number of words in independent claim 2 Number of claims Number of categories (objects or methods) of independent claims 3 Independent port number Number of independent claims 4 The number of words in the description of the invention Number of words in the detailed description part of the specification 5 Claims Number of claims 6 Number of family patents in the United States Number of family patents in the United States 7 Number of Reexamination Reexamination Count 8 Interference Count Interference count 9 Reissue number Reissue number 10 Total number of citations Total number of citations 11 Number of IPC Number of IPC classification codes 12 Overseas family patent number Number of overseas country family patents 13 Annual registration number Number of registration year 14 First, Whether there was a priority claim 15 Certification of Correction Number Certification of Correction claimed 16 Rights holder change Number of rights holder change 17 Litigation information If there is a lawsuit, the number of times 18 Prior Art Literature Number of prior art documents presented during the review process

On the other hand, the above evaluation factors (evaluation indexes) are merely illustrative, and any information directly derived from the patent information can be utilized as evaluation factors (evaluation indexes). Furthermore, any information that can be indirectly derived or calculated by processing patent information can be utilized as an evaluation factor (evaluation index).

On the other hand, the evaluation engine unit (116, 136) evaluates each patent based on the evaluation factors and the evaluation mechanism stored in the evaluation element DB (193, 194) for each predefined evaluation item, and calculates the evaluation result. Further, the evaluation engine units 116 and 136 may store the calculated evaluation results in the evaluation result DB 197 and 198, respectively.

The evaluation items may be defined as rights, technicalities, and usability. Alternatively, the evaluation items may be defined as rights property and commercial property. This definition may change depending on what the primary purpose of evaluating a patent is. Accordingly, the scope of the present invention is not limited to the above-described embodiments, and can be extended to any one to which the spirit of the present invention can be applied.

The evaluation mechanism may include a weight and a learning model. The weight may be a value calculated based on a result of an expert's preliminary evaluation of several sample patents. The evaluation engine units 116 and 136 will be described later in detail with reference to FIG.

The domestic / foreign patent service servers 120 and 140 may include at least one of an evaluation report generation unit 121 and a portfolio analysis unit 122 and a portfolio analysis unit 142. The evaluation report generation units 121 and 141 generate evaluation reports based on the evaluation results stored in the evaluation results DB 197 and 198. [ The portfolio analyzing units 122 and 142 can analyze the portfolio of the patents held by the owner based on the information stored in the similar patent DBs 195 and 196. [ Also, the portfolio analyzing units 122 and 142 may analyze the status of the patent by technology field or IPC classification by the right holder. In addition, the portfolio analysis units 122 and 142 can perform various analyzes based on the similar patent DBs 195 and 196 and the evaluation result DBs 197 and 19. For example, the portfolio analysis units 122 and 142 may perform various analyzes such as analysis of patent trends, analysis of the total amount of the annual fee for each patentee, and the like.

The domestic / foreign patent service servers 120 and 140 may provide an evaluation result of the specific patent when a user requests an evaluation service for the specific patent from the user device. In addition, the evaluation report may be provided in the form of a web page, an Excel file, a PDF file, or a word file according to a user's request, or may provide the analysis result. For this service, the user authentication / authority management unit 160 may be required.

FIG. 4 is an exemplary view showing in detail the configuration of the domestic / foreign patent evaluation servers 110 and 130 in FIG.

As can be seen from FIG. 4 and the above description, the description processing units 111 and 131 receive the patent specification information from the patent information DBs 191 and 192, and parse the specification. The specification may be written in XML, for example. In order to parse the XML, the specification processing units 111 and 131 may include an XML tag processing unit.

The evaluation element processing units 115 and 135 may include a first evaluation element processing unit that processes the evaluation element based on the parsed specification, and a second evaluation element processing unit. The first evaluation element processing unit extracts an evaluation element that does not need a natural language processing result based on the parsed specification. For example, the first evaluation element processing unit calculates the values of evaluation elements that do not require natural language processing, such as the length of the independent claim, the number of claims, the number of claims, the number of independent factors, the number of domestic family patents, And stores it in the evaluation element DB 193/194.

The natural language processing units 112 and 132 perform natural language processing based on the parsed specification. The natural language processing units 112 and 132 include a morpheme analysis unit for performing an operation based on a dictionary DB and a TM analysis. Morpheme is the smallest unit of meaning and is the smallest semantic element that can no longer be analyzed. Morpheme analysis is the first step in natural language analysis. It converts input strings into morpheme strings. The TM analysis is a two-level analysis, expressed as Tm = (R, F, D), where R is a rule set, F is a finite automaton converter, and T is a triad dictionary.

When the natural language processing is completed, the second evaluation element processing unit of the evaluation element processing units 115 and 135 calculates remaining evaluation elements based on the natural language processing result. For example, the evaluation element values such as " keyword consistency with similar foreign patent families " summarized in Table 1 above are calculated and stored in the evaluation element DB 193/194.

Meanwhile, the keyword extracting unit 113/133 for extracting a keyword based on the result of the natural language processing may include a keyword candidate selecting unit, a stopword removing unit, and a key word selecting unit. The key word candidate selection unit selects candidates of key words that best represent the subject of the individual patent. The abbreviated term eliminating unit removes the insignificant words having low importance from the extracted key word candidates. The key word selection unit finally selects a proper number of key words from the remaining key word candidates from which the stopwords have been removed, and stores them in the evaluation element DB 193/194.

The accuracy according to the number of key words is summarized in the following table.

Keyword count 10 things 50 Recall frequency 22.7% 54.1% Precision 20.6% 10.9%

Referring to Table 3, the probability of being recalled (i.e., the probability that a keyword will be reused) is 22.7% when 10 keywords are used, and 54.1% when the keyword is increased to 50 keywords. However, when the number of keywords is 50, the accuracy is as low as 10.9%. On the contrary, when the number of keywords is 10, the accuracy is 20.6%. As described above, if the number of key words is simply increased, the frequency of recalling may increase but the accuracy becomes lower. Therefore, the most optimal number of key words can be calculated based on the calculated accuracy.

The similar patent extracting units 114 and 134 serve to search for similar patents on the basis of the keyword, and may include a document clustering unit, a document similarity calculating unit, and a similar patent generating unit. The document clustering unit clusters similar patents primarily based on the keyword. The document similarity calculation unit calculates the degree of similarity among the patent documents among the clustered patents. Based on the calculated similarity, the similar patent generation unit generates actually the most similar patents among the primarily clustered patent documents, and stores the results in the similar patent DBs 195 and 196.

The patent evaluation engine sections (116, 136) include a learning model section and a patent evaluation section. And the learning model unit performs learning based on the expert evaluation result DB. To do this, experts in each technology field can receive evaluation results on sample patents.

The sample patent is a set for learning, and extracts hundreds to thousands of cases from the patent information DBs 191 and 192 and generates them. The sample patent may be extracted so that the values of the evaluation factors shown in Table 1 and Table 2 are uniformly included. For example, if the value of the above-mentioned evaluation factors such as invalid judgment, active right range judgment judgment, passive rights judgment judgment, and first judgment are not 0, the patent can be applied to all registered patents (approximately several tens to millions of patents) It is quite rare. Therefore, it is preferable to extract the sample patents so that the patents having non-zero values of the respective evaluation elements are distributed at a certain ratio. When the sample patents are selected, they can be roughly divided into a plurality of sets (for example, ten sets). Learning is performed using a plurality of sets of the plurality of sets, and the learning result is verified using the remaining sets.

In order to receive the expert evaluation result, the service server 120, 140 may provide a web page screen in which an expert can log in, as shown in FIG. When the expert logs in by inputting an account and a password, a list of patents to be evaluated by the expert can be provided as shown in FIG. When the expert clicks on any patent on the list, the value of the evaluation element for each sample patent and the evaluation item described above are provided to the expert as shown in FIG. For example, the service server 120 or 140 provides the computer of the expert with a web page listing the evaluation items (for example, the rightness, the technicality, and the usability) described above. In addition, the service server 120 or 140 provides the computer of the expert with a web page listing the evaluation factors as listed in Table 1 or Table 2. At this time, the service server 120 or 140 may map candidates of evaluation factors related to each evaluation item. For example, candidates for evaluation factors related to each evaluation item may be mapped and shown as shown in Table 4 below.

Evaluation items Evaluation Criteria Evaluation factor Right Right-of-way Independent claim length, independent number, number of claims Claim Variety Number of independent claims, number of claim series, number of claims, Technical Technology Leadership Filing date Technology persistence Citation / cited relationship Usability Possibility of commercialization Priority review, determination of active rights range Judgment Possibility of exercise of rights Actively ascertain scope of rights Judgment, litigation information

In Table 4 above, there are two sub-evaluation items for the evaluation item "rightness" as an example, namely, "the scope of the scope of the rights" and "the claims diversity" There are two lower evaluation items, namely "commercialization possibility" and "possibility of exercise of rights", in the presence of the lower evaluation items, namely, "technology leading" and "technology sustainability" However, this is merely an example and can be variously changed.

Then, the expert inputs the score for each evaluation item on the web page while viewing the value of the relevant evaluation element candidates for each evaluation item, and the service server 120, 140 receives the score , And stores it in the expert evaluation DB.

Then, the learning model unit extracts evaluation factors actually related to each evaluation item based on the expert evaluation DB. Specifically, the learning model unit analyzes the correlation between each evaluation item and each evaluation element based on the expert evaluation DB. For example, if the result of the expert evaluation result is based on DB, if the value of the evaluation factor increases, the score of the evaluation item input by the expert increases, or when the value of the evaluation factor decreases, .

For example, the results of analyzing the correlation of the evaluation factors with respect to the evaluation item "rightness" are shown in Table 5 below.

Evaluation factor Correlation Claims 0.442313457 Independent port number 0.43624915 Number of claims 0.369720889 Independent claim length -0.331545077 Support specification -0.21671485 Number of licensees -0.149643054 The number of words in the description of the invention 0.148732909 First, -0.145481466 Rights holder change -0.114275841 Third party review request 0.114180544 Early public availability -0.093078214 Number of invalid referees 0.092559136 Number of comments submitted 0.090248983 Litigation information 0.083141351 Decision of Refusal 0.061068447 Number of IPC 0.05544511 Annual registration number 0.053290108 Filing fee 0.049645717 Domestic family patent number -0.046240513 Total number of citations 0.045334312 First ref 0.038343259 Number of judges who confirm passive scope of right 0.02519305 Information Provided 0.015832255 Number of active judges 0.014261697 Overseas family patent number -0.009559996

In the correlation value summarized in Table 5, a negative value indicates that the value of the evaluation item increases when the value of the evaluation element decreases, and a positive value indicates that the value of the evaluation item increases when the value of the evaluation element increases Respectively.

For example, the results of the analysis of the evaluation factors for the evaluation item "technicality" are shown in Table 6 below.

Evaluation factor Correlation Total number of citations 0.371199951 Keyword consistency with leading patent 0.313839149 Mean Depth of Dependence 0.228073655 Independent port number 0.214881486 Claims 0.198064827 Personality ratio of related patentee 0.170751806 Independent claim length -0.155430986 Authority Agency Attributes 0.145374949 Number of IPC 0.125438684 Number of claims 0.121824863 Rights holder change -0.119199481 Early public availability -0.117456589 Annual registration number -0.107145378 Number of invalid referees 0.092747643 Litigation information 0.087185784 Filing fee 0.085308543 Information Provided -0.069417918 First, -0.06285232 Number of comments submitted 0.054409496 First ref 0.048091418 Number of judges who confirm passive scope of right 0.046281442 Decision of Refusal -0.044629864 Number of licensees -0.033172901 Domestic family patent number -0.032396341 The number of words in the description of the invention 0.023315445 Number of active judges -0.021026967 Overseas family patent number -0.01527722 Third party review request 0.005756606

For example, Table 7 shows the results of analyzing the correlation of the evaluation factors for the evaluation item "usability".

Evaluation factor Correlation Number of invalid referees 0.451065838 First ref 0.386848273 Number of judges who confirm passive scope of right 0.358071363 Litigation information 0.313920588 Annual registration number 0.307371492 Number of active judges 0.3035969 Total number of citations 0.285179065 Difference between citation and filing date 0.274485287 Independent claim length -0.196663177 Rights holder change 0.156737637 Support specification -0.141657162 Mean Depth of Dependence 0.139758293 Claims 0.135037121 Early public availability 0.132799209 Independent port number 0.120920898 Number of IPC -0.105520543 Number of licensees 0.102959263 Keyword consistency with leading patent 0.081065843 First, 0.053285091 Number of claims 0.05240555 Decision of Refusal 0.050829491 Third party review request 0.049764894 Filing fee -0.044103946 Information Provided 0.040184183 Domestic family patent number -0.025321814 Holder-like patent share -0.018127852 Number of comments submitted 0.007443184 The number of words in the description of the invention -0.006710761 Overseas family patent number 0.003790473

Overseas family patent number 0.003790473

Based on the correlation values listed in Tables 5, 6 and 7, the learning model unit extracts evaluation elements having a high correlation with the evaluation item " rightness ". The correlation value is a value between -1 and +1. If the value is usually 0.2 to 0.6, the correlation is high. Therefore, the learning model unit can select, for example, the number of claims, the number of independent factors, the number of claims, and the like as evaluation factors for the evaluation item " rightness ".

In order to solve such a problem, according to one embodiment of the present invention, it is possible to additionally calculate the correlation between experts.

Field expert Correlation Right Technical Usability Electronic A B 0.64 0.39 0.83 C D 0.29 0.32 0.48 machine E F 0.60 0.23 0.55 G H 0.59 0.23 0.63 chemistry I J 0.66 0.71 0.60 K L 0.59 0.34 0.50 physics M N 0.50 0.35 0.48 O P 0.81 0.15 0.80 Bio Q R 0.64 0.66 0.19 S T 0.51 0.45 0.38

As summarized in Table 8 above, for example, the technical field can be divided into electronic, mechanical, chemical, physical, and bio fields. Then, experts can be assigned to a plurality of experts in each field, and then experts in each field can be paired. For example, in the case of the electronic field, experts A, B, C, D are assigned to experts A and B, and then experts C and D are evaluated Pairing is performed on the same patent. In this case, as a result of calculating the degree of correlation between experts A and B in the electronic field, as shown in Table 5, the evaluation item "rightness" was 0.64, the evaluation item "technicality" was 0.39, and the evaluation item "usability" 0.89. If the correlations between the pair of experts are low, one may use the results of the more highly correlated pair of experts, or may assign a higher weight to one of the alternatively paired experts.

In this manner, after the learning model unit defines the evaluation factors for each evaluation item, the learning model unit performs learning based on the evaluation scores of the experts stored in the expert evaluation DB. Specifically, the learning model unit calculates a weight value based on the evaluation score of the expert stored in the expert evaluation DB, and performs the learning using the calculated weight value. At this time, the learning can be performed by technology field. As described above, since the evaluation of the sample patent by the expert is made by the technical field, similarly, the above-mentioned learning is also performed for each technology field. For example, in the case of the mechanical field and the electronic field, the longer the independent claim length is, the narrower the claim range. In the case of the chemical field, however, the length of the independent claim and the scope of the claim range may be irrelevant. Learning is done on a technology-by-technology basis. Therefore, the above-described weight values can also be separately calculated for each technical field.

The patent evaluation section of the patent evaluation engine section (116, 136) evaluates the patents according to the learned results, and stores the evaluated results in the evaluation result DB (197, 198).

5 is a flowchart illustrating a method of constructing an evaluation engine by learning an expert evaluation result according to an embodiment of the present invention.

As can be seen from FIG. 5 and the above description, the evaluation items can be defined in advance (S110). As described above, the evaluation items can be defined as rights, technicalities, and usability. Alternatively, the evaluation items may be defined as rights property and commercial property. This definition may change depending on what the primary purpose of evaluating a patent is.

Then, the service servers 120 and 140 firstly map the evaluation items and evaluation factors for the sample patent, and provide them to the expert computer (S120). The primary mapping may be to map candidates of the inferred evaluation factor associated with each item of evaluation.

Then, the evaluation result of the sample patent may be received from the expert computer (S130). The evaluation result may be a score evaluated by the expert on the evaluation item. In order to provide information to the expert computer and receive the evaluation result, the service server 120 or 140 may prepare a web page in advance.

Subsequently, based on the evaluation results of the experts on the sample patent, the degree of correlation of the evaluation factors with respect to one or more predefined evaluation items can be calculated (S140). The correlation may be a value between -1 and +1 as described above.

Then, each evaluation item and the evaluation element can be remapped based on the calculated degree of correlation (S150). Some of the evaluation elements that are primarily mapped to the respective evaluation items by the remapping may be excluded from the mapping, and other evaluation elements may be mapped to the arbitrary evaluation items.

As such, when the mapping is completed, the evaluation result mapped to the evaluation item is used to learn the evaluation result of the expert, and an evaluation engine is constructed (S160).

6 is a flowchart illustrating a method of providing a patent evaluation service using an evaluation engine according to an embodiment of the present invention.

As can be seen from FIG. 6, the service server 120, 140 receives information on a specific patent from the user device (S210), and receives an evaluation request for the specific patent from the user device (S220). To this end, the service server 120, 140 may provide a web page to the user's computer.

Then, the service server 120, 140 may provide an evaluation result calculated using an evaluation engine constructed in advance for a specific patent identified using the information, and output the evaluation result to the user's computer (S230) .

At this time, the service server 120 or 140 may merely provide an evaluation result, but may generate an evaluation report and provide the evaluation report to the user's computer. The evaluation report may include an evaluation result that is calculated and a detailed description of the evaluation result. Such an evaluation report may be in the same format as the PDF, or it may be based on a web page.

The above-described method can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

For a hardware implementation, the method according to embodiments of the present invention may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs) , Field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

7 shows physical configurations of the evaluation servers 110 and 130 and the service servers 120 and 140, respectively, which are shown in an embodiment of the present invention.

7, the evaluation servers 110 and 130 may include transmission / reception units 110a and 130a, controllers 110b and 130b, and storage units 110c and 130c. Each of the service servers 120 and 140 May include transmission / reception units 120a and 140a, controllers 120b and 140b, and storage units 120c and 140c.

The storing means stores the method shown in Figs. 3 and 6 and the above-mentioned contents. For example, the storage means 110c and 130c of the evaluation servers 110 and 130 are respectively provided with the specification processing units 111 and 131, the natural language processing units 111 and 132, the keyword processing units 113 and 133, The processing units 114 and 134, the evaluation elements (or evaluation indexes) 115 and 135, and the evaluation engine units 116 and 136 may store the respective implemented programs. The storage means 120c and 140c of the service servers 120 and 140 may store at least one of the evaluation report generating units 121 and 141 and the portfolio analyzing units 122 and 142. [

The controller controls the transmission / reception unit and the storage unit. Specifically, the controllers execute the method or the programs stored in the storage means, respectively. The controller transmits and receives signals through the transceiver.

The embodiments disclosed herein have been described with reference to the accompanying drawings. Here, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings, but should be construed as meaning and concept consistent with the technical idea disclosed in the present specification.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only examples described in the present specification, and not all of the technical ideas disclosed in the present specification are described. Therefore, various modifications It is to be understood that equivalents and modifications are possible.

110: domestic patent evaluation server
120: Domestic patent service server
130: Overseas (e.g., US) patent evaluation server
140: Overseas (e.g., US) Patent Service Server
191, 192, Patent information DB
193, 194: evaluation element (or evaluation index) DB
195, 196: Similar patent DB
197, 198: Evaluation results DB

Claims (10)

A method for receiving an evaluation result of a sample patent in advance from an expert in order to construct a patent evaluation engine in a server,
Mapping one or more predefined patent evaluation items and candidates of correlated evaluation elements to a server of an expert;
Receiving, by the server, an evaluation result for each sample item from the expert computer;
A step of the server calculating a degree of correlation between each evaluation item and each evaluation element based on the evaluation results of the experts by the evaluation items;
Remapping each evaluation item and each evaluation element based on the calculated correlation;
And constructing an evaluation engine by learning an evaluation result of an expert using an evaluation factor mapped to the evaluation item. 2. The method of claim 1,
Wherein the information is information extracted from at least one of bibliographic information, progress information, specification, and claims of the registered patent. 3. The system of claim 2, wherein the evaluation element
Wherein the information includes information extracted by processing the specification of the registered patent and claims in a natural language, and receiving the patent evaluation from an expert. The method according to claim 1,
A method of receiving a patent evaluation from an expert that is characterized by including one or more of: The method according to claim 1,
Receiving information about a particular patent from a user device;
Receiving an evaluation request for the specific patent from the user device;
Further comprising the step of outputting an evaluation result calculated using the evaluation engine constructed for a specific patent identified using the information. 6. The method according to claim 5, wherein in the step of outputting the evaluation result
And outputting the evaluation result calculated in advance if the evaluation result of the specific patent confirmed using the information is calculated in advance,
The evaluation result is immediately calculated and output in response to the request of the user for a specific patent identified using the information
A method for automatically evaluating a patent using an evaluation engine. 6. The method according to claim 5, wherein in the step of outputting the evaluation result
Wherein the evaluation result is immediately calculated and output in response to the user's request for a specific patent identified using the information. 6. The method of claim 5,
The results of the evaluation by the experts are performed for each technical field,
The evaluation engine is constructed for each technical field,
Wherein the step of outputting the evaluation result uses an evaluation engine of a technical field corresponding to the technical field of the specific patent. The system of claim 1, wherein the evaluation engine
Wherein the correlation is calculated based on a result of evaluation by a plurality of experts in each technology field and is constructed based on an evaluation result of an expert having an excellent correlation based on the calculated correlation. Assessment Methods. A server that receives evaluation results of sample patents from an expert in advance to construct a patent evaluation engine,
A transmitting and receiving section for mapping the one or more predefined patent evaluation items and the candidates of the correlated evaluation elements to provide the server to the computer of the expert and receiving the evaluation result for each evaluation item from the expert computer about the sample patent, Wow;
Calculating a degree of correlation between each of the evaluation items and each of the evaluation factors based on the evaluation results of the experts by the evaluation items, re-mapping each of the evaluation items and each of the evaluation factors based on the calculated degree of correlation, And a controller for building an evaluation engine by learning an evaluation result of an expert using an evaluation element mapped to the item.

Images