KR20110068278A - Method on patent rating - Google Patents

Abstract

PURPOSE: A patent automatic evaluation method of an automatic patent evaluation system is provided to supplement patent weakness in a patent portfolio of one manager by generating evaluation information according to various evaluation items about one patent. CONSTITUTION: An evaluation basis DB is generated(S21). A sample patent which is as a evaluation population which performs expert investigation is selected(S22). By enforcing expert evaluation about the sample patent, evaluation reference information is generated as an expert evaluation score(S23). By setting an expert evaluation result as a true value, evaluation model is generated through the evaluation element measuring value of the sample patent(S24). The generated estimation model is verified(S25). The verified estimation model is confirmed(S26).

Description

Patent automatic evaluation method of patent automatic evaluation system {Method on Patent Rating}

The present invention relates to a method for automatically evaluating patents in an automatic evaluation system for patents, and using an optimized evaluation algorithm, establishes an evaluation model according to various criteria, such as by technical field and property of owner, and by various evaluation items such as rights, technology, and marketability. The present invention relates to a patent automatic evaluation method of an automatic patent evaluation system for generating an evaluation score and an evaluation grade of an evaluation target patent.

Patent evaluation has largely expert evaluation and automatic evaluation. Expert evaluation is the evaluation of the value of the patent to be evaluated by experts in each technical field. Expert evaluation has the advantage that the expert's individual expertise can be utilized, and qualitative value evaluation is possible, while expert subjectivity can be strongly involved in the evaluation, and it is costly and time-consuming.

In order to overcome the shortcomings of expert evaluation, a patent automatic evaluation system is being developed. Representative overseas patent automatic evaluation system includes the PatentRatings system of OceanTomo of the United States, the PatentScore system of the IPB of Japan.

OceanTomo's PatentRatings system extracts the evaluation factors that have a positive effect on the annual registration based on the annual registration (update) of patent holders, and designs the evaluation model using the extracted evaluation factors. The reason why annual registration is important is that it is reasonably judged from the various perspectives by comparing the expected benefit of maintaining with the cost required to maintain, and on the average, a valuable patent lasts longer than a less valuable one, and the value of a patent It reflects the basic assumption of having a lognormal distribution.

The system consists of six groups of evaluation elements, which are divided into six groups: Technology, Disclosure, Claim, Prosecution, Priority, and Ownership.

 Explaining each of these, Technology is a result of patent retention / abandonment rate of the similar patent group of the same technology group. The unit of description group is a unit of a cluster of USPC Class / Subclass or related technology. Disclosure refers to the length of the word count and the number of drawings included in the patent specification as described in the patent specification, and Claims is the length of the patent claim or the number of independent and dependent claims, the use of qualifiers, and the form of the claim (method, apparatus Prosecution is a value obtained through the litigation history of a patent, and it uses information on the duration of litigation, the number and type of litigation, litigation agent, law firm, and hearing officer. The scope of prior literature is related to the number of backward citations, the type of prior literature, the timing of prior literature, and the number of field of searches by the examiner. Ownership is the information of the patent owner, whether or not it is a company, size, and domestic. It is a numerical value that can be obtained through the status of foreign right holder, etc. The information also includes. According to the disclosure, the system includes: Number of Independent Claims, Number of Dependent Claims, Average Length of Independent Claims, Shortest Independent Claim, Type of Claims (Meth / Sys / App / Comp) (Type of Claim), Patent Class / Subclass, Patent Pendency Period, Scope and Content of Cited Prior Art Scope and content), Relative Earliness of Priority Date, Forward Citation Rate, File History Details, Number of Related US It is understood that Patents (number of related US patents) and the like are used as evaluation elements.

Based on the most capitalist view of maintenance costs and the benefits of maintenance and disposal, the system derives the evaluation factors that affect the maintenance rate and incorporates them into the evaluation model. As a result, it directly helps decision-making regarding patent retention and retirement, includes the expected life expectancy and its probability information in the results of patent evaluation, and the percentiles by total / technical field / authority in the evaluation factor. It provides information. In addition, it has the advantage of high reliability of related patents because it replaces text mining by using abundant citation information, and provides a relevancy score that predicts the possibility of infringement of other rights by investigating whether the patent technology is directly or indirectly cited. It can be used to predict risks on patents, find licensing partners, and activate technology transactions. However, in patent evaluation, there is a problem that the view of retention rate is excessively acting, and the US patent environment (quotation information) is specialized in the US patent environment (abundance of citation information, high economic view of retention rate, etc.). In terms of abundance, high economic share of retention rates, etc.).

The Japanese IPB's PatentScore system evaluates the historical information as an important evaluation factor on the assumption that the historical information related to the actions of applicants, third parties and examiners over the entire life cycle of the patent has a great influence on estimating the value of the patent. Reflected. Accordingly, there is a feature in which not only the information on the specification but also an evaluation element based on such historical information is introduced into the patent evaluation. Furthermore, since there is a lot of accumulated historical information in the old patents, the necessity to aggregate and adjust by the filing year is considered, and since there is a difference in the technical fields, the necessity of aggregation and adjustment by the technical field is also considered.

This system derives various evaluation factors from the historical information that records various actions of three subjects surrounding patents such as applicant's actions, examiner's actions, and third party's actions, and similar patents have text mining around summary and claims. Derived using the technique. According to the published data, it is suggested that each of the derived evaluation factors shows a similar pattern to the annual registration rate, and a recent patent application issued by IPB (WO20080054001) explains the relationship between the annual registration rate and the evaluation factor. Although the assessment factors are not extracted based on the annual registration rate, it is recognized that the evaluation factors are closely related to the annual registration rate.

The evaluation elements of this system are largely divided into two axes: the office action series and the bibliographic / specific series. This system is the information on the applicant's behavior, and the actions performed by the applicant include: overseas application, request for examination, presence of exception proof of loss of novelty, domestic priority claim, divisional application, appeal judgment, and the existence of transposition, For example, there is an early review request, PCT application, etc., and the examiner's OA, such as the examination, the documents cited at the time of judgment, the number of times citations to evidence of the request for judgment, etc. Examples of historical information include actions that can be taken by third parties, particularly competitors, in patent law procedures such as reading, objection, invalid trial request, presence or absence of citations, etc. as a competitor's progress information. In addition, the time elapsed information considers "the elapsed days from an application, the elapsed days from a request for examination, the elapsed days from a registration date", etc., and the number of claims, independent claims, length of a specification, claims of a specification-related evaluation factor. The length, the number of drawings, the number of inventors, the number of applicants, the number of types of IPC, the number of domestic preferences, the average number of claims, and the like.

While this system has the advantage of an evaluation model that reflects dynamism with rights holders, reviewers, and competitors surrounding a patent, the view of historical information is excessively overwhelming in patent evaluation. There is.

These systems are 1) optimized for each country's patent environment, which makes it difficult to apply them directly to a patent evaluation system that conforms to the Korean patent environment, 2) an evaluation element that reflects the structural features of the patent specification, and 3) the patent. There was a problem in that the evaluation factor considering the relative environment between the patents and the technically similar patents was not considered.

Accordingly, while reflecting the structural features of the patent specification, the evaluation factors considering the relative environment between technically similar patents are appropriately reflected, and for each evaluation field and the right holder for each evaluation item from various viewpoints such as rights, technology, and marketability. There is an urgent need for the development of the next-generation patent automatic evaluation system that reflects the characteristics of each property and the patent evaluation method of the system.

The technical problem to be solved by the present invention is to propose an evaluation information processing method of the automatic patent evaluation system.

In order to achieve the technical problem to be achieved by the present invention, at least one or more designated patent identification information is obtained from at least one user computer, and a predetermined evaluation model is applied to an evaluation target patent corresponding to the patent identification information. An automatic patent evaluation method of an automatic patent evaluation system for generating evaluation information, the method comprising: receiving, by the automatic computer evaluation system, evaluation target patent identification information from the user computer; Selecting an evaluation model for evaluating the patent to be evaluated from the user computer or by predetermined criteria of the automatic patent evaluation system; Generating evaluation information by applying the selected evaluation model; And transmitting the evaluation information to the user computer. The generating of the evaluation information includes generating and storing evaluation information in advance for the evaluation target patent for each evaluation model, and then storing the evaluation information in the evaluation target patent. It is carried out by any one of the first method of obtaining the evaluation information for the evaluation method and the second method of generating the evaluation information in real time for each evaluation model, the evaluation model is composed of selected registered patents while satisfying a predetermined standard Obtaining evaluation criteria scores for the registered patents included in the evaluation criteria population; Obtaining an evaluation element measurement value for at least two evaluation elements for the registered patent; And generating a first evaluation model tree from the evaluation element measurement values, and repeatedly generating an i + 1 evaluation model tree based on a difference between the predicted value and the actual value of the i-th evaluation model tree, and the generated evaluation model tree. Comprising a weighted average of to complete the evaluation model; i is generated through, i is an integer greater than or equal to 1, the predicted value is the value of evaluating the registered patent with the i-th evaluation model tree, the actual The value suggests a patent evaluation method of the automatic patent evaluation system, characterized in that the reference score for the registered patent.

Preferably, the evaluation model is generated by applying any one or more of at least two technical fields and at least two rights holder attributes to which predetermined criteria are applied.

The evaluation model is generated for each evaluation item, and the evaluation item is preferably one or more of rights, technology, and marketability.

The evaluation element may be generated by clustering and analyzing key keywords extracted from patent contents included in a patent specification, and may obtain similar patent group information including at least one similar patent for the patent, and include the similar patent group. And at least one or more similar patent group based evaluation elements generated by processing meta information of similar patents, and evaluation element values for the similar patent group based evaluation elements are obtained from the user computer for evaluation target patent identification information. It is pre-created and saved before you receive it.

The evaluation information is a first method for generating one comprehensive evaluation score for the patent to be evaluated, 2) a second method for generating an evaluation score for each evaluation item including at least one of rights, technology, and marketability And 3) a third method of generating an evaluation score for each detailed evaluation item constituting the evaluation item.

Preferably, the evaluation information is an evaluation score and an evaluation grade, and the evaluation grade is given according to a preset rating model using the evaluation score as an input variable.

And providing similar patent group information of the evaluation target patent to the user computer, wherein the similar patent constituting the similar patent group further includes a similarity score with the evaluation target patent. desirable.

When performing the step of generating evaluation information by applying to the selected evaluation model, when there is information corresponding to the preset special event, it is preferable that the evaluation information is generated by applying a preset deduction point. Do.

Implementing the present invention has the following effects.

A large number of patents can be evaluated quickly and inexpensively, under objective evaluation criteria, so that high-quality patents can be selected and the decision can be made as to whether or not to maintain the patent registration of the patent rights.

Evaluation information can be generated for each patent by various evaluation items such as rights, technology, and marketability, thereby creating an opportunity to compensate for patent weaknesses in a patent portfolio of a right holder.

A separate evaluation model can be created for each technical field and property of an owner, and evaluation information can be generated by applying a separate specialized evaluation model, which can be used for various purposes such as comparative analysis with competitors.

The present invention can be objectively evaluated for the collective patent portfolio of the owner unit can be utilized for the qualitative evaluation of the patent of the owner unit.

Similar patent groups are provided that are technically similar to the patents to be evaluated, so that a large number of patents that are technically similar to the patents to be evaluated can be effectively reviewed. By analyzing similar patent groups, patent trends of competitors and new entrants can be identified. do.

EMBODIMENT OF THE INVENTION Hereinafter, this invention is demonstrated in detail, referring drawings.

1 is an exemplary configuration of a patent automatic evaluation system 1000 of the present invention.

In the present invention, the patent automatic evaluation system 1000 and the user computer 2000 are connected to the wired / wireless network 4000. The automatic patent evaluation system 1000 may optionally be connected to at least one external data source group 3000. The external data source group 3000 includes a first country national institution system 3100 such as the Korea Patent Office and the like, an n country national institution system 3200 such as the EPO, a public institution system 3300 having publicity such as the Korea Patent Information Service, etc. ), Public institution system (3300) or private institution system (3400) for providing market information, including corporate information, these are to provide data to the patent automatic evaluation system (1000) via wired and wireless network (4000) Can be. The user computer 2000 is a comprehensive computer including a personal computer, a mobile computer, as well as a server or system using the patent automatic evaluation system 1000 as a user.

The patent automatic evaluation system 1000 may include a specification analysis engine 1100 for analyzing and processing a specification, an evaluation element processing unit 1200 for processing an evaluation element, and an evaluation engine 1300 for generating, managing, and evaluating evaluation models. At least one of a reporting processing unit 1400 for providing an evaluation result, a DB unit 1500 for storing various data, a user interface unit 1600, and a system manager function unit 1700 for an administrator for managing a system. It may include. Then, separate them one by one.

The automatic patent evaluation system 1000 includes a specification analysis engine 1100 that analyzes a patent specification, extracts key technology key keywords from the patent specification, and extracts evaluation elements through pattern processing. This will be described in detail. As illustrated in FIG. 2, the specification analysis engine 1100 includes a morpheme analyzer 1111, a terminology recognizer 1112 for recognizing a technical term, a key keyword processor for extracting a technical term, and processing a weighting for each field. A core keyword extractor 1110 including a 1113, a similar patent group generator 1120 including a cluster analyzer 1121 for performing cluster analysis, a synonym / thesaurus dictionary 1131, a morpheme dictionary 1132, There is a dictionary 1130 that includes a terminology dictionary 1133, a user dictionary 1134, a post-processing rule 1135, and the like, and includes a pattern / rule-based evaluation element extractor 1141 and a claim-related evaluation element extractor 1142. There is an evaluation element extractor 1140 comprising a. The key keyword processor 1113 may include a technical term extractor 1113-1 and a field weight application unit 1113-2 for applying different weights for each field in which the key keywords appear. The data processed by the specification analysis engine 1100 is stored in the specification analysis result DB 1510. How the specification analysis engine 1100 processes information is illustrated by way of example in FIG. 4.

The specification analysis engine 1100 obtains a specification (S11), processes the specification data for each field (S12), extracts a preliminary keyword for each field (S13), and the stemmer 1111 analyzes the patent specification in natural language. During processing, preliminary keywords are selected for each field on the patent specification and stored (S14).

For example, as follows. When the sentence "the present invention relates to a visitor notification service system through the network 4000 system for listing the image information and identity information for the visitors of the apartment house and transmits to the resident terminal." 1111 performs morpheme analysis and compound noun analysis to generate the following preliminary keywords.

"Public housing", "visitor", "public housing visitor", "image", "information", "image information", "identity information", "listing", "resident", "terminal", "resident terminal" , "Transfer", "network (4000)", "system", "network (4000) system", "visitor", "service", "system", "service system"

Subsequently, the core keyword extractor 1110 of the present invention extracts preliminary air pair information. Air pair information refers to information about key keywords that appear together but are spaced apart. In the above example sentence, the following preliminary air pairs can be extracted.

"Community Visitor | Image Information", "Community Visitor | Identity Information", "Identity Information | Image Information", "Listing | Image Information", "Listing | Identity Information", "Resident Terminal | Identity Information", "Resident Terminal | Listing", "Listing | Send", "Resident Terminal | Send", "Resident Terminal | Network (4000) System", "Network (4000) System | Transfer", "Visitor | Send", " Network (4000) System | Visitor "," Network (4000) System | Service System "," Visitor | Service System "

Subsequently, the key keyword extractor 1110 generates a term frequency (TF) for the preliminary keyword and the preliminary air pair. For example, there are one resident, one resident terminal | network 4000 system, and one visitor two times.

The following table shows an example of information on the frequency and weight of the preliminary keyword for each patent specification field of each preliminary keyword for the patent application No. 1019900000891.

Application number IPC Filing date Key keywords F1 F2 F3 F4 TF weight 1019900000891 A61K7 / 42 19900125 External preparation One 0 One 0 2 3 1019900000891 A61K7 / 42 19900125 Kojisan 0 0 2 0 2 3 1019900000891 A61K7 / 42 19900125 Desaturation 0 0 2 0 2 2 1019900000891 A61K7 / 42 19900125 derivative 0 0 2 0 2 2

F1: Title F2: Claim F3: Prior art F4: Rest of patent specification, TF: Frequency of term

The preliminary keyword included in the preliminary keyword set may be further weighted, and the preliminary keyword and the weight information are stored in the specification analysis result DB 1510. The key keyword extractor 1110 included in the specification analysis engine 1100 applies a key keyword extraction algorithm to the preliminary keyword and / or the preliminary air pairs (S15) to extract the preliminary keyword and / or the preliminary air pairs. It generates and stores a key keyword and / or a core air pair that can represent the technical content of the patent specification (S16). The key keyword extraction algorithm may use various methods proposed in the field of natural language processing technology. In the present invention, Equation 1 below uses one such algorithm. In Equation 1 below, a field weight Fwi specialized in the patent specification is applied.

tf is the term frequency where the key keywords (indexes) appear in the current document

N : the total number of documents

df : the number of documents in which a key keyword appears in document frequency

slope : slope (arbitrary value, adjustable)

ut : unique terms for the entire document set

pivot : average length of literature (ut / N)

uf : ut of the document

Fwi: weight by field

The key keyword extraction algorithm illustrated above is extracted in consideration of the frequency of a term extracted from one patent specification and the frequency of a document including the term. In the patent specification, since the name of the invention, the abstract is particularly high in weight, and the claims can be given more weight than the detailed description of the invention, the key keywords extracted from these high-weight fields may have a special weight.

At this time, it will be apparent that the core air pair may be generated by applying the above algorithm to the preliminary air pair. Hereinafter, since the core air pairs can also be treated as the same as the core keywords, the core keywords below include the core air pairs in addition to the core keywords of the consultation.

Meanwhile, the evaluation element extractor of the specification analysis engine 1100 may generate the evaluation base DB 1520 by pattern matching or rule-based information processing. For example, if a pattern corresponding to the prior patent document number is found in the description of the prior art information in the background art on the patent specification, it can be obtained, and the independent claim / dependent claim of each claim also in the claims. , The depth of the dependent term (the dependent term directly quoting the independent term is depth 2, the depth of the dependent term quoting the dependent term depth n is n + 1), the length of the claim (the claim term, in particular the number of words constituting the independent term, etc.) ), Information about the length of the specification, etc. may be generated to generate specification analysis result data. When the evaluation element value is generated by using the data included in the evaluation basic DB 1520 including the specification analysis result data, the generated evaluation element value is stored in the evaluation element value DB 1530 of the present invention.

The evaluation element processor 1200 of the present invention includes an evaluation element value generator 1210. The evaluation element value generation unit 1210 refers to a generation rule corresponding to each evaluation element, and refers to data for each data source included in the evaluation basic DB 1520 as a result of language processing of the specification analysis engine 1100. To generate an evaluation element value corresponding to each evaluation element. For example, the bibliographic data includes information about the number of claims, the number of applicants, the inventor's nationality, and the like for each patent. ), An evaluation element value (for example, five claims, two co-applicants, three inventors' nationalities, etc.) is generated. There may be many different types of evaluation elements. In this invention, it demonstrates centering around the evaluation element peculiar to this invention.

When there is a patent to be evaluated, the patent is subjected to natural language processing (primarily using clustering based on key keywords) to provide a similar patent group composed of patents that are technically similar to the patent to be evaluated. And, providing various analysis information for the similar patent group also exists in the US OceanTomo system or Japan IPB system.

The present invention proposes a technique for processing the similar patent group information to derive an evaluation element based on the similar patent group information, and use the derived evaluation element in the establishment of an evaluation model and automatic patent evaluation. Generating an evaluation element value based on the similar patent group is processed by the evaluation element value generator 1210 of the present invention.

Similar patents constituting the similar patent group may have various meta information as in general patents. The meta information of the similar patent may include bibliographic information, historical information, family information, citation information, citation information, key keyword information extracted from the similar patent, and market information related to the applicant of the similar patent. Can be. The meta information may be included in the patent specification, but may be obtained and processed from an external data source (particularly, market information). External data sources can include DocDB. The market information may be, for example, the applicant type information of the applicant, applicant nationality information, applicant size information, accounting information including financial information, stock price information, handling item information, and affiliation industry information.

Hereinafter, an evaluation element based on a similar patent group will be described.

There is an average turnover of relevant market participants. The average sales volume of the relevant market participants is calculated by an exemplary formula as follows for the right holder (mainly a company) having financial information in a similar patent group. This evaluation factor has a positive effect on patent evaluation (particularly, marketability evaluation of patents) as the number of firms in a similar patent group and the firm's sales scale are large. In general, in terms of marketability of patent technology, patents of corporations (large and foreign companies) tend to be more marketable than patents of universities, research institutes, and individuals, and this trend should be reflected as an evaluation factor.

1) Average sales volume of relevant market participants = sum (Revenue of company i belonging to similar patent group / Number of registered patents of company i * Number of patents in similar patent group of company i)

The above formula is illustrative and the average sales volume of the relevant market participants may also be calculated as follows.

2) Average sales volume of relevant market participants = sum (Revenue of company i belonging to similar patent group * Number of patents in similar patent group of company i) / Number of similar patent group

On the other hand, the average sales increase rate, average profit rate, etc. of the right holder (company) constituting the similar patent group may also be an evaluation factor using the similar patent group information of the present invention.

On the other hand, if the proportion of companies among the rights holders belonging to the similar patent group is higher than that of universities, researches, or individuals, or if the proportion of large companies is higher than that of small and medium-sized companies, the marketability of the technology in the technical field that characterizes the similar patent group is greater. Can be good. Therefore, the ratio of the owner attribute constituting the similar patent group may also be an important evaluation factor. For example, the following evaluation factors could be introduced.

3) Ratio of Large Company Patents Constituting Similar Patent Groups = Number of Large Company Patents in Similar Patent Groups / Total Number of Corporate Patents in Similar Patent Groups * 100

4) Ratio of large company patents in similar patent group = Number of large company patents in similar patent group / total number of similar patent group

5) Proportion of company patents in similar patent group = Number of large company patents in similar patent group / total number of similar patent group

On the other hand, when a large number of foreign companies are included in the similar patent group, if the average technology gap with foreign companies or other market leadership can be expected, the ratio of foreign companies in the similar patent group is also a good evaluation factor. Can be The following is an example.

6) Percentage of foreign companies in the similar patent group = Number of foreign company patents in the similar patent group / total number of company patents in the similar patent group * 100

7) Percentage of Foreign Companies in Similar Patent Group = Number of Foreign Enterprise Patents in Similar Patent Group / Total Number of Patents in Similar Patent Group * 100

8) Percentage of foreign companies in the similar patent group = Number of foreign company patents in the similar patent group / Number of company patents in a specific country (for example, Korea) in the similar patent group * 100

In this way, various evaluation factors are utilized by utilizing various meta-information (right holder nationality, property of right holder (company, university, individual, research institute, etc.), sales size, profit rate, sales growth rate, etc.) included in similar patents that make up similar patent group. Can be envisioned. The above is merely an example. As illustrated for each evaluation element, a formula defining an evaluation element may correspond, and when processing information on each component constituting the formula (mainly, the basic data for processing such information in the evaluation basic DB 1520). The evaluation element values for each evaluation element can be generated. On the other hand, it is a matter of course that the above-described evaluation element may be configured in a complex. For example, the following equation can be considered.

9) Proportion of high commercial right holders in similar patent groups = (Number of large company patents in patent group / Number of total company patents in similar patent group * 100) * (Number of foreign company patents in similar patent group / All company patents in similar patent group Number * 100)

On the other hand, the various ratios exemplified above may be an evaluation factor of similar patent group utilization that is also important to increase or decrease the ratio. For example, if the proportion of large companies with similar patent groups is increasing over time, the activity of patents (applications or registrations) of large companies in this technology field is active, which usually has a positive impact on the marketability of this technology field. Gives. For example, the rate of increase or decrease of the rate may be obtained by calculating a rate in units of years and calculating the degree of increase or decrease of the rate for each year.

On the other hand, companies generally tend to apply for a large number of research results with commercialization in mind, and universities tend to apply for leading and pioneering research results independently of commercial performance. If the proportion of companies is very low and the proportion of universities is quite high, the technology belonging to the similar patent group is likely to be the technology of the introduction stage. Therefore, it is also possible to estimate whether the evaluation target patent technology is likely to belong to one of the technology change cycles such as the introduction phase, the growth phase, the mature phase, and the decline phase, based on the property of the right holder included in the similar patent group of the evaluation target patent.

The advantage of introducing an evaluation element based on such a similar patent group is to look at the collective properties of similar patent group constituent patents that are technically similar to the patent to be evaluated, thereby estimating properties that are difficult to estimate in the evaluation target patent itself. You can do it.

Next, an evaluation model generation method for patent evaluation of the present invention will be described in more detail with reference to FIG. 5. In order to generate an evaluation model, for patent sample selection and other various purposes, 1) an evaluation basic DB 1520 is generated (S21), 2) a sample patent as an evaluation population to perform expert investigation (S22), 3 ) Perform evaluation of experts on sample patents to generate evaluation criteria information with scores evaluated by experts (S33), and 4) create evaluation models from evaluation factor measurement values of sample patents with expert evaluation results as true values (S33). S24), 5) verify the generated evaluation model (S25), and 6) confirm the evaluation model (S26).

When selecting sample patents, selecting too many samples increases the cost of expert evaluation, so it is necessary to select the appropriate number of samples. For this purpose, a sampling method of statistics can be used, and a stratified sampling method is commonly used. For the stratification extraction method, evaluation element measurement value information of each patent is required for each preliminary evaluation element, which should be based on the generation of the evaluation base DB 1520. Evaluation Factor Select the appropriate number of samples to reflect various evaluation factors by referring to the measured value information. Expert evaluation is a detailed evaluation item that constitutes a score or evaluation item for each evaluation item (for example, rights, technicality, and marketability) of each sample patent to each expert, such as experts in each technical field and market evaluation experts, for the sample patent ( For example, scores by sub-assessment of the above evaluation items, such as strength of rights, fidelity of rights, strength of patent network formation, and / or tax assessment items (eg, under the strength of rights). You can ask for more detailed scores. Although one expert may evaluate one patent, it is preferable that it is 2-3 people (3 are more preferable). As a result of the expert evaluation, one or more of the total score, the score for each evaluation item, the score for the detailed evaluation item, and the score for the detailed evaluation item are generated for one sample patent, which constitutes the evaluation criteria information.

Next, an evaluation model is generated. Evaluation model generation is one of the most important elements of the present invention. The generation of an evaluation model may also perform the simplest regression analysis (polynomial regression analysis with the true value (the expert evaluation score for the sample patent) set to Y value and the value of each evaluation factor as an independent variable), but in this case However, there is a problem such as multilinearity between the evaluation elements, and there is a disadvantage in that the problem when each evaluation element has a nonlinear characteristic cannot be solved. Thus, the present invention does not exclude polynomial regression analysis, but applies advanced statistical analysis techniques. This will be described in more detail with reference to FIG. 6.

First, a first optimal evaluation model tree (classifier) f ₁ (X) is generated using the evaluation factor measurement value of the evaluation factor X and an evaluation criterion score Y evaluated by an expert (S31). F ₁ (X) may be in the following form.

f ₁ (X) = {m ₁₁ * I ₁ (X ₁ ) + m ₁₂ * I ₂ (X ₁ ) + m _1i * I _i * (X ₁ )} +. . . . (m _j1 * I ₁ (X _j ) + m _j2 * I ₂ (X _j ) + m _ji * I _i * (X _j ). . . to be.

I is an indicator function that has a value of 0 or 1 depending on the condition.

mji refers to the average of the evaluation scores of the patents when the i-th indicator function is applied with respect to the evaluation factor j. The indicator function may vary depending on each evaluation factor, but may have the same value as 2 or 3. i corresponds to the number of branches in the tree.

The above equation will be described with a simple example. For example, if the number of evaluation elements is one in the number of claims, j = 1, and if there are two branches in the tree, the maximum value of i is 2. For example, if X ₁ is the number of claims and the number of claims is optimally diverged at 10, the above equation may be expressed as follows.

f ₁ (X) = m ₁₁ * I ₁ (number of claims = <10) + m ₁₂ * I ₁ (number of claims> 10)

I ₁ (claim number = <10) is the case where the number of claims is 10 or less, m ₁₁ is the average of the evaluation criteria scores of patents having the number of claims 10 or less, and m ₁₂ is the average of the evaluation criteria scores of patents having more than the number of claims 10 Becomes

Finding the number of claims being a branch = 10 may branch by several branch candidates and repeatedly calculate the minimum value of the error to find the optimal branch by evaluation factor. Since this will be easy for those skilled in the art to find the optimal value, detailed descriptions are omitted.

Next, the difference value (residual) between the first evaluation model tree value generated from the true value Y and the predicted value f ₁ is obtained. A second evaluation model tree f ₂ is generated from the difference value Yf ₁ (X), and a difference value (residual) Yf ₁ (X) -f ₂ (X) is obtained. (S32) The difference value Yf ₁ (X) Create a third evaluation model tree f ₃ (X) with -f ₂ (X), and find the difference value (residual) Yf ₁ (X) -f ₂ (X) -f ₃ (X). In this way, f _i (X) is generated, and then the Mth evaluation model tree f _M (X) is generated using the values Yf ₁ (X) -f ₂ (X)-.. f _{M - 1} (X). Perform iteratively. A weighted average of f ₁ (X) to f _M (X) completes the evaluation model. On the other hand, when generating a _{f M (X), f M} -1 (X) takes a value obtained by multiplying the _{shrinkage parameter s Y-sf 1 (} X) -sf 2 (X) to the - .. to sf _M (X) It can also be processed. In this case, the weights may be the same, or may be differently given in consideration of the magnitude of the error. The following final evaluation model is an example in which the weights become equal.

f (X) = f ₁ (X) + f ₂ (X) +. . . + f _M (X)

The evaluation model generation of the present invention is equally applied not only to the evaluation model for the entire patent but also to the evaluation model for each evaluation item (rights, technicality, marketability, etc.) and the detailed evaluation items constituting subordinate items of the evaluation item. That is, the Y value may be an expert's evaluation score for the entire patent, but may be an expert's evaluation score for each evaluation item or an expert's evaluation score for each detailed evaluation item. According to the property of the Y value, the entire evaluation model, the evaluation model for each evaluation item, and the evaluation model for the detailed evaluation item are generated.

On the other hand, the present invention introduces an evaluation model for each technical field, owner attribute, and time. In order to generate an evaluation model for each technical field, sampling is performed to limit the sample itself to a specific technical field, and after the sampling process, the evaluation model for the specific technical field is completed. On the other hand, sampling for each property of the rights holder, and if the process after sampling, the evaluation model for each property of the rights owner is generated. The same applies to the time-based evaluation model. Generating several evaluation models is in charge of the evaluation model generator 1310 of the present invention. The generated evaluation model is managed by the evaluation model manager 1320.

On the other hand, once the evaluation model is established, the evaluation model can be verified. Verification is based on 1) the method of verifying the difference between the evaluation criteria score evaluated by the expert and the prediction score generated by the evaluation model for the patent to be verified, 2) generating a ranking by the score, and the difference of the ranking of the patent to be verified. Verification method, 3) create a ranking by the score, give a grade according to the ranking, and verify the difference by the grade. The patent to be verified may be a patent for expert evaluation, but may also be a separate patent for verification. On the other hand, by dividing the sample patent into a training set used for establishing an evaluation model and a test set for verification n times, and performing the verification for the patent of the test set n times with an evaluation model generated through the training set. The results of these n verifications may test the performance of the evaluation model. In this case, the shrinkage parameter may be tuned in the evaluation model.

When the evaluation model is established, the evaluation factor is reflected in the evaluation model, and when the evaluation element measurement value for each evaluation element is input, the evaluation element measurement value for each evaluation element can be processed to generate evaluation information. It demonstrates centering on FIG. The generation of the evaluation information includes: 1) receiving selection information on the evaluation target patent and the evaluation model to be used from the user computer 2000 (S41), obtaining evaluation element value data on the evaluation target patent (S42), and selecting the selection. The evaluation element value data is applied to the evaluation model that has received the information (S43), and the evaluation engine 1300 generates evaluation information including the evaluation result value (S41), and the evaluation result in the user computer 2000. Reporting the evaluation information including the value (S45). In this case, the default evaluation model provided by the patent automatic evaluation system 1000 may be selected as the selection of the evaluation model. Below, it is divided into necessary steps.

The evaluation information may be of various types according to an evaluation model such as an overall evaluation score, an evaluation score for each evaluation item, and an evaluation score for each detailed evaluation item. Since the patent evaluation information may vary depending on the evaluation model, the selection of the evaluation model may precede the generation of the patent evaluation information. The determination or selection of an evaluation model (a general evaluation model, an evaluation model for each technical field, an evaluation model for each attribute of the right holder, and a time-based evaluation model may be selected or selected) for evaluating the patent to be evaluated. The user of 1000 may do so, but the patent automatic evaluation system 1000 may do so. In the latter example, the patent automatic evaluation system 1000 may automatically designate which technical field-specific evaluation model is applied with reference to the IPC information of the patent to be evaluated (it may be appropriate to use the main IPC). In addition, it is possible to designate an appropriate evaluation model for each property holder by identifying the property of the right holder with information on the right holder of the patent to be evaluated, and a reasonable timely evaluation model may be specified by referring to the filing date or registration date of the patent to be evaluated. The evaluation model manager 1320 of the present invention may provide a user with a complete set of evaluation models managed by the user and induce selection.

Meanwhile, for special evaluation experts, the evaluation model manager 1320 may provide an interface for manually adjusting the weights of the evaluation elements constituting the evaluation model, and when the weights are input from the experts, An expert-specific evaluation model may be established by reflecting the weight, and the patent evaluation may proceed with the established evaluation model.

When the evaluation target patent and the evaluation model are determined, evaluation element measurement value information may be obtained for each evaluation element of the evaluation target patent. If the evaluation element measurement value information is input to the evaluation model for each evaluation element, the evaluation model may be obtained. Generates preset evaluation information. Generation of the evaluation information is in charge of the evaluation information generation unit 1330 of the present invention. The evaluation information may include an evaluation score and an evaluation grade. When the evaluation score is determined by the evaluation model, the evaluation grade is assigned according to the evaluation grade granting model corresponding to the evaluation score. The evaluation grade may be any grade, but it would be reasonable to be at least 5 and less than 15 grades. There are two methods for assigning an evaluation grade: 1) a score section corresponding rating section method, and 2) a ranking section corresponding rating section section method. 1) is a method of assigning a grade to a score from a few points to a few points, and 2) is an evaluation score for all patents belonging to the population based on the population to which the evaluation target patent belongs. The sequence of is determined, and then the evaluation grade corresponding to the sequence percentile interval is determined.

Next, a point provision evaluation element is demonstrated. Progress information, such as invalid judgments and informational provisions, is a scarce evaluation factor, and because of frequency problems, there is room for statistical models to be missing from the evaluation model. In this case, after generating an evaluation score by applying an evaluation model, if it is confirmed that a special event such as a special event or a license setting according to the progress information such as the invalid judgment is generated in the evaluation target patent, the special event is generated. One point to a predetermined score may be added or deducted for each kind / count and / or for each kind of result of the patent event (for example, determination of partial invalidity, etc.). In the present invention, the special event processing unit 1331 is in charge of adding or subtracting the special event. The above points or points may be granted to the entire patent evaluation score, but may also be given for each evaluation item, such as rights, technology, and marketability. Accordingly, when a special event occurs, an evaluation grade may be calculated as an evaluation score after performing a point or deduction process on the evaluation score of the evaluation model. Of course, conceptually, since the evaluation model may be a reflection of a point or a penalty according to the patent event, in this case, the evaluation grade may be calculated as the evaluation score of the evaluation model.

The generated evaluation score and evaluation rating may be transmitted to the user computer 2000. The evaluation information may be delivered in a web page or a report (eg, a report in pdf format).

On the other hand, the automatic patent evaluation system 1000 of the present invention can provide similar patent group information for the evaluation target patent. When providing the similar patent group information, the similarity scores / information of the evaluation target patent and the similar patent included in the similar patent group may be further provided. The similarity score / information is information on how similarity or relevance of the patent to be evaluated with the similar patent in terms of key keywords is generated during cluster analysis, which is common in natural language processing, and thus detailed description is omitted. do.

Data processed in the patent automatic evaluation system 1000 is stored in the DB unit 1500 of the present invention. The DB unit 1500 includes bibliographic data 1511, historical information data 1512, market information data 1513, DocDB data 1514, and hit data 1515 containing the number of inquiries of a patent specification or a registration register. An evaluation base DB 1510 is included. Meanwhile, the DB unit 1500 includes a specification analysis result DB 1520 related to specification analysis including a specification analysis result, which includes index terms 1521 for each field, frequency data 1522 for each index term, and key keyword accumulation information for each period. 1523, technical terminology information 1524, associated air pair information 1525, evaluation element information 1526, similar patent group data 1527, and extracted key keyword data 1528 may be included. . The DB unit 1500 may further include an evaluation element value DB 1530 including evaluation element value data 1531 which is an evaluation element value for each evaluation element of each patent. Meanwhile, the DB unit 1500 includes various evaluation models of the present invention (by technical field, by owner attribute, etc.), which are stored in the evaluation model DB 1540. Meanwhile, the information generated by the user of the automatic patent evaluation system 1000 is included in the user DB 1550, which includes user information data 1551, which is information about a user as a user, and data managed by the user. There may be user management data 1552.

The automatic patent evaluation system 1000 may further include a user interface 1600 that receives patent identification information about an evaluation target patent from a user and provides evaluation information. The user interface 1600 may include an input UI for receiving the patent identification information and an output UI for providing evaluation information. FIGS. 8 and 9 respectively illustrate the input UI and the output UI. One embodiment of the is shown.

As shown in FIG. 8, the input UI may further include any one or more of a number input UI 810, an applicant input UI 820, and a file input UI 830. 9, at least one of a bibliographic output UI 910, an evaluation information output UI 920, an evaluation item-specific evaluation information output UI 930, and a report output UI 940 may be included in the output UI. It may further include.

The present invention can be utilized in the patent industry, evaluation industry, and intellectual property consulting industry.

1 is a view of an embodiment configuration for a patent automatic evaluation system of the present invention,

2 is a diagram of an exemplary configuration of a specification analysis engine of the present invention;

3 is a view of an exemplary configuration for a DB unit of the present invention,

4 is a diagram of an embodiment of an information processing method of a specification analysis engine according to the present invention;

5 is a view of a flow for generating an evaluation model in the patent automatic evaluation system of the present invention,

6 is a diagram illustrating an embodiment related to generation of an evaluation algorithm of the evaluation model of the present invention.

7 is a diagram illustrating an example of a flow in which the patent automatic evaluation system of the present invention generates evaluation information.

8 is a diagram illustrating an exemplary configuration of an input UI of the patent automatic evaluation system of the present invention.

9 is a diagram illustrating an exemplary configuration of an output UI of the patent automatic evaluation system of the present invention.

Claims (8)

Patent automatic of the automatic patent evaluation system that obtains at least one or more designated patent identification information from at least one user computer, and generates predetermined evaluation information by applying a predetermined evaluation model to the evaluation target patent corresponding to the patent identification information In the evaluation method, the patent automatic evaluation system Receiving evaluation target patent identification information from the user computer; Selecting an evaluation model for evaluating the patent to be evaluated from the user computer or by predetermined criteria of the automatic patent evaluation system; Generating evaluation information by applying the selected evaluation model; And Transmitting the evaluation information to the user computer; Generating the evaluation information Among the first method of generating and storing evaluation information on the evaluation target patent for each evaluation model in advance, and the second method of generating evaluation information in real time for each evaluation model. Is carried out in either way, The evaluation model is Obtaining evaluation criteria scores for the registered patents included in the evaluation criteria group composed of selected registered patents while satisfying a predetermined standard; Obtaining an evaluation element measurement value for at least two evaluation elements for the registered patent; And A first evaluation model tree is generated using the evaluation factor measurement values, the i + 1 evaluation model tree is repeatedly generated by the difference between the predicted value and the actual value of the i-th evaluation model tree, and the generated evaluation model tree is generated. Comprising a weighted average to complete the evaluation model; will be generated through, i is an integer greater than or equal to 1, And wherein the predicted value is a value obtained by evaluating the registered patent with the i-th evaluation model tree, and the actual value is a reference score for the registered patent. The method of claim 1, The evaluation model is a method for processing patent evaluation information of an automatic patent evaluation system, characterized in that generated by applying any one or more of at least two or more technical fields, at least two or more rights holder attributes, and timely attributes by applying a predetermined standard. . The method of claim 1, The evaluation model is generated for each evaluation item, the evaluation item is patent evaluation information processing method of the automatic patent evaluation system, characterized in that any one or more of rights, technology and marketability. The method of claim 1, The evaluation element includes at least one similar patent group based evaluation element, The similar patent group based evaluation element is generated by obtaining similar patent group information including at least one similar patent for the evaluation target patent, and processing meta information of the similar patent included in the similar patent group, The similar patent group is generated by clustering analysis of key keywords extracted from patent contents included in the patent specification. And an evaluation element value for the similar patent group-based evaluation element is previously generated and stored before receiving evaluation target patent identification information from the user computer. The method of claim 1, The evaluation information is a first method for generating one comprehensive evaluation score for the patent to be evaluated, 2) a second method for generating an evaluation score for each evaluation item including at least one of rights, technology, and marketability And 3) a third method of generating an evaluation score for each detailed evaluation item constituting the evaluation item, using one or more methods of generating patent evaluation information. The method of claim 1, The evaluation information is to be the evaluation score and evaluation grade, The evaluation grade is a patent evaluation information processing method of the automatic patent evaluation system, characterized in that given as the input variable in accordance with a predetermined rating model. 5. The method of claim 4, Providing similar patent group information of the evaluation target patent to the user computer; The similar patent group information further includes a similarity score between the evaluation target patent and the similar patent, The similarity score is calculated by the relationship between the key keywords extracted from the patent specification of the similar patent and the key keywords extracted from the patent specification of the evaluation target patent, characterized in that the patent evaluation information processing method of the automatic patent evaluation system. The method of claim 1, When the evaluation information is generated by applying to the selected evaluation model, when the evaluation target patent has information corresponding to a predetermined special event, the evaluation information is generated by applying a preset deduction point. Patent evaluation information processing method of patent automatic evaluation system.

Images