CN108694462B

CN108694462B - Patent retrieval result sorting method and computer-readable storage medium

Info

Publication number: CN108694462B
Application number: CN201810296511.0A
Authority: CN
Inventors: 郭永红; 何佳; 陈伟然; 辛莹; 王希桢; 姜庭欣; 杨冠梅
Original assignee: Beijing Incopat Co ltd
Current assignee: Beijing Incopat Co ltd
Priority date: 2018-03-30
Filing date: 2018-03-30
Publication date: 2022-03-15
Anticipated expiration: 2038-03-30
Also published as: CN108694462A

Abstract

A patent retrieval result sorting method and a computer-readable storage medium, the patent retrieval result sorting method comprising: obtaining multi-dimensional parameters of a plurality of patent files; a patent value degree evaluation model is adopted, and the multidimensional parameters are summed and converted into a single patent value degree according to respective weights; and sequencing the plurality of patent files according to the patent value degree, wherein the weight of each multidimensional parameter is a calculated value obtained by operation according to the result of big data analysis of a patent database, and the patent value degree evaluation model adopts the multidimensional parameter as an evaluation index and regularly carries out adaptive adjustment according to a dynamic index acquired in real time in the patent database. Objective and accurate evaluation models can be generated in a targeted manner to evaluate the value degree of the patent, and the accuracy of evaluation of the value degree of the patent is improved.

Description

Patent retrieval result sorting method and computer-readable storage medium

Technical Field

The present disclosure relates to the field of patent analysis technologies, and in particular, to a method for sorting patent search results and a computer-readable storage medium.

Background

Chinese researchers have conducted a great deal of research on evaluation indexes and models of Chinese high-value patents, and the basic idea is to use three major factors that affect the patent value: starting from technology, market and law, an index system for evaluating patent value is constructed. The evaluation model calculation method mainly adopts an expert scoring method, a hierarchical method, a decision tree method, a fuzzy comprehensive evaluation method and the like. And constructing a patent evaluation index system from the technical, market and right angles by using the pancakes and the vermilion, wherein the system comprises 17 evaluation indexes, and an evaluation model is established by using a hierarchical method and a fuzzy comprehensive theory. In 2012, patent value analysis indexes were established from the perspective of patent legal value, technical value and economic value by the "patent value analysis index system operation manual" jointly published by the national intellectual property office and the chinese technical exchange, and included 18 evaluation indexes in total. However, the determination of the value-affecting index and its importance in the above patent value evaluation system mainly comes from the logical guess of the degree of importance of the index by experts without examination, and its subjective factor has a large influence. Therefore, a more targeted high-value patent evaluation system is needed to be established.

Disclosure of Invention

In view of this, the present disclosure provides a patent value evaluation method and a computer-readable storage medium, which can generate an objective and accurate evaluation model to evaluate the value of a patent in a targeted manner, and improve the accuracy of evaluation of the value of the patent.

The disclosure provides a patent value degree evaluation method, which is characterized by comprising the following steps: constructing a patent value degree evaluation model; evaluating the target patent according to the constructed patent value degree evaluation model; and optimizing a patent value degree evaluation model according to the evaluation result of the target patent, wherein the construction of the patent value degree evaluation model comprises the following steps: selecting parameters influencing the patent value degree; selecting a patent data sample according to a preset patent value evaluation standard; and establishing an evaluation formula reflecting the relationship between the selected parameters and the patent value degree based on the selected patent data samples.

Preferably, the constructed patent value degree evaluation model includes a first model and a second model different from the first model, and the optimizing the patent value degree evaluation model according to the evaluation result of the target patent includes: taking the target patent evaluated as having the value degree in the expected range by the first model and/or the second model as a reference patent; and updating a patent data sample used to construct the first model and/or the second model with the reference patent.

Preferably, the constructed patent value degree evaluation model comprises a first model, and the optimizing the patent value degree evaluation model according to the evaluation result of the target patent comprises: taking the target patent which is evaluated as having the value degree in the expected range by the first model as a reference patent; and updating a patent data sample used to construct the first model with the reference patent.

Preferably, the constructed patent value degree evaluation model includes a first model and a second model different from the first model, and the patent value degree evaluation method further includes: evaluating the preselected verification data by using a first model and a second model respectively to obtain a first evaluation result and a second evaluation result; comparing the first evaluation result with the second evaluation result; the first model and/or the second model is adjusted based on the comparison result.

Preferably, the constructed patent value degree evaluation model includes a first model, and the patent value degree evaluation method further includes: evaluating the preselected verification data using a first model to obtain a first evaluation result; evaluating a patent data sample used for constructing the first model to obtain a second evaluation result; comparing the first evaluation result with the second evaluation result; the first model is adjusted based on the comparison.

Preferably, the patent value degree evaluation method further includes: after the target patent is evaluated according to the constructed patent value degree evaluation model, the evaluation result is subjected to normalization processing to be converted into an evaluation score value within a preset range.

Preferably, there are a plurality of selected parameters, and the establishing an evaluation formula representing a relationship between the selected parameters and the patent value degree based on the selected patent data samples includes: aiming at each selected parameter, establishing a sub-evaluation formula reflecting the relationship between the parameter and the patent value degree based on the selected patent data sample; and taking the weighted sum of the sub-evaluation formulas as an evaluation formula for the selected overall parameters.

Preferably, the selected parameters are divided into a plurality of groups, and the establishing of the evaluation formula reflecting the relationship between the selected parameters and the patent value degree based on the selected patent data samples includes: aiming at each selected group of parameters, respectively establishing a sub-evaluation formula reflecting the relationship between the group of parameters and the patent value degree based on the selected patent data samples; and taking the weighted sum of the sub-evaluation formulas as an evaluation formula for the selected overall parameters.

Preferably, the patent value degree evaluation method further includes: and if the target patent meets the preset condition, increasing or decreasing the weight of the specified sub-evaluation formula when evaluating the target patent according to the constructed patent value degree evaluation model.

Preferably, the selecting a patent data sample according to a preset patent value evaluation criterion includes: selecting a first group of patent data samples and a second group of patent data samples according to a preset patent value evaluation standard, wherein establishing an evaluation formula reflecting the relationship between the selected parameters and the patent value based on the selected patent data samples comprises: calculating the distribution of the selected parameters in the first group of patent data samples; calculating the distribution of the selected parameters in the second group of patent data samples; calculating a comparison result between the distribution of the selected parameter in the first set of patent data samples and the distribution of the selected parameter in the second set of patent data samples; and fitting a relational expression between the selected parameter and the comparison result based on the comparison result to serve as an evaluation formula for embodying the relationship between the selected parameter and the patent value degree.

Preferably, the establishing an evaluation formula reflecting the relationship between the selected parameter and the patent value degree based on the selected patent data sample includes: extracting a patent corresponding to each value of the parameter; evaluating the patent value degree of the patent corresponding to each value, and setting the weight of the influence of the value of the parameter on the patent value degree based on the evaluation result; and obtaining an evaluation formula reflecting the relationship between the parameters and the patent value degree based on the weight of the influence of each value of the parameters on the patent value degree.

Preferably, the target patent comprises a patent group, and the evaluating the target patent according to the constructed patent worth evaluation model comprises: evaluating each patent in the patent group according to the constructed patent value degree evaluation model; and performing mathematical operation on the evaluation results of all patents in the patent group to obtain the evaluation results aiming at the patent group.

According to another aspect of the present disclosure, there is also provided a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the above-described patent value degree evaluation method.

Drawings

To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below, and it is apparent that the drawings in the following description only relate to some embodiments of the present disclosure and do not limit the present disclosure.

Fig. 1 shows a schematic flow diagram of a method of constructing a patent merit evaluation model according to an embodiment of the present disclosure.

Fig. 2A shows a schematic diagram of probabilities of patents of different claim numbers appearing in the first patent data sample in chinese patents according to an embodiment of the present disclosure.

Fig. 2B shows a schematic diagram of the probability of patents with different claim numbers appearing in the second patent data sample in chinese patents according to an embodiment of the present disclosure.

Fig. 2C shows a comparison of the probability of the occurrence of patents of different claim numbers in the first and second patent data samples in chinese patents according to an embodiment of the present disclosure.

Fig. 3A illustrates the probability of different claim numbers of patents appearing in the first patent data sample in a U.S. patent according to an embodiment of the disclosure.

Fig. 3B shows probabilities of different claim numbers of patents appearing in the second patent data sample in U.S. patents according to an embodiment of the disclosure.

Fig. 3C shows a comparison of probabilities of patents of different claim numbers appearing in the first and second patent data samples in U.S. patents according to an embodiment of the disclosure.

Fig. 4 shows a fitting curve of the relationship between the number distribution of the high-value patents and the common patent claims in china.

Fig. 5 shows a schematic flowchart of a patent value degree evaluation method according to an embodiment of the present disclosure.

Fig. 6 shows a schematic diagram of the comprehensive evaluation result of the patent worth degree according to the embodiment of the present disclosure.

Fig. 7A shows a value degree analysis chart of the first to eighteenth chinese patent prize winning patents by using the patent value degree evaluation method according to the embodiment of the present disclosure.

Fig. 7B shows a value degree analysis chart of the patent value degree evaluation method of the embodiment of the present disclosure for the essential patents of ETSI chinese standards.

Fig. 7C shows a presentation of the evaluation of the comprehensive value degree of a single patent and the evaluation of the patent value degrees of each dimension by the patent value degree evaluation method according to the embodiment of the present disclosure.

Fig. 8A shows a schematic diagram of a user interface for patent worth evaluation using a patent worth evaluation method according to an embodiment of the present disclosure.

Fig. 8B shows a schematic diagram of a patent layout of an applicant of a high-value patent screened by the patent merit evaluation method according to the embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described below in detail and completely with reference to the accompanying drawings of the embodiments of the present disclosure. It is to be understood that the embodiments described are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the disclosure without any inventive step, are within the scope of protection of the disclosure.

The patent value degree evaluation model is constructed by the method, the patent value degree evaluation method and the computer-readable storage medium, the influence of each parameter on the patent value is obtained by a big data analysis method without adopting an expert scoring method, so that an evaluation formula of the patent value degree is established, and the patent value degree model is obtained. On one hand, objective and accurate evaluation models can be generated in a targeted manner, and on the other hand, the accuracy of the evaluation of the patent value degree can be improved.

In step S101, parameters that affect the patent value degree are selected.

Table 1 below gives examples of parameters that affect the patent value degree. Desired parameters may be selected from table 1 as needed. In order to ensure the comprehensiveness of model establishment, various factors possibly influencing the patent value degree should be found out as much as possible at the beginning of establishing a patent value degree evaluation model, wherein the factors include evaluation indexes which are commonly adopted internationally, indexes summarized according to the characteristics of Chinese patent protection conditions, direct parameters directly related to the patent itself, and indirect parameters related to the technical fields to which an applicant, an agency and the patent belong.

TABLE 1

In step S102, a patent data sample is selected according to a preset patent value evaluation criterion.

In this step, patents meeting a first preset condition may be selected as a first set of patent data samples, and patents meeting a second preset condition may be selected as a second set of patent data samples. For example, samples of known high-value patents and samples of general patents can be selected according to general criteria for patent value evaluation by people.

For example, patents meeting at least one of the following criteria may be selected as a sample of patents of known high value: 1. patents that participate in infringement litigation and obtain substantial complains; 2. maintaining long-life patents which are paid to the expiration of the patent period; 3. the patentees put the patents into operation and obtain high-value patents through modes such as implementation, permission or transfer and the like; 4. a patent for which an invalidation request is made and which is judged to remain valid; a basic patent that is incorporated into a standard or forms a de facto standard by actual application to become an industry specification; 6. patents used more than a preset number of times; 7. patents filed in multiple countries, such as patents filed in multiple countries by the international partnership project PCT, paris convention, or other means; 8. other patents have had a significant impact on the technology or market of the industry.

On the one hand, the development level of the Chinese patent litigation is far from that of the United states, and the analysis requirement cannot be met from the selectable data sample quantity and quality. Taking the maintenance period of the patent as an example, 2156 patents are kept in the period of the Chinese invention paying to the expiration of the patent period as 12 months in 2017, wherein 20% of the patents belong to the Chinese petrochemical group, so that the current patents paying to the expiration of the patent period in China mainly come from national enterprises and have small relation with specific values. On the other hand, the lack of related data also causes obstacles to the extraction of high-value patent data samples. For example, patentees can determine that the number of patents with patent application situations and operation values is not large in China, and lack sufficient effective data support. Therefore, we should start from reality and cannot move the theory when selecting the high-value patent data sample in China.

As an illustration, the embodiment uses 2305 patent inventions selected from the chinese patent database that are invalid and maintained valid as simplified high-value patent analysis samples. In contrast, chinese patents are requested to be invalid and are judged to maintain sufficient patent quantity samples, and the manpower and financial resources are spent on invalid patent lifting, which indicates that the patents are applied in the industry or have certain influence on the industry, and the value of the patents can be verified from the side.

As a common patent sample for comparison, all inventions and utility model patents of the Chinese patent database can be directly selected, and the statistical data of the patents comprehensively reflects the distribution condition of the number of claims in the patents in the whole database.

Of course, the embodiments of the present disclosure are not limited thereto, and the selection criteria of the patent data samples may be adjusted as needed, for example, all patents of a certain applicant may be selected as the common value patent data samples, and patents meeting the preset conditions may be selected from all patents of the applicant as the high or low value patent data samples. In some embodiments, the selection of the high or low value patent data samples may not be limited to the range of the common value patent data samples, and may be selected from other ranges.

In addition, the patent data sample can be a data sample of a common single patent, and can also be a data sample of a composite patent processed by a plurality of patents in the form of a single patent. For example, a family of patents typically includes multiple patents associated with each other, such as patents that are filed separately for multiple countries based on the same priority. In some cases, the characteristic parameters of each patent in the same family of patents can be integrated in a database and converted into a single patent form for easy retrieval and analysis. For example, the applicants of a single patent transformed from a family of patents may be a collection of individual applicants of the family of patents, the number of claims may be the sum or average of the number of claims of individual patents in the family, and so on. The patent data sample of the embodiment covers the situation, for example, the patent data sample can be extracted for the patent family converted into the single-piece patent form, so as to establish a patent value degree evaluation model to evaluate the value degree of the patent family. In this case, because there are differences between a single patent and a patent family in terms of parameters, weights, and relationships between the parameters, the present embodiment can construct a merit assessment model for patents of the same family on the basis of statistics and analysis of patent family data of the whole world, rather than simply accumulating the value scores of individual patents in the family or simply accumulating the corresponding parameters of all patents in the patent family and then evaluating according to the merit assessment model of the individual patents.

In step S103, a relationship map of the selected parameters and the patent value degree is established.

After the analysis samples of the high-value patents are determined, the correlation between the high-value patents and the patent value degree can be calculated one by one according to the previously selected evaluation parameter list. In this embodiment, the number of claims is selected as a parameter that affects the patent value, and this is taken as an example for explanation.

The scope of the invention or the utility model patent right is subject to the content of the claims according to the provisions of the patent laws. Indeed, the scope of each claim is literally set forth, so that it is difficult to particularly quantify it. Although the amount of claims may not directly reflect the scope of patent protection, the amount of claims may reflect the scope of patent protection and the stability of the claims to some extent. On the one hand, the patent technical scheme is more complex or the invention content is richer due to the large number of the claims. The larger the number of independent claims, the larger the angle of view of protecting the same inventive concept, or the more technical solutions derived from this inventive concept, the larger the scope of protection of the patent relatively; the greater the number of dependent claims, the greater the technical solutions derived from the respective independent claims, or the more elaborate modifications thereof, the greater the scope of protection which can be maintained as far as possible and the stability of the claims as far as possible in the case where the independent claims cannot be maintained. On the other hand, as mentioned above, according to the regulations of the chinese patent law, when the number of the claims exceeds 10, the additional fee needs to be paid when applying for the patent, and at the same time, the labor of the agent is more in the writing process, and the paid patent agent fee may be increased, so that the cost paid by the applicant when applying for the patent is relatively increased. Therefore, theoretically, the probability that a patent with a large number of claims will form a high-value patent is also higher.

To verify the above theory, the following description will be made by taking chinese patent and us patent as examples.

In this step, a comparison relation table between high-value patents and common patents can be established according to the quantity distribution of the claims. The number of patents with claim number equal to n (n is a natural number greater than or equal to 1) in the high-value patent sample and the full-base patent (note: the full-base patent adopted in the embodiment includes all the invention applications and all the utility model patents in China 25 days 10 months before 2017), and the proportion of the number of patents in the high-value patent and the normal patent respectively are calculated.

Fig. 2A shows a schematic diagram of probabilities of patents of different claim numbers appearing in a first patent data sample (a common patent) in chinese patents according to an embodiment of the present disclosure. Fig. 2B shows a schematic diagram of the probability of patents with different claim numbers in chinese patents appearing in the second patent data sample (high-value patent sample) according to an embodiment of the present disclosure. Fig. 2C shows a comparison of the probability of the occurrence of patents of different claim numbers in the first and second patent data samples in chinese patents according to an embodiment of the present disclosure.

As can be seen from fig. 2A and 2B, the icons show a trend of ascending before descending as the number of claims increases from 1, both for normal patents and for high-value patents. Only ordinary patents begin to decline when the number of claims is greater than 3, while high value patents begin to decline when the number of claims is greater than 10.

Next, the probability of the high-value patents with the number of patent claims n appearing in the high-value patents and the probability multiple of the ordinary patents with the number of patent claims n appearing in the full database are respectively calculated to obtain the relationship between the two, as shown in fig. 2C. As can be seen from fig. 2C, when the claim number is 6 or less, the claim number of the high-value patent is smaller than the number of the ordinary patents, and when the claim number is more than 6, the total of the claim number of the high-value patent is higher than the ordinary value, and the increase factor is in a substantially linear increasing trend. The specification shows that the number of the claims of the high-value patent is relatively larger than that of the ordinary patent, and the probability of the high-value patent occurring is higher when the number of the claims is larger, which is consistent with the expected trend of the high-value patent.

To more clearly show the patent value versus the number of claims, we can simply fit the above scatter plot to a one-dimensional linear curve: and y is 0.085x +1.0695, and the slope of 0.085 represents the correlation of the parameter of the claim quantity to the patent value degree.

In fact, the relationship of patent value to the number of claims does not always show a linear growth. As can be seen from fig. 2A and 2B, when the number of claims is small (e.g., when the number of claims is less than 10), the patent value changes more steeply with the increase of the number of claims, and as the number of claims continues to increase, the influence of the number on the patent value decreases gradually and becomes gentle. Therefore, when the relation between the patent parameters and the patent values is constructed, a more complex and accurate fitting strategy is provided.

As can be seen from fig. 2A to 2C, when the number of claims rises to a certain number (in this example, when n is greater than 23), the distribution of data shows a discrete tendency. This is because the high-value patent samples are only used for case demonstration, and therefore the number of samples is small, and when the number of claims is large, the number of samples is missing. If the data sample of the high-value patent is large enough, the above-mentioned dispersion phenomenon will gradually weaken, but will not disappear, because in practice, the high-value patent also has the situation of fewer claims, and the low-value patent also has the situation of more claims, and the number of claims can only reflect the probability of the high-value patent occurring, and cannot absolutely represent the patent value.

In addition, as can be seen from fig. 2A to 2C, when the claim number is 10, the high-value patent distribution ratio has the phenomenon of "diving" compared with the ordinary patent, and according to the trend of the front and back data, when the claim number is 10, the ratio of the high-value patent should be about 2 times that of the ordinary patent, and the ratio in the statistical chart is only 0.75, which is smaller than the claim values of

claim numbers

6, 7, 8 and 9, and is smaller than the high-value patent ratio of which the claim number is greater than 10. The value of not all patents with claim number 10 must be small in probability and the problem still arises from the patent sample. According to the screening conditions of the high-value patents, most of the obtained samples are issued patent inventions. Patents of the invention may require substantial examination prior to filing, and in substantial examination, the claims of a number of patents may be subject to local modification by an examiner, resulting in a discrepancy (usually a reduction) in the number of claims in the final filed patent from the number originally filed. The "full library" patents representing the general patent samples include all the invention applications and utility model patents in China, which are not essentially examined. In view of the specification of the claim surcharge in the chinese patent charging standard, many applicants or agents are used to limit the number of claims to 10 or 10, so that the probability of the distribution of the patents with the claim number of 10 in the unexamined patents is high, and the statistical data looks abnormal. Similarly, in the above scatter diagram, statistics of

claim numbers

20, 30, 40, etc. with 0 bits are more or less abnormal, and have some relation with this or more (some people like "whole numbers" when writing claims). When a patent value degree evaluation formula is actually constructed, the relation between the patent parameters and the patent values can be more truly restored by neglecting the abnormal data points.

Of course, the function reflected by only one simple high-value patent sample model is not necessarily very accurate, and the solutions are as follows: firstly, the number of high-value patent samples and the purity of high-value patents contained in the high-value patent samples are improved; secondly, constructing different high-value patent sample models, respectively carrying out back-to-back calculation, then respectively comparing, finding out reasons and carrying out corresponding adjustment; thirdly, grouping a plurality of groups of randomly sampled patent samples by using the preliminarily constructed high-value patent model, and manually adjusting the weight of the related parameters by analyzing the accuracy of a grouping result. In summary, the construction of the sample model of the high-value patent is a very complicated process, and requires repeated calculation, comparison, adjustment and verification.

In addition, considering that the patent database is a dynamically changing and continuously increasing data set as a whole, the statistical result of the big data and the reflecting rule are continuously changed along with the time, so that the model needs to be checked and adjusted periodically when the patent value evaluation model is established.

In the process of establishing the relationship map of the number of the patents and the evaluation parameters, the relationship map is not specially set by an expert scoring method according to the parameter characteristics of the Chinese patents, but the relationship map truly reflects the true characteristics of the Chinese patents, and the reason is that the basis for establishing the map is Chinese patent data and high-value patents extracted from the Chinese patent data, so that the relationship map is not influenced by the characteristics of other countries and regional patents.

In the same way, a relation map of the number of the U.S. patents and the evaluation parameters can be created, and through comparison, the difference between the number of the patents and the value degree relation of the patents in the two different countries is found. This is explained below with reference to fig. 3A to 3C. By way of example, we selected all patents that were filed and published since 2000 in the united states as base samples and among them the standard essential patents (total 18393) related to the ETSI (European Telecommunications Standards Institute) standard as high-value patent samples, and analyzed using the above method, the following relationship maps were obtained:

fig. 3A illustrates the probability of different claim numbers of patents appearing in the first patent data sample in a U.S. patent according to an embodiment of the disclosure. Fig. 3B shows probabilities of different claim numbers of patents appearing in the second patent data sample in U.S. patents according to an embodiment of the disclosure. Fig. 3C shows a comparison of probabilities of patents of different claim numbers appearing in the first and second patent data samples in U.S. patents according to an embodiment of the disclosure.

As can be seen from fig. 3C, the probability relationship between the number distribution of the high-value patent and the number distribution of the common patent claims is:

y＝0.0297x+0.5452 (1)

from equation (1), it can be derived that the degree of correlation of the parameter of the number of claims to the degree of patent worth is 0.0297.

Comparing fig. 2 and 3, it can be seen that:

(1) the probability that the patents with the claim number of n in the United states and the United states appear in all samples can be seen, most of the Chinese patents are distributed in the interval that the claim number is less than or equal to 10, and the patent number proportion of the claim numbers of 1 and 2 respectively reaches 9.6 percent and 9.8 percent respectively. Except for claim 10, the patent with claim number equal to 3 with the highest probability is the patent with claim number equal to 3, the patent number is gradually reduced in the interval of claim number equal to 3 to 9, when the claim number is more than 10, the patent number is suddenly reduced in a cliff type, and when the claim number is more than 12, the percentage is basically kept below 1%. And the U.S. patent shows a relatively slow ascending trend of the patent number when the claim number is less than 10, shows a substantially gentle straight line in the interval of the claim number equal to 10 to 20, shows a slow descending trend when the claim number is more than 20, and shows a patent proportion in the interval of the claim number 20 to 30 which is substantially more than 1%. The number of claims describing the united states general patent is mostly kept in a larger range of values and the overall patent writing level is higher than that of the chinese patent. Furthermore, the distribution of high value patents and generic patents in the united states exhibits a significant sudden rise when the number of claims equals 20, which is clearly directly related to the rules for charging surcharges for united states patent claims.

(2) Comparing the distribution probability of the high-value patent with the claim n in the two countries in China, it can be seen that the number of the claims of the high-value patent in China basically increases within 10, when the number of the claims is more than 10, although the number distribution has a more obvious sudden decrease, the number will slowly decrease from about 3%, and when the number of the claims is equal to 21, the number still remains over 1%. The high-value patents in the united states still basically keep the trend of ascending and descending along with the increase of the number of claims, but generally, the ratio of the patents with more claims is higher than that of the high-value patents in China. Although the high-value patents in China are provided in larger quantity than the ordinary patents in China in terms of average claim number, a certain gap is still left between the high-value patents in China and the high-value patents in the United states.

(3) In general, the average occupation ratio of the high-value patents in the united states is improved with the increase of the number of claims, but the increase speed (slope) is obviously lower than that of the Chinese patents, and particularly when the number of claims is less than 20, the numbers of the high-value patents and the ordinary patents have no obvious advantages in probability distribution, which shows that the number of claims represents more obviously in the aspect of patent value for the Chinese patents, or the difference of the number of claims in the Chinese high-value patents is more obvious compared with the ordinary patents.

In step S104, an evaluation formula of the relationship between the patent parameters and the patent values is constructed

Although theoretically, the more relevant parameters contained in the model, the more detailed the generalization and description of the patent value will be, and the more accurate the evaluation result should be. However, it should be considered that the complexity and the calculation amount of the model are increased by increasing the evaluation indexes, and the correlation between some indexes (such as the patent examination duration) and the patent value is very complex, so that specific analysis needs to be performed on specific situations, and the judgment on the patent value is sometimes contradictory by merely mechanically applying the result of system data statistics; some indicators that affect the study but are temporarily difficult to obtain in their entirety (e.g., the importance of the invalidated claims in the partial invalidation review) may also reduce the feasibility of this study. These patents need to be selectively discarded based on careful analysis of experimental results. Therefore, the evaluation parameters adopted by the model need to be selected by comprehensively considering the factors and the actual map test result.

After the selected evaluation parameters are determined, the actual characteristics of each parameter are analyzed, and an evaluation formula of the relationship between the parameters and the patent value is constructed. The following describes the construction process of the claim quantity and patent value evaluation formula by taking the claim quantity parameter as an example.

(1) Construction of patent value evaluation formula of direct parameters satisfying curve relation

Fig. 4 shows a fitting curve of the relationship between the number distribution of the high-value patents and the common patent claims in china. By performing a depth analysis on the scatter plot of fig. 4, the following non-linear mathematical model can be fitted:

wherein, Vi means the contribution of the ith patent parameter to the patent value. M is the maximum value that Vi can get, p is the parameter value, and K is the value of p when Vi tends to slow.

For the fitted curve of the relationship between the distribution of the number of the chinese high-value patent and the number of the general patent claims shown in fig. 4, p may represent the number of the current patent claims, and M represents the maximum value of the influence of the claim parameters on the patent value degree, that is, when the number of the claims p is greater than the threshold K, the influence of the increase of the number of the claims on the patent value degree is no longer reflected. The value of K in this example is 50.

Compared with the linear fitting result of the equation (1), the equation (2) can more accurately reflect the influence of the claim quantity parameter of the Chinese patent on the patent value.

The fitting method can also be used for other parameters such as patent citation quantity, patent family quantity and the like, and it should be noted that the shapes of the influence curves on the patent value presented by different parameters and the reasons for the influence are different, and specific analysis and design are required to be performed respectively, and in some cases, corresponding functional relationships need to be given in a segmented manner.

(2) Construction of patent value evaluation formula of direct parameters not meeting curve relation

Not all parameters that may affect patent merit can be constructed by fitting the curves described above. For example, the applicant type (enterprise, university, other institution, individual), the type of patent (invention, utility model), the validity (valid, pending, invalid), or the legal status (litigation, licensing, transfer, invalid, etc.) all have an impact on the value of the patent, but these parameters are difficult to analyze by statistics on the variables. For the parameters of the type, the distribution probability of the parameters in high-value patents and general patents can be counted in a big data analysis mode, the influence of each situation on the value degree is estimated, and corresponding influence weight is given. For example, taking the applicant type as an example, a patent corresponding to each value of the parameter may be extracted, a patent merit degree evaluation may be performed on the patent corresponding to each value (here, the patent merit degree evaluation may be performed using an already established model, for example, a preliminary model constructed from the other parameters satisfying the curve relationship, or may be performed in another manner), a weight of an influence of the value of the parameter on the patent merit degree may be set based on an evaluation result, and an evaluation formula representing a relationship between the parameter and the patent merit degree may be obtained based on the weight of the influence of each value of the parameter on the patent merit degree.

Taking the applicant type parameter as an example, the patents may be grouped according to whether the applicant is an enterprise, a university, another institution, or an individual, and the value degree of each patent in each group is obtained by using a preliminary model constructed by parameters such as the number of claims, the number of patent citations, the number of patents in the same family, and the like, so as to obtain the comprehensive value degree of the patents in the group, and the following contents described in step S105 may be referred to specifically for obtaining the comprehensive value degree. Influence weights of corresponding parameter values can be designed according to the comprehensive value degree of each group, and influence factors of the parameters are incorporated into a patent value degree evaluation model.

Still taking the applicant type parameter as an example, assuming that the four applicant types of enterprises, universities, other institutions and individuals are set to have four values of a, b, c and d respectively, the influence on the patent value is evaluated according to the following rules when evaluating the patent value:

if applicant type is enterprise: v1 ═ a

If applicant type ═ college & universities: v2 ═ b

If applicant type other mechanisms: v3 ═ c

If applicant type is personal: v4 ═ d

Wherein V1, V2, V3 and V4 represent the contributions of the four applicant types of enterprises, universities, other institutions and individuals to patent value, respectively.

(3) Construction of indirect parameter patent value evaluation formula

In addition to the direct parameters, 14 indirect parameters such as "patent agency's comprehensive patent application quality index" are listed in table 1. Each indirect parameter is obtained by performing a comprehensive calculation on a plurality of sub-parameters affecting the value of the indirect parameter, and in some embodiments, each indirect parameter is obtained by a different value evaluation model. For example, the indirect parameter "patent agency comprehensive patent application quality index" is obtained by comprehensively evaluating a plurality of parameters such as a comprehensive value degree score of a patent of an agency, the number and the proportion of invented patents of the agency, an authorization rate of the invented patents, and the average claim number of the patents. When an evaluation formula is constructed for indirect parameters, factors influencing the patent value degree can be selected, sub-parameters for evaluating the factors can be selected, the factors are evaluated according to the sub-parameters, and evaluation results are used as the selected indirect parameters. Specifically, the indirect parameter can be obtained by the following equation:

wherein p is_iDenotes the ith indirect parameter, s denotes the number of subparameters that influence the number of indirect parameters, r_tRefers to the contribution of the t-th sub-parameter to the value of the indirect parameter, w_tRepresenting the weight of the t-th sub-parameter.

The relationship between the indirect parameter and the patent merit figure can be expressed as:

V_i＝f(p_i) (4)

wherein V_iRepresents the contribution of the ith indirect parameter to the patent value degree, and f () represents a functional relationship.

It should be noted that the indirect parameters listed in the attached table in this section also include "preliminary value evaluation index for individual patent", and the parameters are obtained by the following method: 1. constructing a preliminary evaluation model of the patent value by using other indexes except the index to obtain a preliminary value evaluation index of the patent; 2. Iterating and optimizing part of direct parameters or indirect parameters aiming at the preliminary value evaluation index to obtain an optimization model for evaluating the patent value degree; 3. and obtaining the final patent value degree of the patent by using the optimization model.

In step S105, a comprehensive evaluation formula of the patent value degree is established.

After the evaluation formulas of the direct parameters and the indirect parameters are established, a comprehensive evaluation formula of the patent value degree can be established.

The existing parameter system for evaluating the patent value degree is roughly divided into a plurality of structural types such as a unitary structure, a linear structure, a tower structure and the like. The unitary structure is evaluated by adopting a single parameter, and is obviously not suitable for evaluating the worth degree of the patent. The linear structure employs a plurality of parameters, but these parameters are in a parallel relationship with each other. Due to many factors influencing the patent value degree judgment, the system is too complicated by adopting a linear structure, the relation among all parameters is difficult to grasp, and the situation that the situation is approximate to the situation may be caused, so that the reliability of the analysis result is reduced. According to the characteristics of parameters for evaluating the patent value degree, a tower-type evaluation structure can be selected, namely, all evaluation parameters are divided into a plurality of layers according to different angles for representing the patent value. For example, the parameters may be divided into a plurality of groups, each group corresponding to one evaluation dimension. In this embodiment, the parameters may be divided into three groups according to three dimensions such as technology (technical advancement), law (technical stability), market (protection scope), etc., wherein the technical stability mainly considers whether the patent is substantially examined and authorized or maintained effectively, whether the patent is examined by market operation activities such as patent infringement litigation, pledge, etc., and whether there is a patent of the same family authorized overseas, etc.; the technical advancement mainly takes the global citation condition of patents and patents in the same family, the number of research and development personnel invested in the patents, the technical field related to the patent technology (IPC classification number), the assignment or permission condition of the patents and the like into consideration; the protection scope mainly refers to the number of claims of a patent, the remaining protection period, the number of countries or regions of a patent layout, and the like. Because the selected parameters have hierarchy, the weight of each item in each hierarchy is controllable, and the parameters can be correspondingly expanded along with the increase of the parameters, the method is more flexible and has smaller limitation.

The comprehensive value degree of the patent is obtained by the sum of the comprehensive values of the three dimensions:

wherein, when the value range of i is respectively [1, n1 ]]、[n1+1,n2]，[n2+1,n]When, V_iRespectively represents the influence contribution of the patent parameters in three dimensions of technology (technical advancement), law (technical stability) and market (protection range) on the patent value degree. The value of each dimension can be controlled to be within a desired range by an extremum-defined method.

In addition, on the basis of the Chinese patent value degree evaluation model, a value degree evaluation model for one patent set (such as all Chinese patents applied by a certain applicant) can be established, so that the comprehensive patent values in a plurality of different sets can be evaluated and compared.

In addition, user-defined parameters may be set in addition to the parameters listed in table 1, in which case, the user may select other desired parameter types as needed, and a corresponding data list may be provided as part of the patent data sample if necessary. When the patent value degree evaluation model is constructed, part or all of parameters can be selected from the table 1 by default to construct a preliminary patent value degree model, and another patent value degree evaluation model is constructed based on the user-defined parameters and combined with the preliminary patent value degree model, so that the finally constructed patent value degree evaluation model is obtained.

In addition, the patent value degree is embodied in a plurality of layers of strategic value, technical value, legal value, market value, economic value and the like of the patent, the patent value degree evaluation model of the embodiment can give the value degree evaluation result at least in part of layers, for the layer which cannot be covered by the embodiment, the patent value degree evaluation model can be given in other modes or even directly obtain the evaluation result, and the final patent value degree evaluation is obtained by combining with the patent value degree evaluation model of the embodiment. For example, the evaluation of the relevant evaluation dimension of strategic value or economic value can be added on the basis of the evaluation results of the patent value evaluation model on technical value, legal value and market value, so that the application range can be expanded from macro analysis to individual case value analysis. The strategic and economic value of a patent depends largely on some special attributes of the patent itself, such as the implementation and application conditions of the patent-related technology or product, the evasiveness, confidentiality or replaceability of the patent technology, the difficulty of acquiring evidence of patent infringement, the asset resetting cost of the applicant and the patent, the recent transaction price of the same or similar assets in the market, the excess income brought by the patent assets, and the like, and many of these attributes are personalized, are often difficult to acquire through traditional patent information or legal information, and need to be individually assigned for specific situations.

It should be noted that the above embodiments are only for illustrating the principle of the patent merit evaluation model construction, and more complicated influencing factors should be considered in the actual construction process. For example, there may be strong correlation between some parameters (e.g., authorization of a patent and authorization of a patent family, etc.), which may result in repeated weighting. In addition, the model in this embodiment is a comprehensive evaluation model for inventions and utility model patents, in which the parameters related to the unexamined patent applications, the utility model patents and the examined patent patents are different, and the patent value conditions reflected by the same parameter are also different, so that for these complicated conditions, multi-angle judgment can be performed on the logic relationship, and selective combination or appropriate adjustment of the occupied weight can be performed according to different conditions. Therefore, in actual operation, the correlation indexes can be selected and integrated in combination with the results of the correlation test.

In step S201, a patent value degree evaluation model is constructed. In this step, a patent merit degree evaluation model may be constructed by the method described above with reference to fig. 1 to 3. The number of models constructed may be one or more. For example, a first model may be constructed based on one set of patent data samples, while a second model is constructed based on another set of patent data samples for use in subsequent steps.

In step S202, the patent merit evaluation model is optimized. Ways of optimization include, but are not limited to, iteration and comparison.

For example, assuming that one model, i.e., the first model, is constructed in step S201, a target patent evaluated by the first model as having a degree of merit in a desired range may be taken as a reference patent with which to update a patent data sample used to construct the first model. The desired range can be selected as desired, for example above a predetermined value, below a predetermined value and/or within a predetermined range. For the model for screening the high-value patents, the patents higher than the preset value can be used as reference patents to update patent data samples; for the model for screening low-value patents, patents below a preset value can be used as reference patents to update patent data samples. In some embodiments, also in case a model is built, the model can be optimized by: the method includes evaluating preselected validation data using the model to obtain a first evaluation result, evaluating a sample of patent data used to construct the model to obtain a second evaluation result, comparing the first evaluation result with the second evaluation result, and adjusting the model based on the comparison result. Here, the operation of evaluating the patent data sample to obtain the second evaluation result may be performed according to experience or a preset standard, or may be performed by other means or with other tools. After comparison, the model is optimally adjusted according to the deviation between the two evaluation results, so that the accuracy of the model is improved.

Similarly, assuming that two models, i.e., a first model and a second model, are constructed in step S201, a target patent evaluated by the first model and/or the second model as having a degree of merit in a desired range may be taken as a reference patent with which to update a patent data sample used to construct the first model and/or the second model. Similarly, the so-called desired range may also be selected as desired, and will not be described further herein. As another example, it is also possible to prepare a set of patent data for verifying a model, evaluate the patent data using the first model and the second model, respectively, and obtain evaluation results, compare the evaluation results of the first model and the evaluation results of the second model, find the cause of the difference, and adjust the first model and/or the second model accordingly.

In some embodiments, when a target patent is evaluated by using a model, if the target patent meets a preset condition, the weight of a corresponding sub-evaluation formula in the model used for evaluating the target patent is increased or decreased. For high value patents, for example, it is clear that the longer the remaining useful life, the higher the patent value. However, in the case of low-value patents, there is no obvious relationship between the value of the patent and the length of the remaining effective period, and if the low-value patents are weighted in the same manner as in the case of high-value patents, the objectivity of the evaluation result is inevitably affected. Therefore, through the iterative method, after the target patent is evaluated as a high-value patent, it is considered that the contribution of the parameter of the residual life to the patent value degree needs to be enhanced, the weight of the corresponding sub-formula is increased, and the weight of the corresponding sub-formula is reduced, and then the optimized model is used for re-evaluating the target patent, so that a more accurate value degree evaluation result is obtained.

In step S203, the value degree of the target patent is evaluated using the constructed patent value degree evaluation model. For example, if the patent merit degree evaluation model obtained in step S201 is constructed based on three sets of parameters, which respectively correspond to the evaluation criteria of the three levels of technical stability, technical advancement, and protection scope. In this step, the model can be used to obtain the evaluation results of the target patent in the three levels, and the evaluation results are weighted and summed to obtain a comprehensive evaluation result, so that the retrieval results can be sorted according to the patent value degree.

In this step, the value degree of the target patent can be evaluated by using an appropriate model as needed. The subject patent herein includes, but is not limited to, a single patent or a group of patents.

For example, if the target patent is a normal single-patent, the worth of the target patent can be evaluated by using a patent worth evaluation model constructed by using data samples of the single-patent. If the target patent is a composite patent processed into a single piece by a family patent, the value degree of the target patent can be evaluated by utilizing a patent value degree evaluation model constructed by corresponding composite patent data samples. If the target patent is a patent group (e.g., a patent group divided by the applicant, the IPC classification number, the applicant region, the patent family, etc.) it is possible to evaluate each patent in the group individually and perform mathematical operations on the evaluation results to obtain the evaluation results for the patent group, where the mathematical operations include, but are not limited to, averaging, geometric averaging, square averaging (root mean square average, rms), harmonic averaging, weighted averaging, etc. The evaluation results of the patents in the group can also be processed by using a statistical tool such as a box chart statistical tool, so as to obtain more accurate and reasonable evaluation results for the patent group.

Patent classification can be performed before or after the mathematical operation (i.e., the calculation result is normalized to a score in a preset range as described above), if the calculation result is the score in the preset range, the value degree of each patent is classified, then the value degree score of each patent is subjected to the mathematical operation to obtain an evaluation result for a patent group, and then the evaluation result for the patent group is rounded; if the evaluation result is the latter, firstly, the evaluation result of each patent is subjected to mathematical operation to obtain the evaluation result aiming at the patent group, and then the evaluation result of the patent group is normalized to be a score in a preset range, namely, the value of the patent group is graded.

Fig. 6 shows a schematic diagram of the comprehensive evaluation result of the patent worth degree according to the embodiment of the present disclosure. As shown in fig. 6, in order to enable the user to more intuitively and quickly obtain the evaluation result of the value degree and to realize the screening and statistics of the patent value, the comprehensive evaluation score is normalized. For example, the comprehensive evaluation scores of the patents can be segmented according to a 10-point scoring mechanism according to the distribution of the comprehensive evaluation scores of the patents in the whole database, so that the proportion of the patent data of each score in the whole database is distributed according to a certain rule, and the probability distribution of the patent values is decreased from 10 points to 1 point. Similarly, the 10-point score for each category may be processed in the same manner.

In addition, the model in this embodiment is an evaluation model designed for the comprehensive value of the chinese invention and the utility model patent, and although the obtained patent value degree reflects the probability distribution of the patent value as a whole, if the core influence factor of the patent value is to be judged quickly, the comprehensive evaluation score of the three evaluation dimensions of the patent value can be also designed in a hierarchical manner of 10 scores in a manner similar to the above. On the basis of acquiring the comprehensive value degree of the patent, a user can comprehensively know the specific value condition of the patent by combining the comprehensive values of all the dimensionality values and the list display of relevant evaluation bases.

Compared with the invention and the utility model, the Chinese design patent has different protected objects, has differences in parameter selection and value judgment, can consider the differences in model construction and evaluation, but has the same basic design idea and method, and is not repeated herein.

Although some example implementations of optimizing the patent merit evaluation model are given above, embodiments of the present disclosure are not limited thereto, and the optimization of the model will be described in further detail below. The above embodiment introduces the process of establishing the patent value degree evaluation model for the example of the actual model of the comprehensive value evaluation of the high-value patents in china. In the embodiment, aiming at the requirement of quickly identifying high patent value in the retrieval process of a user, the patent document information disclosed since the implementation of a patent system in 1985 and related patent laws and operation information are fully utilized, a basic model of the invention patent authorized in China is established on the basis of the invention patent authorized in China on the basis of a large number of experiments and analysis, and on the basis, design adjustment is respectively carried out on the Chinese patent application, the Chinese utility model and the Chinese appearance design patent, so that the objective evaluation and classification of the comprehensive value of the Chinese patent are realized. From the actual verification effect of the model, the model can achieve the expected identification and distinguishing effect on the Chinese high-value patent.

In addition, in the process of searching, the user also wants to be able to grade the value of the patents in other countries and regions and show the value in the search result at the same time, besides expecting to quickly identify the high-value patents in China. Therefore, the parameters in the model and the setting of the weight thereof can be adjusted according to the patent characteristics of different countries and regions, so that the parameters and the setting of the weight thereof are integrated with the Chinese patent value degree evaluation model to form a complete system for evaluating the comprehensive value of the patent.

On the basis, more refined personalized model design can be carried out, for example, aiming at a special database in the field of Chinese chemical materials, after the basic database is built, high-value patent samples can be selected from the special database, and a patent value degree evaluation model in the special field is formed through analysis.

Specifically, the patent information can be applied to various scenes such as technical information analysis, novelty analysis of technology or patent, free implementation (FTO) analysis, competition information analysis, patent operation analysis, anti-infringement analysis, technical cooperation, talent introduction and the like. Different application scenarios have different retrieval purposes, and the understanding or the requirement of the high-value patent is different. Therefore, on the basis of the comprehensive evaluation model, corresponding parameters and weights in the model can be correspondingly adjusted according to different application scenarios to form a refined value degree evaluation model.

When the contribution of a certain parameter to the patent merit degree is emphasized, the weight of the parameter can be increased in the evaluation formula, and conversely, the weight of the parameter is decreased. The following briefly describes the different application scenarios and the emphasis points of the patent merit evaluation model.

(1) Patented technology

The patent technology value analysis is mostly used for research activities aiming at acquiring technical solutions and technical evolution processes and development trends thereof, such as basic research and analysis of special technologies, technical investigation before project establishment, technical solution selection in the project establishment process, related technical solution query in project research and development and the like. In such application scenarios, the emphasis of the user is the innovation of the technical scheme itself and the origin, evolution, application situation thereof, especially those patents whose technology has a short time of origin, high novelty, rapid development and great influence. Therefore, when designing a patent value evaluation model for the purpose of utilizing technical information, direct parameters should focus on the application date (patent age), novelty, type and authorization of a patent, average age of cited documents, number of IPC classifications, number of cited patents or patent families (particularly number of patents cited by others) and number of related IPC classifications, number of IPC classifications related to cited patents, number of dependent claims corresponding to independent claims, number of specifications, number of inventors, and the like of a patent; the indirect parameters focus on the comprehensive technology advancement index of the applicant, the technology association degree, the technology emerging degree, the technology heat degree, the technology life cycle and the like of the industry.

(2) Infringement risk analysis

The infringement risk analysis comprises patent infringement retrieval analysis aiming at a specific technical scheme and patent infringement risk early warning analysis aiming at a specific technical field.

Patent infringement retrieval is also called patent infringement investigation, namely, a pointer is used for searching for a specific technical scheme, a patent possibly having infringement risk is found out through retrieval, and the process of judging whether the existing technical scheme falls into the protection range of the technical scheme is analyzed. Patent infringement retrieval also belongs to microscopic retrieval aiming at specific technical contents, and the main content concerned is whether a patent similar to the analyzed technical contents exists, whether the current legal state of the patent is valid, and whether the related technical scheme falls into the protection scope if the patent is valid.

Although the infringement search is also a nondirectional microscopic search performed for a specific technical scheme, the content of infringement analysis relates to the protection of patent rights, and the writing quality of a patent, the protection range of the rights, the property of a right subject, the protection of the patent, the maintenance strategy of the patent and the like all affect the probability that the patent infringement risk may occur. The model is designed with more attention paid to the effectiveness of the current patent, the type of the patent, the number of independent claims, the number of dependent claims, the number of words or technical features of the independent claims, the number of pages of the specification, the cases of patent litigation and patent litigation of the same family, the cases of patent licensing and assignment, the cases of patent invalidation and patent grant of the same family, and the like, which are related to legal factors, and the indirect parameters are more attention paid to the technical relevance of the applicant, the comprehensive patent application quality index, the comprehensive offensiveness index of the applicant, and the like.

Patent risk early warning refers to a process of carrying out patent retrieval analysis aiming at the concerned technical field, finding out patents with possible risks, classifying, monitoring and taking corresponding measures in order to discover or avoid the possible infringement risks in advance by a user. Unlike patent infringement retrieval, patent risk early warning is not directed to a specific technical solution, but to all technologies related to the technology, product or business concerned. However, the same as the infringement search, the important points of concern are the right protection condition of the relevant patent and the probability of risk occurrence, and the patent value analysis model of the patent risk early warning can refer to the model of the patent infringement search.

(3) Patent operation analysis

The purpose of patent operation analysis is to quickly find high-value patents with operation value, so that the operation conditions of the patents which should be paid attention to firstly, namely activities such as licensing, assignment, pledge and the like, wherein the licensing and the assignment of the patents are divided into the assignment and the licensing inside an organization and the licensing and the assignment outside the organization, the former focuses more on policy management inside the organization, and the latter can reflect the market operation value of the patents more objectively. The actions such as litigation and invalidation of patents can reflect the influence of patents on the market from the other side, and further reflect the operation value of the patents, and the actions should also be taken as the focus of attention. In addition, the cited condition of the patent reflects the technical influence of the patent, and the patent with great technical influence is also a patent with higher operation value; in addition, the effectiveness, the protection range, the invalid conditions of operation and litigation of the patent family, the comprehensive operation capacity index of the applicant and the like of the patent also have important reference function on the operation value of the patent and can be used as an important index of an analysis model of the operation value of the patent.

(4) Patent application quality control

The patent value is the comprehensive embodiment of the technical value, the legal value and the market value. The technical value is the basis for forming high-value patents, the patent application is the key for converting the technical value into the legal value, and the combination of the technical value and the legal value becomes the support of market value. Therefore, the management and monitoring of patent application quality is also one of the major concerns of the public. The patent application quality evaluation model can be used for inspecting the quality of patent applications of an applicant or a competitive partner, and particularly can be used for inspecting the service quality of a patent agency.

When constructing the evaluation model of the patent application quality value degree, the direct parameters related to patent writing, such as the number of independent claims, the number of dependent claims corresponding to the independent claims, the number of words (or technical features) of the independent claims, the number of pages of the specification, the number of drawings of the specification, and the like, can be considered in an important manner, and if possible, the average index of the indexes in the corresponding technical field can be referred to. The comprehensive patent application quality index of the applicant, the comprehensive patent application quality index of the patent agency and the indirect parameter are all related to the application quality of the patent, but belong to comprehensive evaluation indexes of the patent application quality, and the indexes can be used as reference of the patent quality capability of a related party when carrying out comprehensive value analysis, competitor analysis and operation analysis of the patent, but the application quality evaluation of a specific patent is not recommended to be endowed with too high weight.

(5) Failure patent

Patents are legal exclusive rights which can be granted to patentees by law, but such rights are subject to limitations. Firstly, the patent protection is regional, after a patent is applied abroad, if the application protection is not applied in China within the time specified by law, the priority in China is lost, and if the application is disclosed before, the corresponding technical scheme of the patent loses the opportunity of authorization in China; secondly, after the patent application, the authorization can be realized through examination, and if the patent is rejected or withdrawn after the patent is published, the applicant can also lose monopoly rights; thirdly, after the patent is granted, the applicant must pay annual fee according to the term to maintain normal rights, and the protection period of the patent is limited, and once the protection period expires, the applicant loses the corresponding rights; in addition, issued patents may be challenged by others to become invalid, and patentees may lose all or part of their rights if they become invalid in whole or in part. These patents, which are not entitled to them, are now in the state of the art and are intended for the public at will. We refer to these situations collectively as a failed patent (note: the loss of the right of a certain applicant or right does not mean that the patent must be a public technology, the public should pay attention to the fact that before use, the corresponding right is legally and effectively owned by the applicant or another person, and that the failure of a patent in one country or region is not equivalent to the loss of the right in another country or region in view of the regional protection of the patent).

In some application scenarios (such as patent infringement risk analysis and patent operation analysis), the failed patent has little value in utilization. On the other hand, the patent system is set up to make the public obtain the results of technical innovation and promote social progress through application and improvement. A large number of invalid patents are free knowledge treasuries which can be conveniently obtained by the public. The invalid patent can be directly utilized without risk under the condition that other legal rights are not determined, so that the research and development time and the research and development investment are greatly saved, and the value of the invalid patent is not too thick. Therefore, it is also very significant to specifically grade and evaluate the utility value of the failed patent.

The evaluation of the value of the invalid patent is greatly different from the analysis of the patent operation in the aspects of application purpose and scene, but except that the judgment condition of whether the patent is invalid in the target country at present is just opposite, other principles for judging the high-value patent are basically consistent, and the evaluation model of the value degree can be obtained by slightly modifying the evaluation model of the value degree of the patent operation on the basis of the evaluation model of the value degree of the patent operation.

It should be particularly emphasized that the optimization of the value degree evaluation model just adds the weight of some parameter or parameters in the contribution of the patent value degree according to the practical application scene of the user, and the priority levels of the patents are respectively set to facilitate quick retrieval and browsing, which should not be understood as a mode of patent screening or exclusion. In view of the complexity and uncertainty of patent literature information and the patent value, any patent value evaluation system formed by a mathematical model established by using the patent parameters is used for evaluating the occurrence probability of related events, and is not used for absolutely judging the patent value. In addition, the models can be used independently or combined with each other for comprehensive use. For example, the patent and technical value analysis can be performed on the search result first, then the operation value analysis can be performed on the search result, and the results of the two are combined, so that the patent with higher operation value and higher patent and technical value can be found out quickly.

In addition, the adaptability summation and the fine design of the patent value degree evaluation model are correspondingly designed according to the patent types, application regions, application scenes and the like of different objects, so that the evaluation result is more targeted, and is more scientific and reasonable.

In the above-mentioned patent merit evaluation model, the selection of each parameter and the setting of the weight thereof are obtained by adopting a corresponding analysis assignment method on the basis of analyzing sample data. On the whole, the evaluation and grading results conform to the probability distribution rule of high-value patents, so that most common retrieval and analysis scenes can be met. In general, once a valuation model is determined, the parameters and their weights are generally fixed, and the user only has the right to use and no right to freely select adjustments, except for occasional local optimization adjustments. Although the value degree evaluation model can meet the general search and analysis requirements, for higher-end users, the model also needs to be personalized and adjusted according to different application scenes and different requirements, so that the flexible design of the model is realized.

(1) Flexible design of patent classes for different features

The development of technology is unbalanced, for example, the technology heat, the technology life cycle, the market competition situation, etc. may be very different at different periods of the same technical field, and if the same model is used to analyze the patent value, the bias may not be generated. At this time, the user can be allowed to input the retrieval conditions by himself, specify the basic data and the high-value patent samples therein, respectively count the corresponding patent parameters, compare the parameters with the original model, and then give new weights again, thereby forming a more scientific and accurate value evaluation model for the technical field.

Similarly, characteristics of patents applied by different applicant types (such as enterprises, universities, individuals and the like), different periods and even different applicants can be analyzed, so that a corresponding personalized value degree evaluation model is formed.

(2) Flexible design for personalized needs of different users

The demands of users are various, and no matter how perfect the curing model can not adapt to different personalized demands of users. For example, when a user performs an analysis of the value of a patent technology, if the user is concerned about a new patent technology, the user may wish to raise the weight of the application date and novelty of the patent; if the influence of the technology is of greater concern, it is desirable to increase the weight of the cited cases of a patent or patent family. Therefore, on the basis of the existing value degree evaluation model, the user is allowed to adjust the corresponding indexes, the flexible design of the patent value degree evaluation model is realized, the model is more humanized, and the method and the device are flexibly suitable for various application scenes.

Characteristics, limitations and applicable application scenarios of patent value degree evaluation model

(1) Objectivity

All indexes adopted in the evaluation model are objective parameters extracted from patent information, the influence weight of each parameter on the value degree is calculated according to the result of big data analysis of the current Chinese patent database, the patent value reflected by the patent value degree is the real embodiment of each objective parameter of the patent, and the evaluation model is basically not influenced by human factors except for fine adjustment according to the actual situation in the later iteration stage.

(2) All-round

In the construction process of the evaluation model, all parameters which may influence the patent value degree are comprehensively tested and verified, all dimensions of the patent value are comprehensively analyzed, all influence factors of the patent parameters are fully considered in the construction process of the model, the finally obtained evaluation result is the comprehensive application of a plurality of parameters, and the effect is obviously superior to that of the evaluation of a single parameter.

(3) Real-time property

The evaluation indexes in the evaluation model are dynamic indexes acquired in real time in the database, are updated in real time along with events such as patent bulletins, legal state changes, new citation information and the like, and the patent value model can be adjusted adaptively according to the change condition of the database at regular intervals, so that the current value condition of the patent is reflected really.

(4) Accuracy of

The evaluation model is elaborately designed according to the characteristics and rules of Chinese patent information on the basis of an advanced credit card value degree evaluation model in the world, can truly reflect the value distribution condition of Chinese patents, and the accuracy of the evaluation model is verified in long-term practical application.

(5) Convenience of use

The weight of each parameter in the evaluation model is obtained through automatic calculation of a computer, and under the condition that a high-value patent sample is properly selected, the model can be constructed basically without the participation of experts (particularly, the experts do not need to evaluate indexes of each parameter respectively), so that the method is convenient and fast.

(6) Flexibility

The evaluation model is obtained by comparing the selected high-value patent sample parameters with the average parameters of the database patents, so that a user can select different analysis targets and high-value patent samples according to actual needs to quickly construct a corresponding model. For example, the user may use the chinese invention and the utility model patent application whose applicant type is college and university after 2000 as basic data, and select a part of high-value patent samples from the basic data, thereby constructing a model for evaluating the worth of the patents after 2000 years of chinese college and university, and objectively comparing and analyzing the worth of the patents.

The influence weight of each parameter on the value degree in the model depends on the distribution condition of the related parameter of the high-value patent sample, the weight of the parameter in the model is different when the selected high-value patent sample is different, and therefore the evaluation result of the patent value degree is different. Therefore, the strategy for selecting the high-value patent samples is a key factor in the process of constructing the value model. The weight adjustment mentioned in the above embodiments may be implemented by selecting a suitable implementation manner as needed, for example, the weight adjustment may be performed manually or automatically.

The patent merit evaluation model of the embodiment of the present disclosure is a merit evaluation model designed to measure characteristics of chinese patent information by using statistical analysis results of big data of a chinese patent database according to the accepted standards of the industry for patent merit evaluation. The method can truly screen and sort the patent value degrees according to the distribution probability of the high-value patents, can enable users to quickly focus on the high-value patents in massive patents, and obviously improves the utilization efficiency and effect of patent information.

According to the embodiment of the disclosure, some important information hidden in the patent retrieval result is quickly known by analyzing the quantity and the distribution rule of high-value patents in the patent retrieval result, so that the overall situation of an analysis object is more accurately and comprehensively grasped, and strategic reference is provided for further making relevant decisions or coping measures. For example, a company high-value patent can be screened and analyzed to obtain the application quantity and trend, technical field distribution, technical influence, application country, inventor investment, litigation, invalidity, permission and transfer characteristics, and through analyzing and knowing the information of the company in the aspects of technology, market and the like, such as competitive strength, development strategy, core talents, attack and defense characteristics and the like, the company high-value patent can objectively evaluate the competitive threat of the company high-value patent to the market itself, or discover the cooperation opportunities among the company high-value patents in time, and make corresponding countermeasures or take corresponding actions.

In the aspect of value analysis for a specific patent, a user can quickly know the patent value level scores of all relevant dimensions of the patent and the detailed conditions of all important parameters influencing the patent value level scores by checking the detailed information of the patent value degrees on the basis of generally knowing the comprehensive value degrees of the patent, so that the actual value embodiment of the patent is quickly preliminarily judged.

The verification results of the embodiments of the present disclosure are described below with reference to fig. 7A to 7C.

And when a patent value degree evaluation model is constructed, the evaluation effect can be verified through an actual application scene. In this embodiment, a high-value patent is selected to verify the patent value evaluation model. In order to ensure the objectivity of the verification result, when the data sample for effect verification is selected, the data sample is prevented from being coupled with the establishment rule of the model, namely, the high-value patent sample selected when the model is established is not used as the verification sample, and the related parameters in the model are not used for searching to obtain the verification sample. In addition, it is ensured that the selected high patent value should have a certain degree of confidence, for example, the patent value has been verified by the market or widely recognized by the industry. In this embodiment, consider that the validation sample is chosen from a different sample selection criteria than at model build time.

Fig. 7A shows a value degree analysis chart of the first to eighteenth chinese patent prize winning patents by using the patent value degree evaluation method according to the embodiment of the present disclosure. The Chinese patent award is a selection activity jointly developed by the Chinese intellectual property office (-CNIPA) and the United nations World Intellectual Property Organization (WIPO), is also the only government award specially awarded for the invention creation of the patent right in China, and has been successfully held eighteenth times so far. The prize evaluation standard not only emphasizes the patent technology level and the innovation height of the project, but also emphasizes the application condition of the project in the market conversion process, and simultaneously, demands are also made on the protection condition and the management condition of the project. The reward is recommended by intellectual property offices, state institute units, national industry associations or academists of Chinese academy of sciences/academists of Chinese engineering academies, and is selected after being subjected to multi-level evaluation by the intellectual property offices and the senior experts of various industries respectively, so that the reward has high public credibility and can be used as a verification sample of the value degree model.

The number of inventions and utility model patents for obtaining the first to eighteenth Chinese patent prizes (including the Chinese patent prize and the Chinese patent gold prize) is 3175. After grading the patent value, an analysis chart is derived as shown in fig. 7A. As can be seen from fig. 7A, 871 patents with a full score account for 27.4% of the total number of samples (only 4.5% of the patents with a score of 10 in the general patents), 2664 patents with a score of 8 or more account for 83.9% of the total number of samples, and 127 patents with a score of 5 or less account for only 4% of the total number of samples (55.5% of the patents with a score of 1 to 5 in the general patents), and these 127 patents have all failed at present by checking the current legal status of the patents (these patents should all be authorized and valid according to the condition of winning Chinese patent prize). The cases fully illustrate that the patent value degree model constructed currently has very good matching degree with the evaluation result of the Chinese patent prize, and can be used for screening high-value patents.

Of course, some so-called medium-value patents with patent value degrees of 6, 7 and 8 exist, part of the reasons are that the patent value degrees are reduced due to the failure of some patents, and meanwhile, factors such as economic benefits, market conversion and the like which cannot be predicted only by a patent value degree model are included in the selection conditions of the Chinese patent prize. On the other hand, the patent merit evaluation model is not a "artifact" for identifying whether a specific patent is absolutely a high-value patent, and although the model cannot guarantee the hundreds of the high-value patents, the probability of judging the high-value patent is still quite spectrum-dependent, and the model can be used as a help for identifying the high-value patent in work.

With respect to standard-Essential Patents (SEPs), the International Telecommunications Union (ITU) defines it as "any patent or patent application that may cover a standard draft, in whole or in part". The necessary patents of the standard can cover the draft of the standard in whole or in part, so the application value of the necessary patents of the standard is very remarkable. We selected the necessary patents (6851 total) related to the ETSI (European Telecommunications Standards Institute) standard from the patents of the invention and utility model granted in china, and analyzed the current value distribution as shown in fig. 7B. As can be seen from fig. 7B, 6129 patents with patent worth degrees of 9 or more account for 89.5% of the total number of samples, only 151 patents with patent worth degrees of 7 or less account for 2.2% of the total number of samples, and 64.9% of these patents are patents that have failed currently.

Fig. 7C shows a presentation of the evaluation of the comprehensive value degree of a single patent and the evaluation of the patent value degrees of each dimension by the patent value degree evaluation method according to the embodiment of the present disclosure. In the example of fig. 7, an invention patent with patent publication number CN102868498B and patent name "codebook generating method, data transmission method and apparatus" is extracted from a patent with patent value degree equal to 10, and its relevant key parameters and scoring conditions of technical stability, technical advancement and protection scope are shown in fig. 7C. As can be seen from fig. 7C, the indexes displayed in the list are only some common parameter indexes adopted in the patent value degree evaluation model, but the "standard necessary patent" factor can be automatically identified as a high-value patent on the premise that the influence parameter of the patent value degree evaluation model is not considered, which indicates that the model can reasonably use the big data statistical information of each patent parameter and scientifically and reasonably assemble the big data statistical information in the construction process, thereby forming a credible patent value degree evaluation system.

From the previous grading effect verification experiment of the high-value patent, it can be seen that the patent value degree model of the embodiment of the disclosure has a very significant effect on rapidly identifying the high-value patent objectively. Next, the model is used for carrying out statistics on data of Chinese patent operation, litigation and the like, and some distribution rules are known from the statistics.

The number of patent application documents (including invention application, utility model and appearance design, not including the authorized invention text) in the chinese patent database is counted to obtain the number ratio of each legal event, as shown in table 2 below.

TABLE 2

As can be seen from table 2, the high-value patents with patent value degrees of 9 or 10 account for less than 10% of all patents, but include more than half of patents in which litigation, licensing, or pledge has occurred, more than 40% of patents in which review or customs filing has occurred, and more than 30% of patents in which transference or invalidation has occurred. Further, the above-mentioned patents having a low patent merit (a patent merit of 4 or less) were analyzed, and many of them were found to have problems such as rejection or complete ineffectiveness, poor patent stability, and narrow protection range. Therefore, the patent value degree model can help the user to effectively screen high-value patents.

The probability of various legal events occurring for each degree of value patent is further analyzed with reference to table 3 below:

TABLE 3

As is clear from table 3, the probability of occurrence of legal events such as litigation, pledge, transfer, approval, customs filing, review, invalidation, etc. of a patent with a high patent value is far greater than that of a patent with a relatively low value.

The application of patent worth mentioning in patent retrieval and analysis practice is briefly described below with reference to fig. 8A and 8B in conjunction with specific examples.

Fig. 8A shows a schematic diagram of a user interface for patent worth evaluation using a patent worth evaluation method according to an embodiment of the present disclosure. Assuming that an enterprise producing a soymilk machine intends to understand important patents in the soymilk machine in the industry, patents containing the keyword "soymilk machine" can be searched in titles or abstracts, and sorted according to the patent value degree in the retrieval results, as shown in fig. 8A. The utility model 'a double-layer lower cover soymilk machine' (application number: CN201414387Y) is arranged at the head in figure 8A. It can be seen in the label alongside a patent that the patent has experienced various legal events such as litigation, transfer, licensing, customs docketing and revocation scrutiny. Further investigation of this patent revealed that it was a step-high patent purchased by jiuyang corporation, who later in patent infringement litigation acquired a victory or reimbursed 540 ten thousand yuan. The small utility model is not invented, and plays a great role in patent right.

Next we analyze the value degree distribution of the applicant patent.

Fig. 8B shows a schematic diagram of a patent layout of an applicant of a high-value patent screened by the patent merit evaluation method according to the embodiment of the present disclosure. As shown in fig. 8B, among the chinese patents applied by the jiuyang stockings limited company, there are 4 cases, 19 cases, 21 cases, 6 cases and 8 cases in which litigation, transfer, approval, customs filing, review and invalidation occur; screening patents with the patent value degree of 9 or 10 in Chinese patents applied by Jiuyang company, and finding that cases in which the legal events happen are respectively 4, 8, 20, 3, 4 and 8, and comprise most of the important cases in which the legal events happen. It can be seen that the applicant has more high-value patents.

In practice, after a patent is carefully searched as required, the patent value degree model is used for screening search results, and the actual content and technology of the patent and the application condition of the market are combined for deep and specific analysis.

Embodiments of the present disclosure also provide a computer-readable storage medium for storing instructions that, when executed by a processor, cause the processor to perform the above-described method.

By way of example, embodiments of the disclosure may also be described in the context of machine-executable instructions, such as those included in program modules, being executed in devices on target real or virtual processors. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In various embodiments, the functionality of the program modules may be combined or divided between program modules as described. Machine-executable instructions for program modules may be executed within local or distributed devices. In a distributed facility, program modules may be located in both local and remote memory storage media.

Computer program code for implementing the methods of the present disclosure may be written in one or more programming languages. These computer program codes may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the computer or other programmable data processing apparatus, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. The program code may execute entirely on the computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or entirely on the remote computer or server.

In the context of this disclosure, a machine-readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More detailed examples of a machine-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical storage device, a magnetic storage device, or any suitable combination thereof.

The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for sorting patent retrieval results is characterized by comprising the following steps:

obtaining multi-dimensional parameters of a plurality of patent files;

a patent value degree evaluation model is adopted, and the multidimensional parameters are summed and converted into a single patent value degree according to respective weights; and

sorting the plurality of patent documents according to the patent value degree,

the multidimensional parameters at least comprise an ith patent parameter, the ith patent parameter is any one of claim quantity, patent citation quantity and patent family quantity, the contribution of the ith patent parameter is a calculated value obtained by fitting the distribution condition of patent data samples by adopting the following nonlinear data model according to the distribution probability of the patent data samples selected from a patent database:

wherein, V_iNamely the contribution of the ith patent parameter to the patent value, and M is V_iThe maximum value that can be taken, p is the parameter value, K is the threshold value of the parameter value,

the patent value degree evaluation model adopts the multi-dimensional parameters as evaluation indexes and carries out adaptive adjustment periodically according to dynamic indexes acquired in real time in the patent database.

2. The sorting method according to claim 1, further comprising: and constructing a patent value degree evaluation model according to the big data statistical analysis result of the patent database.

3. The ranking method of claim 2, wherein constructing a patent merit assessment model comprises:

selecting parameters influencing the patent value degree;

selecting a patent data sample according to a preset patent value evaluation standard;

and establishing an evaluation formula reflecting the relationship between the selected parameters and the patent value degree based on the selected patent data samples.

4. The sorting method of claim 3, further comprising: and optimizing a patent value degree evaluation model according to the screening and sorting results.

5. The ranking method according to claim 4, wherein the constructed patent value degree evaluation model includes a first model and a second model different from the first model, and the optimizing the patent value degree evaluation model includes:

taking the target patent evaluated as having the value degree in the expected range by the first model and/or the second model as a reference patent; and

the reference patent is utilized to update the patent data samples used to construct the first model and/or the second model.

6. The ranking method according to claim 3, wherein the constructed patent value degree evaluation model includes a first model and a second model different from the first model, and the ranking method of the patent search results further includes:

calculating the value degree of the preselected verification data by using the first model and the second model respectively to obtain a first evaluation result and a second evaluation result;

comparing the first evaluation result with the second evaluation result;

the first model and/or the second model is adjusted based on the comparison result.

7. The ranking method according to claim 3, wherein the constructed patent value degree evaluation model includes a first model, and the ranking method of the patent retrieval results further includes:

calculating a value degree for the preselected verification data using the first model to obtain a first evaluation result;

calculating the value degree of a patent data sample used for constructing the first model to obtain a second evaluation result;

comparing the first evaluation result with the second evaluation result;

the first model is adjusted based on the comparison.

8. The ranking method according to claim 3, wherein the ranking method of patent search results further comprises: after the value degree is calculated for the target patent according to the constructed patent value degree evaluation model, the evaluation result is subjected to normalization processing to be converted into an evaluation score value within a preset range.

9. The ranking method of claim 3 wherein there are a plurality of selected parameters, and wherein the establishing an evaluation formula that embodies the relationship between the selected parameters and the patent value based on the selected patent data samples comprises:

aiming at each selected parameter, establishing a sub-evaluation formula reflecting the relationship between the parameter and the patent value degree based on the selected patent data sample;

and taking the weighted sum of the sub-evaluation formulas as an evaluation formula for the selected overall parameters.

10. The ranking method of claim 3 wherein the selected parameters are divided into a plurality of groups, and establishing an evaluation formula that embodies the relationship between the selected parameters and the patent value based on the selected patent data samples comprises:

aiming at each selected group of parameters, respectively establishing a sub-evaluation formula reflecting the relationship between the group of parameters and the patent value degree based on the selected patent data samples;

11. The sorting method according to claim 3, wherein the selecting patent data samples according to a preset patent value degree evaluation criterion comprises: selecting a first group of patent data samples and a second group of patent data samples according to a preset patent value evaluation standard,

the establishing of the evaluation formula reflecting the relationship between the selected parameters and the patent value degree based on the selected patent data samples comprises the following steps:

calculating the distribution of the selected parameters in the first group of patent data samples;

calculating the distribution of the selected parameters in the second group of patent data samples;

calculating a comparison result between the distribution of the selected parameter in the first set of patent data samples and the distribution of the selected parameter in the second set of patent data samples; and

and fitting a relational expression between the selected parameters and the comparison result based on the comparison result to serve as an evaluation formula for reflecting the relation between the selected parameters and the patent value degree.

12. The ranking method of claim 3 wherein the establishing an evaluation formula that embodies the relationship between the selected parameters and the patent value based on the selected patent data samples comprises:

extracting a patent corresponding to each value of the parameter;

evaluating the patent value degree of the patent corresponding to each value, and setting the weight of the influence of the value of the parameter on the patent value degree based on the result;

and obtaining an evaluation formula reflecting the relationship between the parameters and the patent value degree based on the weight of the influence of each value of the parameters on the patent value degree.

13. A computer readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the sequencing method of any of claims 1 to 12.