KR20200013130A - Device and Method on Data Processing of Descriptions of the Figure Numbers Corresponding to Figure Numbers on Patent Drawings Using Machine Learning Methodology Based on Using Artificial Intelligence Technology - Google Patents

Abstract

The present invention relates to a method for processing descriptive data of reference numerals corresponding to reference numerals of an image of a patent drawing using machine learning based on artificial intelligence technology and a device thereof. The method comprises the steps of: (a) obtaining a set of descriptive data (DFNijk(Dij(Pi))) of reference numerals for each drawing identifier (Dij(Pi)) of patent (Pi); (b) generating at least one of quantitative information (Ftn(DFNijk(Dij(Pi)))) for each description of reference numerals in patent units with a descriptive data set of the reference numerals for each drawing identifier and quantitative information (Ftn(DFNijk(Pi))) for each description of reference numerals in patent units; and (c) generating a description usage association data set with quantitative information (Ftn(DFNijk(Dij(Pi)))) for each description of reference numerals in patent units and quantitative information (Ftn(DFNijk(Pi))) for each description of reference numerals in patent units. According to the present invention, the method and the device may search for a drawing including a description of reference numerals through search for description of reference numerals.

Description

Device and method on data processing of descriptions of the figure numbers corresponding to figure numbers on patent drawings using machine Learning Methodology Based on Using Artificial Intelligence Technology}

The present invention relates to a method and apparatus for processing description data of reference numerals corresponding to reference numerals of patent drawing images using machine learning based on artificial intelligence technology. Drawings included in the patent drawing image are recognized using the learning model generated by learning the parameters of the learning model under hyper parameter conditions optimized through validation, and the recognized drawings include the drawing. The description data of a reference number corresponding to the reference number is extracted from the specification and is mapped based on a statistical method or rule to generate and process the description data set of the reference number in the drawing unit. Description The present invention relates to a data processing method and apparatus.

Patent applications include patent specifications and drawings. Many patent drawings contain a plurality of marks for reference numerals. However, drawings are provided in the form of images, and reference numerals are often in the form of simple numbers or numbers + letters. In this case, there are the following problems.

First, there is a problem that it is difficult to accurately grasp the object referred to by the reference only by the reference number.

Second, it is not known how much description of any reference number is used in a particular patent drawing unit. Thus, it is not possible to retrieve other drawings of the same specification and other drawings of the same patent in which descriptions of specific reference numerals are used.

Third, precise metering / evaluation data generation for the relationship between the description ij of the patent i vs. reference sign cannot be generated, and the description of the reference sign associated with the patent drawing cannot be used in the patent search results.

Fourth, descriptions of other reference numerals appearing together in the same drawing are not known. For this reason, it becomes impossible to search the patent drawing which the description of two or more code | symbols used together in the same drawing of the same patent, or the patent containing the patent drawing. Furthermore, relationship information (for example, co-occurrence data, etc.) between description i of the reference numeral and description j of the reference numeral used together in a specific drawing cannot be generated in patent drawing units.

In some countries, it may be recommended to separately describe only the explanation of the reference numerals. There are a number of patents in these countries that have regions (eg, 'description of reference' regions) in which descriptions of reference numerals have been collected. However, the data in this area has the following problems. First, there is no information on how much a description of a particular reference number appears in which figure. Second, there is no information indicating how many descriptions of particular reference numerals appear in each drawing or the entire drawing. Third, only the description of reference numerals considered to be important by the specification creator is often described, and a problem may occur in the information processing for the entire description of the reference numerals.

In addition, there are many paragraphs describing the invention idea while quoting the reference numbers from the patent specification. In this case, there are also the following problems. First, even if a particular reference number is cited in a particular paragraph, there is no guarantee that the reference number and the description of the reference number used in that paragraph are necessarily included in that reference number. Second, there is a case in which a specific paragraph refers to two or more reference numbers, in which case it is not possible to determine which drawing contains the reference numerals and the descriptions of the reference numerals used in the paragraph. Third, many paragraphs use reference numerals and descriptions of reference numerals without quoting reference numerals. In this case, it becomes impossible to know which figure corresponds to the description of the reference numeral and the reference numeral used in the paragraph. Therefore, even if the paragraph of the specification is parsed, the specific reference numerals used in the specific drawings and the description of the reference symbols cannot be grasped with high accuracy.

KR1020120032773 A1 KR1020160147950 A1

The problem to be solved by the present invention is to solve the above problems, learning the parameters of the learning model under the hyperparameter (hyper parameter) conditions that have been optimized by the cross-validation (cross-validation) of the character-adaptive image data for each character Recognizing the reference numerals included in the patent drawing image using the learning model generated by using the method, extracting the description data of the reference numerals corresponding to the recognized reference numerals from the specification and mapping by statistical methods or rules A method and apparatus for generating an explanatory data set of reference numerals in units of drawings are provided. On the other hand, the present invention discloses a method and apparatus for secondary processing and using a description data set of reference numerals in the generated drawing units.

In order to achieve the technical problem to be achieved by the present invention, a method of processing the description information of the reference numerals referred to by the reference numerals of the provided patent drawings, (a) i) to be associated with the patent identifier by the patent drawing OCR apparatus After obtaining at least one patent drawing image file corresponding to a patent drawing identifier that may be obtained, at least one drawing containing reference number is recognized in the patent drawing image file, and the recognized drawing includes position information of the drawing containing reference number. A first state in which a drawing containing reference information set corresponding to the drawing containing reference number is generated; And ii) a specification string including a reference number and a description of the reference number is parsed by a reference-description recognition device, and at least one specification-containing reference number and a description of the at least one reference number included in the specification string are recognized. And a patent reference code-description processing server in a state in which a second state in which a specification-specific reference code-description mapping information set for each patent identifier is generated including a mapping relationship between the recognized reference code and the description of the code is performed. (A) obtaining a set of description data (DFNijk (Dij (Pi))) of reference numerals for each drawing identifier Dij (Pi) of the patent Pi; (b) Descriptive weighing information (Ftn (DFNijk (Dij (Pi)))) of the reference numerals in the unit of the drawing as explanatory data sets of the reference numerals by the drawing identifiers and the description-based weighing of the reference numerals in the patent unit (Ftn (DFNijk) (Pi))) generating at least one of the information; and (c) the weighing information (Ftn (DFNijk (Dij (Pi)))) by description of the reference numerals of the drawing units and the reference numerals of the patent units. Generating a description use associated data set of reference numerals by description-specific metering (Ftn (DFNijk (Pi))), wherein the reference numerals including the figures and the reference numerals including the specification are the same character string or the same. A data processing method of a patent reference-description processing server characterized in that it is a string.

In order to achieve the first state, a method of recognizing the reference numerals including the drawings may include: (i1) performing sampling on collected character-corresponding image data; (i2) dividing the sampled character-corresponding image data for each character into n (natural numbers of n> = 3) folders; (i3) A crossover that trains a model using character-corresponding image data for each character in the n-1 folder as training data, and obtains a tester error n times using character-corresponding image data for each character in the remaining folders as test data. Optimizing hyper parameters through cross validation; And (i4) using the collected character-specific image correspondence image data and learning the parameters of the learning model under optimized hyperparameter conditions.

The learning model is at least one selected from a discriminant deep learning model and a generative deep learning model, and the learning hyperparameter is a number of layers and a hidden variable. It is preferably at least one selected from the group consisting of a number, a drop out ratio, a filter size, and an activation function.

In the step (b), the description-specific weighing information Ftn (DFNijk (Pi)) of the reference numeral of the patent unit includes the number (expression frequency) of the description (DFNijk (Pi)) of the reference numeral and the description of the reference numeral (DFNijk). (Pi)) It is preferable that it is any one or more of the number of description existence figures, the description of reference numerals (DFNijk (Pi)) presence drawing density (ratio of the number of DFNijk (Pi) presence figures among all figures).

The association data set of step (c) is to generate an explanatory data set of associative reference numerals, and the generation of the explanation data set of associative reference numerals comprises: (c11) constructing an elementary matrix Aij in unit of drawing; Composition of Ai (row): Dij (Pi), Composition of Aj (column): DFNk

Aij = Ftn (DFNijk (Dij (Pi))), (c12) performing AA ^T processing; And (c13) generating explanatory data of the association reference number including the association information between the description pairs of the reference number.

The associative dataset of step (c) is to generate a similarity pair dataset, and the generation of the similarity pair dataset comprises (c21) constructing an elementary matrix Aij in unit of drawing; Composition of Ai (row): Dij (Pi), Composition of Aj (column): DFNk

Aij = Ftn (DFNijk (Dij (Pi))), (c22) A ^T A Performing a process; And (c23) generating a similarity drawing dataset.

The association data set of step (c) is to generate a similar patent data set, and the generation of the similar patent data set comprises: (c31) constructing a base matrix Bij in patent units;

Configuration of Bi (row): Pi, Bj (column) Configuration: Keyword Bij extracted from patent = Keyword attribute extracted from patent, (c32) performing BB ^T processing; And (c32) performing a BB ^T treatment; And (c33) generating a patent dataset having high similarity.

The keyword extracted from the patent is a description of reference numeral (DFNk) alone or includes a description (DFNk) of the reference numeral, and the keyword attribute extracted from the patent is an attribute (Ftn (DFNijk ( Pi))) alone, or it is preferable that the attribute (Ftn (DFNijk (Pi))) of the description of the reference numeral for each patent is added to the attribute of the keyword.

In recognizing reference numerals for achieving the first state, a description of reference numerals recognized for achieving the second state or a specification including reference numerals for describing the reference state generated to achieve the second state It is preferred that any one or more of the three be used.

The specification-included reference-description mapping information set includes probabilistic mapping data for the description-included reference number and the description of the reference number. It is preferable that the information on the importance and the sequence processing information is included.

In order to achieve another technical problem to be achieved by the present invention, there is provided a patent reference-description processing server for processing descriptive information of reference numerals referred to by reference numerals of provided patent drawings. I) after the at least one patent drawing image file corresponding to the patent drawing identifier that can correspond to the patent identifier is obtained by the patent drawing OCR device, at least one drawing containing reference number is recognized in the patent drawing image file and A first state in which a drawing-included reference information set corresponding to the recognized drawing-included reference code including position information of the drawing-included reference number is generated; And ii) a specification string including a reference number and a description of the reference number is parsed by a reference-description recognition device, and at least one specification-containing reference number and a description of the at least one reference number included in the specification string are recognized. In the state in which a second state in which a specification-specific reference code-description mapping information set for each patent identifier is generated including a mapping relationship between the recognized specification-containing reference number and the description of the reference number is performed, the figure of the patent Pi is performed. A communication unit for obtaining a set of explanatory data DFNijk (Dij (Pi)) of reference numerals according to the identifier Dij (Pi); (I) Descriptive weighing information (Ftn (DFNijk (Dij (Pi))) by description of reference numerals in drawing units as explanatory data sets of reference numerals by drawing identifiers and description-based weighing by reference numerals in patent units (Ftn (DFNijk) (Pi))) a process for generating any one or more of information; and (II) the weighing information (Ftn (DFNijk (Dij (Pi)))) by description of the reference numerals of the drawing units and the reference numerals of the patent units. A process for generating a description use associated data set of reference numerals by description-based metering (Ftn (DFNijk (Pi))); wherein the reference numeral includes the reference numeral and the reference numeral includes the same character string; A patent reference-description processing server is characterized in that it is a character string in which identity is recognized.

According to the present invention, a patent drawing image using a learning model generated by learning a parameter of a learning model under a hyper parameter condition in which character-corresponding image data of each character is optimized through cross validation. Recognizes the reference numerals included in the reference numerals, maps the description of the reference numerals corresponding to the recognized reference numerals by a statistical method, and generates the corresponding relation data between the description of the reference numerals and the figures with high accuracy. Description of reference numerals by using reference data of reference numerals for each reference, generating related information between recommendation of similar drawings, description of reference numerals, description of reference numerals for generating similar patents, or searching for explanation of reference numerals. This included drawing can be searched. Through this, it is possible to effectively search the reference numerals included in the patent drawings, it is possible to support the promotion of the technical idea of the patent through the patent drawings. In addition, when recognizing the reference numerals included in the drawings, the recognition accuracy of the reference numerals may be improved by querying the reference included reference numeral list extracted from the specification. When processing based on statistical probability, even if a description element is included in the specification, an accurate description of the reference numerals can be placed on the reference numerals including the drawings.

1 is an exemplary embodiment of the overall configuration in which the spirit of the present invention is implemented.
2 is an embodiment of a patented OCR device of the present invention.
FIG. 3 is an exemplary embodiment of a symbol-description recognition apparatus of the present invention.
4 is one embodiment implementation of the patent reference-description processing server of the present invention.
5 is an exemplary embodiment of a sign recognition learning apparatus of the present invention.
6 is one embodiment implementation of the data storage device of the present invention.
7 is an exemplary process for implementing the spirit of the present invention.
8 is an exemplary process for recognizing reference numerals including drawings of the patented OCR device of the present invention.
9 is an exemplary process related to the processing of a recognition string containing a number in the patented figure OCR device of the present invention.
Fig. 10 is an exemplary process relating to the synthesis processing of the recognized character string of the patented figure OCR apparatus of the present invention.
FIG. 11 is an exemplary process for improving recognition accuracy by inquiring a recognized drawing containing reference number of a patented drawing OCR device in a specification containing reference number list.
Fig. 12 is an exemplary process relating to the recognition processing of the specification including reference numerals and descriptions of the reference numerals of the reference numeral-description recognition apparatus of the present invention.
FIG. 13 is an exemplary process related to the mapping process between the specification-included reference numeral and the description of the reference numeral in the reference-description recognition apparatus of the present invention.
Fig. 14 is an exemplary process relating to the mapping process of the reference numerals and descriptions of the reference numerals before generation of the mapping data of the reference numerals and the description of the reference numerals of the reference numerals-description recognition apparatus of the present invention.
15 is an exemplary process related to the mapping data generation process for the description of the reference numerals and the reference numerals via the reference numerals of the reference numeral-recognition recognition apparatus of the present invention.
Figure 16 is one embodiment implementation of a process for generating explanatory metering information of a reference numeral of a patent reference-explaining processing server of the present invention.
17 is another exemplary embodiment of a process for generating explanatory data of an associated reference number of a patent reference-explaining processing server of the present invention.
FIG. 18 is another exemplary embodiment of a process for generating highly similar drawing pair data of a patent reference-description processing server of the present invention.
19 is an exemplary embodiment of the process of generating similar patent information of the patent reference-description processing server of the present invention.
20 is an exemplary process for preprocessing a composite view of a patented drawing OCR device of the present invention.
21 is an exemplary configuration of a learning process of the code recognition learning apparatus of the present invention.
22 is an exemplary method in which the patented drawing OCR device of the present invention uses the learning result of the code recognition learning device.
FIG. 23 is an exemplary visualization of data generated after a patent processing OCR apparatus parses patches in a drawing file. FIG.

DETAILED DESCRIPTION OF THE INVENTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced to clarify the objects, technical solutions and advantages of the invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

When applying a technology such as OCR in the present specification, machine learning based modeling such as deep learning, boosting, SVM, and the like may be included. In this case, a person skilled in the art understands that 'modeling' or 'model generation' refers to performing machine running according to a procedure, and is not intended to refer to a mental action such as human educational activity. Could be.

Also, throughout the description and claims of this invention, the word 'comprises' and variations thereof are not intended to exclude other technical features, additives, components or steps. Other objects, advantages and features of the present invention will become apparent to those skilled in the art in part from this description and in part from the practice of the invention. The following examples and drawings are provided by way of illustration and are not intended to limit the invention.

Moreover, the present invention encompasses all possible combinations of the embodiments indicated herein. It is to be understood that the various embodiments of the invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

Unless otherwise indicated herein or clearly contradicted by context, an item referred to in the singular encompasses the plural unless the context otherwise requires. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

As used herein, the term “patent” is a broad concept, and includes not only a patent of a specific country, but a concept including all patents of patents of each country, bibliographic information contained in individual patents or published / registered patents around the world, It should be understood to include processing bibliographic information, information contained in the specification, drawing information, or processing information of this information.

Apparatuses of the present invention typically include computer hardware (eg, client computers and server computers, which may include components of a computer processor, memory, storage, input and output devices, and other conventional computer systems; electronic communication lines, routers, switches). An electronic communication device such as an electronic information storage system such as network attached storage (NAS) and a storage area network (SAN) and computer software (ie, instructions that cause computer hardware to function in a particular way). To achieve the desired performance.

It will be described below in more detail with reference to the drawings.

1 is an exemplary embodiment of the overall structure in which the spirit of the present invention is implemented.

As illustrated in FIG. 1, an implementation of the present invention includes a patent drawing OCR device 100, a reference sign-description recognition device 200, a patent reference sign-description processing server 300, a reference sign recognition learning device 400, The data storage device 500 or the like is used. The patent reference-description processing server 300 may obtain or transmit data while communicating with at least one other device 900 through the wired / wireless network 800. An example of the other device 900 is a user's computer (personal computer or server or other device 900) using the patent reference-explaining processing server 300 or the patent reference-explaining processing server 300. ) May be a server or device that provides or receives data. The patent drawing OCR apparatus 100, the reference numeral-description recognition apparatus 200, the patent reference numeral-description processing server 300, the reference numeral recognition learning apparatus 400, and data storage in one server or cloud system physically The devices 500 may be mounted or connected to each other via a wired or wireless network 800.

Meanwhile, the patent drawing OCR processor 1000 included in the patent drawing OCR apparatus 100, the reference sign description recognition processor 2000 included in the reference sign-description recognition apparatus 200, and the patent reference sign. Any one or more of the patent reference-description combining processor 3000 included in the description processing server 300 and the reference recognition recognition processor 4000 included in the reference recognition recognizing learning apparatus 400 are physically present. You can share the same or a single processor.

It will be readily understood by those skilled in the art that the method of the present invention will be implemented using a combination of computer hardware and software as detailed below.

2 is an exemplary embodiment of a patented OCR device 100 of the present invention.

The patented drawing OCR device 100 includes a patented drawing OCR processor 1000 for performing optical character recognition (OCR) processing specialized in the patented drawing and a patented drawing OCR device 100 for supporting communication of the patented drawing OCR device. 100a).

The patent drawing OCR processor 1000 is a drawing file acquisition process 1100 that performs a function of obtaining at least one individual drawing image file, parsing a drawing file that decomposes at least one drawing image component from the drawing image. Process 1200, and a drawing string recognition process 1300 for recognizing a drawing string in the drawing image.

The drawing string recognition process 1300 includes a drawing text classification process 1310 for classifying individual characters (that can be treated as letters and other characters, such as numbers, alphabets, and the like) among parsed drawing image elements. Generate a string string synthesis process 1320 for synthesizing individual characters in the form of a reference number, a reference recognition process 1330 for recognizing a reference number including a drawing, and generating all information about the recognized reference number including a reference number; It may include storing reference information set generation process (1400).

The reference character recognition process 1330 may include a synthesized character noise remover 1331 for removing noise from the synthesized character, the synthesized character in a specification-containing reference number-description mapping information set including a specification-containing reference number or a specification-containing reference number. It can be recognized as a reference number using the synthesized character inquiry process 1332 to inquire, a reference size predictor 1333 and a reference size predictor 1333 for predicting the size of a reference number in the unit of a drawing image or a patent identifier. However, it may further include a reference noise canceller 1334 that removes reference noise that is not substantially reference numeral.

The drawing file parsing process 1200 is a process of parsing candidate groups of patches that need to determine text from the drawing file. The drawing file parsing process 1200 finds all closed contours in the non-white portion of the drawing file. FIG. 23 shows a part (box processed in green) corresponding to a result of parsing patches in a drawing file.

The drawing character classification process 1310 is a process of classifying drawing characters among the patches of the drawing by using a model trained from the sign recognition learning apparatus 400. The probability value for each character of each patch is shown. For example, the output of up to four characters per patch can be generated.

Tables 1 to 3 are examples of data generated by the drawing character classification process 1310 for drawing images corresponding to FIG. 10 of Korean Patent Application No. 1020167002395.

Each row in Table 1 is information about one patch. x and y are the coordinates of the lower left coordinate of the patch, w is the size of the patch width, and h is the height of the patch. x + w is the x coordinate of the left and right ends of the patch, and y + h is the y coordinate of the top left of the patch.

In Table 2, result is the character with the highest probability among the recognition values for the patch, and neighbor is the character with the low recognition value.

In Table 3, result_prob is the probability of the result recognition value for the patch, and nb number_prob is the probability of the recognition value of the neighbors.

The drawing string synthesizing process 1320 is a process of synthesizing a drawing character having a close position among drawing characters recognized from a patch of each drawing into a single character string. For example, the method of determining whether the positions of the drawing characters are close to each other may be determined to be closer if the distance between the center positions of the drawing characters is smaller than the preset distance (eg, 350% or more of the average width of the characters). have.

The information processing method of the synthesized character noise removing process 1331 is well illustrated in FIG.

The synthesized character noise removing process 1331 determines whether the synthesized character is in the list of reference numerals including the specification for each patent identifier through the synthesized character inquiry process 1332 among the synthesized characters, and if so, the synthesized character is valid. And process. If there is no (No), the compound character is valid when it is determined from the candidate group set of compound characters that it is in the list including reference numerals for each patent identifier, and the compound character is treated as noise.

개 조합의 합성 문자가 될 수 있다.By way of example, one glyph may be up to four glyphs from the glyph classification process 1310. Therefore, if the compound character is two letters long,

Can be a composite of dog combinations.

The synthesized character inquiry process 1332 operates by retrieving a list of specification containing reference numerals for the patent identifier, and then inquiring whether the synthesized character is in the list of specification containing reference numerals for each recognized synthesized character.

The reference sign size prediction process 1333 predicts the reference sign reference size using a composite character valid as the reference reference sign. If there is no valid compound character as a drawing-included reference number in one drawing, the prediction is made based on the previously learned reference size among other drawing images of the patent identifier. An example of code size prediction logic is as follows.

The reference noise removal process 1334 performs a function of determining whether the recognized reference characters are drawing reference codes or noise based on the reference size prediction process 1333. An algorithm for determining noise is as follows.

The reference information set generation process 1400 is a process of generating and storing information of a reference numeral, such as position information and font size of the recognized reference numerals.

FIG. 3 is an exemplary embodiment of the apparatus 200 for explaining a reference sign-description of the present invention.

The reference description description processor 2000 of the reference description description apparatus 200 obtains a specification file for obtaining partial data of a specification including a reference number and a character string corresponding to the description of the reference number in the specification file or the description. Reference-description mapping for generating and storing mapping information between the processor 2100, a reference-containing reference number, and a description of the reference number, and a reference-description recognition process 2110, and a description between the specification-containing reference number and the description of the reference number. An information set generation process 2400 is included. The reference-description recognition processor 2000 may optionally perform a process of processing importance information 2200 for carrying out importance information of a description including reference numerals and / or descriptions of reference numerals, and a specification in which a series relationship is established among the specification inclusion reference numeral sets. A sequence processing process 2300 may be performed to extract and identify an included reference subset and perform a sequence process on the specification included reference number included in the specification included reference subset.

The reference-description recognition process 2110 includes a description recognition process 2120 of reference numerals for recognizing the description of the reference numerals, and a reference-description mapping process 2130 for mapping the description including reference numerals and the description of the reference numerals. It is included. The importance processing process 2200 includes a specification component part (name of the invention, abstract, claims (full, independent claim, dependent claim, specific part of dependent claim, depth of citation of dependent claim, etc.), description of the invention (background, summary, The frequency of appearance of the position recognition processing process 2210 and the description of the reference numerals for recognizing and processing the position where the description of the reference numerals appear in the detailed description of the invention, the brief description of the drawings, the embodiments, the problems, the effects, etc.) A recognition frequency processing process 2220 may be included. The sequence processing process 2300 may be included in a sequence recognition process 2310 and a specification include reference code subset which extract and identify a specification include reference code subset in which a series relationship is established among the specification include reference code set. It may include a sequence categorization process 2320 that performs a sequence process on the specification included reference numeral.

Table 4 below is an exemplary form of data generated by reference numeral-description recognition process 2110 for Korean Patent Application No. 1020167002395 (Korean Patent Application corresponding to US Patent Application No. 14 / 249,716).

In the reference numeral 26 of Table 4, the probability recognized as a light emitting diode is 0.8, and the probability recognized as a diode is 0.2. The reference-description mapping process 2130 recognizes the specification including reference numeral 26 as a light emitting diode.

4 is an exemplary embodiment of a patent reference-description processing server 300 of the present invention.

The patent reference-explanation processing server 300 is a description of the reference information generation process 3100 of reference numerals for generating various weighing information for each description of the reference numerals, the description of the reference numerals as a search target, Description search support process 3200 for providing a search result of a patent having a drawing including a description of the reference number or a drawing including a description of the reference number as a search result when a keyword corresponding to the description of? Description of Reference Numbers for Using the Description of Reference Numbers to Create Various Related Data The associated data generation process 3300, which provides various data, information and contents related to the description of the reference numbers to the requestor via wired or wireless networks. The description of the content information providing process 3400 includes a patent reference-description processing processor 3000 is driven. And.

In the description weighing information generation process 3100 of the reference numerals, the description-based weighing information generation process 3110 of the reference numerals for generating the weighing information for each description of the reference numerals, and processing the weighing information for each description of the generated reference numerals in drawing units. Description set generation process (3120) of the description unit reference numeral to be described, description set generation process (3130) of the description of the patent unit reference numeral processing the weighing information by description of the generated reference number in the patent unit An information generation process 3140.

In the description use association data generation process 3300 of the reference numerals, the description data generation process 3310 of the associated reference numerals for generating the description data of the associated reference numerals, and assisting the user to find a drawing with high similarity by using the description of the reference numerals. Similar drawing information generating process 3320, and similar patent information generating process 3330 to assist in finding a patent having high similarity by using the description of the reference numerals.

5 is an exemplary configuration of a sign recognition learning apparatus 400 of the present invention.

The code recognition learning apparatus 400 includes a code recognition learning processor 4000, a learning data storage device 4100, and a communication unit 400a of the code recognition learning apparatus 400. The reference recognition learning processor 4000 includes a learning process 4100, which includes a hyperparameter optimization process 4110 and a learning model parameter learning process 4120. The hyperparameter optimization process 4110 includes a training data object recognition modeling learning unit 4111 and a validation data object recognition modeling verification unit 4112. The learning model parameter learning process 4120 includes an entire data object recognition modeling learner 4121. The learning data storage device 4100 includes a learning image data unit 4110 for each character and a learning result storage unit 4100.

The letter-by-character learning image data unit 4110 stores dozens to thousands of character images for each character (number, alphabet, other characters, etc.) used in the patent drawings. The text image is preferably cut out from the patent drawing, and may be cut out by numbers or alphabets from other image sources. In this case, it is preferable to prepare a character image for each font type of the reference numeral used in the drawing image. In particular, in the case of handwriting, even if deep learning is used, it is more preferable to prepare as much as possible because more character images are advantageous for recognition. In order to recognize image data and handwriting of all computer fonts for each character well, the learning images for each character may be used by directly tearing down each character from the written reference numerals in the patent drawings.

The learning model can use discriminant deep learning model (ex: deep convolutional neural network, deep neural network, etc.) and can create learning model using generative deep learning model (ex: variational autoencoder, generative adversarial network). On the other hand, other machine learning models such as boosting or SVM can be used. Boosting is a nonlinear model, an algorithm that creates a powerful classifier by ensemble several simple decision tree models.

In the deep learning model, hyper parameters such as the number of layers, the number of hidden variables, the drop out ratio, the filter size, and the type of activation function may be used. The hyperparameters of boosting include the depth of crystal tree model, how many crystal tree models to use (number of trees), and the shrinkage parameter for how much to move in the gradient direction when learning. have. These hyper parameters are optimized through the hyper parameter optimization process 4110.

In the present invention, three methods may be used singly or in combination to optimize hyperparameters. The first method is grid search. Since Grid Search performs cross validation on all possible combinations of hyperparameters, you can most certainly optimize the hyperparameters. Grid search is an example of hyperparameter 5 ~ 20 layers, hidden variable 0.3 ~ 0.9 times input data size, drop out ratio 0 ~ 0.9, activation function relu, sigmoid, Leaky ReLU, PreLU, You can perform cross validation by assigning it like RreLU, ELU, etc. That is, grid search is performed for each combination of the granular range and type of each hyperparameter (activation function only). At this time, the number of layers and hidden variables are changed one by one, the drop out ratio is changed by 0.05 or 0.1, and cross validation is performed using each activation function for each variation.

The second method is random search. Random search is equivalent to grid search in terms of method. The difference is that Grid Search performs cross validation using the assigned hyperparameters, while Random Search performs cross validation using random values randomly selected for each hyperparameter.

The third method is Bayesian optimization. This method is a method of assigning and optimizing a prior distribution of hyperparameters.

In the hyper parameter optimization process 4110, the hyper parameter may be optimized by mixing a random search and a grid search. In the first step, randomly assign hyperparameters, optimize using cross validation, and find the range of primary values for each hyperparameter that performs the best in the validation set. Hyperparameters can be finely subdivided within the primary range for each hyperparameter, and grid search can be optimized by subdividing hyperparameters.

One of the cross validations is validation data and the other is training data. The training data object recognition modeling learning unit 4111 performs modeling on the training data for each set hyperparameter. Meanwhile, validation model object recognition modeling verification unit optimizes hyperparameters by applying deep learning models modeled on training data to validation data.

The entire data object recognition modeling learner 4121 learns the parameters of the actual machine learning model under optimized hyperparameter conditions.

If the specific learning is carried out using a language package such as R, Physon, or a machine learning program library such as Google Tensorflow or Caffee, this part is obvious to those skilled in the art, and detailed description thereof will be omitted.

21 shows an exemplary embodiment of a learning process of the code recognition learning apparatus 400 according to the present invention.

The reference recognition recognition apparatus 400 performs sampling (S161) from the collected character-corresponding image data stored in the character-based learning image data unit 4110. Subsequently, the hyperparameter optimization process 4110 of the reference recognition apparatus 400 divides the sampled character-corresponding image data for each character into n (n = n3 natural numbers) folders (S162), and then (S162). , the model is trained using the character-corresponding image data of each character of the n-1 folder as the training data, and the test data is used as the test data of the character-corresponding image data of the remaining folders as the test data. Hyper parameter optimization (S163) through cross validation). Next, the learning model parameter learning process 4120 of the reference recognition apparatus 400 uses the collected character-corresponding image data for each character and under the optimized hyperparameter condition, a deep learning model, a boosting model, or an SVM. Learning the parameters of the model (S164). n is preferably 5 or 10.

Next, learning the parameters of a learning model is demonstrated.

In the case of the neural network model of deep learning model, when X is input data, X moves from the bottom layer to the upper layer through “reLu (X% *% W + b)”, where W and b belong to parameters. You will learn this. In the case of the boosting model, on the other hand, when the depth of the decision tree and the number of decision trees to be used are determined, the decision tree suitable for the data is learned.

The learning result storage unit 4100 includes a processing statement for image recognition processing, which is a learning result.

The method of using the learning result by the patent drawing OCR apparatus 100 is illustrated by way of example in FIG. 22.

The patent drawing OCR device 100 executes a processing statement S171 for image recognition processing, which is a learning result generated by the reference sign recognition learning device 400. Subsequently, the drawing character classification process 1310 of the patent drawing OCR apparatus 100 inputs patch data generated from the drawing image into a processing statement (S172), and generates one or more recognition values for each patch (S173). Next, the patch character classification processing result for each patch is stored (S174).

The drawing character classification process 1310 of the drawing character classification process 1310 applies a processing statement for the image recognition processing to a patch generated by the drawing file parsing process 1200 from the drawing image. One example of the result of applying a processing statement to a patch is in Tables 1 to 3 (particularly Table 2).

6 is an exemplary embodiment of a data storage device 500 of the present invention.

The data storage device 500 of the present invention includes a file storage device and a DB device. The file storage device may include a drawing file storage unit and a specification file storage unit, the drawing file storage unit may include an original drawing file storage unit, and optionally, a processed drawing file storage unit. Patent full text data typically includes a specification file in a markup language such as XML, and, if there is a drawing, consists of one or more drawing files. In the drawing file, two conceptually separated drawings (eg, fig1 and fig2) may be physically included in one image file. The processed drawing file is a drawing file processed according to the present invention. The machining drawing file may include two machining drawings. First, a processing drawing in which the descriptive arrangement data of the reference numerals is displayed directly or indirectly (to connect, interlock, or combine) with reference to the drawing including the reference included in the original drawing file according to the present invention. File. Second, when a physical drawing includes two or more drawings that can be conceptually separated, the drawings are physically divided and the processed drawing file to which the present invention is applied.

The DB apparatus includes a drawing-based data unit 5210, a specification-based data unit 5220, and a patent reference code processing data unit 5230, and optionally, a processing rule data unit 5240 or an aggregate data unit 5250. It may further include. The drawing-based data unit 5210 includes data extracted from a drawing for each drawing file of each patent. In the drawing-based data unit 5210, attribute information of a drawing including reference number and a drawing including reference number for each drawing including reference number is stored. The specification-based data unit 5220 includes a specification including reference numerals, descriptions of reference numerals, explanatory attribute data of reference numerals, and the like.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

7 is an exemplary process for implementing the spirit of the present invention.

As illustrated in FIG. 7, the inventive concept of the present invention is a patent inputted by the reference numeral-explanation recognition apparatus 200, in which a reference numeral including a drawing is processed in a drawing input from the patent figure OCR apparatus 100. In the specification, a description including reference numerals and descriptions of reference numerals are recognized (S12), and description batch data of reference numerals are generated (S13) by the patent reference-explanation processing server 300, and patent reference numerals- Description The processing server 300 is implemented in such a manner that the processing drawing data is generated by the real-time request or batch processing method from another apparatus 900 or the base information that can be generated the processing drawing data is provided (S14).

As illustrated in FIG. 8, the patent drawing OCR processor 1000 performs character recognition processing on individual patent drawings to generate character recognition information for each individual character (S21), and synthesizes the recognized individual characters. In operation S22, the synthesized drawing reference numeral candidate is mapped with the specification including reference numeral set to determine the reference numeral including the reference numeral (S22), and the attribute information of the reference numeral corresponding to the determined reference numeral is generated (S26). .

The patent drawing OCR processor 1000 optionally calculates a mapping ratio mapped to a specification including reference code set among drawing reference code candidates, inquires whether the mapping rate is equal to or greater than a set reference, and otherwise performs orientation processing on the drawing. In addition, the process of S21 to 23 is performed on the oriented drawing. The reason for the alignment process is necessary because most of the drawings are arranged in the horizontal direction, but the drawings arranged in the vertical direction can also be considerable. This is because, in the case of drawings arranged in the horizontal direction, the reference numerals including the drawings are mostly written in the horizontal direction, and in the case of the drawings arranged in the vertical direction, most of the drawings including the drawings are written in the vertical direction. The orientation process first determines the angle to be oriented among the right turn 90 degrees, the right turn 180 degrees, and the right turn 270 degrees according to the drawing characteristics of each country.

Optical character recognition recognizes characters including numbers in image data. Detailed techniques for the OCR itself are natural to those skilled in the art (a large amount of books, papers, published / commercial software, etc.) for the OCR are omitted from the description of the present invention. However, in the present invention, when recognizing character strings included in patent drawings through conventional OCR, the following three problems occur.

First, although the recognized character string may be a reference numeral including a drawing, there may be a plurality of non-drawing reference numerals. A good example is the string content in a box in a process diagram. These are strings, but not reference numerals, including drawings. Therefore, it is a key problem to distinguish between the drawing reference numeral and the non-drawing reference numeral among the recognized character strings.

Second, various lines or shapes constituting the drawings are often looked like numbers or letters when separated in large or small block units. OCR often scans an image to determine whether the block being scanned is a letter or a number. At this time, the vertical line is often recognized as a lowercase letter of "I" or "L", and the line that bends vertically to the left is recognized as "J" and the line bent to the left and right as "S". In many cases. In fact, it is not a reference number, but a misrecognition of a character due to the shape of a part occurs not only in the image constituting the drawing but also in a line connecting the description of each part of the drawing with the reference number.

As such, when the conventional OCR technology is applied to the patent drawing as it is, a great deal of error recognition occurs. The present invention includes a technical idea that systematically solves this problem.

Of course, when performing optical character recognition, machine learning may be used to increase the recognition rate for numbers, letters and symbols that appear frequently in patent drawings. To this end, first, a large amount of data is collected by numbers, letters, and symbols (e.g., partial images that can correspond to the number "2" to be recognized as the number "2" (extract only the portion corresponding to the number 2 in the drawing image). Are created and collected by storing them as separate image files). Then, the collected data can be divided into a training set and a test set (in some cases, a validation set can be prepared separately), and a dip such as a convolution neural network (CNN) can be used. Learning is applied by applying a learning algorithm or other machine learning algorithm to generate an image recognition algorithm (model) specialized in the recognition of reference numerals appearing in patent drawings. The generated image recognition algorithm is then included in the optical character recognition routine. In the optical character recognition process, an image recognition algorithm (model) specialized for the recognition of reference numerals may be used to determine whether a particular image block is a letter or number and which character or number. Through an image recognition algorithm (model) specialized in the recognition of reference numerals, it is possible to recognize the reference numerals more accurately and comprehensively than conventional optical character recognition algorithms.

Third, the present invention may use a KNN (K nearest neighborhood) algorithm. In order to use KNN, a large amount of character-adaptive image data is prepared. Image recognition algorithms, such as the KNN algorithm, often do not recognize handwriting or characters using unusual fonts. To this end, it is highly necessary to use learning algorithms such as deep learning, boosting, and SVM. In the case of handwriting or reference numerals in various fonts, image recognition through machine learning is more effective.

FIG. 9 is an exemplary implementation of the string recognition process including numbers in the figure string recognition process 1300 of the present invention.

As illustrated in FIG. 9, the drawing string recognition process 1300 searches for at least one second individual character to be synthesized with the recognized first individual character using a composition processing rule to generate a drawing-containing reference candidate ( S31), it is determined whether a number is included in the recognized character string (S32), and if yes, the recognized character string is extracted (S33), and if No, the recognized character string is discarded (S38). The patent drawing OCR apparatus 100 may optionally perform the division (S34) process of the string including numbers after step S33. The patent drawing OCR apparatus 100 determines whether the number-containing string divided after S33 or S34 belongs to the decision pattern rule of the drawing reference numeral (S35). Only the split string matching the pattern rule is extracted (S36), and when No, the string including the number is discarded (S37).

Many character recognition algorithms recognize most characters one by one. Therefore, in order to generate one character string, it is necessary to combine the individual recognized characters. Drawing Included Reference numerals are typically not very long (within 8 to 10 characters), most of which have no spaces, and sometimes contain special characters.

In the drawing string recognition process 1300, when a specific block is a character (a character such as a character block, a number, or an alphabet becomes one character block), the character block and another block at a predetermined position are characters. If is, perform the process of connecting the two blocks. A character of a character block within a predetermined distance in consideration of the size of a character recognized in the character block in the direction of the character recognized in the character block may be an example of a preset position. Size information may be generated for each recognized character. The size information may be determined as the size (coordinate value) of the recognized character block (square).

Since the set of coordinate values corresponds to the character block, the center coordinate value can correspond to the character block. The distance between characters is also character spacing, which is typically much smaller than the size of one character (for example, a space). When the recognized character is a character that writes horizontally, such as English or Korean, the patent figure OCR device 100 is within a preset position to the right if the direction of the character is a normal direction (eg, oriented in the horizontal direction). For example, the characters in the chapter 33-200%) can be connected to the process. For example, a character block at 33 to 200% of the longest length among the horizontal length or the vertical length of the character block may be synthesized based on the average value of the character block. On the other hand, for example, another character block that comes within 10 to 150% of the length value of the length or length of a specific character block may be synthesized. These rules can be part of the synthesis processing rule.

On the other hand, when the recognized character is oriented in the vertical direction, the concatenation process is performed on the character in the block in the vertical direction. When the above concatenation process is performed, a string block is generated. The patent figure OCR apparatus 100 may recognize a character string when the connection between the character blocks is completed.

On the other hand, not only characters but also marks (for example, "-" or "_", etc.) frequently appearing in the reference numerals in the connection process can be treated as characters. The drawings are made up of a myriad of lines, many dotted lines, etc., so that there can be a large number of parts of the drawings that can be recognized as "-" or "_". Therefore, when a specific part of the drawing is recognized as "-" or "_", the synthesis process with the character blocks before and after is primarily performed, and the synthesized character string is inquired into the reference code set including the specification. Only the character string included in the specification included reference numeral set may be treated as the reference numeral included.

A block to be synthesized for each character block is searched for, and the synthesis process can be performed for the longest within the limit to which the synthesis process rule is applied. For example, when there is a reference numeral "153" including the drawing, it is preferable to perform the synthesis process up to "153" rather than "15". The binding direction may be treated as the basic direction from the left-> right direction. Meanwhile, the preferred bundle direction may vary according to the orientation of each country or drawing (horizontal drawing, vertical drawing, etc.). Of course, the bundle direction may be left <-> right. When a specific character block is included in the bundle, the grouping process from the character block may be omitted. That is, when "5" is tied to "1", it will be desirable not to attempt a new bundle with "5" as the initial starting point. On the other hand, it will be obvious that the synthesis processing rule can be applied in the left-right direction based on "5".

On the other hand, there are a lot of spaces between "153" and "267," such as "153 267", etc. At this time, "5" is grouped based on "1", and "3" is grouped based on "5". The processing can be performed, and since there are a lot of spaces between 3 and "2", after the binding of "5" and "3", the binding process can be aborted. The process of rebinding by reference could begin.

The drawings contain a significant number of non-reference characters. A typical example is a word or word that refers to a string or a specific object contained within a block diagram. Reference numerals are usually composed of numbers only or numbers are major parts, but such strings do not contain numbers or numbers are not major parts. These strings need to be excluded from the mapping.

An information process for generating a candidate of a reference numeral including a drawing by the drawing string recognition process 1300 will be described with respect to a string that has undergone a string recognition process. The drawing string recognition process 1300 searches for at least one second individual character to be synthesized with the recognized first individual character by using the combining processing rule to generate a drawing-containing reference candidate (S31), and includes the drawing. It is determined whether the number is included in the reference numeral candidate (S32), and if it is Yes, the recognized character string is extracted (S33), and if it is not, the recognized character string is discarded (S38). In this case, when the recognized character string is a long string (when a space is included, etc.), a process of extracting or splitting only a string including a number from the string is performed (S34). If a space is not included in the recognized string, or if the unit of recognition or extraction is a single word representation in the string recognition step or the string extraction step, the process of dividing the string (S34) may be unnecessary. The single word expression is a string without spaces, and the string is a string consisting of only numbers, alphabets, or letters, or a string including a predetermined drawing including a reference sign such as "-" or "_". Preferably, in step S31, the recognition processing of the character string may be performed in a single word representation unit.

The drawing string recognition process 1300 performs an inquiry (S35) processing of whether the number-bearing string or single word representation belongs to the decision pattern rule of the drawing-bearing reference number, and when Yes, the decision pattern rule of the drawing-bearing reference number. Only split strings matching the extracted string are extracted (S36), and in case of No, the string including numbers is discarded (S37).

The decision pattern rule of the drawing reference numeral may include at least one of the following rules.

First, strings that do not contain numbers are not reference numerals, including drawings.

Second, if a string contains numbers and letters, the following sub-criteria applies:

1) In the case of starting with a letter, the number of letters of the starting letter is less than or equal to the number of letters composing the number.

2) In the case of ending with a character, the number of characters of the ending character is less than or equal to the number of characters constituting the number.

Third, the drawing-included reference sign may not be the beginning or the end of a character string.

Fourth, the total length of the string is within a preset limit (eg, 6-8 characters).

Fifth, punctuation or symbols that are not symbols including drawings are excluded from the drawings including drawings.

The decision pattern rule of the reference numeral including the drawing may be stored in the processing rule data unit 5240.

On the other hand, when the characters are connected using individual characters recognized in a specific drawing, candidates of more than one kind of drawing reference numerals may be generated according to a connection method or a policy. For example, there may be a string "25" very close to a string recognized as "A12" in a particular drawing. Ambiguity exists at a distance, such as when the distance between "A12" and "25" is longer than the flatness between "A", "1", and "2" that make up the interior of "A12". Frequently there are cases. In this case, the reference numerals including drawings may be two, such as "A12" and "25", respectively, but may also be "A1225". If such ambiguity exists, it is a question of how far each string is concatenated. In the present invention, unless the specification-containing reference numeral is preferentially used in the character string recognition step, the patented OCR apparatus 100 is a candidate for the character string "A12" and "25" and "A1225". Create In this case, it will be apparent that the attribute information of the drawing reference numerals, such as the position and font size, is generated for each of the strings "A12" and "25" and "A1225".

The drawing string recognition process 1300 processes a string that passes the decision pattern rule of the drawing inclusion reference numeral as a candidate of the drawing inclusion reference numeral. In one image, candidates of n reference numerals may be generated, and these constitute a candidate set of reference numerals.

On the other hand, among the strings that pass the decision pattern rule of the drawing reference numerals, there are many cases that are not the drawing reference numerals. Some of the reasons are as follows.

First, it is a sentence type and syntax string in the drawing. For example, many words including numbers among the words constituting the various sentences, phrases, and the like in the process diagram pass through the decision pattern rule of the reference numeral including the drawing. For example, when there is a character string "A380", even if the character string represents an aircraft model of Boeing Corporation, it passes the decision pattern rule of the reference numeral including the drawing.

Second, even if an image mistaken as a number or letter is connected with a number or letter, it is highly likely to pass the decision pattern rule of the reference numeral including the drawing. As illustrated in FIG. 30, when there are many reference numerals in one drawing, the drawing also often has a complicated configuration. At this time, in the drawing, a plurality of drawing reference numerals are arranged throughout the image (high density of drawing reference numerals in the drawing image), and the diagram connecting the drawing reference numerals with components becomes complicated. In this case, the part of the image that is mistaken as a character is connected with the actual character to form an invalid string.

Third, the misrecognized image part portions are connected to each other and output as strings, which can pass the decision pattern rule of reference numerals including drawings. For example, a case in which two vertical lines are bent to the left and right by the numbers "1" and "1" is recognized as "3" and a string "113" is produced.

The present invention proposes two methods for solving the above problems.

The first method utilizes the specification including reference number extracted from the specification, and processes the candidates of the extracted specification including reference number and the obtained reference including reference number, and processes only the candidates of the mapped reference including reference number as the reference including reference number. This is a post-mapping method. This method is illustrated in FIG.

The second method is a previous-stage filtering method that utilizes the specification-included reference numeral in the step of generating the candidate set of the reference-included reference numeral. By using the specification reference numerals, 1) the type of the string to start is determined, and 2) the character string that can come after a specific character. Therefore, the possibility of false recognition from the character recognition stage can be much reduced. Not heard. In particular, the concatenation of a string block has a special effect when deciding whether to connect a specific string block with another string block. This ensures that strings that do not become drawing containing reference numbers without a connection (drawings that are not in a specification containing reference number) are treated as if they were accidentally treated as a drawing containing reference (by means of a drawing containing a reference number in a specification containing reference number and consequently The same string representation can be avoided as a result of which the description of reference numerals is combined. To this end, the step of querying the specification including reference code set in the drawing reference code candidate generation step must be processed, and for this purpose, the specification including reference code set must be generated in advance.

Accordingly, it may be preferable to perform the processing of the reference numeral-explanation recognition apparatus 200 before performing the processing of the patent drawing OCR apparatus 100.

On the other hand, when there is an expression referring to a figure such as "figure + number" or "degrees + number" in the paragraph on the specification constituting the specification containing reference code candidate set, the specification including reference number coming out of the paragraph is specialized for a specific drawing. It is treated as a high specification reference number, the reference number and the reference number may be mapped and stored. Through this, the specification candidate reference set may be constructed in drawing units. If there is a reference included candidate set in drawing units, utilizing this data enables more precise processing of the previous stage filtering of the candidate set of included reference numbers.

In the case of mapping the specification inclusion reference numeral to the candidate set of actual reference inclusion reference numerals, there are many cases of ambiguity. In particular, various ambiguities exist when the drawing is blurred, the resolution is low, the font is not clean, or the drawing includes a lot of reference numerals. In this case, the specification-included reference code candidate set generated for each drawing may serve as an additional weighting factor in the processing of the candidate of the drawing-included reference code extracted from the specific drawing.

For example, if the accuracy of one character of the candidate character string of a specific drawing reference numeral recognized in a specific drawing is low (e.g., the letter "I" or the number "1" is not clearly classified), The accuracy of the recognized character can be improved by referring to the specification including reference numerals generated in the description part of the specification related to the specific figure. As a result, the recognition rate of the string is increased.

10 is one embodiment implementation of the synthesis processing process of the code recognition process 1330 of the present invention.

The reference recognition process 1330 searches for the recognized first individual character and at least one or more second individual characters to be synthesized using the combining processing rule (S41). When there is a first individual character, the recognition value of the first individual character or the recognition value of the second individual character is not very important in the search for the second individual character to be synthesized. For example, if the recognition value for the first individual character is the number "1" or the lowercase letter "i", there is no significant effect in searching for the second individual character to be combined with the first individual character. This is because there may be drawing reference numbers starting with the number "1", but there may also be drawing reference numbers starting with the lowercase letter "i".

The reference recognition process 1330 searches for whether there is at least one second individual character to be synthesized with the first individual character by applying a synthesis processing rule (S42). The second individual character may be two or more characters. The finally synthesized character string may have the form of "first individual character + second individual character + third individual character + ... nth individual character". When the nth individual character is included, "the first individual character + the second individual character + the third individual character + ... the nth -1 individual character" is at least excluded from the result of the combining process. In other words, the longest one becomes a candidate of the reference numeral including the drawing when the composition process is performed. For example, when there is "S1234" at a particular position in the figure, "S" is recognized, and "1", "2", "3" and "4" are next to "S", respectively, and this is " If it is synthesized by S1234 ", it means that S12 and S123 are excluded. Of course, if there is "S123" elsewhere in the same figure, it will be natural that the reference numeral candidates generated by the reference recognition process 1330 include "S1234" and "S123".

The reference recognition process 1330 includes at least one drawing by combining the first recognition value of the first individual character and the second recognition value of the at least one second individual character, if any, as a result of the search for the second individual character. A reference numeral candidate is generated (S43). Since the character recognition may not always be correct, one or more recognition values and recognition evaluation information (recognition probability / accuracy / recognition level, etc.) may correspond to each character recognition object. For example, a portion of an image having a shape of "1" may be recognized as a number "1", may be recognized as a lowercase "i", and the recognition algorithm may generate recognition evaluation information for each recognition value. . Therefore, when a plurality of recognition objects each having at least one or more recognition values are subjected to the synthesis process, at least one or more drawing candidates can be generated. Ranking or comprehensive recognition evaluation information may correspond to each drawing reference numeral candidate.

The reference recognition process 1330 inquires (S44) from the high priority drawing containing reference code candidate based on the ranking or the comprehensive recognition evaluation information (S44). Is the generated drawing include reference candidate in the specification containing reference set? Inquired (S45), if there is one, it is determined by reference numerals including drawings (S46). At this time, a case where one candidate with reference numerals may correspond to one or more specification reference numerals may of course occur. Subsequently, the patented drawing OCR apparatus 100 performs the mapping data (1: 1 or 1: n) generation (S47) of the drawing containing reference numeral and the description of the reference numeral through the reference numeral. If the query result drawing back reference numerals are not included in the specification inclusion reference numerals, the mapping data may not be generated, or a greed search may be selectively performed (S48). Greedy search refers to processing all combinations of all recognition values for the object to be stingy or higher than a predetermined criterion as drawing reference numeral candidates, and performing inquiry (S46) processing for the specification including reference numeral sets.

Normally, reference numerals including drawings of one character / character are not used much, but they are not used at all. Therefore, care must be taken when there are no second individual characters as targets. Because there are a myriad of errors in character recognition, many image elements (lines, dashed lines, curved portions, etc.) constituting the drawing are separated into small boxes so that they are recognized as characters. This causes a lot of noise. For example, if a part of the vertical line is recognized in the form of a box, it may be recognized as a number "1", an uppercase letter "I", or a lowercase letter "L", and in particular, in a set of numbers, the number "1", "I" or " When there is a lowercase letter "L", noise is generated in many parts of the figure.

If there is no second individual character, the reference recognition process 1330 queries whether the first individual character deviates much from the average size (S49-1). The patent drawing OCR apparatus 100 may calculate the average size of the drawing-containing reference numerals to be recognized. The patented drawing OCR apparatus 100, when there is a specification-containing reference number that is mapped to the drawing-containing reference number candidate 1: 1 in the step S22, the size information of individual characters for the drawing-containing reference number candidate Obtain the average size. This is because the font sizes of the drawings including the drawings included in one drawing or one patent do not differ greatly.

When the first individual character deviates much from the average size, the reference character recognition process 1330 performs the noise processing (S49-2). Otherwise, the reference character candidate includes the generated drawing reference numeral candidate. The set is inquired (S49-3). Is the generated drawing include reference candidate in the specification containing reference set? If (S49-4) is inquired, it is determined by reference numerals including drawings (S49-5). If not, it is processed as noise (S49-6).

FIG. 11 is an exemplary embodiment of a process for generating attribute information of a reference numeral including the reference information set generation process 1400 of the present invention. The reference information set generation process 1400 may query a candidate string of a reference numeral including a reference number to a reference candidate reference set in operation S51, and then refer to a candidate string of a reference numeral including a reference numeral or a reference numeral reference candidate It is determined whether there is a set (S52), and if present, the candidate character string of the drawing reference numeral is recognized as the drawing reference numeral (S53), and if not, the candidate string of the drawing reference numeral is discarded (S54). Can be processed.

Processing of attribute information of reference numerals, including drawings, will be described. The reference information set generation process 1400 generates and stores attribute information of the reference numeral reference candidate when recognizing a character string corresponding to the reference numeral reference candidate. Attribute information of the drawing-containing reference numerals includes the position of the drawing-containing reference numerals (four vertex coordinate values corresponding to the rectangular drawing-containing block), the size of the drawing-containing reference numeral font, the orientation (rightward, upward, downward, etc.), Configuration (numbers only, letters + numbers, etc.), lengths of character strings (number of characters), specific reference numbers in which drawing reference numerals appear, and the total number in which drawing reference numerals appear.

Next, generation of description-included reference numerals and description data of the reference numerals of the reference-description recognition apparatus 200 will be described. It is preferable that the generation of reference numerals and description data of reference numerals is processed before the recognition of reference numerals in the figures.

FIG. 12 is an exemplary embodiment of a process for generating a specification including reference numerals and description information of reference numerals of an apparatus 200 for describing a description.

Reference numerals and description data of the reference numerals are generated by processing the specification data. In some cases, the specification-indicated reference data is in a markup language such as XML, or in the case of simple text without tags. The apparatus for recognizing reference numerals 200 may obtain a specification (S71) and determine whether the reference numerals are written in a markup language (S72). When the specification include reference number is in markup language, the tag portion of the specification include reference number corresponding to the reference number is recognized, and the specification include reference number is extracted from the tag portion of the specification include reference number (S73), and the specification is included. The character string at a predetermined position with the tag portion of the reference numeral is extracted (S74), and a description candidate of the reference numeral is determined from the extracted character string (S75).

For example, suppose that the following specification (content extracted from the markup language file of US Patent Application No. 14 / 249,716) is given.

Hybrid oxide-silicon thin-film transistor structures such as Illustrative thin-film transistor structures <b> 302 </ b> of <figref idrefs = "DRAWINGS"> FIG. 10 </ figref> may be used in forming CMOS-type circuitry in display driver circuitry such as gate driver circuitry <b> 18 </ b> and demultiplexer circuitry <b> 20 </ b>. As shown in <figref idrefs = "DRAWINGS"> FIG. 10 </ figref>, structures <b> 302 </ b> may have a polysilicon layer <b> 308 </ b> that is formed on substrate <b> 24 </ b>. P-channel active area <b> 310 </ b> may be formed under gate <b> 312 </ b>. Gate insulator layer <b> 306 </ b> (eg, silicon oxide) may separate gate <b> 312 </ b> from silicon channel region <b> 310 </ b> in silicon layer <b> 308 </ b >. Dielectric layer <b> 302 </ b> (e.g., sublayers of silicon oxide and silicon nitride) may cover gate <b> 312 </ b>. Dielectric layer <b> 306 </ b> may separate gate <b> 312 </ b> from overlapping oxide layer <b> 312 </ b>. Oxide layer <b> 312 </ b> may be a semiconducting oxide such as IGZO material.

From the string "<figref idrefs =" DRAWINGS "> FIG. 10 </ figref>", we can see that this paragraph relates to "Fig. 10". On the other hand, in the above example, it can be seen that the reference numerals exist between the <b> tags, and "<b> string </ b>" becomes the tag portion of the reference numerals. The apparatus for recognizing reference numerals 200 extracts the reference numerals by processing the tag part of the reference numerals. Subsequently, the apparatus for recognizing the reference numeral-description extracts a character string at a predetermined position with the tag part of the reference numeral reference numeral S74. For example, the apparatus for recognizing reference numeral 200 may read a character string in a reverse direction from a starting point (the preceding <b> tag) of a reference numeral including a specification and satisfy a description candidate condition of a reference numeral as in the following example. Extract the string to the position

If the string is English,

1) String just before meeting the first indefinite article

2) the string before the first said said, said, or the

3) a string up to immediately before the description candidate related expression of a predetermined reference number, such as at least a number, one or more, etc.

4) If there is no corresponding string in the condition 1) or 2) up to the nth word, the string up to the first plural noun

5) String up to the beginning of the sentence, without 1), 2) and 3)

Description candidate conditions of reference numerals may be stored in the processing rule data unit 5240.

If the string is Korean, not English, the expression corresponding to the or the said (for example, "above") is applied instead of the definite article or the said.

The apparatus for recognizing reference numeral 200 may obtain a specification (S71) and determine whether the specification is written in a markup language (S72). When the specification is not in a markup language, the reference sign-explanation recognizing apparatus 200 applies the specification-containing reference identification rule, extracts the specification-containing reference number (S76), and references the specification-containing reference number. In operation S77, a character string at a predetermined position with the specification reference number is extracted (S77), and a description candidate of the reference number is determined from the extracted character string (S78).

The above-mentioned reference numeral identification rule is 1) when there is an explicit regularity such as a method of expressing the reference numeral reference code on the specification such as "parenthesis + specification included reference number + brackets", etc., the rule is applied and processed. do. (In a Korean patent specification, reference numerals are conventionally expressed in parentheses.) Since a parenthesis often includes other character strings other than the specification reference numerals, among the character strings included in parentheses, reference numerals are included. To determine, the following conditions apply: On the other hand, when there is no conventional specification code reference identification rule, such as parentheses, the same conditions can be applied even in the case of a non-type method such as "space code + reference code + space".

First, a string is not a reference sign with a specification of two words or more (when there is a space).

Second, strings that do not contain numbers are not reference numerals in the specification.

Third, if the string contains both numbers and letters, the following substandards apply:

1) In the case of starting with a letter, the number of letters of the starting letter is less than or equal to the number of letters composing the number.

2) In the case of ending with a character, the number of characters of the ending character is less than or equal to the number of characters constituting the number.

Third, the specification inclusion reference sign cannot be the beginning or the end of a character string.

Fourth, the total length of the string is within a preset limit (eg, 6-7 characters).

Fifth, punctuation or symbols that are not symbols including specifications are excluded from the specification including references.

The specification identification code identification rule may be stored in the processing rule data unit 5240.

When the specification including reference numerals are determined in the specification that is not in the markup language, the description of the reference numerals is the same as in the specification in the markup language.

However, in principle, a 1: 1 relationship should be established between the reference numerals and the explanation candidates of the reference numerals in principle, but the 1: 1 relations may not be established in principle. have. Reasons include the following.

First, it is because of the mistake or error of the person who prepares the specification. The same specification includes the same reference numerals in different expressions, or different expressions are used in the same specification included reference numerals.

Second, there may be a mistake or an error in the specification writing program or the specification computerization program or the specification processing program.

Third, it is because of incompleteness of the description candidate condition of the reference numeral or incompleteness of the program to which the description candidate condition of the reference numeral is applied. No matter how diverse and strict the preparation of the description candidate conditions of reference numerals, 1) there is always the possibility of an expression outside the description candidate conditions of the reference numerals, and 2) exceptions in the application of conflicts or priorities between the description candidate conditions of the reference numerals. This can happen.

At this time, it is important to determine which description reference numerals will be mapped to which reference numerals. A description will be given with reference to FIGS. 13 to 15.

FIG. 13 is an exemplary embodiment of a process for determining an explanation of reference numerals of the apparatus for recognizing reference numerals 200 of the present invention.

As illustrated in FIG. 13, the apparatus for recognizing reference numerals 200 obtains an explanation candidate of reference numeral S81, applies an explanation determination rule of reference numerals to an explanation candidate of reference numerals S82. The description of the processing is performed in a manner of determining (S83).

14 is a diagram illustrating a mapping process of a description including reference numerals and descriptions of reference numerals before generation of mapping data of the reference numerals and descriptions of the reference numerals of the reference numeral-recognition recognition apparatus 200 according to the present invention. to be.

As illustrated in FIG. 14, the apparatus for recognizing reference numerals 200 obtains pair data of a description including reference numerals and descriptions of reference numerals generated in the entire specification (S91), and refers to the entire reference numerals based on the specification inclusion reference numerals. If the description is unique (1: 1 correspondence), it is inquired (S92). If Yes, 1: 1 mapping is performed between the specification including reference numerals and the description of the reference numerals (S93). On the other hand, the reference description description apparatus 200 applies a description determination rule of reference numerals when the 1: 1 correspondence does not hold. The apparatus for recognizing reference numeral 200 obtains the description of the least common reference numeral from the description of the reference numeral, processes the frequency ratio value as a probability value, and includes 1: n mapping including the reference information and the probability information of the description of the reference numeral. Processing (S94), and generating the mapping data (1: 1 or 1: n) for the specification reference numeral and the description of the reference numeral (S95).

FIG. 15 is a diagram illustrating a process of generating mapping data (1: 1 or 1: n) with reference to reference numerals and descriptions of reference numerals through the specification-included reference numerals of the apparatus 200 for describing the present invention. Example Embodiments.

As illustrated in FIG. 15, the apparatus for recognizing the reference numeral 200 obtains the reference numeral including the drawing (S101), and inquires whether the reference numeral corresponding to the reference numeral corresponding to the (diamond) drawing exists (S102). If yes, the mapping between the reference numerals and the reference numerals (S103) is performed, and the mapping data (1: 1 or 1: n for the description of the reference numerals and the reference numerals) is provided through the specification reference numerals. (S104), and if No, stores that the specification include reference number does not exist, so that this reference number is excluded from the mapping process between the reference number and the description of the reference number. Processing is performed in a manner of (S105).

Subsequently, data generation by the reference sign-explanation recognition apparatus 200 according to the present invention will be described by way of example.

For example, the following specification (content taken from the markup language file of US patent application 14 / 249,716) will be described by way of example.

Hybrid oxide-silicon thin-film transistor structures such as Illustrative thin-film transistor structures <b> 302 </ b> of <figref idrefs = "DRAWINGS"> FIG. 10 </ figref> may be used in forming CMOS-type circuitry in display driver circuitry such as gate driver circuitry <b> 18 </ b> and demultiplexer circuitry <b> 20 </ b>. As shown in <figref idrefs = "DRAWINGS"> FIG. 10 </ figref>, structures <b> 302 </ b> may have a polysilicon layer <b> 308 </ b> that is formed on substrate <b> 24 </ b>. P-channel active area <b> 310 </ b> may be formed under gate <b> 312 </ b>. Gate insulator layer <b> 306 </ b> (eg, silicon oxide) may separate gate <b> 312 </ b> from silicon channel region <b> 310 </ b> in silicon layer <b> 308 </ b >. Dielectric layer <b> 302 </ b> (e.g., sublayers of silicon oxide and silicon nitride) may cover gate <b> 312 </ b>. Dielectric layer <b> 306 </ b> may separate gate <b> 312 </ b> from overlapping oxide layer <b> 312 </ b>. Oxide layer <b> 312 </ b> may be a semiconducting oxide such as IGZO material.

When the above description candidate condition of the reference number is applied, a description candidate set of the reference number shown in Table 5 may be generated from the above-mentioned cited contents.

Data such as Table 5 is stored in the specification-based data unit 5220.

As can be seen in Table 5, there may be cases in which descriptions of other reference numerals correspond to the same reference numerals as in reference numerals 302, 308, 310, and 312. This 1: n (n is a natural number) correspondence relationship is frequently generated in reality apart from the possibility of being described in the specification. For this reason, there arises a problem in that the description of the reference numeral corresponding to the reference numeral included in the specification cannot be mechanically determined.

The apparatus for recognizing reference numeral 200 generates data as shown in Table 5 for the entire specification, and generates data as shown in Table 6 below when information is processed based on reference numerals including the specification.

In the present invention, when the description candidate of at least two reference numerals corresponds to the reference specification including reference numerals, the apparatus for recognizing the description of the reference numerals is as follows. Decision rules can be applied.

1) Description candidate of the reference code with the highest reference frequency

2) If there is information in which a mapping relationship between the reference numerals included in the specification and the description of the reference numerals has been processed, the candidates for explanation of the reference numerals that preferentially reflect the mapping relationship processing information.

3) If there is no 1) or 2), the candidate for explanation of the longest reference numeral

An explanation determination rule of reference numeral may be stored in the processing rule data unit 5240.

The apparatus for recognizing the reference numeral-description generates mapping data of the specification including reference numerals, the description of the reference numerals, and the description attribute information of the reference numerals as shown in Table 7 below. Data shown in Table 7 below is an example of the reference sign-description mapping information set generated by the reference sign-description mapping information set generation process 2400 of the present invention.

Data as shown in Table 7 may be stored in the specification-based data unit 5220, and the aggregate information may also be stored in the aggregate data unit 5250. The constituent number of characters is a description of reference numerals for arranging the description of the reference numerals. It is important for generating batch data. The longer the description of the reference numerals, the larger the size of the space for displaying the description of the reference numerals. In the case of the description of the long reference numerals, there is a high possibility that collisions / overlaps with other explanations will occur. In the case where a large amount of description is included, the arrangement between the descriptions of the reference numerals becomes very important.

FIG. 15 is a diagram illustrating a process of generating mapping data (1: 1 or 1: n) with reference to reference numerals and descriptions of reference numerals through the specification-included reference numerals of the apparatus 200 for describing the present invention. Example Embodiments.

As illustrated in FIG. 15, the apparatus for recognizing reference numeral 200 obtains the reference numeral including the drawing (S101), and inquires whether the reference numeral corresponding to the reference numeral includes the drawing (S102), and Yes. Is a mapping between the drawing reference numeral and the specification inclusion reference numeral (S103), and generating mapping data (1: 1 or 1: n) for the description of the reference numeral and the reference numeral through the specification inclusion reference numeral ( S104). Meanwhile, the apparatus for recognizing reference numerals 200 stores the reference numerals corresponding to the reference numerals when the reference numerals do not exist. Processing is performed in such a manner that the reference numerals can be excluded from the mapping process of the reference numerals including the reference numerals and the description of the reference numerals.

In the above-described method, description DFNijk (Dij (Pi)) of reference numerals included in a specific drawing Dij (Pi) of a specific patent Pi is specified. At this time, when there are two or more identical DFNijk (Dij (Pi)) in the drawing Dij (Pi), (Count (DFNijk (Dij (Pi))) which is the number of (DFNijk (Dij (Pi))) is treated as 1, It can be processed as many as Count (DFNijk (Dij (Pi))) should be treated as 1.

As illustrated in FIG. 16, the description weighing information generation process 3100 of reference numerals is Pi vs. Phi, which is explanatory data of reference numerals included in a specific drawing of a specific patent Pi. Dij (Pi) vs. DFNijk (Dij (Pi)) correspondence data is obtained (S111).

The description weighing information generation process 3100 of the reference numerals generates the description-specific weighing information (Dij (Pi): Ftn (DFNijk (Dij (Pi)))) of the reference numerals in the drawing unit with the corresponding relationship data obtained (S112). )do. Explanation of reference numerals included in the drawing Dij (Pi) is a function value measured, measured and processed for each drawing Dij (Pi) for each DFNijk (Dij (Pi)). The simplest form of a function value is a weighed value, the simplest form of a weighed value is a number, and the simplest form of a number is the presence or absence. Through this, it is possible to know various function values including whether there is a corresponding relationship with DFNijk (Dij (Pi)) for each specific patent drawing Dij (Pi) of a specific patent. The generation of the description-specific weighing information Dij (Pi): Ftn (DFNijk (Dij (Pi))) of the reference numeral is performed by the description-based weighing information generation process 3110 of the reference numeral of the present invention.

The description information generation process 3100 described with reference numerals may generate primary derived data based on this value.

First, the description set generation process 3120 of the drawing unit reference number is the description set DFNSet (Dij (Pi)) of the drawing unit reference number, and generates description full set data of the reference number included in Dij (Pi) (S113). do. As a result, data about the entire description of the reference numeral corresponding to the drawing is generated for each drawing.

Second, the description set generation process 3130 of the patent unit reference number is the description set DFNSet (Pi) of the patent unit reference number, and generates description set data of the reference number included in Pi (S114). Through this, description-specific data of reference numerals included in the patent is generated for each patent.

Third, the description-specific metering information generation process 3140 of the patent unit reference number generates the description-specific metering information (DFNijk (Pi), Ftn (DFNijk (Pi))) of the patent unit reference number (S115). Ftn (DFNijk (Pi))) is the description value data of each description of reference numerals included in Pi. In the basic Ftn (DFNijk (Pi)), the number (frequency) of DFNijk (Pi), the number of drawings with DFNijk (Pi) present (the number of drawings with DFNijk (Pi) among all drawings, the density of drawing with DFNijk (Pi) (all In the drawings, the ratio of the number of DFNijk (Pi) being present) may be obtained, etc. On the other hand, the Ftn (DFNijk (Pi)) is based on the entire DFNijk (Pi) in Pi for each base Ftn (DFNijk (Pi)). The calculated DFNijk (Pi) ranking (rank) may be included When the DFNk is obtained as a search word, the Ftn (DFNijk (Pi)) may be used to determine a search ranking of Pi that appears as a search result. When using Ftn (DFNijk (Dij (Pi))), at least DFNijk (Pi)) value is provided when representatively representing a specific drawing among several drawings of Pi (this drawing is independent of the representative drawing provided by the applicant). It is avoided that the drawings without this are provided.

When there is such data, secondary derived data may be generated.

Description of the Reference Search Support Process 3200 The description search support index generation process 3210 generates the description search index data of the reference number for the search service targeted for the description of the reference number. The searcher may input descriptions of two or more reference numerals as search terms. In this case, a patent drawing corresponding to a search condition or a search term inputter's intention may be provided as a search result. To this end, it is necessary to include a corresponding relationship between the patent drawings Dij (Pi) of at least one specific patent including DFNk for each DFNk in the index. In this case, it is more preferable that Ftn (DFNijk (Dij (Pi))) is included in the index. On the other hand, in the case of providing the patent Pi as a search result, the corresponding relationship between at least one patent Pi whose DFNk is included in the description of the reference numeral for each DFNk should be included in the index. In this case, it is more preferable that Ftn (DFNijk (Pi)) is included in the index.

The explanatory search index data of the reference numeral may include a DFNijk (Pi) ranking (rank) that is counted by the base Ftn (DFNijk (Pi)).

Descriptive Use of Descriptive Reference The descriptive data generating process 3310 of the denoting reference data generation process 3300 may generate co-occurrence data. It is possible to generate the relationship data between DFNijk (Dij (Pi)) used together based on the patent drawing Dij (Pi) of a specific patent.

The explanatory data generation process 3310 of the associative reference numeral generates co-occurrence relational data by utilizing a matrix operation or a join function of a DBMS. In the following, the matrix operation is exemplarily described, but the essence is equivalent.

The explanatory data generation process 3310 of the associated reference numerals first configures Aij constituting the base matrix A in the drawing unit as follows (S121).

The composition of Ai (row): Dij (Pi),

Composition of Aj (Columns): DFNk

Aij = Ftn (DFNijk (Dij (Pi)))

The simplest form of Ftn (DFNijk (Dij (Pi))) is frequency. In this case, Aij may correspond to 1 when a specific DFNk exists in a specific drawing Dij (Pi) of a specific patent, and 0 when not present.

Subsequently, the description data generation process 3310 of the associated reference number performs AA ^T processing (S122). Through this, the description data generation process 3310 of the associated reference number may generate co-occurrence data of the DFNk perspective (S123). At this time, the patent set Pset (DFNki, DFNkj) including both DFNki and DFNkj can be specified for the pair of DFNk (DFNki, DFNkj) with co-occurrence, so that the description data generation process 3310 of the associated reference number ) May generate quantitative analysis information on (DFNki, DFNkj) as quantitative analysis information on Pset. Representative quantitative analysis information is based on the bibliographic information (time, owner, inventor, citation / citation, family, litigation, transaction, standard information, etc.) of the Pi constituting the Pset. For example, time series of (DFNki, DFNkj), time series or share by owner / inventor of (DFNki, DFNkj), patent transaction trends or litigation trends by (DFNki, DFNkj), number of family patents generated in the last n years, Ratios and the like can be analyzed.

The quantitative analysis information may include an eventual metric analysis metric such as new (appears only after a specific point in time), a sudden increase (above a predetermined reference level) or a sharp decrease. For example, descriptions of reference numerals newly coupled to drawings related to specific parts / products may be extracted through time series data of (DFNki, DFNkj). In particular, it may be more useful if the description of the reference numeral to be newly combined is the description of the reference numeral including the product, the part, the material, the functional expression, and the like. Further, in the case of limiting a specific company (such as a leading company in the industry), it may be more useful if the description of reference numerals including a product, a part, a material, a functional expression, etc., is newly incorporated into a specific product / part of the company. In particular, the idea of the present invention is to receive a preset information for each member, such as a new or skyrocketing DFN in a co-occurrence relationship for each specific DFN, periodically search and provide notification information to the member if the notification conditions It can also be used for services.

As illustrated in FIG. 17, the similar drawing information generation process 3320 may perform A ^T A (S132) to generate highly similar pair data in terms of a description set of reference numerals included in a specific drawing unit (S133). do. The similar drawing information generation process 3320 generates (Dij (Pi), Djk (Pj)) pair data using only Pi is not Pj among pairs of (Dij (Pi), Djk (Pj)). Through this data, we can find similar Djk (Pj) drawings from Dij (Pi) perspective. In particular, the similar drawing information generation process 3320 is a special weight when the description of a particular reference number is the name of a product, the name of a part, a functional description keyword, or a specially managed keyword, or the description of the reference number is a representation including them. A value can be assigned and a function value can be processed in consideration of this weight value in Ftn (DFNijk (Dij (Pi))). In this case, similar drawings which share a large number of specific products, parts or functional technology keywords can be effectively extracted. For example, if a particular drawing has a description of a reference number containing a name of a product or a part called a display, a description of a reference number containing a part name of a temperature sensor, and a description of a reference number including a part name of an inertial sensor. Similar figures can be found with the display, the temperature sensor and the inertial sensor all together or containing at least two.

The similar patent information generation process 3330 configures Bij constituting the base matrix B in patent units as follows (S141).

The composition of Bi (row): Pi,

Composition of Bj (Column): Keyword Extracted from Patent

Bij = keyword attribute extracted from patent

At this time, the keyword extracted from the patent is DFNk alone or will include DFNk,

The keyword attribute extracted from the patent may be Ftn (DFNijk (Pi)) alone or may be one in which Ftn (DFNijk (Pi)) is added to the keyword attribute.

As described above, only the description of reference numerals may be used to generate the similar patent, or may be used together with other keywords. Of course, when using the description of the reference numerals, the frequency or the like shown in the drawings can be used to generate the similar patent as an attribute of the description of the reference numerals. That is, when a specific keyword KWi is used n times in the specification (the entire specification or the claim, etc.), the KWi is also used for the description of the reference numeral, and when there are m drawings including the description of the reference numeral, the term frequency of the KWi. (term frequency, which is one of Ftn (DFNijk (Pi))) may be n + m.

The simplest form of Ftn (DFNijk ((Pi)) is frequency, in which case Bij can map the frequency of a particular DFNk to a specific patent Pi or its importance in a patent document. The importance of the entire DFNk patent document may be a TFIDF value, for example, a method of measuring the term frequency of DFNijk (Pi) is i) Pi. A method of quantifying only the term frequency throughout the specification, ii) a method of quantifying only the frequency value (the number of figures in which DFNijk (Pi) is shown) throughout the drawing, iii) i) and ii) by simply adding or ii) There may be a method for weighted addition by adding a specific weight.

The similar patent information generation process 3330 performs BB ^T processing (S142). In this manner, the similar patent information generation process 3330 may generate DFN co-occurrence data in which DFNk alone or DFNk is treated as a keyword.

B ^T when performing the B process, DFNk it is possible to independently or DFNk generates similar patents in the collective similarity aspect of the description of the reference numerals included in the similar patents or drawings that are treated as a kind of a keyword (S143).

Description search support process 3200 of the reference numerals support to perform a specialized drawing search utilizing the spirit of the present invention.

Description of References The search support process 3200 includes a description of the reference vs. the reference numerals. Patent Drawings, Explanation of References vs. The description of reference numerals can be used as a search target using an index including the correspondence of patent data. For this reason, the explanatory search field of the code | symbol can be included in a keyword search field group. Existing keyword search field groups include the name of the invention, the abstract, claims and description of the invention. Therefore, a description search field of reference numeral can be added as a new search field, and the description search field of reference numeral can be used in combination with another search field.

On the other hand, utilizing the idea of the present invention, it is possible to effectively search for a patent corresponding to the product. For example, it can be used to find a patent or a patent drawing that is most relevant to an image of a product or a part included in a catalog image or a blueprint. In the search system to which the present invention is applied, one or more (preferably several) names of specific parts / parts of a product-part image or a design drawing are entered in the description search field of the reference numeral, and optionally the applicant / current owner search field. When a manufacturer or a related company is inputted, an optimal patent drawing or an optimal patent can be found. This function is also performed through the description search support process 3200 of the reference numeral.

As illustrated in FIG. 20, the patent drawing OCR processor 1000 identifies the number of drawing numbers in the drawing (S151), and if the number of drawing numbers is two or more, cut the drawing by the number of drawing numbers and make independent drawing data. Process in the manner of generating (S152).

The number of drawing numbers refers to identifying how many representations with independent drawing numbers such as "Fig. + Number" or "degree + Number" exist in a single drawing image file. The patent reference-description processing server 300 may physically divide a drawing into a representation unit having an independent reference number, and apply the inventive concept to the divided drawing.

On the other hand, when the machine translation of the description of the reference numerals in a specific language to another language, it will be obvious that the present invention is applied to the description of the machine-translated reference numerals. Table 8 below is an example of data in which pairs of descriptions of original reference numerals and descriptions of reference numerals translated in other languages are shown.

The present invention can be widely used in the patent information industry, the patent information service industry, the technology information business, and the technology information service industry of the present invention.

100: OCR device with patent drawings
200 reference numeral-description recognition device
300: Patent reference code-Description Processing Server
400: reference sign recognition learning device
500: data storage device
800: wired and wireless network
900: other devices
910: user computer
1000: Patent Drawings OCR Processor
1100: drawing file acquisition process
1200: Drawing File Parsing Process
1300: Drawing String Recognition Process
1310: Drawing Text Classification Process
1320: Drawing String Synthesis Process
1330: Sign Recognition Process
1331: Synthetic Character Noise Reduction Process
1332: Composite Character Lookup Process
1333: Sign Size Prediction Process
1334: Sign Noise Removal Process
1400: Process for creating reference information set
100a: Patent drawing OCR device communication unit
2000: Reference Mark-Description Recognition Processor
2100: statement acquisition process
2110: Reference sign-description recognition process
2120: Description Recognition Process of Reference Numbers
2130: reference sign-description mapping process
2200: Importance Handling Process
2210: Location Aware Processing Process
2220: frequency recognition processing process
2300: Series Processing Process
2310: family recognition process
2320: Series Categorization Process
2400: Process for generating reference sign-description mapping information set
200a: reference numeral-description recognition device communication unit
300: Patent reference code-Description Processing Server
3000: patent reference-description processing processor
3100: Description of reference numerals Process for generating weighing information
3110: process for generating weighing information by description of reference numerals
3120: Process for creating a description set of drawing unit reference numbers
3130: Process for generating an explanatory set of reference numerals for patent units
3140: Weighing Information Generation Process by Description of Patent Unit Reference Code
3200: Description Search Support Process Of Reference Number
3210: Process for creating an explanatory search support index of reference numerals
3220: description search ranking information generation process of the reference numeral
3300: Description of reference numerals Use association data generation process
3310: Process of generating explanatory data of associative reference numbers
3320: Process for generating similar drawing information
3330: Process for generating similar patent information
3400: Process for providing description content information of reference numerals
300a: Patent Reference Code-Description Processing Device Communication Unit
4000: Sign Recognition Learning Processor
4100: learning process
4110: hyperparameter optimization process
4111: training data object recognition modeling learning unit
4112: validation data target recognition modeling verification unit
4120: Learning Model Parameter Learning Process
4121: learning model for all data object recognition
4100: Learning Data Storage
4110: learning image data part by character
4100: learning result storage unit
400a: reference sign recognition learning unit communication unit
5100: File Storage
5110: drawing file storage
5111: Original drawing file storage unit
5112: Machine drawing file storage
5120: specification file storage unit
5200: DB device
5210: Drawing-based data section
5211: Reference information set
5220: specification based data part
5221: Detailed Description Data of the Invention
5222: Reference code-description mapping information set
5230: Description layout data portion of the reference numeral
5231: Explanation of Reference Weighing Data
5232: Explanation of reference numerals used association data
5240: machining rule data part
5250 aggregate data
5260: policy data part
5270 training data part
500a: data storage communication unit

Claims (11)

In the method for processing the descriptive information of the reference numeral indicated by the reference numeral of the provided patent figure,
(a) After the patent drawing OCR device obtains at least one patent drawing image file corresponding to the patent drawing identifier that can correspond to the patent identifier, the at least one drawing containing reference number in the patent drawing image file is obtained. A first state in which a drawing-included reference information set corresponding to the recognized drawing-included reference code including position information of the drawing-included reference number is generated; And ii) a specification string including a reference number and a description of the reference number is parsed by a reference-description recognition device, and at least one specification-containing reference number and a description of the at least one reference number included in the specification string are recognized. And a patent reference code-description processing server in a state in which a second state in which a specification-specific reference code-description mapping information set for each patent identifier is generated including a mapping relationship between the recognized reference code and the description of the code is performed. (A) obtaining a set of description data (DFNijk (Dij (Pi))) of reference numerals for each drawing identifier Dij (Pi) of the patent Pi;
(b) Descriptive weighing information (Ftn (DFNijk (Dij (Pi)))) of the reference numerals in the unit of the drawing as explanatory data sets of the reference numerals by the drawing identifiers and the description-based weighing of the reference numerals in the patent unit (Ftn (DFNijk) (Pi))) generating any one or more of information; and
(c) Weighing information by description (Ftn (DFNijk (Dij (Pi)))) of the reference numeral in the drawing unit and Weighing description (Ftn (DFNijk (Pi))) of the reference numeral in the patent unit. Generating a usage-related dataset;
The reference numeral with reference numeral and the reference numeral with reference specification are the same character string or a character string which is recognized the sameness. The method of claim 1,
The manner in which the drawing reference numerals are recognized to achieve the first state is
(i1) performing sampling on the collected character-corresponding image data for each character;
(i2) dividing the sampled character-corresponding image data for each character into n (natural numbers of n> = 3) folders;
(i3) A crossover that trains a model using character-corresponding image data for each character in the n-1 folder as training data, and obtains a tester error n times using character-corresponding image data for each character in the remaining folders as test data. Optimizing hyper parameters through cross validation; And
(i4) using the collected character-corresponding image data for each character and learning the parameters of the learning model under optimized hyperparameter conditions; and a data processing method of a patent reference-explaining processing server, characterized in that performed. The method of claim 2,
The learning model is at least one selected from a discriminant deep learning model and a generative deep learning model.
The hyperparameter may be one or more selected from the number of layers, the number of hidden variables, the drop out ratio, the filter size, and the type of activation function. A data processing method of a patent reference-description processing server, characterized in that. The method of claim 1,
In the step (b), the description-specific weighing information Ftn (DFNijk (Pi)) of the reference numeral of the patent unit includes the number (expression frequency) of the description (DFNijk (Pi)) of the reference numeral and the description of the reference numeral (DFNijk). (Pi)) A patent drawing characterized in that it is any one or more of the number of the existing drawings and the description of the reference signs (DFNijk (Pi)) the existing drawing density (ratio of the number of DFNijk (Pi) present drawings among all the drawings). Code-Description Processing server's data processing method. The method of claim 1,
The association data set of step (c) is to generate a description data set of an associated reference number.
(c11) constructing an elementary matrix Aij in the drawing unit;
The composition of Ai (row): Dij (Pi),
Composition of Aj (Columns): DFNk
Aij = Ftn (DFNijk (Dij (Pi)))
(c12) performing AA ^T treatment; And
(c13) generating explanatory data of the associative reference numeral including the association information between the pair of description of the reference numeral;
Data processing method of the patent reference-description processing server, characterized in that it comprises a. The method of claim 1,
The associative dataset of step (c) is to generate a similarity pair dataset, and the generation of the similarity pair dataset
(c21) constructing an elementary matrix Aij in the drawing unit;
The composition of Ai (row): Dij (Pi),
Composition of Aj (Columns): DFNk
Aij = Ftn (DFNijk (Dij (Pi)))
(c22) A ^T A Performing a process; And
(c23) generating a drawing data set having high similarity; the data processing method of the patent reference-description processing server, characterized in that it comprises a. The method of claim 1,
The associated dataset of step (c) is to generate a similar patent dataset, and the generation of the similar patent dataset
(c31) constructing a base matrix Bij in patent units;
The composition of Bi (row): Pi,
Composition of Bj (Column): Keyword Extracted from Patent
Bij = keyword attribute extracted from patent
(c32) performing BB ^T processing; And
(c32) performing BB ^T processing; And
(c33) generating a patent data set having high similarity; data processing method of a patent reference-description processing server, characterized in that it comprises a. The method of claim 7, wherein
The keyword extracted from the patent is a description of reference numeral (DFNk) alone, or includes a description of reference numeral (DFNk),
The keyword attribute extracted from the patent is the attribute of the description of the reference numeral for each patent (Ftn (DFNijk (Pi))) alone, or the attribute of the explanation of the reference numeral for each patent (Ftn (DFNijk (Pi))) is added to the attribute of the keyword. A data processing method of a patent reference-description processing server, characterized in that. The method of claim 1,
In recognizing reference numerals for achieving the first state, a description of reference numerals recognized for achieving the second state or a specification including reference numerals for describing the reference state generated to achieve the second state The data processing method of the patent reference-description processing server, characterized in that any one or more of the three is used . The method of claim 9,
The specification-included reference-description mapping information set includes probabilistic mapping data for the specification-included reference number and the description of the reference number,
And optionally, the specification-included reference-description mapping information set includes the importance and sequence processing information for the description of the reference number. In the patent reference-description processing server which processes the description information of the reference number to which the reference number of the provided patent drawing refers,
The patent reference-description processing server
i) after obtaining at least one patent drawing image file corresponding to a patent drawing identifier that may correspond to a patent identifier by the patent drawing OCR apparatus, at least one drawing containing reference number is recognized in the patent drawing image file, A first state in which a drawing-bearing reference information set corresponding to the recognized drawing-bearing reference code including position information of the drawing-bearing reference sign is generated; And ii) a specification string including a reference number and a description of the reference number is parsed by a reference-description recognition device, and at least one specification-containing reference number and a description of the at least one reference number included in the specification string are recognized. In the state in which a second state in which a specification-specific reference code-description mapping information set for each patent identifier is generated including a mapping relationship between the recognized specification-containing reference number and the description of the reference number is performed, the figure of the patent Pi is performed. A communication unit for obtaining a set of explanatory data DFNijk (Dij (Pi)) of reference numerals according to the identifier Dij (Pi);
(I) Descriptive weighing information (Ftn (DFNijk (Dij (Pi)))) of the reference numerals in the unit of drawing as explanatory data sets of the reference numerals according to the drawing identifiers, and the description-based weighing of the reference numerals in patent units (Ftn (DFNijk). (Pi))) a process for generating any one or more of information; and (II) the weighing information (Ftn (DFNijk (Dij (Pi)))) by description of the reference numerals of the drawing units and the reference numerals of the patent units. A process for generating a description use association data set of reference numerals by description-specific metering (Ftn (DFNijk (Pi)));
And said specification containing reference numeral is the same character string or a character string in which the sameness is recognized.

Images