KR102638388B1 - Object recognition system, object recognition method and computer program stored in a recording medium to execute the method - Google Patents

Abstract

An object recognition method according to an embodiment of the disclosed invention includes the steps of: (a) extracting objects including the symbols from the diagram image by an object extractor; (b) calculating, by an overlap ratio calculation unit, an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and (c) determining, by a duplicate object determination unit, whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of the objects.

Description

OBJECT RECOGNITION SYSTEM, OBJECT RECOGNITION METHOD AND COMPUTER PROGRAM STORED IN A RECORDING MEDIUM TO EXECUTE THE METHOD}

The present invention relates to an object recognition system and object recognition method that can improve object recognition performance for symbols in diagram images by applying different criteria depending on the symbol type of the extracted object to determine whether the object is an overlapped object. will be.

A representative diagram used in the process plant industry is the P&ID (piping and instrumentation diagram). P&ID is prepared based on PFD (process flow diagram) and expresses in detail the flow of major equipment and materials and fluids that make up each process.

This P&ID is the basic medium that conveys the basic design of the plant and is used as basic data for detailed design, purchasing and procurement, construction, and commissioning.

Meanwhile, even in newly constructed plants, the P&ID created in the FEED (front end engineering and design) stage or the P&ID provided by the equipment manufacturer often has an image format due to contractual relationships or intellectual property security issues.

Plant operators continuously improve and expand the plant during the plant operation process, and often receive P&IDs in image format from external partners. Plant operators may apply digital P&ID, but in the case of old plants operating for a long time, a large amount of P&ID may be stored in image format.

Accordingly, in the plant industry, there may be a need to convert P&ID in image format to digital P&ID. At this time, in order to convert the image format P&ID into a digital P&ID, technology is needed to better recognize the symbols included in the P&ID.

The present invention can accurately determine whether objects extracted from diagram images correspond to duplicate objects, prevent objects that do not correspond to duplicate objects from being incorrectly recognized as duplicate objects, and improve symbol recognition accuracy. It is intended to provide an object recognition system, object recognition method, and computer program.

In addition, the present invention is an object recognition system, object recognition method, and computer program that can compensate for performance degradation due to symbol size deviation in diagram images and can increase recognition performance for high-density symbols through a separate post-processing process. It is intended to provide.

An object recognition method of a diagram image for recognizing an object in a diagram image including a plurality of symbols according to an aspect of the disclosed invention includes (a) extracting objects including the symbols from the diagram image by an object extractor; step; (b) calculating, by an overlap ratio calculation unit, an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and (c) determining, by a duplicate object determination unit, whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of the objects.

Additionally, step (c) includes: (c1) analyzing the size and shape of the objects to determine symbol types of the objects and comparing the symbol types of the objects; (c2) determining whether the symbol types of the objects correspond to an overlapping symbol set determined based on the frequency of overlapping occurrences; and (c3) determining the overlapping object according to a comparison result of the symbol types and whether it corresponds to the overlapping symbol set.

Additionally, step (c3) includes: determining that the objects are not duplicate objects when the symbol types of the objects are not the same; When the symbol types of the objects are the same and correspond to the overlapping symbol set, determining the overlapping object by comparing the overlapping ratio with a first reference threshold determined according to the symbol type; And when the symbol types of the objects are the same and do not correspond to the overlapping symbol set, determining the overlapping object by comparing the overlapping ratio with a second reference threshold lower than the first reference threshold. can do.

Additionally, step (c) includes: generating a bounding box containing an object extracted from the diagram image; calculating an IOU ratio between the bounding boxes from a ratio of the area of the overlapping area to the area of the combined area of the overlapping bounding boxes; determining a symbol type based on the size and shape of the extracted object; determining the reference threshold based on the determined symbol type; removing one of the overlapping bounding boxes if the IOU ratio is greater than or equal to the reference threshold; and recognizing an object included in the bounding box that has not been removed as a symbol corresponding to the bounding box.

In addition, the method may further include generating at least one of position information of the symbol in the diagram image or a list of the symbols based on a bounding box including an object recognized as the symbol.

In addition, it further includes the step of learning an artificial intelligence model by setting the characteristics of the symbol as an input variable and the types of the objects as an output variable by an artificial intelligence learning unit, wherein the step (c) includes the extraction. extracting features of the object from the object; and determining the type of the extracted object using the artificial intelligence model based on the characteristics of the extracted object.

In addition, the step of generating a size-changed diagram image by changing the size of the diagram image by a pre-processing unit, wherein the step of extracting the object includes adding the size-changed diagram image to the size-changed diagram image by the object extraction unit. A step of extracting the included object whose size has changed may be included.

In addition, the step of generating a plurality of segmented diagram images by dividing the diagram image by a preprocessor, wherein the step of extracting the object includes dividing the diagram image into a plurality of segmented diagram images by the object extracting unit. A step of extracting an object may be further included.

In addition, the step of generating a size-changed diagram image by changing the size of the diagram image by a preprocessor, and generating a plurality of split diagram images by dividing the diagram image, and extracting the object. may include extracting, by the object extraction unit, an object included in the diagram image based on an object whose size has been changed and included in the size change diagram image and an object included in the segmented diagram image. there is.

In addition, the method may further include extracting text included in the segmented diagram image by a text extraction unit and recognizing the text of the diagram image.

In addition, the preprocessor extracts a horizontal line from the original diagram image, determines the horizontal line with the longest length among the extracted horizontal lines, extracts all black pixels connected to the longest horizontal line, and extracts the horizontal line from the original diagram image. The method may further include generating the diagram image by removing black pixels.

A computer program according to one aspect of the disclosed invention may be stored in a computer-readable recording medium to execute the object recognition method.

An object recognition system for a diagram image that recognizes an object in a diagram image including a plurality of symbols according to an aspect of the disclosed invention includes an object extraction unit that extracts objects including the symbols from the diagram image; an overlap ratio calculation unit that calculates an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and a duplicate object determination unit that determines whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of objects.

In addition, the duplicate object determination unit: determines symbol types of the objects by analyzing the size and shape of the objects, and compares the symbol types of the objects; Determine whether the symbol types of the objects correspond to an overlapping symbol set determined based on the frequency of overlapping occurrences; And, the overlapping object may be determined according to a comparison result of the symbol types and whether it corresponds to the overlapping symbol set.

Additionally, the duplicate object determination unit: determines that the objects are not duplicate objects when the symbol types of the objects are not the same; If the symbol types of the objects are the same and correspond to the overlapping symbol set, determine the overlapping object by comparing the overlapping ratio with a first reference threshold determined according to the symbol type; And when the symbol types of the objects are the same and do not correspond to the overlapping symbol set, the overlapping ratio is compared with a second reference threshold lower than the first reference threshold to determine the overlapping object. .

Additionally, the duplicate object determination unit: generates a bounding box including an object extracted from the diagram image; Calculate the IOU ratio between the bounding boxes from the ratio of the area of the overlapped area to the area of the combined area of the overlapping bounding boxes; determine a symbol type based on the size and shape of the extracted object; determine the reference threshold based on the determined symbol type; if the IOU ratio is greater than or equal to the reference threshold, remove one bounding box of the overlapping bounding boxes; Additionally, it may be configured to recognize an object included in the bounding box that has not been removed as a symbol corresponding to the bounding box.

In addition, it may further include a processor configured to generate at least one of position information of the symbol in the diagram image or a list of the symbols, based on a bounding box including an object recognized as the symbol.

In addition, it further includes an artificial intelligence learning unit configured to learn an artificial intelligence model by setting the characteristics of the symbol as an input variable and the types of the objects as an output variable, wherein the duplicate object determination unit: selects the extracted object from the extracted object. extract features of the object; And it may be configured to determine the type of the extracted object using the artificial intelligence model based on the characteristics of the extracted object.

In addition, it further includes a preprocessor configured to change the size of the diagram image to generate a size-changed diagram image, and to divide the diagram image to generate a plurality of split diagram images, wherein the object extractor is configured to change the size. It may be configured to extract objects included in the diagram image based on objects included in the diagram image and whose size has been changed and objects included in the split diagram image.

In addition, it may further include a text extraction unit configured to extract text included in the segmented diagram image and recognize the text of the diagram image.

Additionally, a horizontal line is extracted from the original diagram image, a horizontal line with the longest length among the extracted horizontal lines is determined, all black pixels connected to the longest horizontal line are extracted, and the extracted black pixels are removed from the original diagram image. It may include a pre-processing unit configured to generate the diagram image.

According to one aspect of the disclosed invention, it is possible to accurately determine whether objects extracted from a diagram image correspond to duplicate objects, prevent objects that do not correspond to duplicate objects from being incorrectly recognized as duplicate objects, and symbolize recognition. Accuracy can be improved.

In addition, according to an embodiment of the present invention, performance degradation due to symbol size deviation of the diagram image can be compensated, and recognition performance for high-density symbols can be improved through a separate post-processing process.

Additionally, according to an embodiment of the present invention, string recognition performance can be improved by recognizing a string by distinguishing whether each string area is a rotated string or an unrotated string through a rotation estimation process.

1 is a configuration diagram of an object recognition system according to an embodiment.
Figure 2 is a diagram for explaining a case where object recognition fails.
Figure 3 is a table classifying symbol types according to one embodiment.
Figure 4 is a diagram to explain incorrectly judging a recognized object as a duplicate object.
FIG. 5 is a diagram illustrating differences when different reference thresholds are applied to recognized objects according to an embodiment.
Figure 6 is a diagram illustrating a list of symbols created according to one embodiment.
Figure 7 is a diagram showing the data structure of an artificial intelligence model according to an embodiment.
FIG. 8 is a diagram illustrating a process for extracting objects included in a size change diagram image and a split diagram image according to an embodiment.
FIG. 9 is a diagram illustrating a process for recognizing text included in a diagram image according to an embodiment.
Figure 10 is a diagram illustrating a process for extracting text according to one embodiment.
Figure 11 is a diagram showing the process of preprocessing an original diagram image.
Figure 12 is a flowchart of an object recognition method according to an embodiment.
13 is a flowchart of an object recognition method according to another embodiment.

Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the disclosed invention pertains is omitted. The term '~unit' used in the specification may be implemented as software or hardware, and depending on the embodiments, multiple '~units' may be implemented as one component, or one '~unit' may be implemented as a plurality of components. It is also possible to include elements.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the disclosed invention will be described with reference to the attached drawings.

1 is a configuration diagram of an object recognition system according to an embodiment.

Referring to FIG. 1, the object recognition system 100 according to an embodiment of the present invention includes an object extraction unit 110, an overlap ratio calculation unit 120, a duplicate object determination unit 130, a preprocessor 140, It may include a processor 150, an artificial intelligence learning unit 160, and a memory 170.

The diagram image 200 may be input to the object recognition system 100 in the form of image data.

The diagram image 200 may be a diagram used in the process plant industry. For example, the diagram image 200 may be a piping and instrumentation diagram (P&ID), but is not limited thereto.

The diagram image 200 can be created based on a process flow diagram (PFD) and can express in detail the flow of major equipment and materials and fluids that make up each process. In other words, the diagram image 200 may be a basic medium that conveys the basic design of the plant, and may be an image used as basic data for detailed design, purchase, procurement, construction, and commissioning.

The object recognition system 100 may recognize the object 400 in the diagram image 200 including a plurality of symbols 300.

The object extraction unit 110 may extract objects 400 from the diagram image 200.

Object 400 may include symbol 300. That is, when the object recognition system 100 extracts the object 400 from the diagram image 200, it can recognize the symbol 300 included in the extracted object 400.

Meanwhile, the object extraction unit 110 extracts a plurality of objects 400 from the diagram image 200, and at this time, the areas occupied by each of the two extracted objects 400 may overlap each other.

The object extractor 110 may detect the location of the object 400 existing in the diagram image 200 through conventional object detection.

The overlap ratio calculation unit 120 may calculate the overlap ratio between the overlap area 501 of the extracted objects 400 and the area 500 of the objects.

The overlap area 501 may be the area of the overlapped area when two extracted objects 400 overlap.

Additionally, the area 500 of objects may be the total area occupied by the two extracted objects 400 when they overlap. That is, the area 500 of the objects may be the sum of the areas of each object 400 minus the overlapping area 501.

The overlap ratio may be the ratio of the overlapping area 501 to the area 500 of objects. That is, the overlap ratio may be a value representing the ratio of the overlapped objects 400 when any two objects 400 overlap.

The duplicate object determination unit 130 may determine whether the objects 400 correspond to the duplicate objects 401 by comparing the overlap ratio with a reference threshold determined according to the type of the objects 400.

The types of objects 400 may be types of objects 400 that can be extracted from the diagram image 200. Additionally, the type of the objects 400 may be a type of the shape of an object 400 that can be extracted from the diagram image 200. It may be a type of the form formed by a plurality of objects 400.

When the object 400 is a symbol 300 included in the diagram image 200, the type of the object 400 may be the symbol type of the object 400. These symbol types can be determined by pre-classifying each symbol 300 according to its size and shape.

The reference threshold may be a standard value for determining whether the two extracted objects 400 correspond to the overlapping object 401 when the areas of the two extracted objects 400 overlap. That is, the reference threshold may have a value between 0 and 1. The reference threshold may be set in advance.

Although the two extracted objects 400 should in fact be recognized as one object 400, they may be recognized as separate objects 400. At this time, the duplicate object determination unit 130 may determine that the two extracted objects 400 are duplicate objects 401.

If the overlap ratio between objects 400 detected as the same object 400 exceeds the reference threshold, the duplicate object determination unit 130 may determine that the objects 400 are duplicate objects 401. At this time, since the reference threshold is different for each type of object 400, even if the overlap ratio is the same, the duplicate object determination unit 130 determines that any two objects 400 are duplicate objects 401, and It may be determined that the other two objects 400 are not duplicate objects 401.

The processor 150 recognizes only the more recognizable object 400 among the two objects 400 determined to be duplicate objects 401 as the symbol 300, and recognizes the remaining object 400 as the symbol 300. You may not recognize it. As a result, the processor 150 can prevent one symbol 300, which was originally supposed to be extracted as one object 400, from being mistakenly recognized as two symbols 300.

The pre-processing unit 140, the object extraction unit 110, the overlap ratio calculation unit 120, the duplicate object determination unit 130, and the artificial intelligence learning unit 160 include a plurality of processors included in the object recognition system 100 ( 150) may include any one processor 150. Additionally, the object recognition method according to the embodiments of the present invention described so far and the embodiments to be described in the future may be implemented in the form of a program that can be driven by the processor 150.

Here, the program may include program instructions, data files, and data structures, etc., singly or in combination. Programs may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described method for modifying the code, or may be implemented using various functions or definitions known and available to those skilled in the art in the field of computer software. A program for implementing the above-described information display method may be recorded on a recording medium readable by the processor 150. At this time, the recording medium may be the memory 170.

The memory 170 can store programs that perform the operations described above and operations described later, and the memory 170 can execute the stored programs. When the processor 150 and the memory 170 are plural, they may be integrated into one chip or may be provided in physically separate locations. The memory 170 may include volatile memory such as Static Random Access Memory (S-RAM) or Dynamic Random Access Memory (D-Lab) for temporarily storing data. In addition, the memory 170 includes read only memory (ROM), erasable programmable read only memory (EPROM), and electrically erasable programmable read only memory (EEPROM) for long-term storage of control programs and control data. May include non-volatile memory.

The processor 150 may include various logic circuits and operation circuits, process data according to a program provided from the memory 170, and generate control signals according to the processing results.

FIG. 2 is a diagram for explaining a case where object recognition fails, FIG. 3 is a table classifying symbol types according to an embodiment, and FIG. 4 is a diagram for explaining incorrectly judging a recognized object as a duplicate object. am.

Referring to FIG. 2, it can be seen that for symbols 300 that are large in size in the diagram image 200, there are cases where the object 400 cannot be extracted from the beginning and thus the symbol 300 cannot be detected. Such a large symbol 300 can be successfully detected if there is sufficient learning data.

It can be seen that small-sized symbols 300 in the diagram image 200 are often misrecognized as other symbols 300 having similar shapes. Therefore, in the case of small symbols 300, it can be confirmed that errors may occur not simply because the object 400 cannot be extracted, but because the extracted object 400 is incorrectly recognized as another symbol 300. .

Referring to FIG. 3, it can be seen that symbols 300 that can be included in the diagram image 200 can be classified by symbol type. That is, the symbols 300 may be classified into five symbol types as shown, but the classification method or type is not limited to this, and may be classified in various other ways depending on the characteristics of each symbol 300. .

Depending on the symbol type of the symbol 300 recognized from the extracted object 400, the degree to which it is mistakenly recognized as a single symbol 300 may vary even though it is a plurality of symbols 300. In other words, although it should originally be recognized as two objects 400 and consequently recognized as two symbols 300, the extent to which it is mistakenly recognized as a single symbol 300 may vary depending on the shape and type of the symbol 300. .

For example, the symbols 300 corresponding to 'PIPING COMPONENT' may be more often mistakenly recognized as a single symbol 300 than the symbols 300 corresponding to 'EQUIPMENT'. .

Referring to FIG. 4, one can see the process in which two objects 400, which should be recognized as two symbols 300, are ultimately misrecognized as one symbol 300.

The duplicate object determination unit 130 may determine the objects 400 to be duplicate objects 401 if the overlap area 501 between the two objects 400 is greater than the reference threshold. Meanwhile, this process can continue to occur repeatedly for the two objects 400 on which the overlapping area 501 is formed. For example, after determining whether two objects 400 are duplicate objects, it is also determined whether one object 400 and the third object 400 of the two objects 400 correspond to duplicate objects 401. You can judge.

Since the processor 150 can recognize only the more recognizable object 400 among the two objects 400 determined to be overlapping objects 401 as the symbol 300, by repeating the above-described process, the areas are finally overlapped. In the case of objects 400, only one object 400 may be used to recognize the symbol 300. This process may be necessary to prevent one symbol 300 from being mistakenly recognized as multiple symbols 300.

However, depending on the type of symbol 300, when two separate symbols 300 are adjacent, a case may occur where they are ultimately misrecognized as one symbol 300 even though they are separate symbols 300.

Therefore, it may not be desirable to apply the same reference threshold to the extracted plurality of overlapping objects 400 to determine whether the corresponding object 400 is a duplicate object 401. That is, for symbol types in which a plurality of symbols 300 are more often mistakenly recognized as a single symbol 300, a higher standard threshold is applied, so that even if there is some overlap, the extracted object 400 It may be desirable to determine that they are not duplicate objects 401.

For example, referring to FIG. 4, when there are two symbols 300 representing two adjacent 'flanges' in the diagram image 200, the overlapping area 501 where the two recognized objects 400 overlap is the reference. Because it is larger than the threshold, it may be incorrectly recognized as one symbol (300).

FIG. 5 is a diagram illustrating differences when different reference thresholds are applied to recognized objects according to an embodiment.

Referring to FIG. 5 , it can be seen that when determining whether a plurality of extracted objects 400 are duplicate objects 401, the results may vary depending on the applied reference threshold.

The duplicate object determination unit 130 may analyze the size and shape of the objects 400 to determine symbol types of the objects 400 and compare the symbol types of the objects 400.

The duplicate object determination unit 130 may determine the type of object 400 existing in the diagram image 200 through conventional object detection.

Object detection is a method for understanding images and is a technology that has been majorly researched in the field of computer vision. Traditional machine learning can extract features from the object 400. Afterwards, classifiers such as SVM and Adaboost that can robustly classify the distribution of features extracted from different objects 400 can be trained. In the subsequent application stage, features can be extracted from the input image for which detection is to be performed and the object 400 can be detected by passing it through a classifier.

Meanwhile, the object extraction unit 110 does not necessarily have to extract the object 400 existing in the diagram image 200 through machine learning-based object detection.

For example, deep learning-based object detection technology can be used instead of machine learning-based object detection. Deep learning-based object detection technology learns the features of the object 400 extracted from the image based on data. To learn how to extract features from images, the CNN (Convolutional Neural Networks) structure, which stacks multiple convolution layers, is widely used.

The overlapping object determination unit 130 may determine whether the symbol types of the objects 400 correspond to an overlapping symbol set.

The overlapping symbol set may be a set determined based on the frequency of overlapping occurrences.

The duplicate object determination unit 130 may be configured to determine the duplicate object 401 according to a comparison result of symbol types and whether it corresponds to an overlapping symbol set.

[Equation 1]

Referring to [Equation 1], the overlapping symbol set is a set of symbols 300 with a high frequency of overlapping when recognizing the object 400 ( ) can be.

corresponds to each of the two extracted objects 400. and is the overlap ratio of Is It may be a function representing the symbol type of .

If the symbol types of the objects 400 are not the same, the duplicate object determination unit 130 may determine that the objects 400 are not duplicate objects 401 .

That is, referring again to [Equation 1], if the symbol types of the two objects 400 are different (ie , the duplicate object determination unit 130 may determine that the two objects 400 are not duplicate objects 401.

If it is determined that the two objects 400 are not duplicate objects 401, the duplicate object determination unit 130 recognizes the symbols 300 corresponding to each of the two objects 400 based on the two objects 400. can do.

If the symbol types of the objects 400 are the same and correspond to an overlapping symbol set, the duplicate object determination unit 130 compares the overlap ratio with a first reference threshold determined according to the symbol type to determine the duplicate object 401. You can judge.

The first reference threshold may be a reference threshold that is used as a standard for determining whether an object 400 of a symbol type that does not easily overlap is a duplicate object 401 .

Referring to [Equation 1], if the symbol types of the two objects 400 are the same and the two objects 400 correspond to an overlapping symbol set ( ), if the overlap ratio is greater than the first reference threshold ( ), the two objects 400 may be determined to be duplicate objects 401.

If the symbol types of the objects 400 are the same and do not correspond to an overlapping symbol set, the duplicate object determination unit 130 compares the overlap ratio with a second reference threshold lower than the first reference threshold to determine the duplicate object 401 ) can be configured to judge.

The second reference threshold may be a reference threshold that serves as a standard for determining whether an object 400 of a symbol type that easily overlaps is an overlapping object 401.

Referring again to [Equation 1], if the symbol types of the two objects 400 are the same and the two objects 400 do not correspond to an overlapping symbol set ( ), if the overlap ratio is greater than the second reference threshold ( ), the two objects 400 may be determined to be duplicate objects 401.

As a result, according to an embodiment of the present invention, object recognition determines whether or not a duplicate object 401 is a duplicate object 401 by applying a uniform reference threshold, and recognizes only one object as a symbol 300 if it is determined to be a duplicate object 401. The symbol 300 can be recognized better than this method.

That is, the object recognition system 100 of the present invention applies deep learning technology to the diagram image (P&ID) 200, which has high density and consists of various types of symbols 300 with different sizes and shape complexities. The symbol 300 can be recognized at a level applicable to industry.

Referring again to FIG. 5 , the duplicate object determination unit 130 may generate a bounding box 600 including the object 400 extracted from the diagram image 200.

One bounding box 600 may correspond to one object 400 and may be used to generate location information of the object 400. For example, the location information of the object 400 included in the bounding box 600 may be coordinate information of the upper left and lower right corners of the bounding box 600.

The overlapping object determination unit 130 may calculate an intersection over union (IOU) ratio between the bounding boxes 600 from the ratio of the area of the overlapping area to the area of the combined area of the overlapping bounding boxes 600. At this time, the IOU ratio may be the overlap ratio between the two bounding boxes 600.

The overlapping object determination unit 130 determines a reference threshold based on the symbol types of the two extracted objects 400, and if the IOU ratio is greater than the determined reference threshold, one of the overlapping bounding boxes 600 is selected as a bounding box ( 600) can be removed.

The duplicate object determination unit 130 may be configured to recognize the object 400 included in the bounding box 600 that has not been removed as the symbol 300 corresponding to the bounding box 600.

That is, the duplicate object determination unit 130 generates two bounding boxes 600 even though it is one symbol 300, and in order to prevent it from being mistakenly recognized as two symbols 300, the overlap ratio between the two bounding boxes 600 is set. If the size is larger than a certain size, only one bounding box 600 can be left so that the corresponding symbol 300 is normally recognized as a single symbol 300.

Figure 6 is a diagram illustrating a list of symbols created according to one embodiment.

Referring to FIG. 6, it can be confirmed that the objects 400 included in the diagram image 200 are recognized, and one bounding box 600 is created for each object 400.

Additionally, it can be confirmed that a list of recognized objects 400 has been created and a list of symbols 300 has been created.

The processor 150 may generate a list of symbols 300 based on the bounding box 600. Specifically, the processor 150 generates location information or symbol 300 of the symbol 300 in the diagram image 200 based on the bounding box 600 containing the object 400 recognized as the symbol 300. It may be configured to generate at least one of the list.

Figure 7 is a diagram showing the data structure of an artificial intelligence model according to an embodiment.

Referring to FIG. 7, the learning data storage structure used for learning the artificial intelligence model for recognizing the object 400 can be confirmed.

In the learning data, the types of objects 400 may be defined in consideration of the symbol type and direction of the symbol 300.

The top folder of the learning data folder structure may be the P&ID folder. This folder may contain class list files for symbols 300 and characters. A subfolder of the P&ID folder may be a project folder. The project folder may contain diagrams and diagram list files created in the project. Since the P&ID standard used for each project is different in the project folder, learning data can be stored separately depending on the project. The subfolders of the project folder include the folder necessary for detection of the symbol 300, the folder storing the learning data necessary for classification of the symbol 300, the folder storing the learning data necessary for character recognition, and the learning data necessary for line recognition. This may be a saved folder. An annotation file containing necessary information may be stored for each diagram in the learning data folder of each object 400. Symbol image files extracted from each diagram may be stored in the folder storing the learning data required for classification of the symbol 300.

The artificial intelligence learning unit 160 may be configured to learn an artificial intelligence model by setting the characteristics of the symbol 300 as input variables and the types of objects 400 as output variables.

The characteristics of the symbol 300 may be information representing various characteristics of the symbol 300. For example, the characteristics of the symbol 300 may be information about the size, shape, etc. of the symbol 300.

At this time, learning the artificial intelligence model may be done through machine learning.

Machine learning can mean using a model composed of multiple parameters and optimizing the parameters with given data. The artificial intelligence learning unit 160 can learn an artificial intelligence model using the output, which is the final result (output) produced through the artificial intelligence model according to the input, and the label (correct answer) for the input.

The artificial intelligence learning unit 160 can perform personal authentication not only through machine learning but also through deep learning.

The artificial intelligence model may be a model in which the characteristic information of the symbol 300 is set as an input variable and the symbol type is set as an output variable. The artificial intelligence model may include the above-described learning data folder and may be stored in the memory 170.

The duplicate object determination unit 130 may extract characteristics of the object 400 from the object 400 extracted from the diagram image 200. The characteristics of the object 400 may be information about the size, shape, etc. of the object 400.

The duplicate object determination unit 130 may be configured to determine the type of the extracted object 400 using an artificial intelligence model based on the characteristics of the extracted object 400.

As a result, the object recognition system 100 of the present invention can classify the type of the object 400 extracted from the diagram image 200 more robustly through machine learning or deep learning methods, and can more robustly classify the object 400 extracted from the diagram image 200. The symbol type of the robustly extracted object 400 can be determined.

FIG. 8 is a diagram illustrating a process for extracting objects included in a size change diagram image and a split diagram image according to an embodiment.

It may be difficult for the object extractor 110 to extract a relatively large object 400 included in the diagram image 200. Therefore, additional preprocessing may be required to extract the object 400 from the diagram image 200.

Meanwhile, for a relatively small object 400, it may be easier to extract it from a plurality of divided diagram images 200 than to simply extract it from the diagram image 200. However, the relatively large object 400 may be divided together when the diagram image 200 is divided, so it may not be easy to extract it from the divided diagram image 200.

Therefore, it may be desirable to extract the object 400 from the diagram image 200 preprocessed using different methods for the relatively large symbol 300 and the relatively small symbol 300.

Referring to FIG. 8, the object recognition system 100 uses a size change diagram created by changing the size of the diagram image 200 and a split diagram image 201 created by dividing the diagram image 200 to recognize an object ( 400) can be extracted.

The pre-processing unit 140 may be configured to change the size of the diagram image 200 to generate a size-changed diagram image 202. At this time, the size change diagram image 202 may be an image created by reducing the horizontal and vertical lengths of the diagram image 200. At this time, it may be desirable to reduce the ratio of the horizontal length and the vertical length so that it remains constant, but it is not necessarily limited to this.

The object extraction unit 110 may be configured to extract the object 400 included in the diagram image 200 based on the object 400 whose size has been changed and included in the size-changed diagram image 202.

That is, the object extractor 110 may extract the object 400 of the diagram image 200 by extracting the object 400 whose size has been reduced included in the size change diagram image 202 whose size has been reduced. As a result, the object extractor 110 can better recognize the object 400, which was relatively large and difficult to recognize before changing the size.

The pre-processing unit 140 may be configured to generate a plurality of split diagram images 201 by dividing the diagram image 200. At this time, the segmented diagram image 201 may be an image representing a certain area of the diagram image 200. For example, the preprocessor 140 may divide the diagram image 200 horizontally into three and vertically into three to generate nine split diagram images 201. However, generating the split diagram image 201 The method is not limited to this.

The object extraction unit 110 may be configured to extract the object 400 included in the diagram image 200 based on the object 400 included in the segmented diagram image 201. That is, the object extractor 110 may extract the object 400 of the diagram image 200 by dividing it and extracting the object 400 included in a partial area of the diagram image 200.

FIG. 9 is a diagram illustrating a process for recognizing text included in a diagram image according to an embodiment, and FIG. 10 is a diagram illustrating a process for extracting text according to an embodiment.

Referring to FIG. 9, the process of recognizing an object 400 according to an embodiment includes preprocessing the original diagram image 203 to generate a diagram image 200, recognizing a symbol 300, and text. It may consist of a recognition step and a step of storing the recognition result.

The text extractor may be configured to extract text included in the segmented diagram image 201 and recognize the text of the diagram image 200.

Specifically, an area where characters included in the segmentation diagram image 201 exist and a recognized result can be derived. Afterwards, by inputting the information and recognition score of the text area of the derived result, it is possible to distinguish whether each string area is a rotated or non-rotated string through a rotation estimation process. Afterwards, after recognizing the string for the image inside the text area, converting the result to fit the diagram image 200 before division and merging, the final recognition result of the string included in the diagram image 200 can be obtained. .

Referring to FIG. 10, rotation determination can be accomplished in two steps. Specifically, it can be divided into rotation judgment based on the aspect ratio of the bounding box 600 and rotation judgment based on the text recognition score. The ratio-based rotation judgment of the bounding box 600 determines whether the text is rotated when the ratio of the text area bounding box 600 (horizontal length of the bounding box 600/vertical length of the bounding box 600) is 0.5 or less. It can be judged by the string. According to the score-based rotation judgment, when the ratio of the bounding box 600 is 0.5 or more, string recognition is performed on both the non-rotated image and the rotated image, and if the result in the rotated image has a high score, the rotated string It can be judged as follows.

Referring to Figures 10 (a) and (b), in the case of a long string, it is possible to easily determine that the string has been rotated using the bounding box 600 ratio-based method. However, when the length of the vertical string is short, as shown in (c) of Figure 10, it may be difficult to determine the string rotated in this way. In this case, considering the score information of the text recognition result in the rotated image as shown in (d) of Figure 10, if the rotated image shows a higher score, it is judged to be a rotated string, and the result recognized in the rotated image is the final It can be used as a character recognition result.

Figure 11 is a diagram showing the process of preprocessing an original diagram image.

Referring to FIG. 11, the object recognition system 100 of the present invention can generate a diagram image 200 from the original diagram image 203.

The original diagram image 203 is a diagram before the diagram image 200 used for object extraction is preprocessed, and may be a diagram including an outer border and a title.

The shape of the outer border and title area of the original diagram may vary from project to project. The outer boundary consists of a single or double solid line and may have supplementary marks. Headings can be placed at the bottom left, center, or right. Additionally, there is a note area in the title area, which can mainly be located on the right side of the original diagram.

If these outer boundaries and headings are included in the diagram image 200, there is no significant difference from the symbol 300, text, and line objects, so a problem may occur in which accuracy is lowered in the object recognition process. Therefore, it is necessary to identify and remove the outer borders and headings from the original diagram image 203.

The preprocessor 140 may cut off the bottom 1/5 of the original diagram as shown in (b) of FIG. 11 to reduce the amount of calculations required in the image processing process.

The preprocessor 140 can extract all horizontal lines in the cut diagram.

The preprocessor 140 may determine the horizontal line with the longest length among the extracted horizontal lines. That is, the preprocessor 140 can identify the line with the longest length among the extracted horizontal lines, as shown in (b) of FIG. 11.

At this time, the preprocessor 140 can additionally identify lines with a length error of 5% or less from the additionally identified lines and find out one point of each line forming the outer boundary regardless of the shape of the outer boundary.

The preprocessor 140 may generate the diagram image 200 by extracting all black pixels connected to the determined longest horizontal line and removing the black pixels extracted from the original diagram image 203.

Since the title is connected to the outer border, if all black pixels connected to a point of the identified outer border are extracted, the title and outer border can be extracted as shown in (c) of FIG. 11.

At this time. The preprocessor 140 can identify the outer boundary and title area by applying the erosion operation and dilation operation of the morphology operation to the extracted black pixels, and finally, the original diagram image. The identified region within (203) can be removed.

Morphology operation is an image processing technique used to transform the shape of a specific object 400 existing in an image, and is widely used as pre-processing or post-processing in various image processing systems. Among the morphology operations, the most basic operations include erosion operations and expansion operations.

The erosion operation is an operation method that substitutes the minimum value among the values of pixels in the kernel area into the value of the current pixel, and can cause the black area to increase and the white area to decrease, as shown in (d) of FIG. 11.

In contrast, the expansion operation is an operation method that substitutes the maximum value of the pixel values in the kernel area into the value of the current pixel, and the white area increases and the black area decreases, as shown in (e) of FIG. 11. This black area is the outer boundary and title area, and performing an expansion operation after an erosion operation is called an open operation.

At least one component may be added or deleted in response to the performance of the components described above. Additionally, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed in response to the performance or structure of the system.

Figure 12 is a flowchart of an object recognition method according to an embodiment. This is only a preferred embodiment for achieving the purpose of the present invention, and of course, some components may be added or deleted as needed.

Referring to FIG. 12, the object extractor 110 may extract objects 400 including symbols 300 from the diagram image 200 (1001). At this time, the object extraction unit 110 may transmit information on the extracted objects 400 to the overlap ratio calculation unit 120.

The overlap ratio calculation unit 120 may calculate the overlap ratio between the overlap area 501 of the extracted objects 400 and the area 500 of the objects (1002). At this time, the overlap ratio calculation unit 120 may transmit information on the calculated overlap ratio to the duplicate object determination unit 130.

The duplicate object determination unit 130 may analyze the size and shape of the colors to determine the symbol types of the objects 400 and compare the symbol types of the determined objects 400 (1003).

The duplicate object determination unit 130 may determine whether the symbol types of the objects 400 are the same (1004).

If it is determined that the symbol types of the objects 400 are not the same ('No' in 1004), the Jungbuk object determination unit may determine that the object 400 is not a duplicate object 401 (1005).

If it is determined that the symbol types of the objects 400 are the same ('Yes' in 1004), the duplicate object determination unit 130 determines that the symbol types of the objects 400 correspond to an overlapped symbol set determined based on the frequency of overlap occurrence. You can determine whether it is done or not (1006).

If it is determined that the symbol types of the objects 400 correspond to the overlapping symbol set ('Yes' in 1006), the overlapping object determination unit 130 compares the overlapping ratio with the first reference threshold to determine whether it is a overlapping object. Can (1007).

If it is determined that the symbol types of the objects 400 do not correspond to the overlapping symbol set ('No' in 1006), the overlapping object determination unit 130 sets the overlapping ratio to a second reference threshold lower than the first reference threshold. By comparing, it is possible to determine whether there is a duplicate object (1008).

If the objects 400 are determined to be duplicate objects 401, the duplicate object determination unit 130 may recognize only one object 400 among the objects 400 as the symbol 300 (1009).

13 is a flowchart of an object recognition method according to another embodiment.

The preprocessor 140 may generate the diagram image 200 by preprocessing the original diagram image 203 (2001).

At this time, the preprocessor 140 may extract a horizontal line from the original diagram image 203 and determine the longest horizontal line among the extracted horizontal lines. Additionally, the preprocessor 140 may generate the diagram image 200 by extracting all black pixels connected to the longest horizontal line and removing the black pixels extracted from the original diagram image 203.

The preprocessor 140 may change the size of the diagram image 200 to generate a size-changed diagram image 202 (2002).

The preprocessor 140 may divide the diagram image 200 to generate a plurality of divided diagram images 201 (2003).

The object extraction unit 110 is based on the object 400 whose size has been changed and included in the size change diagram image 202 and the object 400 included in the segmentation diagram image 201. Object 400 can be extracted (2004).

At this time, the object 400 corresponding to the relatively large symbol 300 can be extracted from the reduced size change diagram image 202, and the object corresponding to the relatively small symbol 300 ( 400) can be extracted from the segmentation diagram image 201.

The text extractor may extract text included in the segmented diagram image 201 and recognize the text of the diagram image 200 (2005).

The processor 150 may generate a list of symbols 300 included in the diagram image 200 based on the extracted object 400 and the extracted text (2006).

In order to verify the performance of the object recognition system 100 of the diagram image 200 that recognizes the object 400 in the diagram image 200 including a plurality of symbols 300 according to an embodiment of the present invention, symbol detection An experiment was conducted.

To measure quantitative performance, 1) precision and recall calculated based on score 0.5 and IOU threshold 0.5 for the derived detections, 2) AP, AP50, AP75, which are average precision indicators used in the COCO dataset, and 3) symbol diagonal length. As a standard, the average precision was calculated by dividing it into 150 or less (small), between 150 and 250 (medium), and 250 or more (large).

In the experiment, the performance of the proposed method was examined by dividing it into four steps. First, the performance using only a small symbol network based on GFL (GFL(default)), the performance applying an adaptive threshold to the small symbol network results (GFL(A.T.)), and the performance when using a large symbol network in parallel (GFL(A.T.)) A.T.) + large) and finally, the performance when data augmentation is applied (GFL + large w/ augmentation). In the last case, experiments were conducted by adding crop, up & down scaling, blur, and noise in addition to basic flip augmentation.

The performance measurement results of the above four steps are shown in [Table 1] below. For each performance indicator, the highest performing item was indicated in bold and underline, and the second highest performing item was indicated in bold and italic.

[Table 1]

The results table showed that the use of a large symbol network, the use of adaptive threshold, and augmentation all contributed to increasing precision/recall performance. Ultimately, when all methodologies were applied, the highest performance was achieved with precision of 0.9718 and recall of 0.9827.

According to an embodiment of the present invention, research was conducted to utilize deep learning-based object detection technology to detect symbols in the diagram image 200. The previously proposed deep learning-based object detection technology measured and reported its performance based on commonly known datasets. However, it cannot be guaranteed that these performance standards appear equally in all diagram images 200. Due to the nature of deep learning technology, the characteristics of the learning data and its distribution have a significant impact on performance, so the resulting performance may vary when applied to other datasets.

According to an embodiment of the present invention, the detection performance of the symbol 300 for the diagram image 200 was investigated through experiments under various conditions, focusing on the one-stage approach, and a method showing optimal performance was proposed. Two networks were configured to compensate for performance degradation due to deviation in symbol size of the diagram image 200. A separate post-processing process was configured to improve recognition performance for the high-density symbol 300, and the final performance was derived through data augmentation.

Additionally, experiments were conducted on a separate diagram image (200) other than the dataset constructed through this study, and similar results were observed. In the diagram image 200, the symbols 300 were arranged more densely than the dataset used for learning. Due to the nature of the diagram image 200, if the diagram is drawn using the same system, the shape and text composition of the symbol 300 do not change significantly even if the overall shape of the diagram is different. Therefore, it was confirmed that the symbol detection and text recognition network trained using the learning dataset constructed through this study can be used for other diagram images (200).

As a result of the experiment, both the proposed symbol recognition and text recognition methods showed satisfactory performance. In symbol recognition, the two proposed methods, the use of a large symbol network and the use of an adaptive threshold, were found to gradually increase performance.

It was experimentally confirmed that applying augmentation improves performance if it is necessary to increase the number of correct answer instances and produce more accurate box regression results. Through quantitative performance indicators and visualized results, it was seen that the proposed GFL-based symbol detection method shows high performance.

In text recognition, existing text recognition methodologies showed satisfactory performance.

To verify the proposed method, a symbol (300) and text recognition experiment was performed using 81 diagram images (200). As a result, for the symbol (300), the precision and recall were 0.9718 and 0.9827, respectively, and for the text, the precision and recall were 0.9289 and 0.8991, respectively, confirming that the recognition performance was satisfactory for both the symbol (300) and the text. did.

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which the present invention pertains will understand that the present invention can be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

100: Object recognition system
110: Object extraction unit
120: Overlap ratio calculation unit
130: Duplicate object determination unit
140: Preprocessing unit
150: processor
160: Artificial Intelligence Learning Department
170: memory
200: Diagram image
201: Segmentation diagram image
202: Resize diagram image
203: Original diagram image
300: symbol
400: object
401: Duplicate object
500: Area of objects
501: Overlapping area
600: Bounding box

Claims (21)

In an object recognition method of a diagram image for recognizing an object in a diagram image including a plurality of symbols,
(a) extracting objects including the symbols from the diagram image by an object extractor;
(b) calculating, by an overlap ratio calculation unit, an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and
(c) determining, by a duplicate object determination unit, whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of the objects;
Step (c) above is:
(c1) analyzing the size and shape of the objects to determine symbol types of the objects and comparing the symbol types of the objects;
(c2) determining whether the symbol types of the objects correspond to an overlapping symbol set determined based on the frequency of overlapping occurrences; and
(c3) A method for recognizing objects in a diagram image, including determining the overlapping object according to a comparison result of the symbol types and whether it corresponds to the overlapping symbol set. delete According to paragraph 1,
Step (c3) above is:
If the symbol types of the objects are not the same, determining that the objects are not duplicate objects;
When the symbol types of the objects are the same and correspond to the overlapping symbol set, determining the overlapping object by comparing the overlapping ratio with a first reference threshold determined according to the symbol type; and
If the symbol types of the objects are the same and do not correspond to the overlapping symbol set, determining the overlapping object by comparing the overlapping ratio with a second reference threshold lower than the first reference threshold; comprising Object recognition method in diagram image. In an object recognition method of a diagram image for recognizing an object in a diagram image including a plurality of symbols,
(a) extracting objects including the symbols from the diagram image by an object extractor;
(b) calculating, by an overlap ratio calculation unit, an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and
(c) determining, by a duplicate object determination unit, whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of the objects;
Step (c) above is:
generating a bounding box containing an object extracted from the diagram image;
calculating an IOU ratio between the bounding boxes from a ratio of the area of the overlapping area to the area of the combined area of the overlapping bounding boxes;
determining a symbol type based on the size and shape of the extracted object;
determining the reference threshold based on the determined symbol type;
removing one of the overlapping bounding boxes if the IOU ratio is greater than or equal to the reference threshold; and
Recognizing an object included in the bounding box that has not been removed as a symbol corresponding to the bounding box. According to paragraph 4,
Object recognition method further comprising: generating at least one of position information of the symbol in the diagram image or a list of the symbols, based on a bounding box containing the object recognized as the symbol. According to paragraph 1,
It further includes the step of learning an artificial intelligence model by setting the characteristics of the symbols as input variables and the types of the objects as output variables, by the artificial intelligence learning unit,
In step (c),
extracting features of the object from the extracted object; and
Object recognition method including; determining the type of the extracted object using the artificial intelligence model based on the characteristics of the extracted object. According to paragraph 1,
Further comprising: generating a size-changed diagram image by changing the size of the diagram image, by a preprocessor,
The step of extracting the object is,
Extracting, by the object extraction unit, an object whose size has been changed and included in the size change diagram image. According to paragraph 1,
It further includes: generating a plurality of split diagram images by dividing the diagram image by a preprocessor,
The step of extracting the object is,
Object recognition method further comprising: extracting an object included in the segmentation diagram image by the object extractor. According to paragraph 1,
It further includes a step of generating, by a preprocessor, a size-changed diagram image by changing the size of the diagram image, and dividing the diagram image to generate a plurality of split diagram images,
The step of extracting the object is,
Object recognition, comprising: extracting, by the object extraction unit, an object included in the diagram image based on an object whose size has been changed and included in the size change diagram image and an object included in the segmented diagram image. method. According to clause 9,
Further comprising: extracting text included in the segmented diagram image by a text extraction unit to recognize the text of the diagram image. Object recognition method. According to paragraph 1,
By the preprocessor, a horizontal line is extracted from the original diagram image, a horizontal line with the longest length among the extracted horizontal lines is determined, all black pixels connected to the longest horizontal line are extracted, and the extracted black pixels are extracted from the original diagram image. An object recognition method further comprising: generating the diagram image by removing pixels. A computer program stored in a computer-readable recording medium to execute the object recognition method of any one of claims 1, 3 to 11. In an object recognition system for a diagram image that recognizes an object in a diagram image including a plurality of symbols,
an object extraction unit that extracts objects including the symbols from the diagram image;
an overlap ratio calculation unit that calculates an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and
A duplicate object determination unit that determines whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of objects,
The duplicate object determination unit:
Analyzing the size and shape of the objects to determine symbol types of the objects and comparing the symbol types of the objects;
Determine whether the symbol types of the objects correspond to an overlapping symbol set determined based on the frequency of overlapping occurrences; and
An object recognition system in a diagram image, configured to determine the overlapping object according to a comparison result of the symbol types and whether it corresponds to the overlapping symbol set. delete According to clause 13,
The duplicate object determination unit:
If the symbol types of the objects are not the same, determine that the objects are not duplicate objects;
If the symbol types of the objects are the same and correspond to the overlapping symbol set, determine the overlapping object by comparing the overlapping ratio with a first reference threshold determined according to the symbol type; and
If the symbol types of the objects are the same and do not correspond to the overlapping symbol set, compare the overlap ratio with a second reference threshold lower than the first reference threshold to determine the overlapping object. Diagram image. object recognition system. In an object recognition system for a diagram image that recognizes an object in a diagram image including a plurality of symbols,
an object extraction unit that extracts objects including the symbols from the diagram image;
an overlap ratio calculation unit that calculates an overlap ratio between the overlapping areas of the extracted objects and the areas of the objects; and
A duplicate object determination unit that determines whether the objects correspond to duplicate objects by comparing the overlap ratio with a reference threshold determined according to the types of objects,
The duplicate object determination unit:
creating a bounding box containing an object extracted from the diagram image;
Calculate the IOU ratio between the bounding boxes from the ratio of the area of the overlapped area to the area of the combined area of the overlapping bounding boxes;
Determine a symbol type based on the size and shape of the extracted object;
determine the reference threshold based on the determined symbol type;
if the IOU ratio is greater than or equal to the reference threshold, remove one bounding box of the overlapping bounding boxes; and
An object recognition system configured to recognize an object included in the bounding box that has not been removed as a symbol corresponding to the bounding box. According to clause 16,
An object recognition system further comprising: a processor configured to generate at least one of position information of the symbol in the diagram image or a list of the symbols, based on a bounding box containing the object recognized as the symbol. According to clause 13,
It further includes an artificial intelligence learning unit configured to learn an artificial intelligence model by setting the characteristics of the symbol as an input variable and the types of the objects as an output variable,
The duplicate object determination unit:
extracting features of the object from the extracted object; and
An object recognition system configured to determine the type of the extracted object using the artificial intelligence model based on the characteristics of the extracted object. According to clause 13,
It further includes a preprocessor configured to change the size of the diagram image to generate a size-changed diagram image, and to generate a plurality of split diagram images by dividing the diagram image,
The object extraction unit,
An object recognition system configured to extract an object included in the diagram image based on an object whose size has been changed and included in the resized diagram image and an object included in the split diagram image. According to clause 19,
An object recognition system further comprising a text extraction unit configured to extract text included in the segmented diagram image and recognize the text of the diagram image. According to clause 13,
Extract a horizontal line from the original diagram image, determine the horizontal line with the longest length among the extracted horizontal lines, extract all black pixels connected to the longest horizontal line, and remove the extracted black pixels from the original diagram image. An object recognition system comprising: a preprocessor configured to generate a diagram image.

Images