CN114973300B - Component type identification method and device, electronic equipment and storage medium - Google Patents
Component type identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114973300B CN114973300B CN202210922906.3A CN202210922906A CN114973300B CN 114973300 B CN114973300 B CN 114973300B CN 202210922906 A CN202210922906 A CN 202210922906A CN 114973300 B CN114973300 B CN 114973300B
- Authority
- CN
- China
- Prior art keywords
- component
- classification component
- target classification
- recognition
- recognition result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/42—Document-oriented image-based pattern recognition based on the type of document
- G06V30/422—Technical drawings; Geographical maps
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/147—Determination of region of interest
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/18—Extraction of features or characteristics of the image
- G06V30/1801—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/18—Extraction of features or characteristics of the image
- G06V30/18105—Extraction of features or characteristics of the image related to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19173—Classification techniques
Abstract
The application relates to a component category identification method, a component category identification device, electronic equipment and a storage medium, wherein the method comprises the steps of obtaining a target classification component in M components to be classified; obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; acquiring at least one first classification component meeting preset conditions with a target classification component from M components to be classified; acquiring Y second pre-recognition results of the first classification component, and respectively acquiring a correlation probability value of each first pre-recognition result and each second pre-recognition result; calculating to obtain N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; and determining the maximum value in the N calculation results, taking the first pre-recognition result corresponding to the maximum value as the class of the target classification component, and combining the confidence coefficient and the relevance probability value to improve the class recognition rate of the component in the CAD image recognition.
Description
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method and an apparatus for recognizing component types, an electronic device, and a storage medium.
Background
The component identification of the building drawing is a process of detecting and calibrating basic combined objects forming a building in the building drawing, is an important basis for machine understanding of the specific meaning of the building drawing and semantic understanding of the building drawing, and because components belonging to different classes may have similar image characteristics, erroneous judgment is easily caused when two classes of components with similar image characteristics are identified.
The existing component identification methods are mainly divided into four types, one type directly uses a detection model, such as a YOLO model, the end-to-end method does not consider the characteristics of linear components, and the component identification rate is low, wherein the drawing method is complex and the number of interference line segments is large; the second method uses a detection and classification identification component, a detection model is mainly used for improving the component recall rate, a classification model is mainly used for identifying the detected component type, and the classification model uses a classification model similar to VGG (Visual Geometry Group Network), however, the model is the same as the detection model and has low identification rate on the linear bar graph; the third method adopts a vector image identification component, which considers the characteristics of linear components, but needs to acquire vector data in advance, but the cost for accurately acquiring the vector data from a CAD drawing or an image is very high; and fourthly, analyzing the CAD drawing to obtain component primitives, and then identifying the components by adopting a graph and image method.
Disclosure of Invention
In order to solve the technical problem that the class identification rate of the component in the CAD image identification is low, the application provides a component class identification method, a component class identification device, electronic equipment and a storage medium.
In a first aspect, the present application provides a component class identification method, including:
acquiring a target classification component in M components to be classified; m is greater than or equal to two;
obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one;
acquiring at least one first classification component meeting preset conditions with the target classification component from the M components to be classified; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
acquiring Y second pre-recognition results of the first classification component, and respectively acquiring a relevance probability value of each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
calculating N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
determining the maximum value of the N calculation results, and taking the first pre-recognition result corresponding to the maximum value as the category of the target classification component;
optionally, before obtaining N pre-recognition results of the target classification component according to the target classification component and a preset class recognition model, the method further includes obtaining the class recognition model;
wherein the training process of the category identification model comprises the following steps:
acquiring image data and label data corresponding to the component;
training the category identification model according to the image data and the label data; the class identification model is used for identifying the class of the component;
optionally, acquiring image data corresponding to the member comprises:
acquiring a drawing graphic block corresponding to the component;
extracting strokes included in the drawing graphic block, and acquiring the maximum external rectangle of the strokes; taking the graph included by the maximum circumscribed rectangle as the image data corresponding to the component;
optionally, after obtaining the maximum bounding rectangle of the stroke, the method further includes:
acquiring a preset image size specification;
zooming the maximum external rectangle according to the image size specification to obtain an adjusted rectangle; taking the adjusted rectangle as a graph included by the maximum circumscribed rectangle;
optionally, after scaling the maximum bounding rectangle according to the image size specification to obtain an adjusted rectangle, the method further includes:
acquiring pixel values of all pixel points of the rectangle and acquiring a gray threshold;
according to the pixel values and the gray threshold value, carrying out binarization on the rectangle to obtain a binary image of the component; the part of the pixel value which is larger than the gray threshold value is used as the foreground of the binary image;
obtaining the pixel coordinates of each point of the foreground of the binary image according to a contour extraction function to form a coordinate set;
comparing the original image of the rectangle with the binary image, and extracting color information of each coordinate point in the rectangular area corresponding to the foreground of the binary image to form a color set;
carrying out rotation and scaling transformation on the binary image according to a preset rule according to an affine transformation function to obtain a transformation matrix;
obtaining a target image according to the transformation matrix, the coordinate set and the color set; taking the target image as image data corresponding to the component;
optionally, in the obtaining of the correlation probability values of each of the first pre-recognition results and each of the second pre-recognition results, the obtaining of any one of the correlation probability values includes:
acquiring probability values of the first pre-recognition result and the second pre-recognition result;
obtaining a two-dimensional Gaussian function, wherein the two-dimensional Gaussian function is used for representing the distance between the target classification component and the first classification component, and the Gaussian function value of the two-dimensional Gaussian function is in negative correlation with the distance;
multiplying the Gaussian function value and the probability value to obtain a correlation probability value between the first pre-recognition result and the second pre-recognition result;
optionally, any of the confidence levels is greater than a second threshold; the second threshold is greater than zero and less than one;
any of the correlation probability values is greater than or equal to zero and less than or equal to one.
In a second aspect, the present application provides a component category identification device, the device comprising:
the first acquisition module is used for acquiring a target classification component in the M components to be classified; m is greater than or equal to two;
the determining module is used for obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one;
the second acquisition module is used for acquiring at least one first classification component which meets preset conditions with the target classification component from the M components to be classified; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
the third acquisition module is used for acquiring Y second pre-recognition results of the first classification component and respectively acquiring a correlation probability value of each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
the calculation module is used for calculating and obtaining N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
and the identification module is used for determining the maximum value in the N calculation results and taking the first pre-identification result corresponding to the maximum value as the category of the target classification component.
In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
a processor configured to implement the steps of the component category identification method according to any one of the embodiments of the first aspect when executing a program stored in the memory.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the component class identification method according to any one of the embodiments of the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:
according to the method provided by the embodiment of the application, a target classification component in M components to be classified is obtained; m is greater than or equal to two; obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one; acquiring at least one first classification component with the distance from a target classification component smaller than a first threshold value from M components to be classified; acquiring Y second pre-recognition results of the first classification component, and respectively acquiring a correlation probability value of each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one; calculating to obtain N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence degree corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result; and determining the maximum value of the calculation results from the N calculation results, and taking the first pre-recognition result corresponding to the maximum value as the category of the target classification component. When the method is used for classifying the target classification component, the correlation probability existing between the first classification component and the target classification component, the distance between the first classification component and the target classification component is smaller than a first threshold value, the calculation result of each first pre-recognition result is obtained according to the correlation probability values respectively corresponding to N first pre-recognition results and Y second pre-recognition results and the confidence degrees of the N first pre-recognition results, the first pre-recognition result corresponding to the calculation result with the largest numerical value is determined to serve as the class of the target classification component, and the class recognition rate of the component in the CAD image recognition can be improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a system architecture diagram of a component class identification method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart illustrating a component class identification method according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a flow of acquiring training image data in a component class identification method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a component category identification device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The first embodiment of the present application provides a component class identification method, which may be applied to a system architecture as shown in fig. 1, where the system architecture includes at least a terminal 101 and a server 102, and a communication connection is established between the terminal 101 and the server 102, and may be connected through a wired or wireless network, and a specific communication transmission protocol is not limited.
The method can be applied to the terminal 101 or the server 102 in the system architecture, wherein the terminal 101 can be a desktop computer, a tablet computer, a notebook computer, a super computer, and the like, and the server 102 can be a local server, a cloud server, or a server cluster.
It should be noted that the network system architecture shown in fig. 1 is only an example, and may further include more components such as routers and switches.
Based on the system architecture shown in fig. 1, the component category identification method provided in the embodiment of the present application is described in detail below with reference to the related drawings, and fig. 2 is a schematic flow chart of the component category identification method provided in the embodiment of the present application, and as shown in fig. 2, the method includes:
When the target classification member is identified, other members to be classified in a certain range around the target classification member need to be combined, so that at least two members to be classified need to be included, and one target classification member is determined from the members to be classified.
According to the target classification component and a preset class recognition model, a plurality of first pre-recognition results of the target classification component can be obtained, and N first pre-recognition results with confidence degrees larger than a second threshold value can be selected from the plurality of first pre-recognition results, wherein the second threshold value is larger than zero and smaller than one.
The first classification means satisfying the preset condition may be one or more.
Wherein any correlation probability value is greater than or equal to zero and less than or equal to one. For example, if N is 3,Y is 2, and the first classification component satisfying the preset condition is one, the target classification component includes three possible categories (for example, including category 1, category 2, and category 3), and the first classification component includes two possible categories (for example, including category a and category B), at this time, the correlation probability value between category 1 and category a, the correlation probability value between category 1 and category B, the correlation probability value between category 2 and category a, the correlation probability value between category 2 and category B, the correlation probability value between category 3 and category a, and the correlation probability value between category 3 and category B need to be obtained, that is, the total may include N × Y correlation probability values, which is here is 6 correlation probability values.
It should be noted that the relevance probability value may be a preset value representing the relevance between the components, and the relevance probability value is in the range of [0,1], for example, the elevator car component necessarily has an elevator door at the nearest position, so the relevance probability value between the elevator car component and the elevator door component is 1, whereas the probability of having a vertical hinged door beside the elevator car is random, so the relevance probability value may be 0.5.
In one embodiment, the obtaining of the correlation probability value of each first pre-recognition result and each second pre-recognition result respectively includes: and obtaining a probability value of the first pre-recognition result and the second pre-recognition result, and obtaining a two-dimensional Gaussian function, wherein the two-dimensional Gaussian function is used for representing the distance between the target classification component and the first classification component, the Gaussian function value of the two-dimensional Gaussian function is in negative correlation with the distance, and the Gaussian function value is multiplied by the probability value to be used as a correlation probability value between the first pre-recognition result and the second pre-recognition result.
In this embodiment, the variance of the two-dimensional gaussian function may be used as the distance between the target classification component and the first classification component, and the correlation probability value is calculated only when the distance is smaller than the first threshold.
The probability value between the first pre-recognition result and the second pre-recognition result can be preset, the relevance probability value between the two members is related to the probability value between the pre-recognized categories of the two members and the distance between the two members, the distance between the members can be used as a reference in member category recognition more accurately, and the recognition rate of the member categories is improved.
Still taking the target classification component comprising three categories, i.e. category 1, category 2 and category 3, the first classification component comprising category a and category B as an example, each possible category of the target classification component corresponds to one calculated result, and if the confidence of category 1 is 0.9, the confidence of category 2 is 0.85, the confidence of category 3 is 0.8, and the calculated result corresponding to category 1 is =0.9 (the correlation probability value between category 1 and category a) +0.9 (the correlation probability value between category 1 and category B), and similarly, the calculated results corresponding to category 2 and category 3 can be calculated respectively.
Note that the calculation result may also be referred to as a path probability value.
And step 206, determining the maximum value of the N calculation results, and taking the first pre-recognition result corresponding to the maximum value as the category of the target classification component.
The maximum value is determined in three calculation results corresponding to the category 1, the category 2 and the category 3 respectively, the category corresponding to the maximum value is taken as the category of the target classification component, and the category identification rate of the component in the CAD image identification can be improved due to the combination of the correlation probability value existing between the first classification component with the distance of the target classification component smaller than the first threshold value and the target classification component.
In one embodiment, before obtaining N pre-recognition results of the target classification component according to the target classification component and the preset class recognition model, the method further includes obtaining the class recognition model.
The training process of the category identification model comprises the following steps: acquiring image data and label data corresponding to the component, and training a category identification model according to the image data and the label data, wherein the category identification model is used for identifying the category of the component.
The CAD structural member is composed of a series of line strokes, texture information is less compared with a common RGB image, multiple drawing patterns can exist in one structural member, the same drawing pattern can exist in different structural members, and certain challenges are brought to CAD drawing identification. In the embodiment, the component recognition is regarded as a special character and symbol recognition, because the component and the character are also composed of lines and have similar line characteristics.
The category identification model comprises a Convolutional Recurrent Neural Network (CRNN) model, the CRNN model is an existing character identification classical model, a Network structure of the model can slice an input image according to a certain width, extract strokes and combine stroke slice characteristics, and therefore the problem of extraction of line-shaped image characteristics can be solved more finely, the line-shaped image identification rate is improved, and components in a CAD image in a building drawing are generally composed of a series of line strokes, so that the category identification rate of the components is improved by the category identification model. The present application differs from the conventional model training in that the input used in the CRNN model is a single component image block and the label is a single label. The method is mainly used for predicting multiple categories of the CRNN model as much as possible, improving the category recall rate, and then obtaining the final accurate category of the target classification component from the multiple predicted categories through the identification post-processing step, namely, through the correlation probability value between the target classification component and other components in a certain range around the target classification component, and combining the confidence coefficient of the predicted category.
In one embodiment, acquiring image data corresponding to a component includes: acquiring a drawing graphic block corresponding to the component, extracting strokes included in the drawing graphic block, acquiring a maximum external rectangle of the strokes, and taking the graph included in the maximum external rectangle as image data corresponding to the component.
The drawing graphic block corresponding to the component refers to the graphic of the component composed of strokes, or the drawing graphic block can be understood as the concrete form of the component. The input used in the model of this embodiment is a drawing block of a single component, and accordingly, the label is a single label corresponding to the single component. The design is mainly used for predicting multiple types of models as much as possible, improving the type recall rate and then obtaining the final accurate component type through recognition post-processing.
In one embodiment, after obtaining the maximum bounding rectangle of the stroke, the method further comprises: and acquiring a preset image size specification, zooming the maximum external rectangle according to the image size specification to obtain an adjusted rectangle, and taking the adjusted rectangle as a graph included by the maximum external rectangle.
In this embodiment, when the maximum circumscribed rectangle is scaled, the maximum circumscribed rectangle may be scaled according to the long side or the short side of the maximum circumscribed rectangle as a reference, for example, if the pixel value of the preset image size specification is 280 × 46, the long side may be scaled to 280, at this time, if the scaled short side is less than or equal to 46, the adjustment is completed, if the short side is greater than 46, the short side is further scaled to 46, and meanwhile, the long side is adaptively adjusted, so as to ensure that the size of the adjusted rectangle is not greater than the preset image size specification.
In an embodiment, after scaling the maximum bounding rectangle according to the image size specification to obtain the adjusted rectangle, as shown in fig. 3, the method further includes:
301, acquiring pixel values of all pixel points of a rectangle and acquiring a gray threshold;
305, performing rotation and scaling transformation on the binary image according to a preset rule according to an affine transformation function to obtain a transformation matrix;
and step 306, obtaining a target image according to the transformation matrix, the coordinate set and the color set, and taking the target image as image data corresponding to the component.
In this embodiment, the grayscale threshold may be set to 5, the value with the pixel value greater than 5 is set to 255, and the other values are set to 0, a binary image of the component is obtained, that is, the foreground with the pixel value greater than 5 is used as the binary image, the pixel coordinates of each point corresponding to the foreground of the binary image are extracted to form a coordinate set, the color information of each coordinate point in the rectangular region corresponding to the foreground of the binary image is extracted to form a color set, because the angles of the component in the CAD image are different, the binary image may be rotated and scaled according to an affine transformation function to obtain the binary image of the maximum projection area of the component, the relationship between the rotation and the scaling is used as a transformation matrix, the transformed coordinate set of the binary image may be obtained according to the transformation matrix and the coordinate set, the target image may be obtained according to the transformed coordinate set and the color set, and the target image is used as the image data of the component.
It should be noted that the class recognition model can be trained according to the image data of the member, so that the class recognition model can better recognize and classify the class of the target classification member to be recognized, and the recognition rate of the target classification member is improved.
In one embodiment, it is only concerned with how a particular class is identified for the resulting building blocks. The component category identification algorithm flow is divided into three parts, namely training data preparation, model training and identification result post-processing.
First, training data preparation.
Since there are many shapes of a component, such as rectangle, triangle, circle, etc., and the label of a drawing component is not the component itself, it is necessary to convert the component data into a uniform data format in order to facilitate model training.
For the training image portion. In this embodiment, the maximum circumscribed rectangle is obtained for the obtained component drawing graphic block according to the stroke, and the graph included in the rectangle is used as the image data to be trained. Because the members have different sizes, all the members need to be uniformly scaled to a certain size according to the short sides, such as 280 × 46, but the stroke lines of the members are easy to break and distort in the scaling process, in the embodiment, the gray threshold value is 5, the value of the pixel value greater than 5 is 255, the other values are 0, a binary image of the members is obtained, the pixel coordinate of each point in the foreground of the binary image can be obtained through an opencv contour extraction function, a coordinate set of the whole contour is obtained, meanwhile, the binary image is compared with the original RGB image, RGB value information of the corresponding coordinate position is extracted and stored together with the coordinate set, and the coordinate set describes the whole shape and color information of the whole member; rotating the binary image by using an opencv affine transformation function, and carrying out scaling transformation to obtain a transformation matrix; and mapping each coordinate in the obtained pixel coordinate set to a new coordinate position by using the transformation matrix to obtain a new coordinate set, wherein the coordinate set describes the shape of the transformed binary image, and the RGB color value information stored before the corresponding coordinate position is assigned to obtain the RGB image required by training.
A label portion is corresponded to the training data. The member label adopts the real class name of the member, such as a shutter, the number of which is 1, an elevator door, the number of which is 2, and the like. The numbers are used for labels for subsequent model training, for example, label 1 represents a shutter, label 2 represents an elevator door, etc.
It should be noted that, in the process of scaling the component, in order to ensure that the component is not deformed, the length and the width are scaled according to the same proportion, and the size of the scaled component is preferably smaller than or equal to 280 × 46, of course, scaling the component to 280 × 46 in this embodiment is merely an example, and is not limited thereto in practice, it should be understood that the grayscale threshold is set to 5 as an example, and may be set to other values as needed, and the embodiment of the present application is not limited thereto.
And in the second part, model training.
The CRNN model is an existing classical character recognition model, a network structure of the CRNN model can slice an input image according to a certain width, strokes are extracted, and stroke slice characteristics are combined, so that the problem of extraction of line-shaped image characteristics can be more carefully solved, and the line-shaped image recognition rate is improved. The difference from the conventional model training is that the input used in the model of the present embodiment is a single-component image block, and the label is a single label. The design is mainly used for predicting multiple categories of the model as much as possible, improving the category recall rate and then obtaining the final accurate component category through recognition post-processing.
And thirdly, post-processing the recognition result.
In this embodiment, the identification result with the confidence coefficient greater than 0.8 is output by the CRNN model, so that a plurality of results are obtained, and a plurality of results have a certain similarity, and because the drawing diversity of the components and the same drawing of different components exist, the identification of the model is difficult when the model is used for processing such a problem. Therefore, in the post-processing process of the present embodiment, in order to obtain an accurate class corresponding to a component image block, the above problem is solved by referring to classes of other components around a target classification component on the basis of model identification, so as to further improve the component identification rate, and the specific method is as follows:
a relevance probability value is calculated. According to the previously labeled data, the prior knowledge of each component, namely the correlation probability of one component with other types of components, is counted, and the value range is [0,1], in the embodiment, the value is mainly concentrated near 0 and 1. For example, the elevator car member will necessarily have an elevator door in the nearest position, so the probability value of the correlation between the elevator car and the elevator door is 1, whereas the probability of having a side hung door next to the elevator car is random, so the probability value is 0.5. These probability values need to be counted according to the drawing data. However, since the strength of the correlation between the members decreases with the increase of the distance between the members, the embodiment adds a two-dimensional gaussian function on the basis of the probability value, the mean value of the function is the x and y coordinates of the current member, and the variance can be set as required, for example, as 224, which means that the embodiment performs post-processing in the image range with the current member as the center and the radius of 224, and multiplies the gaussian function value by the probability value to obtain the probability value of the correlation between the two members.
A path probability value is calculated. For one object classification component, the model outputs possible classes and corresponding confidences of the object classification component according to a confidence threshold, and other components within a range of 224 pixels of radius centered on the object classification component also obtain a plurality of corresponding classes and corresponding confidences. In order to obtain the final class of the target classification member according to the information, the present embodiment calculates the probability values of the possible classes obtained by the target classification member and the paths of other members within the range of the radius 224, and takes the class with the maximum value as the final class of the target classification member. The path probability value refers to the calculation result in the above embodiment.
The specific process of calculating the path probability value is as follows:
if the classification result obtained by the target classification component to be processed through the CRNN model identification includes three classification results, which are respectively classification 1, classification 2, and classification 3. The confidence of the category 1 is confidence 1, the confidence of the category 2 is confidence 2, the confidence of the category 3 is confidence 3, and the confidence 1, the confidence 2 and the confidence 3 are all greater than 0.8. The path probability value of the category 1 is calculated by multiplying the confidence 1 of the category 1 by the relevance probability value of the other members in the range of the radius of 224 pixels, and the path probability value is the sum of all the values. Category 2 and Category 3 are similar. And respectively calculating path probability values according to the category 1, the category 2 and the category 3, and taking the category corresponding to the highest value in the path probability values as the final category of the target classification member.
It should be noted that, in this embodiment, the recognition result with the confidence of the CRNN model output prediction result being greater than 0.8 is taken as an example, and here, 0.8 may also be set to other values according to needs, and is not limited.
Based on the same technical concept, a second embodiment of the present application provides a component category identification device, as shown in fig. 4, including:
a first obtaining module 401, configured to obtain a target classification component from among M components to be classified; m is greater than or equal to two;
a determining module 402, configured to obtain, according to the target classification component and a preset class identification model, N first pre-identification results of the target classification component and a confidence corresponding to each of the first pre-identification results; n is greater than or equal to one;
a second obtaining module 403, configured to obtain, from among the M members to be classified, at least one first classification member that meets a preset condition with the target classification member; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
a third obtaining module 404, configured to obtain Y second pre-recognition results of the first classification component, and obtain a probability value of a correlation between each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
a calculating module 405, configured to calculate N calculation results of the target classification component according to the confidence and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
an identifying module 406, configured to determine a maximum value of the N calculation results, and use the first pre-identification result corresponding to the maximum value as a category of the target classifying means.
In this embodiment, when the device classifies the target classification component, the device obtains a calculation result of each first pre-recognition result according to the correlation probability values respectively corresponding to the N first pre-recognition results and the Y second pre-recognition results and the confidence degrees of the N first pre-recognition results by combining the correlation probabilities between the first classification component and the target classification component, of which the distance from the first classification component to the target classification component is smaller than the first threshold, and determines the first pre-recognition result corresponding to the calculation result with the largest value as the class of the target classification component, so that the class recognition rate of the component in the CAD image recognition can be improved.
As shown in fig. 5, a third embodiment of the present application provides an electronic device, which includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 complete mutual communication via the communication bus 114,
a memory 113 for storing a computer program;
in one embodiment, the processor 111, configured to execute the program stored in the memory 113, implements the component class identification method provided in any one of the foregoing method embodiments, including:
acquiring a target classification component in M components to be classified; m is greater than or equal to two;
obtaining N first pre-recognition results of the target classification component and a confidence corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one;
obtaining at least one first classification component meeting preset conditions with the target classification component from the M components to be classified; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
acquiring Y second pre-recognition results of the first classification component, and respectively acquiring a relevance probability value of each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
calculating N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
determining the maximum value of the N calculation results, and taking the first pre-recognition result corresponding to the maximum value as the category of the target classification component.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.
The fourth embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the component category identification method as provided in any of the method embodiments described above.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. In the description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of description of the present application, and have no specific meaning by themselves. Thus, "module", "component" or "unit" may be used mixedly.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A component class identification method, the method comprising:
acquiring a target classification component in M components to be classified; m is greater than or equal to two;
obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one;
obtaining at least one first classification component meeting preset conditions with the target classification component from the M components to be classified; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
acquiring Y second pre-recognition results of the first classification component, and respectively acquiring a relevance probability value of each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
calculating N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
determining the maximum value of the N calculation results, and taking the first pre-recognition result corresponding to the maximum value as the category of the target classification component.
2. The method according to claim 1, wherein before obtaining N pre-recognition results of the target classification component according to the target classification component and a preset class recognition model, the method further comprises obtaining the class recognition model;
wherein the training process of the category identification model comprises the following steps:
acquiring image data and label data corresponding to the component;
training the category identification model according to the image data and the label data; the class identification model is used to identify a class of the component.
3. The method of claim 2, wherein acquiring image data corresponding to a component comprises:
acquiring a drawing graphic block corresponding to the component;
extracting strokes included in the drawing graphic block, and acquiring the maximum external rectangle of the strokes; and taking the graph included by the maximum circumscribed rectangle as the image data corresponding to the member.
4. The method of claim 3, wherein after obtaining the maximum bounding rectangle for the stroke, the method further comprises:
acquiring a preset image size specification;
zooming the maximum external rectangle according to the image size specification to obtain an adjusted rectangle; and taking the adjusted rectangle as a graph included by the maximum circumscribed rectangle.
5. The method of claim 4, wherein after scaling the maximum bounding rectangle according to the image size specification to obtain the adjusted rectangle, the method further comprises:
acquiring pixel values of all pixel points of the rectangle and acquiring a gray threshold;
according to the pixel values and the gray threshold value, carrying out binarization on the rectangle to obtain a binary image of the component; the part of the pixel value which is larger than the gray threshold value is used as the foreground of the binary image;
obtaining the pixel coordinates of each point of the foreground of the binary image according to a contour extraction function to form a coordinate set;
comparing the original image of the rectangle with the binary image, and extracting color information of each coordinate point in the rectangular area corresponding to the foreground of the binary image to form a color set;
carrying out rotation and scaling transformation on the binary image according to a preset rule according to an affine transformation function to obtain a transformation matrix;
obtaining a target image according to the transformation matrix, the coordinate set and the color set; and taking the target image as image data corresponding to the component.
6. The method according to claim 1, wherein the obtaining of the correlation probability value between each of the first pre-recognition results and each of the second pre-recognition results comprises:
acquiring probability values of the first pre-recognition result and the second pre-recognition result;
obtaining a two-dimensional Gaussian function, wherein the two-dimensional Gaussian function is used for representing the distance between the target classification component and the first classification component, and the Gaussian function value of the two-dimensional Gaussian function is in negative correlation with the distance;
and multiplying the Gaussian function value and the probability value to obtain a correlation probability value between the first pre-recognition result and the second pre-recognition result.
7. The method of claim 1, wherein either of said confidences is greater than a second threshold; the second threshold is greater than zero and less than one;
any of the correlation probability values is greater than or equal to zero and less than or equal to one.
8. A component category identification device, characterized in that the device comprises:
the first acquisition module is used for acquiring a target classification component in the M components to be classified; m is greater than or equal to two;
the determining module is used for obtaining N first pre-recognition results of the target classification component and a confidence coefficient corresponding to each first pre-recognition result according to the target classification component and a preset class recognition model; n is greater than or equal to one;
the second acquisition module is used for acquiring at least one first classification component which meets preset conditions with the target classification component from the M components to be classified; the preset conditions include: the distance of the target classification component from the first classification component is less than a first threshold;
a third obtaining module, configured to obtain Y second pre-recognition results of the first classification component, and obtain a probability value of a correlation between each first pre-recognition result and each second pre-recognition result; y is greater than or equal to one;
the calculation module is used for calculating and obtaining N calculation results of the target classification component according to the confidence coefficient and the relevance probability value; any calculation result is the sum of the products of the confidence corresponding to any first pre-recognition result and each relevance probability value corresponding to the first pre-recognition result;
and the identification module is used for determining the maximum value in the N calculation results and taking the first pre-identification result corresponding to the maximum value as the category of the target classification component.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the component class identification method of any one of claims 1 to 7 when executing a program stored on a memory.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the component category identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210922906.3A CN114973300B (en) | 2022-08-02 | 2022-08-02 | Component type identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210922906.3A CN114973300B (en) | 2022-08-02 | 2022-08-02 | Component type identification method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114973300A CN114973300A (en) | 2022-08-30 |
| CN114973300B true CN114973300B (en) | 2022-10-11 |
Family
ID=82970272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210922906.3A Active CN114973300B (en) | 2022-08-02 | 2022-08-02 | Component type identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114973300B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030486B (en) * | 2023-02-20 | 2024-02-09 | 中国建筑西南设计研究院有限公司 | Component characteristic data extraction method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113657202A (en) * | 2021-07-28 | 2021-11-16 | 万翼科技有限公司 | Component identification method, training set construction method, device, equipment and storage medium |
| WO2022116744A1 (en) * | 2020-12-03 | 2022-06-09 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for object recognition |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5538967B2 (en) * | 2009-06-18 | 2014-07-02 | キヤノン株式会社 | Information processing apparatus, information processing method, and program |
-
2022
- 2022-08-02 CN CN202210922906.3A patent/CN114973300B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022116744A1 (en) * | 2020-12-03 | 2022-06-09 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for object recognition |
| CN113657202A (en) * | 2021-07-28 | 2021-11-16 | 万翼科技有限公司 | Component identification method, training set construction method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Computer Assisted Recognition of Breast Cancer in Biopsy Images via Fusion of Nucleus-Guided Deep Convolutional Features;Kalpana Georgea,et al.;《Computer methods and programs in biomedicine》;20201231;全文 * |
| 利用Faster R-CNN自动识别建筑平面图功能构件的方法;马焜阳 等;《测绘科学技术学报》;20191207;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114973300A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiao et al. | A configurable method for multi-style license plate recognition | |
| WO2021169161A1 (en) | Image recognition method, recognition model training method and apparatuses related thereto, and device | |
| WO2020238054A1 (en) | Method and apparatus for positioning chart in pdf document, and computer device | |
| EP3806064A1 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
| CN108549870A (en) | A kind of method and device that article display is differentiated | |
| CN109343920B (en) | Image processing method and device, equipment and storage medium thereof | |
| US11915500B2 (en) | Neural network based scene text recognition | |
| Vanetti et al. | Gas meter reading from real world images using a multi-net system | |
| CN113657202B (en) | Component identification method, training set construction method, device, equipment and storage medium | |
| US20220383128A1 (en) | Image-based anomaly detection based on a machine learning analysis of an object | |
| CN110807404A (en) | Form line detection method, device, terminal and storage medium based on deep learning | |
| CN112001362A (en) | Image analysis method, image analysis device and image analysis system | |
| CN114973300B (en) | Component type identification method and device, electronic equipment and storage medium | |
| CN108960246B (en) | Binarization processing device and method for image recognition | |
| Wu et al. | Automatic gear sorting system based on monocular vision | |
| CN110889437A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112784494B (en) | Training method of false positive recognition model, target recognition method and device | |
| Chen et al. | Image segmentation based on mathematical morphological operator | |
| CN111199240A (en) | Training method of bank card identification model, and bank card identification method and device | |
| CN111680680B (en) | Target code positioning method and device, electronic equipment and storage medium | |
| CN111199228B (en) | License plate positioning method and device | |
| CN115345895B (en) | Image segmentation method and device for visual detection, computer equipment and medium | |
| CN112241736A (en) | Text detection method and device | |
| CN114936395A (en) | Household type graph recognition method and device, computer equipment and storage medium | |
| CN112288045B (en) | Seal authenticity distinguishing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |