CN105139388B - The method and apparatus of building facade damage detection in a kind of oblique aerial image - Google Patents
The method and apparatus of building facade damage detection in a kind of oblique aerial image Download PDFInfo
- Publication number
- CN105139388B CN105139388B CN201510494876.0A CN201510494876A CN105139388B CN 105139388 B CN105139388 B CN 105139388B CN 201510494876 A CN201510494876 A CN 201510494876A CN 105139388 B CN105139388 B CN 105139388B
- Authority
- CN
- China
- Prior art keywords
- building facade
- building
- window
- door
- mrow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of method of building facade damage detection in oblique aerial image, including step 1, and building facade is split using the k means clustering algorithms based on rough set theory, obtains the door and window of building facade；Two, rim detection is carried out to the door and window of building facade using canny algorithms, obtains the edge feature of door and window；Three, the edge feature is counted using the Gini coefficient in economics, obtains the Gini coefficient of building facade；Four, judge whether building facade is damaged according to the Gini coefficient.The present invention, which has, not to be needed before prior information and calamity in the case of data, simply can efficiently be carried out the damage detection of building facade, be reduced the complexity of method, saved production cost；The index of Gini coefficient in economics as the damage detection of building facade is introduced, the architectural feature of building facade can be made full use of to judge to damage, this provides solution method for the precision and automaticity of raising building Damage assessment.
Description
Technical field
The present invention relates to remote sensing image applied technical field, more particularly, to building facade in a kind of oblique aerial image
The method and apparatus for damaging detection.
Background technology
Natural calamity makes the life of the mankind and property take a bath for a long time, is the huge of human survival and development
Obstacle.Remote sensing technology has the characteristics that revisiting period is short, investigative range is big, aggregation of data is high, is carried for disaster monitoring with assessment
A kind of favourable means are supplied.With the development of various monitoring means and new and high technology, the detection of traditional disaster is with assessing progressively
Developed from qualitative statistical estimation to the direction quantitatively finely assessed.Key element of the building as people's production and living, it is natural
The detection of information is damaged to it after occurring for disaster and extraction has great importance, and it can be that calamity emergency is responded with recovering after calamity
Rebuild and important decision foundation is provided.In view of the complexity of building damage detection, will not only judge building elevation and area
Etc. the change of information, building top surface and facade damage information are also judged, therefore how comprehensive height is carried out to building
The quantitative Damage assessment of precision is the focus studied at present.
The traditional aviation image of the oblique aerial photography technological break-through of fast development in recent years can only be shot from vertical angle
Limitation, by carrying more sensors on same flying platform simultaneously from a vertical, difference such as three or more are tilted
Angle acquisition image, the multi-faceted information of building can be gathered very well.Many scholars are utilizing oblique photograph e measurement technology
A large amount of in-depth studies work have been done to extract the complete information of building aspect, it is clear that building is damaged using this technology
Detection can make up the deficiency of traditional detection method.Building Top-print information is damaged because texture information is fairly simple using it
It is more to ruin the research of detection, also achieves good achievement；But building facade information texture information is very abundant, building
What door and window and damage occurred collapse, crack and it is damaged there is certain interference from each other, this is to the damage inspection of building facade
Survey causes very big difficulty.
Therefore, damage detection how is carried out to the building facade information in oblique aerial image for improving building damage
The accurate quantification ruined, which is assessed, to have great importance.
The texture of largely abundant information, particularly building facade is contained in high-resolution oblique aerial remote sensing image
Information is very abundant, and domestic and foreign scholars to building facade damage the typical method of detection using oblique aerial image at present
It is included in following two aspects：1) the building facade damage detection based on Mono temporal.Because calamity rear-inclined aviation image is easy to
Obtain, therefore this kind of method more conforms to actual production demand.Such method is divided into based on structural information and textural characteristics again
Damage detection, mainly damage detection is carried out using features such as gray level co-occurrence matrixes or Tamura based on textural characteristics, but by
It is larger in building facade textures roughness, and textural characteristics detection is mainly directed towards in the detection of close grain feature, therefore it is uncomfortable
Close the damage for distinguishing Mono temporal；The information such as building facade crack are mainly extracted based on structural information method to detect damage, but
Because the structural information meeting fracture extraction that building facade enriches causes greatly to disturb, therefore such method is also more difficult.
2) the building facade damage detection based on multidate.The method that such method is based primarily upon change detection carries out building facade
Damage detection, but because the oblique aerial image before calamity is typically difficult to obtain, particularly close on the inclination shadow of disaster-stricken preceding period
It is also current difficult point that as being more difficult to obtain, and before calamity after calamity, how the obliquity effects of multidate, which carry out high registration accuracy,.
Therefore easily obtain, it is necessary to urgently find a kind of data, judge that automaticity is high, extraction result is relatively accurate high and meets reality
Produce the damage detection method needed.
The content of the invention
It is an object of the invention to propose a kind of method and apparatus of building facade damage detection in oblique aerial image,
The present invention takes full advantage of the structural information for tilting building facade in image, in combination with the Gini coefficient conduct in economics
Index is damaged, significantly improves the precision of building facade damage detection, feature is：
(1) this method, which has, does not need before prior information and calamity in the case of data, simply can efficiently carry out building
Facade damage detection, reduces the complexity of method, also saves production cost.
(2) index of the Gini coefficient as the damage detection of building facade in economics is introduced, can be made full use of
The architectural feature of building facade judges to damage, and not only improves the automaticity and precision of judgement, and meet actual life
The needs of production.
To use following technical scheme up to this purpose, the present invention：
A kind of method of building facade damage detection in oblique aerial image, including：
Step 1, building facade is split using the k-means clustering algorithms based on rough set theory, obtains building
The door and window of facade；
Step 2, rim detection is carried out to the door and window of building facade using canny algorithms, the edge for obtaining door and window is special
Sign；
Step 3, the edge feature is counted using the Gini coefficient in economics, obtain building facade
Gini coefficient；
Step 4, judge whether building facade is damaged according to the Gini coefficient.
Wherein, the step 1, building facade is split using the k-means clustering algorithms based on rough set theory,
The door and window of building facade is obtained, is specially：
The gray value of pixel is f, wherein f=0 in S110, image, 1,2 ..., 255, the k obtained using rough set theory
Individual central point is as preliminary classification mean μ1,μ2,μ3,…,μk；
S120, the distance between the gray value f of each pixel and previous step preliminary classification mean μ D in image are calculated, will be every
Individual pixel is assigned to the class nearest apart from initial classes average, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ... k) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；
S130, new cluster centre is calculated for i=1,2 ..., k, update class average:
In formula, NiIt isIn number of pixels,It is the set of certain class pixel, i is the i-th class pixel, and m is iteration time
Number；
S140, all pixels are investigated one by one, if i=1,2 ..., k, hadThen algorithmic statement, terminate,
Otherwise return to S120 and continue next iteration.
Wherein, the step 2, rim detection is carried out to the door and window of building facade using canny algorithms, obtains door and window
Edge feature, be specially：
S210, the door and window progress rim detection using canny algorithms to building facade, obtain the door and window of building facade
Edge；
S220, because most of building facade is all perpendicular to ground, count first parallel to ground parallel lines it
Between range distribution, then calculate distance vector histogram, finally obtain door and window edge feature；Flow is as follows：
A) because building facade may be damaged, therefore the contour line that door and window rim detection obtains may not mutually be put down
OK, therefore statistics is parallel to the method for the distance between the parallel lines on ground distribution use：Along horizontal direction every certain
Step-length counts to building facade to vertical direction, calculates vertical direction and closes on the distance between two pixels, note
For di, whole facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represents the classification number of distance；
B) statistics with histogram function D (d are utilizedi)=niDistance vector histogram is counted, histogram vector of then adjusting the distance
Variable niAscending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤…≤nk；Vector n is to build
Build the edge feature of thing facade door and window.
Wherein, the step 3, the edge feature is counted using the Gini coefficient in economics, built
The Gini coefficient of thing facade, it is specially：
S310, the edge feature for the door and window for assuming to extract in image are g, are distance vector histogram g by g distribution statisticses
=[g1,g2,g3,…,gk], the element in histogram vector of adjusting the distance is sorted from small to large, obtains new Nogata set of graphs
For g '=[g '1,g’2,g’3,…,g’k], then the Gini coefficient formula of measurement image rule degree is：
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, G scope be from 0 to
1, G is bigger, and building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage
Threshold value is that the statistical law in 0.45, and economics is basically identical；
S320, bring into the statistics edge feature vector n in step 2 as g in (2) formula, obtain the Geordie of elevation of building
Coefficient.
Wherein, the step 4, judge whether building facade is damaged according to the Gini coefficient, be specially：
When Gini coefficient G is more than 0.45, represent that building facade is intact；Conversely, when Gini coefficient G is less than 0.45,
Represent that elevation of building is damaged.
The device of building facade damage detection in a kind of oblique aerial image, including：
Building facade cutting unit, for being stood using the k-means clustering algorithms based on rough set theory to building
Face is split, and obtains the door and window of building facade；
Door and window edge feature calculation unit, for carrying out rim detection to the door and window of building facade using canny algorithms,
Obtain the edge feature of door and window；
Gini coefficient computing unit, for being counted using the Gini coefficient in economics to the edge feature, obtain
Obtain the Gini coefficient of building facade；
Judging unit is damaged, for judging whether building facade is damaged according to the Gini coefficient.
Wherein, the building facade cutting unit, the gray value specifically for pixel in a, image are f, wherein f=0,
1,2 ..., 255, it is used as preliminary classification mean μ by the use of the k central point that rough set theory obtains1,μ2,μ3,…,μk；B, calculate
The distance between the gray value f of each pixel and the preliminary classification mean μ D in image, it is initial that each pixel is assigned to distance
The nearest class of class average, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ... k) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；C, for i=1,2 ..., k calculates new poly-
Class center, update class average:In formula, NiIt isIn number of pixels,It is certain class pixel
Set, i is the i-th class pixel, and m is iterations；D, all pixels are investigated one by one, if i=1,2 ..., k, hadThen algorithmic statement, terminate, otherwise return to b and continue next iteration.
Wherein, door and window edge feature calculation unit, the door specifically for a, using canny algorithms to building facade
Window carries out rim detection, obtains the door and window edge of building facade；B, because most of building facade is all perpendicular to ground,
Count and be distributed parallel to the distance between the parallel lines on ground first, then calculate distance vector histogram, finally obtain door
The edge feature of window；Flow is as follows：B10, it may be damaged due to building facade, therefore the wheel that door and window rim detection obtains
Profile may not be parallel to each other, therefore the method that statistics uses parallel to the distribution of the distance between the parallel lines on ground is：Along dampening
Square building facade is counted to vertical direction to every a fixed step size, calculate vertical direction and close on two pixels
The distance between, it is designated as di, whole facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represents the class of distance
Shuo not；B20, utilize statistics with histogram function D (di)=niDistance vector histogram is counted, histogram vector of then adjusting the distance
Variable niAscending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤…≤nk；Vector n is to build
Build the edge feature of thing facade door and window.
Wherein, the Gini coefficient computing unit, the edge feature specifically for a, the door and window for assuming to extract in image are
G, it is distance vector histogram g=[g by g distribution statisticses1,g2,g3,…,gk], the element in histogram vector of adjusting the distance enters
Row sorts from small to large, and it is g '=[g ' to obtain new Nogata set of graphs1,g’2,g’3,…,g’k], then measurement image rule degree
Gini coefficient formula be：
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, G scope be from 0 to
1, G is bigger, and building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage
Threshold value is that the statistical law in 0.45, and economics is basically identical；B, using the statistics edge feature vector n in step 2 as g
Bring into (2) formula, obtain the Gini coefficient of elevation of building.
Wherein, the damage judging unit, specifically for when Gini coefficient G is more than 0.45, representing that building facade is complete
It is good；Conversely, when Gini coefficient G is less than 0.45, represent that elevation of building is damaged.
Beneficial effect：
The method of building facade damage detection in a kind of oblique aerial image of the present invention, including：Step 1, profit
Building facade is split with the k-means clustering algorithms based on rough set theory, obtains the door and window of building facade；Step
Two, rim detection is carried out to the door and window of building facade using canny algorithms, obtains the edge feature of door and window；Step 3, utilize
Gini coefficient in economics counts to the edge feature, obtains the Gini coefficient of building facade；Step 4, according to
The Gini coefficient judges whether building facade is damaged.The present invention proposes in a kind of aviation inclination image and utilizes Gini coefficient
To detect the method for building facade damage, this method is the k- based on rough set theory first with improved k-means algorithms
Means clustering algorithms split the door and window of building facade and metope, obtain the door and window of building；Then canny algorithms are used
Rim detection is carried out to the facade door and window of building, obtains the edge feature of building window；Finally it make use of in economics
Gini coefficient integrally measures building facade damage information as damage index, judges whether building facade damages so as to reach
Ruin.It can be seen that the present invention takes full advantage of the structural information for tilting building facade in image, in combination with the Geordie in economics
Coefficient significantly improves the precision that the damage of building facade detects, feature is as damage index：1) this method, which has, does not need
Before prior information and calamity in the case of data, the damage detection of building facade simply can be efficiently carried out, reduces answering for method
Miscellaneous degree, also saves production cost.2) index of the Gini coefficient as the damage detection of building facade in economics is introduced,
The architectural feature of building facade can be made full use of to judge to damage, not only improve the automaticity and precision of judgement, and
And meet the needs of actual production.
Brief description of the drawings
Fig. 1 is the side of building facade damage detection in a kind of oblique aerial image that the specific embodiment of the invention provides
The flow chart of method.
Fig. 2 is the dress of building facade damage detection in a kind of oblique aerial image that the specific embodiment of the invention provides
The structural representation put.
Fig. 3 is the distance between building facade parallel lines distribution statisticses schematic diagram.
Embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
Technical solution of the present invention is described in detail below in conjunction with drawings and examples.
Embodiment 1
Fig. 1 is the side of building facade damage detection in a kind of oblique aerial image that the specific embodiment of the invention provides
The flow chart of method.As shown in figure 1, the method that the damage of building facade detects in a kind of oblique aerial image of the present invention,
Including：
Step 1, building facade is split using the k-means clustering algorithms based on rough set theory, obtains building
The door and window of facade；
Step 2, rim detection is carried out to the door and window of building facade using canny algorithms, the edge for obtaining door and window is special
Sign；
Step 3, the edge feature is counted using the Gini coefficient in economics, obtain building facade
Gini coefficient；
Step 4, judge whether building facade is damaged according to the Gini coefficient.
The present invention is proposed a kind of aviation and tilts the method for being detected the damage of building facade in image using Gini coefficient,
This method is the k-means clustering algorithms based on rough set theory by building facade first with improved k-means algorithms
Door and window and metope segmentation, obtain the door and window of building；Then edge inspection is carried out to the facade door and window of building using canny algorithms
Survey, obtain the edge feature of building window；The Gini coefficient in economics finally be make use of as the damage overall degree of index
Building facade damage information is measured, judges whether building facade is damaged so as to reach.It can be seen that the present invention takes full advantage of inclination
The structural information of building facade in image, in combination with the Gini coefficient in economics as damage index, significantly improve
The precision of building facade damage detection, feature are：1) this method, which has, does not need before prior information and calamity in the case of data,
The damage detection of building facade simply can be efficiently carried out, the complexity of method is reduced, also saves production cost.2) introduce
Index of the Gini coefficient as the damage detection of building facade in economics, the structure of building facade can be made full use of
Feature judges to damage, and not only improves the automaticity and precision of judgement, and meet the needs of actual production.
In this programme, the step 1, using the k-means clustering algorithms based on rough set theory to building facade
Segmentation, the door and window of building facade is obtained, be specially：
The gray value of pixel is f, wherein f=0 in S110, image, 1,2 ..., 255, the k obtained using rough set theory
Individual central point is as preliminary classification mean μ1,μ2,μ3,…,μk；
S120, the distance between the gray value f of each pixel and previous step preliminary classification mean μ D in image are calculated, will be every
Individual pixel is assigned to the class nearest apart from initial classes average, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ... k) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；
S130, new cluster centre is calculated for i=1,2 ..., k, update class average:
In formula, NiIt isIn number of pixels,It is the set of certain class pixel, i is the i-th class pixel, and m is iteration time
Number；
S140, all pixels are investigated one by one, if i=1,2 ..., k, there is μi (m+1)=μi (m), then algorithmic statement, is tied
Beam, otherwise return to S120 and continue next iteration.
Neat door and window is dispersed with because the building facade in oblique aerial image is noteworthy characterized by, therefore can be extracted
Building vertical door window edge feature is as gini index distribution statisticses feature, by whether differentiating building vertical door window edge
Neat arrangement, judge whether building facade is damaged with this.It is first herein in order to carry out Gini coefficient statistical nature extraction
The door and window of building facade is partitioned into first with k-means clustering algorithms, k-means clustering algorithms are that feature space is divided
A kind of quick, easy sorting technique, can dynamic clustering, there is the advantages of adaptive, belong to the model of non-supervised classification
Farmland, but k-means clustering algorithms are influenceed by initial cluster center selection, if the unreasonable of selection will increase computing
Complexity, cluster process is misled, therefore will directly be obtained using k-means clustering algorithms segmentation building facade irrational
Cluster result.Because rough set, by knowledge abbreviation, can be carried out approximate well in the case where keeping classification capacity constant
Classification, therefore preliminary classification is carried out to image herein by the space-division method of rough set theory, then in preliminary classification
On the basis of using k-means clustering algorithms building facade is divided into door and window and wall two parts, specific algorithm flow is as follows：
Preliminary classification is carried out using rough set.According to rough set theory, can using the information expressed by a width image as
One knowledge system K=(I, R), I represent image, and R is defined in the equivalence relation in image I, utilizes defined equivalence relation R
Mark off the initial center point and its number of cluster.If the gray value of pixel is f, wherein f=0 in image I, 1,2 ..., 255, D
(f)=n represents the number of pixels that statistics gray value is f.The image histogram being made up of D (f) is generally the distribution of paddy peak, passes through
The approximate pixel of gray value can be classified as one kind by histogram, then if image can approximation is divided into Ganlei, therefore define picture
The gray value differences of element can be defined as conditional attribute, then rough set equivalence relation R：If two pixel grey scale value differences are small
In spacing d, then two pixels are related, belong to equivalence class, i.e.,：
R=f | | fi-fj＜ d } (i, j=0,1 ..., 255)
Gray value differences d is determined first, and number of greyscale levels L is obtained by the grey level histogram scope of image.By in tonal range
The largest number of gray values of respective pixel are defined as central point μ.L central point spacing two-by-two is calculated, if between minimum range is less than
Away from d, then respective center point is merged, and the value using 2 points of arithmetic mean of instantaneous value as the central point.It is repeated up to all
The spacing two-by-two of central point is all higher than spacing d, then the number and numerical value of central point are exactly initial required for k-means is clustered
The number and average of class.If it is used as preliminary classification mean μ by the use of the k central point that rough set theory obtains1,μ2,μ3,…,μk。
Therefore, this method is that the k-means clusters based on rough set theory are calculated using the k-means clustering algorithms after improving
Building facade is divided into door and window and wall two parts by method.First, to obtain the initial centers of birdsing of the same feather flock together of K by rough set theory equal
Value, then does cluster segmentation, metope is mainly divided into door and window and the class of wall two.After above cluster process terminates, in order to strengthen
Display effect, each pixel of segmentation result are used as such final gray scale using cluster centre gray value.
In this programme, the step 2, rim detection is carried out to the door and window of building facade using canny algorithms, obtained
The edge feature of door and window is obtained, is specially：
S210, the door and window progress rim detection using canny algorithms to building facade, obtain the door and window of building facade
Edge；
S220, because most of building facade is all perpendicular to ground, count first parallel to ground parallel lines it
Between range distribution, then calculate distance vector histogram, finally obtain door and window edge feature；Flow is as follows：
A) because building facade may be damaged, therefore the contour line that door and window rim detection obtains may not mutually be put down
OK, therefore statistics is parallel to the method for the distance between the parallel lines on ground distribution use：As shown in figure 3, along level side
Building facade is counted to vertical direction to every a fixed step size, vertical direction is calculated and closes between two pixels
Distance, be designated as di, whole facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represents the classification of distance
Number；
B) statistics with histogram function D (d are utilizedi)=niDistance vector histogram is counted, histogram vector of then adjusting the distance
Variable niAscending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤...≤nk；Vector n is to build
Build the edge feature of thing facade door and window.
In this programme, the step 3, the edge feature is counted using the Gini coefficient in economics, obtained
The Gini coefficient of building facade is obtained, is specially：
S310, the edge feature for the door and window for assuming to extract in image are g, are distance vector histogram g by g distribution statisticses
=[g1,g2,g3,…,gk], the element in histogram vector of adjusting the distance is sorted from small to large, obtains new Nogata set of graphs
For g '=[g '1,g’2,g’3,…,g’k], then the Gini coefficient formula of measurement image rule degree is：
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, G scope be from 0 to
1, G is bigger, and building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage
Threshold value is that the statistical law in 0.45, and economics is basically identical；
S320, bring into the statistics edge feature vector n in step 2 as g in (2) formula, obtain the Geordie of elevation of building
Coefficient.
Building facade edge feature is counted using the Gini coefficient in economics, obtains building facade Geordie
Coefficient.Gini coefficient (Gini coefficient), it is Italy economist Geordie early 20th century, according to lorenz curve institute
The index for judging distribution of earnings justice degree of definition, it is to be used for income disparity situation inside integrated survey resident in the world
An important analysis index.The numerical value of Gini coefficient is generally in the range of 0 to 1, and numerical value is bigger to represent that income difference is big, distribution
Unfairness, numerical value is smaller to represent that income difference is small, fairness in distribution.In the world generally using 0.4 as gap between the rich and the poor warning line, greatly
Easily there is social unrest in this numerical value.Gini coefficient is a kind of important measure index for counting uneven distribution, is had very
Good Scale invariant and clone's invariant feature, these characteristics meets six characteristics of sparse measurement well.
Intact building facade normal conditions have good rule degree, and the door and window on surface is distributed into straight uniform, had
Certain sparse distribution characteristic, this method introduces Gini coefficient the metric index damaged as building facade, when Geordie system
Number illustrates that building facade structures are loose when larger, has preferable rule, does not damage；Conversely, explanation building facade knot
Structure is mixed and disorderly, is damaged.
In this programme, the step 4, judge whether building facade is damaged according to the Gini coefficient, be specially：
When Gini coefficient G is more than 0.45, represent that building facade is intact；Conversely, when Gini coefficient G is less than 0.45,
Represent that elevation of building is damaged.
In summary, a kind of aviation proposed by the present invention tilts in image and detects building facade damage using Gini coefficient
The method ruined takes full advantage of the structural information for tilting building facade in image, makees in combination with the Gini coefficient in economics
To damage index, the precision of building facade damage detection is significantly improved, feature is：1) this method, which has, does not need priori letter
Before breath and calamity in the case of data, the damage detection of building facade simply can be efficiently carried out, reduces the complexity of method,
Production cost is saved.2) index of the Gini coefficient as the damage detection of building facade in economics, Ke Yichong are introduced
Divide and judge to damage using the architectural feature of building facade, not only improve the automaticity and precision of judgement, and meet
The needs of actual production.
Embodiment 2
The present embodiment 2 is device embodiment, and embodiment 1 is embodiment of the method, and device embodiment belongs to embodiment of the method
Same technical concept, the content of not detailed description, refers to embodiment of the method in device embodiment.
Fig. 2 is the dress of building facade damage detection in a kind of oblique aerial image that the specific embodiment of the invention provides
The structural representation put.As shown in Fig. 2 the dress that the damage of building facade detects in a kind of oblique aerial image of the present invention
Put, including：
Building facade cutting unit, for being stood using the k-means clustering algorithms based on rough set theory to building
Face is split, and obtains the door and window of building facade；
Door and window edge feature calculation unit, for carrying out rim detection to the door and window of building facade using canny algorithms,
Obtain the edge feature of door and window；
Gini coefficient computing unit, for being counted using the Gini coefficient in economics to the edge feature, obtain
Obtain the Gini coefficient of building facade；
Judging unit is damaged, for judging whether building facade is damaged according to the Gini coefficient.
The present invention is proposed a kind of aviation and tilts the device for being detected the damage of building facade in image using Gini coefficient,
The device first passes through building facade cutting unit and utilizes the improved k-means algorithms i.e. k-means based on rough set theory
Clustering algorithm splits the door and window of building facade and metope, obtains the door and window of building；Then door and window edge feature meter is passed through
Calculate unit and rim detection is carried out to the facade door and window of building using canny algorithms, obtain the edge feature of building window；Most
It make use of the Gini coefficient in economics to be used as damage index by Gini coefficient computing unit afterwards to stand integrally to measure building
Information is damaged in face, judges whether building facade is damaged so as to reach.Built it can be seen that the present invention takes full advantage of to tilt in image
The structural information of thing facade, in combination with the Gini coefficient in economics as damage index, significantly improve building facade
The precision of detection is damaged, feature is：1) device, which has, does not need before prior information and calamity in the case of data, can be simply efficient
Carry out building facade damage detection, reduce the complexity of method, also save production cost.2) introduce in economics
Index of the Gini coefficient as the damage detection of building facade, the architectural feature of building facade can be made full use of to judge
Damage, not only improves the automaticity and precision of judgement, and meet the needs of actual production.
The building facade cutting unit, the gray value specifically for pixel in a, image are f, wherein f=0,1,
2 ..., 255, it is used as preliminary classification mean μ by the use of the k central point that rough set theory obtains1,μ2,μ3,…,μk；B, shadow is calculated
The distance between the gray value f of each pixel and the preliminary classification mean μ D, each pixel is assigned to apart from initial classes as in
The nearest class of average, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ... k) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；C, for i=1,2 ..., k calculates new poly-
Class center, update class average:In formula, NiIt isIn number of pixels,It is certain class pixel
Set, i is the i-th class pixel, and m is iterations；D, all pixels are investigated one by one, if i=1,2 ..., k, there is μi (m+1)
=μi (m), then algorithmic statement, terminates, and otherwise returns to b and continues next iteration.
Door and window edge feature calculation unit, enter specifically for a, using canny algorithms to the door and window of building facade
Row rim detection, obtain the door and window edge of building facade；B, because most of building facade is all perpendicular to ground, first
Statistics is distributed parallel to the distance between the parallel lines on ground, then calculates distance vector histogram, finally obtains door and window
Edge feature；Flow is as follows：B10, it may be damaged due to building facade, therefore the contour line that door and window rim detection obtains
It may not be parallel to each other, therefore count the method used parallel to the distribution of the distance between the parallel lines on ground to be：As shown in figure 3,
Building facade is counted to vertical direction every a fixed step size along horizontal direction, vertical direction is calculated and closes on two
The distance between pixel, it is designated as di, whole facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represent away from
From classification number；B20, utilize statistics with histogram function D (di)=niDistance vector histogram is counted, vector of then adjusting the distance
Histogram variable niAscending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤...≤nk；To
Measure the edge feature that n is building facade door and window.
The Gini coefficient computing unit, the edge feature specifically for a, the door and window for assuming to extract in image is g, by g
Distribution statisticses be distance vector histogram g=[g1,g2,g3,…,gk], the element in histogram vector of adjusting the distance is carried out from small
To big sequence, it is g '=[g ' to obtain new Nogata set of graphs1,g’2,g’3,…,g’k], then the Geordie of measurement image rule degree
Coefficient formula is：
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, G scope be from 0 to
1, G is bigger, and building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage
Threshold value is that the statistical law in 0.45, and economics is basically identical；B, using the statistics edge feature vector n in step 2 as g
Bring into (2) formula, obtain the Gini coefficient of elevation of building.
The damage judging unit, specifically for when Gini coefficient G is more than 0.45, representing that building facade is intact；Instead
It, when Gini coefficient G is less than 0.45, represents that elevation of building is damaged.
The preferred embodiments of the present invention are these are only, are not intended to limit the scope of the invention, it is every to utilize this hair
The equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of method of building facade damage detection in oblique aerial image, it is characterised in that including：
Step 1, building facade is split using the k-means clustering algorithms based on rough set theory, obtains building facade
Door and window；
Step 2, rim detection is carried out to the door and window of building facade using canny algorithms, obtains the edge feature of door and window；
Step 3, the edge feature is counted using the Gini coefficient in economics, obtain the Geordie of building facade
Coefficient；
Step 4, judge whether building facade is damaged according to the Gini coefficient.
2. the method for building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 1
In the step 1, using the k-means clustering algorithms based on rough set theory to the segmentation of building facade, acquisition building
The door and window of facade, it is specially：
The gray value of pixel is f, wherein f=0 in S110, image, 1,2 ..., 255, in k obtained using rough set theory
Heart point is as preliminary classification mean μ1,μ2,μ3,…,μk；
S120, the distance between the gray value f of each pixel and previous step preliminary classification mean μ D in image are calculated, by each picture
Element is assigned to the class nearest apart from initial classes average, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ..k.) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；
S130, new cluster centre is calculated for i=1,2 ..., k, update class average:
In formula, NiIt isIn number of pixels,It is the set of certain class pixel, i is the i-th class pixel, and m is iterations；
S140, all pixels are investigated one by one, if i=1,2 ..., k, there is μi (m+1)=μi (m), then algorithmic statement,
Terminate, otherwise return to S120 and continue next iteration.
3. the method for building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 2
In, the step 2, rim detection is carried out to the door and window of building facade using canny algorithms, obtains the edge feature of door and window,
Specially：
S210, the door and window progress rim detection using canny algorithms to building facade, obtain the door window side of building facade
Edge；
S220, because most of building facade is all perpendicular to ground, count first between the parallel lines parallel to ground
Range distribution, distance vector histogram is then calculated, finally obtain the edge feature of door and window；Flow is as follows：
A) because building facade may be damaged, therefore the contour line that door and window rim detection obtains may not be parallel to each other, because
This statistics is distributed the method used parallel to the distance between the parallel lines on ground：Along horizontal direction every a fixed step size pair
Building facade is counted to vertical direction, is calculated vertical direction and is closed on the distance between two pixels, is designated as di, it is whole
Individual facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represents the classification number of distance；
B) statistics with histogram function D (d are utilizedi)=niDistance vector histogram is counted, histogram vector variable of then adjusting the distance
niAscending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤...≤nk；Vector n is building
The edge feature of facade door and window.
4. the method for building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 3
In, the step 3, the edge feature is counted using the Gini coefficient in economics, obtain building facade base
Buddhist nun's coefficient, it is specially：
S310, the edge feature for the door and window for assuming to extract in image are g, are distance vector histogram g=by g distribution statisticses
[g1,g2,g3,…,gk], the element in histogram vector of adjusting the distance is sorted from small to large, and obtaining new Nogata set of graphs is
G '=[g '1,g’2,g’3,…,g’k], then the Gini coefficient formula of measurement image rule degree is：
<mrow>
<mi>G</mi>
<mrow>
<mo>(</mo>
<mi>g</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mn>1</mn>
<mo>-</mo>
<mn>2</mn>
<mo>&times;</mo>
<munderover>
<mo>&Sigma;</mo>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mi>K</mi>
</munderover>
<mfrac>
<msub>
<mi>g</mi>
<mi>k</mi>
</msub>
<mrow>
<mo>(</mo>
<mo>|</mo>
<mo>|</mo>
<mi>g</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>1</mn>
</msub>
<mo>)</mo>
</mrow>
</mfrac>
<mrow>
<mo>(</mo>
<mfrac>
<mrow>
<mi>K</mi>
<mo>-</mo>
<mi>k</mi>
<mo>+</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
</mrow>
<mi>K</mi>
</mfrac>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, and G scope is from 0 to 1, and G is got over
Greatly, building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage threshold value
It is basically identical for the statistical law in 0.45, and economics；
S320, bring into the statistics edge feature vector n in step 2 as g in (2) formula, obtain the Geordie system of elevation of building
Number.
5. the method for building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 4
In, the step 4, judge whether building facade is damaged according to the Gini coefficient, be specially：
When Gini coefficient G is more than 0.45, represent that building facade is intact；Conversely, when Gini coefficient G is less than 0.45, represent
Elevation of building is damaged.
A kind of 6. device of building facade damage detection in oblique aerial image, it is characterised in that including：
Building facade cutting unit, for being divided using the k-means clustering algorithms based on rough set theory building facade
Cut, obtain the door and window of building facade；
Door and window edge feature calculation unit, for carrying out rim detection to the door and window of building facade using canny algorithms, obtain
The edge feature of door and window；
Gini coefficient computing unit, for being counted using the Gini coefficient in economics to the edge feature, built
Build the Gini coefficient of thing facade；
Judging unit is damaged, for judging whether building facade is damaged according to the Gini coefficient.
7. the device of building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 6
In, the building facade cutting unit, the gray value specifically for pixel in a, image is f, wherein f=0,1,2 ...,
255, it is used as preliminary classification mean μ by the use of the k central point that rough set theory obtains1,μ2,μ3,…,μk；B, calculate every in image
The distance between the gray value f of individual pixel and the preliminary classification mean μ D, each pixel is assigned to apart from initial classes average most
Near class, i.e.,
D|fp-μi|=min D | fp-μi|, (i=1,2 ... k) } (1)
(1) formula is iterated, wherein p is the central point in iterative process；C, for i=1,2 ..., k is calculated in new cluster
The heart, update class average:In formula, NiIt isIn number of pixels,It is the collection of certain class pixel
Close, i is the i-th class pixel, and m is iterations；D, all pixels are investigated one by one, if i=1,2 ..., k, there is μi (m+1)=
μi (m), then algorithmic statement, terminates, and otherwise returns to b and continues next iteration.
8. the device of building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 7
In, door and window edge feature calculation unit, specifically for a, canny algorithms are utilized to carry out edge to the door and window of building facade
Detection, obtain the door and window edge of building facade；B, because most of building facade is all perpendicular to ground, statistics is flat first
Row is distributed in the distance between the parallel lines on ground, then calculates distance vector histogram, and the edge for finally obtaining door and window is special
Sign；Flow is as follows：B10, it may be damaged due to building facade, therefore the contour line that door and window rim detection obtains may not phase
It is mutually parallel, therefore the method that statistics uses parallel to the distribution of the distance between the parallel lines on ground is：Along horizontal direction every
One fixed step size counts to building facade to vertical direction, calculate vertical direction close between two pixels away from
From being designated as di, whole facade image obtains distance vector d=[d1,d2,d3,…,dK], wherein k represents the classification number of distance；
B20, utilize statistics with histogram function D (di)=niDistance vector histogram is counted, then adjust the distance histogram vector variable ni
Ascending sort is carried out, obtains vector n=[n1,n2,n3,…,nK], wherein n1≤n2≤n3≤...≤nk；Vector n is stood for building
The edge feature of face door and window.
9. the device of building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 8
In the Gini coefficient computing unit, the edge feature specifically for a, the door and window for assuming to extract in image is g, by g distribution
Count as distance vector histogram g=[g1,g2,g3,…,gk], the element in histogram vector of adjusting the distance is arranged from small to large
Sequence, it is g '=[g ' to obtain new Nogata set of graphs1,g’2,g’3,…,g’k], then the Gini coefficient of measurement image rule degree is public
Formula is：
<mrow>
<mi>G</mi>
<mrow>
<mo>(</mo>
<mi>g</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mn>1</mn>
<mo>-</mo>
<mn>2</mn>
<mo>&times;</mo>
<munderover>
<mo>&Sigma;</mo>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mi>K</mi>
</munderover>
<mfrac>
<msub>
<mi>g</mi>
<mi>k</mi>
</msub>
<mrow>
<mo>(</mo>
<mo>|</mo>
<mo>|</mo>
<mi>g</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>1</mn>
</msub>
<mo>)</mo>
</mrow>
</mfrac>
<mrow>
<mo>(</mo>
<mfrac>
<mrow>
<mi>K</mi>
<mo>-</mo>
<mi>k</mi>
<mo>+</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
</mrow>
<mi>K</mi>
</mfrac>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, | | g | |1For first normal form, K is the classification sum of distance vector statistics with histogram, and G scope is from 0 to 1, and G is got over
Greatly, building facade is more complete, and G is smaller, and the damage of building facade is serious；Found by statistical experiment, building damage threshold value
It is basically identical for the statistical law in 0.45, and economics；B, the statistics edge feature vector n in step 2 is brought into as g
(2) in formula, the Gini coefficient of elevation of building is obtained.
10. the device of building facade damage detection, its feature exist in a kind of oblique aerial image according to claim 9
In the damage judging unit, specifically for when Gini coefficient G is more than 0.45, representing that building facade is intact；Conversely, work as
When Gini coefficient G is less than 0.45, represent that elevation of building is damaged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510494876.0A CN105139388B (en) | 2015-08-12 | 2015-08-12 | The method and apparatus of building facade damage detection in a kind of oblique aerial image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510494876.0A CN105139388B (en) | 2015-08-12 | 2015-08-12 | The method and apparatus of building facade damage detection in a kind of oblique aerial image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105139388A CN105139388A (en) | 2015-12-09 |
| CN105139388B true CN105139388B (en) | 2017-12-15 |
Family
ID=54724721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510494876.0A Active CN105139388B (en) | 2015-08-12 | 2015-08-12 | The method and apparatus of building facade damage detection in a kind of oblique aerial image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105139388B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200808A (en) * | 2020-12-07 | 2021-01-08 | 领伟创新智能系统(浙江)有限公司 | Strip steel surface defect detection method based on local Gini coefficient |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105931191B (en) * | 2015-12-21 | 2019-05-03 | 南京邮电大学 | Image Blind deconvolution method based on concave-convex mixing canonical priori |
| CN109035206B (en) * | 2018-06-28 | 2020-10-09 | 中国地震局地震预测研究所 | Building damage state detection method |
| CN109703465B (en) * | 2018-12-28 | 2021-03-12 | 百度在线网络技术（北京）有限公司 | Control method and device for vehicle-mounted image sensor |
| CN109886585B (en) * | 2019-02-27 | 2021-01-01 | 广东三维家信息科技有限公司 | Room ventilation evaluation method and device |
| CN111126184B (en) * | 2019-12-09 | 2022-04-12 | 武汉大学 | Post-earthquake building damage detection method based on unmanned aerial vehicle video |
| CN111126183A (en) * | 2019-12-09 | 2020-05-08 | 武汉大学 | Method for detecting damage of building after earthquake based on near-ground image data |
| CN111881915B (en) * | 2020-07-15 | 2022-07-15 | 武汉大学 | Satellite video target intelligent detection method based on multiple prior information constraints |
| CN114782826B (en) * | 2022-06-20 | 2022-11-18 | 绵阳天仪空间科技有限公司 | Safety monitoring system and method for post-disaster building |
| CN114782846B (en) * | 2022-06-20 | 2022-11-08 | 绵阳天仪空间科技有限公司 | Post-disaster rescue guiding system and method |
| CN115424178B (en) * | 2022-09-05 | 2023-07-14 | 兰州大学 | Enhancement method for improving pavement crack data identification |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729853A (en) * | 2014-01-15 | 2014-04-16 | 武汉大学 | Three-dimensional GIS assisted high-resolution remote sensing image building collapse-damage detecting method |
| CN104268879A (en) * | 2014-09-28 | 2015-01-07 | 民政部国家减灾中心 | Physical building quantity damage evaluation method based on remote sensing multi-spectral images |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7007035B2 (en) * | 2001-06-08 | 2006-02-28 | The Regents Of The University Of California | Parallel object-oriented decision tree system |
| US8874454B2 (en) * | 2013-03-15 | 2014-10-28 | State Farm Mutual Automobile Insurance Company | Systems and methods for assessing a roof |
-
2015
- 2015-08-12 CN CN201510494876.0A patent/CN105139388B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729853A (en) * | 2014-01-15 | 2014-04-16 | 武汉大学 | Three-dimensional GIS assisted high-resolution remote sensing image building collapse-damage detecting method |
| CN104268879A (en) * | 2014-09-28 | 2015-01-07 | 民政部国家减灾中心 | Physical building quantity damage evaluation method based on remote sensing multi-spectral images |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200808A (en) * | 2020-12-07 | 2021-01-08 | 领伟创新智能系统(浙江)有限公司 | Strip steel surface defect detection method based on local Gini coefficient |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105139388A (en) | 2015-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105139388B (en) | The method and apparatus of building facade damage detection in a kind of oblique aerial image | |
| CN105404861B (en) | Training, detection method and the system of face key feature points detection model | |
| CN115082467B (en) | Building material welding surface defect detection method based on computer vision | |
| CN107194872B (en) | Remote sensed image super-resolution reconstruction method based on perception of content deep learning network | |
| CN103839065B (en) | Extraction method for dynamic crowd gathering characteristics | |
| CN106127148B (en) | A kind of escalator passenger's anomaly detection method based on machine vision | |
| CN107742099A (en) | A kind of crowd density estimation based on full convolutional network, the method for demographics | |
| CN102509104B (en) | Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene | |
| CN107886507B (en) | A kind of salient region detecting method based on image background and spatial position | |
| CN106683076A (en) | Texture feature clustering-based locomotive wheelset tread damage detection method | |
| CN106127205A (en) | A kind of recognition methods of the digital instrument image being applicable to indoor track machine people | |
| CN104282008B (en) | The method and apparatus that Texture Segmentation is carried out to image | |
| CN103208123A (en) | Image segmentation method and system | |
| CN108280480A (en) | A kind of hidden image vector safety evaluation method based on residual error symbiosis probability | |
| CN104318241A (en) | Local density spectral clustering similarity measurement algorithm based on Self-tuning | |
| CN103324951A (en) | Multiple isomerous water environment monitoring data evaluating and early-warning method | |
| CN107123130A (en) | Kernel correlation filtering target tracking method based on superpixel and hybrid hash | |
| CN106845456A (en) | A kind of method of falling over of human body monitoring in video monitoring system | |
| CN108960142A (en) | Pedestrian based on global characteristics loss function recognition methods again | |
| CN103927516A (en) | Seawater pearl authentication system based on digital image processing | |
| CN104143091A (en) | Single-sample face recognition method based on improved mLBP | |
| CN105205816A (en) | Method for extracting high-resolution SAR image building zone through multi-feature weighted fusion | |
| CN108830882A (en) | Video abnormal behaviour real-time detection method | |
| CN111461003A (en) | Coal-fired working condition identification method based on video image sequence feature extraction | |
| CN107423771A (en) | A kind of two phase method for detecting change of remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |