CN113256714A - Tunnel surface image processing method and system - Google Patents
Tunnel surface image processing method and system Download PDFInfo
- Publication number
- CN113256714A CN113256714A CN202110788036.0A CN202110788036A CN113256714A CN 113256714 A CN113256714 A CN 113256714A CN 202110788036 A CN202110788036 A CN 202110788036A CN 113256714 A CN113256714 A CN 113256714A
- Authority
- CN
- China
- Prior art keywords
- axial direction
- tunnel surface
- coordinate system
- tunnel
- along
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 24
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 230000009466 transformation Effects 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 15
- 201000010099 disease Diseases 0.000 claims description 14
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 14
- 238000001514 detection method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30132—Masonry; Concrete
Abstract
The invention discloses a tunnel surface image processing method and a tunnel surface image processing system, wherein a tunnel surface image and the positions of at least three points on the tunnel surface of a current tunnel section in a world coordinate system are obtained, the transformation relation between the position along a first axial direction in the world coordinate system and the position along a second axial direction of the midpoint in the image coordinate system of the tunnel surface image is obtained, a preset matrix of the tunnel surface image is further obtained, elements of the preset matrix describe the relationship that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface, and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the linear length in the world coordinate system, and then each pixel of the tunnel surface image is stretched according to the preset matrix to obtain the processed tunnel surface image. The method can quickly generate the expansion image of the tunnel surface according to the image of the tunnel surface, and improves the efficiency.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a tunnel surface image processing method and system.
Background
In recent years, with the increase of highway mileage in China and the rapid development of high-speed railways in China, tunnel traffic plays an increasingly important role in the road traffic system in China. The necessity and urgency of tunnel detection maintenance is self-evident. If the tunnel is seriously damaged, the accident rate can rise linearly, and great threat is caused to the safety of passing vehicles. The regular development of the detection work on the tunnel can prolong the service life of the tunnel, reduce the investment of national funds, ensure the passing safety of passing vehicles and reduce the probability of accidents.
The research and development of tunnel detection equipment and systems in China gradually starts to be automated and intelligent, and an automatic tunnel defect detection system becomes an important development trend of tunnel detection and maintenance. The vehicle-mounted dynamic detection equipment can collect the apparent disease image of the tunnel under the condition that the tunnel passes through, thereby being beneficial to the normal operation of traffic and reducing the input of manpower consumption and cost resources.
However, in the prior art, the device is limited to collect and process tunnel images which are nearly planar, because of the arc characteristics of tunnels, the processing effect of the tunnel images with a large curved surface is poor, and the collected images need to be artificially judged and processed in the later period, so that the processing cost is increased, and the detection efficiency is slowed down. In solving this problem, a general approach is to minimize the distance between the photographing device and the tunnel surface, that is, to make the photographed image nearly a planar image by making the photographing device as close to the tunnel surface as possible. But the area of the image shot each time is small, the shooting times are increased, the shooting time is prolonged, and the efficiency is greatly reduced.
Disclosure of Invention
The invention aims to provide a tunnel surface image processing method and system, which can quickly generate an expansion image of a tunnel surface according to an acquired tunnel surface image and improve the efficiency.
In order to achieve the purpose, the invention provides the following technical scheme:
a tunnel surface image processing method comprises the following steps:
acquiring a tunnel surface image and acquiring the positions of at least three points on the tunnel surface of the current tunnel section in a world coordinate system;
according to the positions of the at least three points in the world coordinate system, obtaining a curve equation of the tunnel surface of the tunnel section where the at least three points are located in the world coordinate system;
according to the positions of the at least three points in the world coordinate system and the image coordinate system of the tunnel surface image, obtaining a transformation relation between the position of the midpoint in the world coordinate system along a first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along a second axial direction, wherein the first axial direction is the axial direction of the world coordinate system parallel to the current tunnel section, and the second axial direction is the axial direction of the image coordinate system of the tunnel surface image parallel to the tunnel surface image;
obtaining a preset matrix of the tunnel surface image according to a transformation relation between a position of a midpoint of the world coordinate system along a first axial direction and a position of the midpoint of the image coordinate system of the tunnel surface image along a second axial direction and a curve equation of the tunnel surface of the tunnel section where the tunnel surface is located at present, wherein elements of the preset matrix describe a relation that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the straight line length of the tunnel surface image in the world coordinate system;
and stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image.
Preferably, the acquiring the positions of at least three points on the tunnel surface of the current tunnel section in the world coordinate system comprises:
and acquiring the distance from at least three points on the tunnel surface of the current tunnel section to a preset point and the included angle between the at least three points and the connecting line of the preset point, and acquiring the positions of the at least three points in the world coordinate system according to the acquired distance and included angle.
Preferably, the method specifically comprises the following steps: marking the at least three points on the tunnel surface of the current tunnel section by using light points, wherein the tunnel surface image comprises the light points corresponding to the at least three points.
Preferably, obtaining a transformation relationship between the position of the midpoint in the world coordinate system along the first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along the second axial direction includes:
acquiring position difference of any two points of the at least three points in the world coordinate system along the first axial direction;
acquiring the position difference of any two points of the at least three points in the image coordinate system of the tunnel surface image along the second axial direction;
and obtaining a transformation relation between the position of the point in the world coordinate system along the first axial direction and the position of the point in the image coordinate system of the tunnel surface image along the second axial direction according to the position difference of the world coordinate system along the first axial direction and the position difference of the image coordinate system along the second axial direction.
Preferably, obtaining the elements of the preset matrix includes:
according to the transformation relation between the position of the middle point of the world coordinate system along the first axial direction and the position of the middle point of the image coordinate system of the tunnel surface image along the second axial direction, obtaining the position coordinates of two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction respectively;
obtaining position coordinates of two pixels adjacent along the second axial direction in the world coordinate system along a third axial direction according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and a curve equation of a tunnel surface of a tunnel section where the two pixels are located in the world coordinate system at present;
and according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction, obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system and the arc length of the two pixels corresponding to the tunnel surface, and obtaining the corresponding elements in the preset matrix.
Preferably, obtaining the arc lengths of the two pixels adjacent along the second axial direction corresponding to the tunnel surface comprises:
according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction, obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system;
according to the linear distance between the two pixels adjacent along the second axial direction and the world coordinate system and the curve equation, obtaining the central angle of the tunnel surface arc line corresponding to the point of the tunnel surface corresponding to the two pixels adjacent along the second axial direction;
and obtaining the arc lengths of the two pixels which are adjacent along the second axial direction on the surface of the tunnel according to the central angle.
Preferably, according to the preset matrix, stretching each pixel of the tunnel surface image to obtain a processed tunnel surface image includes:
stretching each pixel of the tunnel surface image according to the preset matrix, and obtaining an original pixel and a stretched pixel corresponding to each pixel;
and sequentially generating each whole pixel by splicing the original pixels or the stretched pixels of the adjacent pixels from the boundary pixels of the tunnel surface image along the second axial direction to obtain the processed tunnel surface image.
Preferably, according to the preset matrix, stretching each pixel of the tunnel surface image to obtain a processed tunnel surface image includes:
and for the generated integer pixel, determining the pixel value of the integer pixel according to the original pixel proportion and the stretched pixel proportion contained in the integer pixel, and the original pixel value and the stretched pixel value.
Preferably, the method further comprises the following steps: and according to the disease area detected in the processed tunnel surface image, obtaining the position or/and size of the actual disease area corresponding to the tunnel surface according to the transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and the curve equation of the tunnel surface of the current tunnel section.
A tunnel surface image processing system for performing the tunnel surface image processing method described above.
It can be known from the above technical solutions that the tunnel surface image processing method and system provided by the present invention obtain a tunnel surface image and obtain the positions of at least three points on the tunnel surface of the current tunnel cross section in the world coordinate system, obtain a curve equation of the tunnel surface of the current tunnel cross section in the world coordinate system according to the positions of the at least three points in the world coordinate system, then obtain a transformation relationship between the position of the midpoint in the world coordinate system along a first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along a second axial direction according to the positions of the at least three points in the world coordinate system and the positions of the image coordinate system of the tunnel surface image, where the first axial direction is an axial direction of the world coordinate system parallel to the current tunnel cross section, the second axial direction is an axial direction of the image coordinate system of the tunnel surface image parallel to the tunnel surface image, and further obtaining a preset matrix of the tunnel surface image according to a transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and a curve equation of the tunnel surface of the tunnel section where the preset matrix is located, wherein elements of the preset matrix describe the arc length of a single pixel of the tunnel surface image along the second axial direction corresponding to the tunnel surface and the relation between the length of the single pixel of the tunnel surface image along the second axial direction and the length of a straight line of the tunnel surface image in the world coordinate system, and then stretching each pixel of the tunnel surface image according to the preset matrix to obtain the processed tunnel surface image, thereby realizing the expansion of the obtained tunnel surface image. The method and the system for processing the tunnel surface image can quickly generate the expansion image of the tunnel surface according to the tunnel surface image, and improve the efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a tunnel surface image processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of acquiring positions of three points on a tunnel surface of a current tunnel section according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for obtaining a transformation between a position of a point in a world coordinate system along a first axis and a position of a point in an image coordinate system of an image of a tunnel surface along a second axis according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for obtaining elements of a predetermined matrix according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method for stretching each pixel of a tunnel surface image according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a tunnel surface image processing method according to the present embodiment, and it can be seen that the tunnel surface image processing method includes the following steps:
s10: acquiring an image of the surface of the tunnel, and acquiring the positions of at least three points on the surface of the tunnel of the current tunnel section in a world coordinate system.
The tunnel surface of the current tunnel section refers to the portion of the tunnel surface through which the current tunnel section passes. And acquiring a tunnel surface image from the tunnel surface at the current position, and acquiring the positions of at least three points on the tunnel surface corresponding to the current position, wherein the at least three points are positioned on the same tunnel section.
S11: and obtaining a curve equation of the tunnel surface of the tunnel section where the tunnel is located currently in the world coordinate system according to the positions of the at least three points in the world coordinate system.
And obtaining a curve equation of the tunnel surface corresponding to the tunnel section where the at least three points are located in the world coordinate system according to the positions of the at least three points on the tunnel surface in the world coordinate system. The at least three points on the tunnel surface are all on the arc of the tunnel surface corresponding to the current tunnel section.
S12: and obtaining a transformation relation between the position of the point in the world coordinate system along the first axial direction and the position of the point in the image coordinate system of the tunnel surface image along the second axial direction according to the positions of the at least three points in the world coordinate system and the position of the point in the image coordinate system of the tunnel surface image.
The first axial direction is an axial direction of the world coordinate system which is parallel to the current tunnel section, and the second axial direction is an axial direction of an image coordinate system of the tunnel surface image which is parallel to the tunnel surface image.
According to the transformation relation of the positions of the two points and the position of the point in the tunnel surface image along the second axial direction, the position of the actual point corresponding to the point along the first axial direction in the world coordinate system can be obtained.
S13: and obtaining a preset matrix of the tunnel surface image according to a transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and a curve equation of the tunnel surface of the tunnel section where the tunnel surface is located at present, wherein elements of the preset matrix describe the relation that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the straight line length of the tunnel surface image in the world coordinate system.
The elements of the preset matrix describe the relationship between the arc length of the corresponding pixel on the tunnel surface in the tunnel surface image and the length of the corresponding straight line parallel to the current tunnel section in the world coordinate system.
S14: and stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image.
And stretching each pixel of the currently acquired tunnel surface image according to a preset matrix, unfolding the currently acquired tunnel surface image to obtain an image which is obtained by unfolding the tunnel surface into a plane, and obtaining a processed tunnel surface image.
The tunnel surface image processing method of the embodiment can rapidly generate the expansion image of the tunnel surface according to the acquired tunnel surface image, and compared with the method of shooting the tunnel surface image by enabling the shooting equipment to be close to the tunnel surface as much as possible in the prior art, the efficiency can be greatly improved.
The tunnel surface image processing method will be described in detail with reference to the specific embodiments. Referring to fig. 1, the tunnel surface image processing method includes the following steps:
s10: acquiring an image of the surface of the tunnel, and acquiring the positions of at least three points on the surface of the tunnel of the current tunnel section in a world coordinate system.
Optionally, distances from at least three points on the tunnel surface of the current tunnel section to the preset points and included angles between the at least three points and the connecting lines of the preset points may be obtained, and the positions of the at least three points in the world coordinate system may be obtained according to the obtained distances and included angles.
Referring to fig. 2 for example, fig. 2 is a schematic diagram illustrating the position of three points on the tunnel surface of the current tunnel cross section obtained in the present embodiment, where the diagram illustrates the obtaining of three points on the tunnel surface. As shown in the figure, three points on the tunnel surface of the current tunnel section are respectively a point a, a point B and a point C, and the distances from the point a, the point B and the point C to a preset point O are respectively measured and are sequentially L1、L2And L3And OA line and OB lineThe included angle between the OB line and the OC line is known and is theta1、θ2。
Preferably, a world coordinate system can be established by taking a preset point O as an origin and taking the current tunnel section as an xoy plane, and the position of each point in the established coordinate system can be obtained according to the corresponding distance and the included angle of each point. FIG. 2 shows a specific example of establishing a world coordinate system, θ, with a predetermined point O as the origin and an OB direction as the y-axis direction1=θ2=30 degrees, the position coordinates of the points a, B and C are sequentially
、
、
。
The distances of at least three points on the surface of the tunnel can be measured by using at least three distance measuring devices respectively, and the connecting lines of the distance measuring devices and the corresponding points on the surface of the tunnel intersect at a preset point. Alternatively, each distance measuring device may be arranged at a preset point, and the distance of each point may be directly obtained according to the distance value measured by each distance measuring device. Alternatively, the distance measuring device may be arranged at a position spaced from the preset point, and the distance from the point on the tunnel surface to the preset point may be obtained according to the distance value measured by the distance measuring device and the distance from the distance measuring device to the preset point.
The direction of each distance measuring device can be set when the distance measuring devices are arranged, so that the included angle between any two distance measuring devices and the connecting line of corresponding points on the surface of the tunnel is known. The size of the included angle can be set according to practical conditions, and is not limited to the 30 degrees.
The number of the selected points on the tunnel surface can be selected according to actual conditions, and the greater the number of the selected points, the more accurate the calculation result can be obtained, but the calculation amount is increased, so the calculation accuracy and the calculation amount are combined to determine in practical application.
S11: and obtaining a curve equation of the tunnel surface of the tunnel section where the tunnel is located currently in the world coordinate system according to the positions of the at least three points in the world coordinate system.
According to the position coordinates of the at least three points in the world coordinate system, a curve equation of the tunnel surface arc line where the at least three points are located can be solved by establishing an equation. For example, points a, B, and C on the tunnel surface as shown in fig. 2 may be listed three equations with three unknowns according to their location coordinates, as follows:
according to the above equation, the coordinates (a, b, 0) of the center of the arc where the tunnel is located and the radius R of the arc where the tunnel is located can be solved. Therefore, the position of the current tunnel section in the world coordinate system becomes known, and the equation of the circular arc coordinate of the current tunnel section is (x-a)2+(y-b)2=R2Here, a, b, and R are all obtained and are known amounts.
S12: and obtaining a transformation relation between the position of the point in the world coordinate system along the first axial direction and the position of the point in the image coordinate system of the tunnel surface image along the second axial direction according to the positions of the at least three points in the world coordinate system and the position of the point in the image coordinate system of the tunnel surface image.
Preferably, the at least three points on the tunnel surface of the current tunnel cross section can be marked by light spots, and the light spots are taken as the shooting objects into the image when the image of the tunnel surface is acquired, so that the acquired image of the tunnel surface contains the light spots corresponding to the at least three points. The positions of the at least three points in the image coordinate system of the tunnel surface image can thus be obtained.
Alternatively, the relative distance and relative angle between the imaging device and each ranging device may be set when the imaging device and the ranging device are arranged, and the position of each point in the acquired tunnel surface image can be obtained from the distance measured by each ranging device without marking each point with a light spot.
Alternatively, the transformation relationship between the position of the midpoint in the world coordinate system along the first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along the second axial direction may be obtained by the following method, referring to fig. 3, including the following steps:
s120: acquiring position difference of any two points of the at least three points in the world coordinate system along the first axial direction;
s121: acquiring the position difference of any two points of the at least three points in the image coordinate system of the tunnel surface image along the second axial direction;
s122: and obtaining a transformation relation between the position of the point in the world coordinate system along the first axial direction and the position of the point in the image coordinate system of the tunnel surface image along the second axial direction according to the position difference of the world coordinate system along the first axial direction and the position difference of the image coordinate system along the second axial direction.
When the world coordinate system is established, at least one axial direction of the world coordinate system is parallel to the current tunnel section, and when the image coordinate system of the tunnel surface image is established, at least one axial direction of the established image coordinate system is parallel to the tunnel surface image.
For example, assuming that the x '-axis of the established image coordinate system is parallel to the tunnel surface image, referring to fig. 2, relationships between the x-axis coordinates of points in the world coordinate system and the x' -axis coordinates of points in the image coordinate system are obtained, and the absolute values of the differences between the points a and B, and between the points B and C in the x-axis coordinates of the world coordinate system are L1/2、L3And/2, the absolute values of the x' -axis coordinate difference in the image coordinate system are m and n respectively. According to the method, the transformation relation between the point x-axis coordinate in the world coordinate system and the point x' -axis coordinate in the image coordinate system can be determined
I.e. the actual distance represented by the unit pixel length. This establishes the correlation of the two coordinate systems.
However, it should be noted that, since the tunnel cross section is a circular arc in the world coordinate system, and the image coordinate system considers the tunnel surface as a plane, this method is not applicable to the position of the point in the image coordinate system along the axial direction perpendicular to the image of the tunnel surface. Alternatively, an image coordinate system of the tunnel surface image may be established with a direction perpendicular to the tunnel surface image as a y 'axis direction, that is, with a direction perpendicular to a lens plane of the image pickup device as a y' axis direction, so that a plane passing through the at least three points is an x 'y' plane.
Can move along the tunnel axial direction, and can sequentially acquire images of the tunnel surface at various positions and acquire the positions of at least three points on the tunnel surface. Preferably, the camera is arranged parallel to the tunnel surface. Can install camera device and range unit on detecting the car, plan the route of marcing of detecting the car well, guarantee to detect the relative position in car and tunnel certainly.
Preferably, the distance measuring device and the imaging device may be mounted in fixed relative positions. For example, the distance measuring device may adopt laser distance measuring instruments, and the light extension lines of the laser distance measuring instruments are arranged to intersect at a point, so that light beams emitted by the laser distance measuring instruments can be projected onto the surface of the tunnel to form light spots. Preferably, the camera device can adopt a high-speed camera with higher resolution, so that images of the tunnel can be conveniently acquired under the condition of high-speed driving, and the detection rate of diseases is increased. The measuring precision of the distance measuring device is kept within 0.1 mm.
S13: and obtaining a preset matrix of the tunnel surface image according to a transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and a curve equation of the tunnel surface of the tunnel section where the tunnel surface is located at present, wherein elements of the preset matrix describe the relation that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the straight line length of the tunnel surface image in the world coordinate system.
Optionally, the elements of the preset matrix may be obtained according to the following method, please refer to fig. 4, where fig. 4 is a flowchart of the method for obtaining the elements of the preset matrix in this embodiment, and includes the following steps:
s130: and obtaining the position coordinates of two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction respectively according to the transformation relation between the position of the midpoint along the first axial direction in the world coordinate system and the position of the midpoint along the second axial direction in the image coordinate system of the tunnel surface image.
In the acquired tunnel surface image, two pixels adjacent in the second axial direction are sequentially selected. For each group of two pixels adjacent along the second axial direction, the position coordinate of each pixel along the second axial direction in the image coordinate system is obtained, and then, according to the transformation relation, the position coordinate of each pixel along the first axial direction in the world coordinate system can be obtained.
And if the pixel rows or the pixel columns of the acquired tunnel surface image are parallel to the tunnel axial direction, the positions of the pixels of each row along the second axial direction in the image coordinate system are obtained by traversing each row of pixels parallel to the tunnel axial direction. Illustratively, assume that in the image coordinate system, the x 'axis thereof is the second axis, and the x' axis coordinate of a row of pixels is m1And the x 'axis coordinate of the row of pixels adjacent to the x' axis coordinate is m2，The x-axis of the world coordinate system is the first axis. The x-axis coordinates in the world coordinate system after transformation are respectively
、
。
S131: and obtaining the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the third axial direction according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the curve equation of the tunnel surface of the tunnel section where the two pixels are located in the world coordinate system currently.
The third axial direction is another axial direction of the world coordinate system which is parallel to the current tunnel section.
Because the points in the real world corresponding to the two pixels adjacent along the second axial direction in the image are both located on the arc where the tunnel surface is located, the position coordinates of the points in the world coordinate system along the first axial direction can be substituted into the obtained curve equation of the tunnel surface, and the position coordinates of the points in the world coordinate system along the third axial direction can be solved.
S132: and according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction, obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system and the arc length of the two pixels corresponding to the tunnel surface, and obtaining the corresponding elements in the preset matrix.
The straight-line distance between two pixels adjacent along the second axial direction and the world coordinate system is specifically the straight-line distance between two pixels adjacent along the second axial direction and the cross section of the tunnel where the two pixels are located in the world coordinate system and are parallel to each other. Illustratively, the linear distance corresponding to the world coordinate system of two adjacent pixels along the second axial direction is obtained as the linear distance corresponding to the world coordinate system along the first axial direction.
In the above embodiment, the two adjacent rows of pixels have a real distance to the points of the world coordinate system along the x-axis
Will correspond to the x-axis coordinate x1、x2Respectively carrying into the surface curve equation of the tunnel to obtain
、
。
Further, according to the obtained tunnel surface curve equation and the position coordinates of two adjacent pixels along the second axial direction in the world coordinate system along the first axial direction and the position coordinates along the third axial direction, the arc length corresponding to the tunnel surface can be obtained. Specifically, the arc length of the tunnel surface corresponding to two pixels adjacent along the second axial direction can be obtained according to the following method, including the following steps:
s1320: and obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction in the world coordinate system respectively.
S1321: and obtaining the central angle of the arc line of the tunnel surface corresponding to the point of the tunnel surface corresponding to the two pixels adjacent along the second axial direction according to the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system and the curve equation.
S1322: and obtaining the arc lengths of the two pixels which are adjacent along the second axial direction on the surface of the tunnel according to the central angle.
According to the solved curve equation of the tunnel surface, the radius of the arc of the tunnel surface can be obtained, and according to a transformation formula of the cosine law, a triangular inner angle formed by the point of the two pixels adjacent along the second axial direction corresponding to the tunnel surface and the circle center of the arc of the tunnel surface is solved, namely the point of the two pixels adjacent along the second axial direction corresponding to the tunnel surface corresponds to the circle center angle of the arc of the tunnel surface, so that the arc length of the two pixels adjacent along the second axial direction corresponding to the tunnel surface is obtained.
For the above specific example, the straight-line distance of the points of the world coordinate system corresponding to the two adjacent rows of pixels in the xoy plane is calculated, and the result is:
accordingly, the arc length corresponding to the central angle can be determined:
. The arc length is the arc length of two pixels adjacent along the second axial direction corresponding to the tunnel surfaceAnd (4) degree.
Preferably, the boundary position of the pixel is used as the pixel position in this specific example.
In the method for obtaining the arc length of the two adjacent pixels corresponding to the tunnel surface, the tunnel surface arc line is taken as an arc line of a standard circle, and in practical application, if the tunnel surface arc line is an arc line of an ellipse or other linear arc lines, other methods can be used to obtain the arc length.
In this embodiment, each element of the preset matrix is a ratio of a length of a single pixel of the tunnel surface image along the second axial direction to an arc length on the tunnel surface, and a length of the single pixel of the tunnel surface image along the second axial direction to a length of a straight line parallel to a current tunnel cross section in the world coordinate system, which may be referred to as a kosmosis ratio. The curve-flat ratio reflects the influence of the arc-shaped property of the tunnel on the distortion of the actually shot image, and the larger the curve-flat ratio is, the more serious the distortion of the image is.
In this embodiment, each element of the preset matrix may specifically describe a relationship between a length of a single pixel of the tunnel surface image along the second axial direction corresponding to an arc length of the tunnel surface and a length of a single pixel of the tunnel surface image along the second axial direction corresponding to a straight line length of the tunnel surface image along the first axial direction in the world coordinate system.
In correspondence with the above specific example, the ratio of the meanders at the pixel is expressed as:
s14: and stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image.
Optionally, each pixel of the tunnel surface image may be stretched by the following method to obtain a processed tunnel surface image, please refer to fig. 5, where fig. 5 is a flowchart of a method for stretching each pixel of the tunnel surface image in this embodiment, and includes the following steps:
s140: and stretching each pixel of the tunnel surface image according to the preset matrix, and obtaining an original pixel and a stretched pixel corresponding to each pixel.
S141: and sequentially generating each whole pixel by splicing the original pixels or the stretched pixels of the adjacent pixels from the boundary pixels of the tunnel surface image along the second axial direction to obtain the processed tunnel surface image.
If the pixel has a corresponding warping-flattening ratio of η, the length of the stretched pixel becomes equal to η times of the original length, i.e. the pixel becomes η pixels, where η pixels include an original pixel and a stretched pixel, where the original pixel is an integer pixel, and the stretched pixel may be a non-integer pixel.
The preset matrix of images can be represented as:
and then, starting from the boundary pixel of the tunnel surface image along the second axial direction, reserving the original pixel of the first pixel, and splicing the stretched pixel of the first pixel and a part of the original pixel of the second pixel into a whole pixel. And continuously splicing the rest part of the original pixel of the second pixel and the stretched pixel thereof, if the whole pixel cannot be spliced, splicing the original pixel of the third pixel and the stretched pixel, and sequentially generating each whole pixel by splicing the original pixels or the stretched pixels of the adjacent pixels. The processing can ensure the consistency of the pixel values of the image before and after the processing in the next step, and can keep the authenticity of the image to the maximum extent.
Non-integer number of pixels may remain at last after stitching, and discarding is performed. Say 4096.5 pixels are generated in total, then the process proceeds by 4096 pixels.
In order to maintain the invariance and consistency before and after image processing, the generated pixels need to be redistributed simultaneously in the process of dividing and combining the pixels. Preferably, when assigning a pixel value rank to each pixel obtained after the stretching process, the pixel value of the generated integer pixel may be determined based on the original pixel proportion and the stretched pixel proportion included in the integer pixel, the original pixel value, and the stretched pixel value. Illustratively, if the integral pixel comprises an original pixel proportion which is greater than or equal to a preset proportion value, assigning the pixel value of the original pixel to the integral pixel; and if the proportion of the original pixel contained in the whole pixel is smaller than the preset proportion value, assigning the pixel value of the stretched pixel contained in the whole pixel to the whole pixel.
And after all the generated pixel value grade assignments are completed, sequentially generating a new image, wherein the image is an expanded image finally generated after the stretching processing. The expansion image completely restores the apparent characteristics of the tunnel curved surface, and provides solid guarantee for subsequent disease detection and treatment. Where the pixel values may be gray scale values, the image may be converted to a gray scale image before assigning the pixel values to the generating pixels.
According to the method, the arc equation and the section size of the section of the tunnel can be calculated according to the obtained image and the shooting parameters by modifying and assembling the shooting equipment, so that the establishment and the mutual conversion of a world coordinate system and an image coordinate system become very convenient and fast.
The influence of the tunnel curve on each pixel is considered by calculating the curve-to-average ratio of each pixel, and the pixels are restored one by one, so that the accurate size correction of the tunnel surface image in the aspect of pixel level is realized, and the size of the restored image has almost the same size as the actual image. The newly generated pixels are accurately assigned in pixel value grade, so that the image is accurately restored in pixel value, the authenticity and the accuracy of image restoration are greatly improved, and a solid guarantee is provided for tunnel disease identification and detection based on the image. The one-to-one corresponding relation between the image pixels and the actual points on the surface of the tunnel is established, point-to-point accurate restoration is achieved, and stretching and correcting of image errors are avoided.
Further, the tunnel surface image processing method of the embodiment further includes: and according to the disease area detected in the processed tunnel surface image, obtaining the position or/and size of the actual disease area corresponding to the tunnel surface according to the transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and the curve equation of the tunnel surface of the current tunnel section.
And according to the detected damaged area in the processed tunnel surface image, combining the transformation relation according to the position of each pixel of the damaged area in the image coordinate system along the second axial direction to obtain the position of each pixel of the damaged area in the world coordinate system along the first axial direction. Then, according to a curve equation of the tunnel surface of the current tunnel section, the position of each pixel of the damaged area in the world coordinate system along the third axial direction can be obtained, and further the actual position of the damaged area corresponding to the tunnel surface can be obtained.
And according to the detected damaged area in the processed tunnel surface image and the size of the damaged area in the image coordinate system, combining the transformation relation to obtain the actual size of the damaged area corresponding to the tunnel surface.
According to the method, the actual quantification result of the apparent tunnel diseases can be obtained by quantifying the disease information contained in the developed tunnel surface image obtained by processing. The method greatly facilitates the quantitative calculation of the tunnel apparent disease real size based on the image, and facilitates the subsequent disease evaluation and repair.
Correspondingly, the embodiment also provides a tunnel surface image processing system, which is used for executing the tunnel surface image processing method.
The tunnel surface image processing system of this embodiment obtains a tunnel surface image and the positions of at least three points on the tunnel surface of the current tunnel cross section in the world coordinate system, obtains a curve equation of the tunnel surface of the current tunnel cross section in the world coordinate system according to the positions of the at least three points in the world coordinate system, then obtains a transformation relationship between the position of the midpoint in the world coordinate system along a first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along a second axial direction according to the positions of the at least three points in the world coordinate system and the positions of the at least three points in the image coordinate system of the tunnel surface image, the first axial direction is an axial direction of the world coordinate system parallel to the current tunnel cross section, the second axial direction is an axial direction of the image coordinate system of the tunnel surface image parallel to the tunnel surface image, and further obtains a transformation relationship between the position of the midpoint in the world coordinate system along the first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image And obtaining a preset matrix of the tunnel surface image by the position transformation relation and a curve equation of the tunnel surface of the current tunnel section, wherein elements of the preset matrix describe the relationship that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the linear length of the tunnel surface image in a world coordinate system, and further stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image, thereby realizing the expansion of the obtained tunnel surface image. The tunnel surface image processing system of the embodiment can quickly generate the expansion image of the tunnel surface according to the tunnel surface image, and the efficiency is improved.
The tunnel surface image processing method and system provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A tunnel surface image processing method is characterized by comprising the following steps:
acquiring a tunnel surface image and acquiring the positions of at least three points on the tunnel surface of the current tunnel section in a world coordinate system;
according to the positions of the at least three points in the world coordinate system, obtaining a curve equation of the tunnel surface of the tunnel section where the at least three points are located in the world coordinate system;
according to the positions of the at least three points in the world coordinate system and the image coordinate system of the tunnel surface image, obtaining a transformation relation between the position of the midpoint in the world coordinate system along a first axial direction and the position of the midpoint in the image coordinate system of the tunnel surface image along a second axial direction, wherein the first axial direction is the axial direction of the world coordinate system parallel to the current tunnel section, and the second axial direction is the axial direction of the image coordinate system of the tunnel surface image parallel to the tunnel surface image;
obtaining a preset matrix of the tunnel surface image according to a transformation relation between a position of a midpoint of the world coordinate system along a first axial direction and a position of the midpoint of the image coordinate system of the tunnel surface image along a second axial direction and a curve equation of the tunnel surface of the tunnel section where the tunnel surface is located at present, wherein elements of the preset matrix describe a relation that the length of a single pixel of the tunnel surface image along the second axial direction corresponds to the arc length of the tunnel surface and the length of the single pixel of the tunnel surface image along the second axial direction corresponds to the straight line length of the tunnel surface image in the world coordinate system;
and stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image.
2. The tunnel surface image processing method according to claim 1, wherein the obtaining of the positions of at least three points on the tunnel surface of the current tunnel section in the world coordinate system comprises:
and acquiring the distance from at least three points on the tunnel surface of the current tunnel section to a preset point and the included angle between the at least three points and the connecting line of the preset point, and acquiring the positions of the at least three points in the world coordinate system according to the acquired distance and included angle.
3. The tunnel surface image processing method according to claim 1, specifically comprising: marking the at least three points on the tunnel surface of the current tunnel section by using light points, wherein the tunnel surface image comprises the light points corresponding to the at least three points.
4. The tunnel surface image processing method according to claim 1, wherein obtaining a transformation relationship between a position of a midpoint in the world coordinate system along a first axial direction and a position of the midpoint in the image coordinate system of the tunnel surface image along a second axial direction includes:
acquiring position difference of any two points of the at least three points in the world coordinate system along the first axial direction;
acquiring the position difference of any two points of the at least three points in the image coordinate system of the tunnel surface image along the second axial direction;
and obtaining a transformation relation between the position of the point in the world coordinate system along the first axial direction and the position of the point in the image coordinate system of the tunnel surface image along the second axial direction according to the position difference of the world coordinate system along the first axial direction and the position difference of the image coordinate system along the second axial direction.
5. The tunnel surface image processing method according to claim 1, wherein obtaining the elements of the preset matrix comprises:
according to the transformation relation between the position of the middle point of the world coordinate system along the first axial direction and the position of the middle point of the image coordinate system of the tunnel surface image along the second axial direction, obtaining the position coordinates of two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction respectively;
obtaining position coordinates of two pixels adjacent along the second axial direction in the world coordinate system along a third axial direction according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and a curve equation of a tunnel surface of a tunnel section where the two pixels are located in the world coordinate system at present;
and according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction, obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system and the arc length of the two pixels corresponding to the tunnel surface, and obtaining the corresponding elements in the preset matrix.
6. The method of claim 5, wherein obtaining the arc lengths of two pixels adjacent along the second axial direction corresponding to the tunnel surface comprises:
according to the position coordinates of the two pixels adjacent along the second axial direction in the world coordinate system along the first axial direction and the position coordinates of the two pixels adjacent along the third axial direction, obtaining the linear distance of the two pixels adjacent along the second axial direction in the world coordinate system;
according to the linear distance between the two pixels adjacent along the second axial direction and the world coordinate system and the curve equation, obtaining the central angle of the tunnel surface arc line corresponding to the point of the tunnel surface corresponding to the two pixels adjacent along the second axial direction;
and obtaining the arc lengths of the two pixels which are adjacent along the second axial direction on the surface of the tunnel according to the central angle.
7. The method according to claim 1, wherein stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image comprises:
stretching each pixel of the tunnel surface image according to the preset matrix, and obtaining an original pixel and a stretched pixel corresponding to each pixel;
and sequentially generating each whole pixel by splicing the original pixels or the stretched pixels of the adjacent pixels from the boundary pixels of the tunnel surface image along the second axial direction to obtain the processed tunnel surface image.
8. The method according to claim 7, wherein stretching each pixel of the tunnel surface image according to the preset matrix to obtain a processed tunnel surface image comprises:
and for the generated integer pixel, determining the pixel value of the integer pixel according to the original pixel proportion and the stretched pixel proportion contained in the integer pixel, and the original pixel value and the stretched pixel value.
9. The tunnel surface image processing method according to claim 1, further comprising: and according to the disease area detected in the processed tunnel surface image, obtaining the position or/and size of the actual disease area corresponding to the tunnel surface according to the transformation relation between the position of the midpoint of the world coordinate system along the first axial direction and the position of the midpoint of the image coordinate system of the tunnel surface image along the second axial direction and the curve equation of the tunnel surface of the current tunnel section.
10. A tunnel surface image processing system for performing the tunnel surface image processing method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110788036.0A CN113256714B (en) | 2021-07-13 | 2021-07-13 | Tunnel surface image processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110788036.0A CN113256714B (en) | 2021-07-13 | 2021-07-13 | Tunnel surface image processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113256714A true CN113256714A (en) | 2021-08-13 |
| CN113256714B CN113256714B (en) | 2021-09-24 |
Family
ID=77191136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110788036.0A Active CN113256714B (en) | 2021-07-13 | 2021-07-13 | Tunnel surface image processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113256714B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103674963A (en) * | 2013-11-15 | 2014-03-26 | 上海嘉珏实业有限公司 | Tunnel detection device based on digital panoramic photography and detection method thereof |
| CN103839268A (en) * | 2014-03-18 | 2014-06-04 | 北京交通大学 | Method for detecting fissure on surface of subway tunnel |
| US20160227126A1 (en) * | 2014-07-25 | 2016-08-04 | West Nippon Expressway Engineering Shikoku Company Limited | Tunnel lining surface inspection system and vehicle used for tunnel lining surface inspection system |
| KR20190049100A (en) * | 2017-11-01 | 2019-05-09 | 한국건설기술연구원 | System and Method for Managing Unexpected Situation in Tunnel |
| CN110033407A (en) * | 2019-03-29 | 2019-07-19 | 华中科技大学 | A kind of shield tunnel surface image scaling method, joining method and splicing system |
| CN112446852A (en) * | 2019-08-30 | 2021-03-05 | 成都唐源电气股份有限公司 | Tunnel imaging plane display method and intelligent defect identification system |
-
2021
- 2021-07-13 CN CN202110788036.0A patent/CN113256714B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103674963A (en) * | 2013-11-15 | 2014-03-26 | 上海嘉珏实业有限公司 | Tunnel detection device based on digital panoramic photography and detection method thereof |
| CN103839268A (en) * | 2014-03-18 | 2014-06-04 | 北京交通大学 | Method for detecting fissure on surface of subway tunnel |
| US20160227126A1 (en) * | 2014-07-25 | 2016-08-04 | West Nippon Expressway Engineering Shikoku Company Limited | Tunnel lining surface inspection system and vehicle used for tunnel lining surface inspection system |
| KR20190049100A (en) * | 2017-11-01 | 2019-05-09 | 한국건설기술연구원 | System and Method for Managing Unexpected Situation in Tunnel |
| CN110033407A (en) * | 2019-03-29 | 2019-07-19 | 华中科技大学 | A kind of shield tunnel surface image scaling method, joining method and splicing system |
| CN112446852A (en) * | 2019-08-30 | 2021-03-05 | 成都唐源电气股份有限公司 | Tunnel imaging plane display method and intelligent defect identification system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113256714B (en) | 2021-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11551341B2 (en) | Method and device for automatically drawing structural cracks and precisely measuring widths thereof | |
| CN108805934B (en) | External parameter calibration method and device for vehicle-mounted camera | |
| CN111260615B (en) | Laser and machine vision fusion-based method for detecting apparent diseases of unmanned aerial vehicle bridge | |
| US7202957B2 (en) | Three-dimensional visual sensor | |
| JP3983573B2 (en) | Stereo image characteristic inspection system | |
| CN108692719B (en) | Object detection device | |
| CN110031829B (en) | Target accurate distance measurement method based on monocular vision | |
| JP4793324B2 (en) | Vehicle monitoring apparatus and vehicle monitoring method | |
| CN102768762B (en) | Digital camera calibration method targeted to shield tunnel defect digital radiography detection and device thereof | |
| CN110378210B (en) | Vehicle and license plate detection and long-and-short-focus fusion distance measurement method based on lightweight YOLOv3 | |
| CN114331924B (en) | Large workpiece multi-camera vision measurement method | |
| CN111707668A (en) | Tunnel detection and image processing method based on sequence image | |
| CN111386530A (en) | Vehicle detection method and apparatus | |
| CN113624158B (en) | Visual dimension detection system and method | |
| CN113256714B (en) | Tunnel surface image processing method and system | |
| CN112784633B (en) | Lane boundary processing method and device, electronic equipment and storage medium | |
| JP5936527B2 (en) | Image processing apparatus and image processing method | |
| CN111429430B (en) | Tunnel detection vehicle size mapping relation calculation method based on machine vision | |
| CN111462243A (en) | Vehicle-mounted streaming media rearview mirror calibration method, system and device | |
| US11477371B2 (en) | Partial image generating device, storage medium storing computer program for partial image generation and partial image generating method | |
| CN112053401B (en) | Chip splicing method, device, equipment and storage medium | |
| CN113963069A (en) | Method, system, computer and storage medium for measuring distance between jumper and iron tower | |
| JP4005228B2 (en) | Optical member inspection apparatus, image processing apparatus, and computer-readable medium | |
| JPH11132763A (en) | Distance measuring method | |
| CN116242316B (en) | Method and device for detecting positioning points of rigid contact net in real time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |