CN114842213A - Obstacle contour detection method and device, terminal equipment and storage medium - Google Patents
Obstacle contour detection method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN114842213A CN114842213A CN202210526559.2A CN202210526559A CN114842213A CN 114842213 A CN114842213 A CN 114842213A CN 202210526559 A CN202210526559 A CN 202210526559A CN 114842213 A CN114842213 A CN 114842213A
- Authority
- CN
- China
- Prior art keywords
- image
- contour
- actual
- point
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 38
- 230000002159 abnormal effect Effects 0.000 claims abstract description 65
- 230000009466 transformation Effects 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims description 41
- 239000011159 matrix material Substances 0.000 claims description 16
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000005260 corrosion Methods 0.000 claims description 8
- 230000007797 corrosion Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 230000003628 erosive effect Effects 0.000 claims description 4
- 230000004888 barrier function Effects 0.000 abstract description 7
- 238000005286 illumination Methods 0.000 abstract description 6
- 238000012360 testing method Methods 0.000 abstract description 5
- 238000012549 training Methods 0.000 abstract description 5
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 240000004050 Pentaglottis sempervirens Species 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000005530 etching Methods 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a method, a device, a terminal device and a storage medium for detecting an obstacle contour, wherein the method for detecting the obstacle contour comprises the steps of acquiring an actual image of an actual scene through a monocular camera, obtaining an actual aerial view through perspective transformation, calculating actual characteristic values of all pixel points in the actual aerial view, obtaining a binary image based on the aerial characteristic values and the actual characteristic values of all the pixel points of the aerial view, denoising abnormal points in the binary image to obtain a first image, and detecting the contour of an obstacle in the actual scene through the contour of the first image. The detection method can accurately identify the size and the appearance characteristics of the barrier, can remove the influence of illumination on image characteristic identification, can be applied to most barrier characteristic identification, and avoids using a large amount of algorithm training and testing, thereby reducing the production cost and the labor cost.
Description
Technical Field
The present invention relates to the field of obstacle detection, and in particular, to a method and an apparatus for detecting an obstacle contour, a terminal device, and a storage medium.
Background
In the prior art, a binocular camera is generally used for detecting obstacles, and an AI (Artificial Intelligence) recognition algorithm is used for operation. However, the binocular camera is complex in structure, high in manufacturing cost and high in requirement for computing capacity, needs to be trained in advance if an AI recognition algorithm needs to be used, and has high requirements for storage and computing capacity of a processor, when the binocular camera is used, the requirement for actual use is high due to the fact that the binocular camera needs to adapt to different illumination and different obstacles, a large number of scenes are needed for algorithm training and testing, and the use is complex.
Disclosure of Invention
In view of the above, the present application provides an obstacle contour detection method, an apparatus, a terminal device and a storage medium.
In a first aspect, the present application provides a monocular camera obstacle contour detection method, where the method includes:
acquiring an actual image of an actual scene through a monocular camera, and carrying out perspective transformation on the actual image to obtain an actual aerial view;
calculating the actual characteristic value of each pixel point in the actual aerial view;
performing abnormal point identification processing based on the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view to obtain a binary image containing abnormal points;
denoising abnormal points in the binary image to obtain a first image;
and carrying out contour detection on the first image, and determining the contour of an obstacle in the actual scene.
In some embodiments, the detecting the contour of the first image and determining the contour of the obstacle in the actual scene includes:
extracting at least one contour in the first image based on a preset algorithm;
calculating the length and the area of the center point of each contour and the minimum oblique rectangle;
if the center point of the current contour is not located at the boundary of the first image, detecting whether the side length is smaller than a preset side length threshold value and whether the area is smaller than a preset area threshold value;
if the side length is smaller than the side length threshold and the area is smaller than the area threshold, determining the current contour as an interference contour and discarding the interference contour, otherwise, keeping the current contour;
if the center point of the current contour is at the boundary of the first image, retaining the current contour;
the remaining contour is taken as the contour of the obstacle.
In some embodiments, said acquiring an actual image of an actual scene by a monocular camera comprises:
acquiring an air-ground image of an air-ground scene through a monocular camera, and measuring the actual size of the air-ground scene to determine the length-width ratio of the air-ground scene;
determining an air-ground coordinate based on the length-width ratio and the resolution of an air-ground aerial view of the air-ground scene to be generated;
setting corner points at the boundary of the space scene, and determining the corresponding corner point coordinates of the corner points in the space image;
calculating parameters of a perspective transformation matrix based on the corner coordinates and the space coordinates to determine the perspective transformation matrix;
and performing perspective transformation on the air-ground image through the perspective transformation matrix to obtain an air-ground aerial view of the air-ground scene.
In some embodiments, the feature value of each pixel point in the corresponding aerial view is calculated by:
carrying out gray processing on the corresponding aerial view to obtain a gray image;
calculating corresponding central brightness and environment brightness based on the pixels of the gray level image as the center;
and taking the ratio of the central brightness to the environment brightness of each pixel point as a characteristic value of the corresponding pixel point.
In some embodiments, the performing, based on the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view image, an abnormal point identification process includes:
comparing the space characteristic value and the actual characteristic value of the pixel point at the same position to obtain a corresponding ratio;
and if the ratio is less than or equal to a preset ratio threshold, determining the pixel point as the abnormal point, otherwise determining the pixel point as the normal point.
In some embodiments, the denoising processing on the abnormal point in the binary image to obtain a first image includes:
performing expansion processing of a first pixel size on the binary image to obtain a second image;
carrying out corrosion treatment of a second pixel size on the second image to obtain a third image;
performing expansion processing of a third pixel size on the third image to obtain a fourth image;
carrying out corrosion treatment of a fourth pixel size on the fourth image to obtain a first image;
wherein the first pixel size, the second pixel size, the third pixel size, and the fourth pixel size are related as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size and smaller than the third pixel size.
In some embodiments, the etching treatment comprises:
sequentially superposing the central point of a preset structural element with each abnormal point in the binary image, and judging whether all pixel points in the structural element are abnormal points;
if all the pixel points are the abnormal points, the pixel points corresponding to the central point are reserved;
and if not, modifying the pixel point corresponding to the central point into a normal point.
In a second aspect, an embodiment of the present application further provides an obstacle contour determination device, including:
the aerial view acquisition module is used for acquiring an actual image of an actual scene through the monocular camera and carrying out perspective transformation on the actual image to obtain an actual aerial view;
the characteristic value calculation module is used for calculating the actual characteristic value of each pixel point in the actual aerial view;
the abnormal point identification module is used for carrying out abnormal point identification processing on the basis of the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view to obtain a binary image containing abnormal points;
the denoising processing module is used for denoising the abnormal points in the binary image to obtain a first image;
and the contour determining module is used for carrying out contour detection on the first image and determining the contour of the obstacle in the actual scene.
In a third aspect, an embodiment of the present application further provides a terminal device, which includes a memory and a processor, where the memory stores a computer program, and the computer program, when running on the processor, executes any one of the above obstacle contour detection methods.
In a fourth aspect, an embodiment of the present application further provides a readable storage medium, which stores a computer program, where the computer program, when executed on a processor, performs any one of the above obstacle contour detection methods.
The embodiment of the application has the following beneficial effects:
the embodiment of the application provides a method and a device for detecting an obstacle contour, a terminal device and a storage medium, wherein an actual image of an actual scene is obtained through a monocular camera, perspective transformation is performed on the actual image to obtain an actual aerial view, an actual characteristic value of each pixel point in the actual aerial view is calculated, abnormal point identification processing is performed on the basis of the aerial characteristic value and the actual characteristic value of each pixel point of the aerial view to obtain a binary image containing abnormal points, the abnormal points in the binary image are subjected to denoising processing to obtain a first image, contour detection is performed on the first image, and the contour of the obstacle in the actual scene is determined. The detection method can judge the size, the appearance characteristic and the like of the recognized barrier more accurately, can remove the influence of illumination on image characteristic recognition by denoising through the abnormal points, can be applied to recognition of most barrier characteristics, avoids using a large amount of algorithm training and testing, and can reduce the production cost and the labor cost.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings required to be used in the embodiments will be briefly described below, and it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope of the present invention. Like components are numbered similarly in the various figures.
Fig. 1 is a schematic flow chart illustrating an obstacle contour detection method according to an embodiment of the present application;
fig. 2 is a schematic flow chart illustrating a process of generating an aerial view of an air space in an obstacle contour detection method according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating a process of calculating feature values in an obstacle contour detection method according to an embodiment of the present application;
fig. 4 is a schematic flowchart illustrating an abnormal point identification process in an obstacle contour detection method according to an embodiment of the present application;
fig. 5 is a schematic flow chart illustrating corrosion processing in an obstacle contour detection method according to an embodiment of the present application;
fig. 6 is a schematic flow chart illustrating obstacle contour determination in an obstacle contour detection method according to an embodiment of the present application;
fig. 7 is a schematic diagram illustrating an obstacle in an obstacle contour detection method according to an embodiment of the present application;
fig. 8 shows a schematic structural diagram of an obstacle contour determination device according to an embodiment of the present application.
Description of the main element symbols:
10-obstacle contour determination means; 11-a bird's eye view acquisition module; 12-a feature value calculation module; 13-an anomaly identification module; 14-a denoising processing module; 15-contour determination module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Hereinafter, the terms "including", "having", and their derivatives, which may be used in various embodiments of the present invention, are only intended to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the existence of, or adding to, one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.
Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present invention belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their contextual meaning in the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments of the present invention.
Example 1
In an embodiment of the present application, as shown in fig. 1, a method for detecting an obstacle contour of a monocular camera provided in the present application includes steps S110 to S150:
s110: and acquiring an actual image of an actual scene through a monocular camera, and carrying out perspective transformation on the actual image to obtain an actual aerial view.
In this embodiment, the object is re-projected onto a new imaging plane by transforming according to the object imaging projection rule, in other words, a top plan view of the air-ground scene, that is, a bird's-eye view, is obtained by performing perspective transformation on a photograph obtained from the angle of view of the monocular camera, so that the target features can be detected on a uniform basis, and the detection of the obstacle can be realized.
As shown in fig. 2, before step S100, steps S210 to S250 are further included:
s210: an air-ground image of an air-ground scene is acquired through a monocular camera, and the actual size of the air-ground scene is measured to determine the length-width ratio of the air-ground scene.
In the embodiment, an air-ground image of an air-ground scene is acquired through a monocular camera, the actual size of the air-ground scene is measured, and the length-width ratio of the actual field is determined by measuring the actual size of the air-ground scene.
S220: determining an air-ground coordinate based on the length-width ratio and a resolution of an air-ground aerial view of the air-ground scene to be generated.
Through the length-width ratio of the air-ground scene and the maximum resolution of the air-ground aerial view to be generated, the length and the width of the air-ground scene at the maximum resolution can be calculated, and therefore the air-ground coordinates corresponding to the air-ground scene can be determined.
Exemplarily, if the actual length-width ratio of the air space scene obtained by measurement is m: n, the length L and the width W for obtaining the maximum resolution of the aerial view of the empty space to be generated can be calculated, and the length L 'and the width W' of the empty space at the maximum resolution can be calculated. By the actual length-to-width ratio m: n and the length L and width W of the space image can determine space coordinates obtained after space scene perspective transformation, wherein the space coordinates comprise: (0, 0), (0, L '), (W', 0), (W ', L'). When W is m/n < L, the length L 'and the width W' are respectively L '═ W is m/n, and W' ═ W is obtained; otherwise, the length L 'and the width W' are L '═ L, W' ═ L × n/m, respectively.
In this embodiment, the cost of obtaining images through the monocular camera is lower, and when the monocular camera is actually used, algorithm training and testing are not required to be carried out on a large number of scenes, so that the workload of workers can be reduced.
S230: and setting corner points at the boundary of the space scene, and determining the corresponding corner point coordinates of the corner points in the space image.
It will be appreciated that a corner point is an extreme point, i.e. a point in the image where certain aspects of the property are particularly prominent. In this embodiment, a plurality of corner points may be set at the boundary of the open space scene, the intersection of two lines may be used as a corner point, and a point located on two adjacent objects with different main directions may be used as a corner point. The coordinates of the corresponding corner points in the air-ground image can be determined by a preset detection algorithm, wherein the detection algorithm includes, but is not limited to, Harris corner point algorithm, Shi-Tomas corner point algorithm, and other detection algorithms.
Exemplarily, at least four corner points may be set at the boundary of the air-ground scene, such as placing a pure black block with a size of 2cm × 2cm at the boundary, and then the pure black block may be a corner point in the image. The Harris corner algorithm automatically identifies four corner coordinates (x1, y1), (x2, y1), (x1, y2), (x2, y2), wherein x represents the length direction and y represents the width direction.
S240: calculating parameters of a perspective transformation matrix based on the corner coordinates and the space coordinates to determine the perspective transformation matrix.
The perspective transformation is to project a two-dimensional picture onto a three-dimensional viewing plane and then convert the two-dimensional picture into two-dimensional coordinates, which is also called projection mapping.
Specifically, the perspective transformation matrix is as follows:
where X, Y and Z represent the three-dimensional coordinates after perspective transformation, and x and y represent the two-dimensional coordinates before perspective transformation. The above perspective transformation matrix formula can be used to obtain:
since (X, Y, Z) is a three-dimensional coordinate, the obtained three-dimensional coordinate needs to be converted into a two-dimensional coordinate (X ', Y', 1) to obtain the following formula:
thus, x 'and y' are the final computed results of the two-dimensional perspective transformation, where c3 is 1. The above-mentioned null field ground coordinates and angular coordinates are substituted into the above-mentioned formula as coordinate points before and after the perspective transformation, that is, (0, 0), (0, L '), (W', 0), (W ', L') and (x1, y1), (x2, y1), (x1, y2), (x2, y2) are substituted into the above-mentioned formula as 4 groups (x, y), (x ', y'), and parameters a1, a2, a3, b1, b2, b3 and c1 of the perspective transformation matrix are calculated.
S250: and performing perspective transformation on the air-ground image through the perspective transformation matrix to obtain an air-ground aerial view of the air-ground scene.
Through the perspective transformation formula, any point (x, y) of the image acquired through the monocular camera can be substituted, and the coordinate point (x ', y') after perspective transformation is obtained through calculation. In other words, the acquired air-ground images are subjected to perspective transformation through the perspective transformation matrix to obtain an air-ground aerial view of the air-ground scene.
The object is re-projected to a new imaging plane by transforming according to the object imaging projection rule, in other words, a top plan view of the air-ground scene, namely a bird's-eye view, is obtained by carrying out perspective transformation on a picture obtained from the visual angle of the monocular camera, so that target features, namely obstacles, can be conveniently detected on a uniform basis.
In step S110, after acquiring the air-ground image of the air-ground scene, the installation position of the monocular camera is kept unchanged, the monocular camera acquires the actual image of the actual scene, and the perspective transformation matrix performs the perspective transformation on the actual image to obtain the top plan view of the actual scene, i.e., the actual bird' S eye view.
S120: and calculating the actual characteristic value of each pixel point in the actual aerial view.
It can be understood that after the actual bird's-eye view is obtained, the actual characteristic value of each pixel point in the actual bird's-eye view corresponding to the actual scene is determined through calculation. And calculating the actual characteristic value and the air-ground characteristic value of each pixel point of the air-ground aerial view corresponding to the air-ground scene in the same calculation mode. As shown in fig. 3, the feature values of each pixel point in the corresponding aerial view of the actual scene and the air-ground scene can be calculated through the following substeps:
substep S121: and carrying out gray scale processing on the corresponding aerial view to obtain a gray scale image.
It can be understood that, after perspective transformation, a bird's-eye view corresponding to an air-ground scene and an actual scene is obtained, and the bird's-eye view is converted into a gray image through gray processing, wherein the gray value corresponding to each pixel point in the gray image ranges from 0 to 255, and the method for performing gray processing on the corresponding bird's-eye view includes: component, maximum, average, and weighted average.
Substep S122: and calculating corresponding central brightness and environment brightness by taking each pixel point of the gray level image as a center.
After the gray image is obtained, taking each pixel point in the gray image as a center, taking the pixel points in a first range around the pixel point as a core area of the pixel point, and calculating the average brightness of the core area of each pixel point as the center brightness of the pixel point, namely taking the pixel point as the core and taking the average value of the gray values of all the pixel points in the core area. And taking the pixel points in the second surrounding range as the environment area of the pixel point, and calculating the average brightness of the environment area of each pixel point as the environment brightness of the pixel point, namely calculating the average value of all gray values in the environment area around each pixel point. And the pixel points in the first range are smaller than the pixel points in the second range.
Exemplarily, when the pixel (30, 30) is a core point and the range value of the core region is 3 × 3, the central luminance of the pixel is an average value of the image gray values of 9 points in total of the pixels (29, 29), (29, 30), (29, 31), (30, 29), (30, 30), (30, 31), (31, 29), (31, 30), (31, 31). When the range value of the environment region is 9 × 9, calculating the average value of the gray values of a total of 81 pixel points in the surrounding environment region as the environment brightness of the pixel point.
Substep S123: and taking the ratio of the central brightness to the environment brightness of each pixel point as a characteristic value of the corresponding pixel point.
After the central brightness and the environmental brightness of each pixel point are obtained, the ratio of the central brightness and the environmental brightness corresponding to each pixel point is calculated and used as the characteristic value of each pixel point. And the number of the characteristic values in the gray level image is equal to the number of the pixel points in the image. After the air-ground characteristic value corresponding to each pixel point in the air-ground aerial view corresponding to the air-ground scene is obtained through calculation, the air-ground characteristic value of each pixel point can be stored in the memory in advance. After the central brightness of each pixel point is compared with the environmental brightness, the interference of illumination environment change on obstacle identification can be removed.
S130: and carrying out abnormal point identification processing based on the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view to obtain a binary image containing abnormal points.
It can be understood that when no obstacle exists in the actual scene, even if the overall illumination brightness changes, the ratio of the brightness of the core area of each pixel point to the brightness of the surrounding environment area does not change significantly. When an obstacle exists in an actual scene, the characteristics of a core area and an environment area are changed, and the brightness ratio of each pixel point in the obstacle to the surrounding environment is obviously changed. By comparing the air-ground characteristic value of each pixel point in the air-ground aerial view with the actual characteristic value of each pixel point in the actual aerial view, the abnormal point in the actual aerial view, namely the pixel point when an obstacle exists on the ground, can be determined. As shown in fig. 4, step S130 includes the following sub-steps:
substep S131: and comparing the space characteristic value and the actual characteristic value of the pixel point at the same position to obtain a corresponding ratio.
The number of pixel points in the corresponding aerial view in the air space scene and the actual scene is the same, and each pixel point in the air space aerial view corresponds to the position of each pixel point in the actual aerial view one by one. And comparing the actual characteristic value corresponding to each pixel point in the actual aerial view with the corresponding air-ground characteristic value in the air-ground aerial view to obtain the ratio of the characteristic values corresponding to each pixel point.
Substep S132: and judging whether the ratio is less than or equal to a preset ratio threshold value.
And judging whether the ratio corresponding to each pixel point is less than or equal to a preset ratio threshold according to a ratio threshold preset according to actual conditions, if the ratio calculated by the pixel points is less than or equal to the preset ratio threshold, executing substep S133, otherwise, executing substep S134.
Substep S133: and determining the pixel points as the abnormal points.
Substep S134: and determining the pixel points as normal points.
And if the ratio obtained by calculating the pixel point is greater than the preset ratio threshold, the pixel point is considered to be matched, namely the pixel point is a normal point. If the characteristic value of the pixel point is less than or equal to the preset ratio threshold, the pixel point is determined to be not matched, and the pixel point is marked as an abnormal point. For example, when the preset ratio threshold is 80%, the ratio of the actual characteristic value of a certain pixel point to the space characteristic value is calculated to be 70%, and at this time, the ratio is smaller than the ratio threshold, and the pixel point is determined to be an abnormal point.
After determining whether each pixel point in the actual aerial view is an abnormal point, namely marking each abnormal point in the image, a binary image containing the abnormal point can be obtained. For example, the labeling result of each pixel point may be represented by 0 and 1, and the actual bird's eye view image may be labeled to obtain a binary image including an outlier, and the binary image may be stored. In this case, 1 represents an abnormal point, 0 represents a normal point, and in this case, 0 is represented by black and 1 is represented by white.
S140: and denoising the abnormal points in the binary image to obtain a first image.
It can be understood that, after the binary image is obtained, the determination result corresponding to each pixel point is a normal point or an abnormal point. When an obstacle exists in an actual scene, significant abnormal points, namely pixel points with obviously changed central brightness and environment brightness, can be identified, but because the binary image has noise, a large number of discrete abnormal points will appear, and the abnormal points need to be further screened to judge whether the abnormal points are the pixel points corresponding to the obstacle. In this embodiment, denoising processing, that is, expansion processing and corrosion processing, is performed on the pixel points marked as outliers in the binary image to obtain the first image. The method comprises the following steps of carrying out denoising processing on abnormal points in the binary image in the following modes:
firstly, the binary image is subjected to expansion processing with the first pixel size to obtain a second image, so that the connected region of the image can be enlarged, and the noise can be prevented from submerging the actual barrier; then, carrying out corrosion treatment on the second image in a second pixel size to obtain a third image, and removing partial pixel points in the second image and partial interference noise; then, performing expansion processing on the third image in a third pixel size to obtain a fourth image, and expanding a connected region in the third image again; and finally, carrying out corrosion treatment on the fourth image according to the fourth pixel size to obtain the first image, so that the binary image characteristic points can be recovered, and the change of the characteristic point size caused by the corrosion treatment and the expansion treatment is avoided. The relationship among the pixel sizes is as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size and smaller than the third pixel size.
Exemplarily, when the first pixel size is n1, the second pixel size is m1, the third pixel size is n2, and the fourth pixel size is m2, m1> n1, n2> m1, m2 ═ n1+ n2-m1, that is, n1+ n2 ═ m2+ m 2.
In this embodiment, the expansion processing and the erosion processing have the same size, and the four-step denoising processing can remove noise interference and retain the size characteristics of the binary image.
As shown in fig. 5, the image erosion process includes the following substeps:
substep S141: and sequentially overlapping the central point of a preset structural element with each abnormal point in the binary image.
In this embodiment, a template matrix for etching operation is first defined, that is, a structural element for etching processing is preset as a first structural element, that is, a structure for control operation, and a center point of the first structural element is sequentially placed in each pixel point marked as an outlier in the binary image.
Substep S142: and judging whether all the pixel points in the structural elements are abnormal points or not.
If all the pixel points are abnormal points, the pixel points corresponding to the central point are reserved. Otherwise, step S143 is executed.
Substep S143: and if all the pixel points in the structural elements are not all abnormal points, modifying the pixel points corresponding to the central points into normal points.
And judging that all pixel points included in the first structural element are abnormal points, if the image pixel points covered by all the elements in the first structural element are the abnormal points, reserving the pixel points corresponding to the center points of the first structural element, and if not, deleting the pixel points corresponding to the center points of the first structural element, namely modifying the pixel points corresponding to the center points from the abnormal points into the normal points. The central point of the first structural element is sequentially overlapped with each pixel point of the binary image, so that whether the pixel point needing to be deleted exists or not is judged, the main area in the image can be reduced, and a small connected domain caused by noise is removed.
Exemplarily, the determination result of each pixel point is represented by 0 and 1, where 0 represents a normal point and 1 represents an abnormal point. The size of the first structural element is 3 × 3 pixels, the first structural element is shaped into a rectangular structure, and the center point of the first structural element of the 3 × 3 pixels is sequentially placed in each pixel point of which the element is 1 in the binary image. If the results of the pixel points covered by all the elements in the first structural element are all 1, retaining the pixel point corresponding to the center point of the first structural element, namely keeping the judgment result mark of the pixel point as 1; otherwise, the judgment result of the pixel point corresponding to the central point is changed from 1 to 0.
The image expansion processing includes the steps of:
first, a template matrix for the dilation operation is defined, in other words, the structural element for the dilation process is set in advance as a second structural element, i.e., a structure for the control operation. For example, the second structural element may be defined as 3 × 3 pixels in size and rectangular in shape. And (3) sequentially superposing the central point of the second structural element with each pixel point marked as an abnormal point of the binary image, judging whether all the pixel points covered by the second structural element are the abnormal points, and if the normal points exist, modifying the normal points into the abnormal points, thereby finishing the expansion action. By the expansion processing, the area of each region can be expanded to fill in the cavity caused by the noise.
S150: and carrying out contour detection on the first image, and determining the contour of an obstacle in the actual scene.
After the first image including a plurality of connected regions is obtained, the shape feature of each connected region can be identified by the contour detection process. Wherein, the image contour refers to the boundary of the image, i.e. the external feature of the target image. As shown in fig. 6, step S150 includes the following sub-steps:
substep S151: extracting at least one contour in the first image based on a preset algorithm.
In this embodiment, an image contour is obtained from the first image after the denoising processing by using a preset algorithm, where the preset algorithm may be a Satoshi Suzuki algorithm, each image may have a plurality of contours, and each contour is composed of a plurality of points.
Substep S152: and calculating the side length and the area of the central point of each contour and the minimum oblique rectangle.
It can be understood that the central point and the minimum oblique rectangle corresponding to each contour are determined, and the side length and the area of the minimum oblique rectangle corresponding to each contour can be calculated through Python, OpenCV, and the like. As shown in fig. 7, the gray part in the graph is a suspected obstacle, the minimum outline size of the obstacle is determined first, in other words, the minimum resolution of the obstacle is determined first, as shown in the box part in fig. 7, the effect of identifying 4 obstacles in the box is that the center point of each outline is marked by a white point in the box in the graph. The coordinates of the center point and the pixel size of each contour can be converted into the actual size in proportion.
Substep S153: determining whether the center point of a current contour is at a boundary of the first image.
Judging whether the central point of each contour is at the boundary of the first image, if the central point of the current contour is not at the boundary of the first image, executing the substep S154, otherwise, executing the substep S156, and keeping the current contour as an in-doubt contour for merging with the boundary image acquired by the adjacent monocular camera.
S154: and detecting whether the side length is smaller than a preset side length threshold value or not and whether the area is smaller than a preset area threshold value or not.
If the side length is less than the side length threshold and the area is less than the area threshold, then substep S155 is performed, otherwise substep S156 is performed.
Substep S155: determining the current profile as an interference profile and discarding.
Substep S156: the current contour is retained.
If the side length of the current contour is smaller than the preset side length threshold and the area of the current contour is smaller than the preset area threshold, the current contour is determined to be an interference contour, and the contour is discarded. Otherwise, the outline is determined to accurately reflect the characteristics of the obstacle, namely the outline of the obstacle.
Exemplarily, the preset side length threshold size is 5 pixels, the preset area threshold size is 30 pixels, and if each side length size corresponding to the current contour is less than 5 pixels and the area is less than 30 pixels, the current contour is considered as an interference contour.
Substep S157: the remaining contour is taken as the contour of the obstacle.
And outputting each reserved contour, wherein the output contour can accurately reflect the characteristics of the obstacle, namely, the reserved contour can be used as the contour of the obstacle.
In the embodiment, the influence of the installation angle of the monocular camera can be eliminated through perspective transformation processing, calculation is performed on the unified top plan view, the contents such as the size and the appearance characteristics of the recognized barrier can be more accurately judged, the recognition influence of different illuminations on the image characteristics can also be eliminated, the method can be applied to recognition of the characteristics of most barriers, and the use of a large amount of pre-algorithm training and testing is avoided, so that the production cost and the labor cost can be reduced.
Based on the obstacle contour detection method of the foregoing embodiment, fig. 8 shows a schematic structural diagram of an obstacle contour determination device 10 provided in the embodiment of the present application.
The obstacle contour determination device 10 includes:
the aerial view acquisition module 11 is used for acquiring an actual image of an actual scene through a monocular camera and performing perspective transformation on the actual image to obtain an actual aerial view;
the characteristic value calculating module 12 is used for calculating the actual characteristic value of each pixel point in the actual aerial view;
the abnormal point identification module 13 is used for carrying out abnormal point identification processing on the basis of the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view image to obtain a binary image containing abnormal points;
the denoising processing module 14 is used for denoising the abnormal points in the binary image to obtain a first image;
and the contour determining module 15 is used for performing contour detection on the first image and determining the contour of the obstacle in the actual scene.
In this embodiment, the obstacle contour determination device 10 is configured to execute the obstacle contour detection method according to the above embodiment through the cooperative use of the bird's-eye view image acquisition module 11, the feature value calculation module 12, the abnormal point identification module 13, the denoising processing module 14, and the contour determination module 15, and the implementation and beneficial effects related to the above embodiment are also applicable to this embodiment, and are not described herein again.
In addition, the present application also proposes a terminal device, which includes a memory and a processor, where the memory stores a computer program, and the computer program executes the obstacle contour detection method according to the above embodiment when running on the processor.
The embodiment also provides a computer storage medium for storing a computer program used in the terminal device.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, each functional module or unit in each embodiment of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention or a part of the technical solution that contributes to the prior art in essence can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention.
Claims (10)
1. An obstacle contour detection method, comprising:
acquiring an actual image of an actual scene through a monocular camera, and carrying out perspective transformation on the actual image to obtain an actual aerial view;
calculating the actual characteristic value of each pixel point in the actual aerial view;
performing abnormal point identification processing based on the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view to obtain a binary image containing abnormal points;
denoising abnormal points in the binary image to obtain a first image;
and carrying out contour detection on the first image, and determining the contour of an obstacle in the actual scene.
2. The method for detecting the contour of an obstacle according to claim 1, wherein the detecting the contour of the first image to determine the contour of the obstacle in the actual scene comprises:
extracting at least one contour in the first image based on a preset algorithm;
calculating the length and the area of the center point of each contour and the minimum oblique rectangle;
if the center point of the current contour is not located at the boundary of the first image, detecting whether the side length is smaller than a preset side length threshold value and whether the area is smaller than a preset area threshold value;
if the side length is smaller than the side length threshold and the area is smaller than the area threshold, determining the current contour as an interference contour and discarding the interference contour, otherwise, keeping the current contour;
if the center point of the current contour is at the boundary of the first image, retaining the current contour;
the remaining contour is taken as the contour of the obstacle.
3. The obstacle contour detection method according to claim 1 or 2, wherein the acquiring of the actual image of the actual scene by the monocular camera is preceded by:
acquiring an air-ground image of an air-ground scene through a monocular camera, and measuring the actual size of the air-ground scene to determine the length-width ratio of the air-ground scene;
determining an air-ground coordinate based on the length-width ratio and the resolution of an air-ground aerial view of the air-ground scene to be generated;
setting corner points at the boundary of the space scene, and determining the corresponding corner point coordinates of the corner points in the space image;
calculating parameters of a perspective transformation matrix based on the corner coordinates and the space coordinates to determine the perspective transformation matrix;
and performing perspective transformation on the air-ground image through the perspective transformation matrix to obtain an air-ground aerial view of the air-ground scene.
4. The obstacle contour detection method according to claim 1, wherein the feature value of each pixel point in the corresponding bird's eye view is calculated by:
carrying out gray processing on the corresponding aerial view to obtain a gray image;
calculating corresponding central brightness and environment brightness based on the pixels of the gray level image as the center;
and taking the ratio of the central brightness to the environment brightness of each pixel point as a characteristic value of the corresponding pixel point.
5. The obstacle contour detection method according to claim 4, wherein the performing of the abnormal point identification processing based on the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view includes:
comparing the space characteristic value and the actual characteristic value of the pixel point at the same position to obtain a corresponding ratio;
and if the ratio is less than or equal to a preset ratio threshold, determining the pixel point as the abnormal point, otherwise determining the pixel point as the normal point.
6. The method for detecting the obstacle contour according to claim 1, wherein the denoising processing of the abnormal point in the binary image to obtain the first image comprises:
performing expansion processing of a first pixel size on the binary image to obtain a second image;
performing erosion processing of a second pixel size on the second image to obtain a third image;
performing expansion processing of a third pixel size on the third image to obtain a fourth image;
performing corrosion treatment of a fourth pixel size on the fourth image to obtain a first image;
wherein the first pixel size, the second pixel size, the third pixel size, and the fourth pixel size are related as follows: the sum of the first pixel size and the third pixel size is equal to the sum of the second pixel size and the fourth pixel size, and the first pixel size is smaller than the second pixel size and smaller than the third pixel size.
7. The obstacle profile detection method according to claim 6, wherein the erosion process includes:
sequentially superposing the central point of a preset structural element with each abnormal point in the binary image, and judging whether all pixel points in the structural element are abnormal points;
if all the pixel points are the abnormal points, the pixel points corresponding to the central point are reserved;
and if not, modifying the pixel point corresponding to the central point into a normal point.
8. An obstacle contour determination device, comprising:
the aerial view acquisition module is used for acquiring an actual image of an actual scene through the monocular camera and carrying out perspective transformation on the actual image to obtain an actual aerial view;
the characteristic value calculation module is used for calculating the actual characteristic value of each pixel point in the actual aerial view;
the abnormal point identification module is used for carrying out abnormal point identification processing on the basis of the air-ground characteristic value and the actual characteristic value of each pixel point of the air-ground aerial view to obtain a binary image containing abnormal points;
the denoising processing module is used for denoising abnormal points in the binary image to obtain a first image;
and the contour determining module is used for carrying out contour detection on the first image and determining the contour of the obstacle in the actual scene.
9. A terminal device, characterized in that it comprises a memory and a processor, the memory storing a computer program which, when run on the processor, performs the obstacle contour detection method according to any one of claims 1 to 7.
10. A readable storage medium, characterized in that it stores a computer program which, when run on a processor, performs the obstacle contour detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526559.2A CN114842213A (en) | 2022-05-16 | 2022-05-16 | Obstacle contour detection method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526559.2A CN114842213A (en) | 2022-05-16 | 2022-05-16 | Obstacle contour detection method and device, terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114842213A true CN114842213A (en) | 2022-08-02 |
Family
ID=82571116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210526559.2A Pending CN114842213A (en) | 2022-05-16 | 2022-05-16 | Obstacle contour detection method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114842213A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152151A (en) * | 2023-10-31 | 2023-12-01 | 新东鑫(江苏)机械科技有限公司 | Motor shell quality detection method based on machine vision |
| CN117911792A (en) * | 2024-03-15 | 2024-04-19 | 垣矽技术(青岛)有限公司 | Pin detecting system for voltage reference source chip production |
-
2022
- 2022-05-16 CN CN202210526559.2A patent/CN114842213A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152151A (en) * | 2023-10-31 | 2023-12-01 | 新东鑫(江苏)机械科技有限公司 | Motor shell quality detection method based on machine vision |
| CN117152151B (en) * | 2023-10-31 | 2024-02-02 | 新东鑫(江苏)机械科技有限公司 | Motor shell quality detection method based on machine vision |
| CN117911792A (en) * | 2024-03-15 | 2024-04-19 | 垣矽技术(青岛)有限公司 | Pin detecting system for voltage reference source chip production |
| CN117911792B (en) * | 2024-03-15 | 2024-06-04 | 垣矽技术(青岛)有限公司 | Pin detecting system for voltage reference source chip production |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110717489B (en) | Method, device and storage medium for identifying text region of OSD (on Screen display) | |
| CN109272489B (en) | Infrared weak and small target detection method based on background suppression and multi-scale local entropy | |
| US10043090B2 (en) | Information processing device, information processing method, computer-readable recording medium, and inspection system | |
| US10614736B2 (en) | Foreground and background detection method | |
| CN114842213A (en) | Obstacle contour detection method and device, terminal equipment and storage medium | |
| CN111091592B (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| CN107404628B (en) | Image processing apparatus and method, and monitoring system | |
| JP7272024B2 (en) | Object tracking device, monitoring system and object tracking method | |
| CN113744142B (en) | Image restoration method, electronic device and storage medium | |
| CN115797300A (en) | Edge detection method and device based on adaptive gradient threshold canny operator | |
| CN114359665A (en) | Training method and device of full-task face recognition model and face recognition method | |
| CN111882578A (en) | Foreground image acquisition method, foreground image acquisition device and electronic equipment | |
| CN113255537A (en) | Image enhancement denoising method for identifying sailing ship | |
| CN114648547A (en) | Weak and small target detection method and device for anti-unmanned aerial vehicle infrared detection system | |
| CN116824166B (en) | Transmission line smoke identification method, device, computer equipment and storage medium | |
| CN106778822B (en) | Image straight line detection method based on funnel transformation | |
| CN110765875B (en) | Method, equipment and device for detecting boundary of traffic target | |
| CN116894849A (en) | Image segmentation method and device | |
| US20190251695A1 (en) | Foreground and background detection method | |
| CN116363753A (en) | Tumble detection method and device based on motion history image and electronic equipment | |
| CN112270693B (en) | Method and device for detecting motion artifact of time-of-flight depth camera | |
| CN113673362A (en) | Method and device for determining motion state of object, computer equipment and storage medium | |
| CN113706505A (en) | Cylinder fitting method and device for removing local outliers in depth image | |
| CN110796050A (en) | Target object identification method and related device in unmanned aerial vehicle inspection process | |
| CN111476821B (en) | Target tracking method based on online learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |