Disclosure of Invention
The invention mainly aims to solve the technical problems that the image edge extracted by the existing edge extraction method is not accurate enough and is difficult to be used for edge enhancement of the image.
The invention provides an edge enhancement method based on a Canny algorithm, which comprises the following steps:
acquiring an original image;
performing edge detection on the original image by using a Canny algorithm;
before determining the edge pixel points, acquiring a threshold empirical value for determining the edge pixel points and inputting the original image into a pre-trained Enet network to acquire a threshold reference value for determining the edge pixel points;
distributing weight coefficients to the threshold empirical value and the threshold reference value and calculating to obtain a threshold correction value;
taking the threshold correction value as a high threshold, and calculating according to a preset proportion based on the high threshold to obtain a low threshold;
and detecting and connecting image edges by using a dual-threshold algorithm, and enhancing the edges in the original image.
In an optional embodiment of the present invention, the performing, by using a Canny algorithm, edge detection on the original image includes:
performing Gaussian filtering processing on the original image;
performing derivation and gradient calculation on the original image subjected to Gaussian processing;
and carrying out non-maximum suppression processing on the obtained gradient amplitude.
In an optional implementation manner of the present invention, the inputting the original image into a pre-trained Enet network to obtain a threshold reference value for determining an edge pixel includes:
segmenting the original image by using an Enet network, and extracting edge features in the image;
and acquiring the gradient amplitude of each pixel point in the edge feature, and taking the gradient amplitude as the threshold reference value.
In an optional implementation manner of the present invention, before inputting the original image into a pre-trained Enet network to obtain a threshold reference value for determining an edge pixel point, the method includes:
an Enet network for extracting image edge features is constructed in advance;
training pictures with different target types are input into the Enet network to train the Enet network, so that the Enet network can meet the recognition of different types of target profiles.
In an alternative embodiment of the invention, the threshold correction value R isxy=α*R1+(1-α)*RP xyWherein R is1Is a threshold empirical value, RP xyThe reference value is a threshold value, alpha is larger than zero and less than or equal to 1, and x and y are coordinates of pixel points in the image matrix.
In an alternative embodiment of the invention, the low threshold T isL=K*THWherein, THFor high thresholds, K is greater than zero and less than 1.
In an optional embodiment of the present invention, the detecting and connecting the image edges by using a dual-threshold algorithm, and the enhancing the edges in the original image includes:
if the gradient value of the pixel point is higher than the high threshold value, marking the pixel point as a strong edge pixel point;
if the gradient value of the pixel point is lower than the high threshold value and higher than the low threshold value, marking the pixel point as a weak edge pixel point;
if the gradient value of the pixel point is lower than the low threshold value, the pixel point is discarded;
when the image edges are connected, the strong edge pixel points are connected first, and then the weak edge pixel points are connected.
The present invention in its second aspect provides an image edge enhancement apparatus, comprising:
the acquisition module is used for acquiring an original image;
the edge detection module is used for carrying out edge detection on the original image by using a Canny algorithm;
the threshold acquisition module is used for acquiring a threshold empirical value for determining the edge pixel points and inputting the original image into a pre-trained Enet network to acquire a threshold reference value for determining the edge pixel points before determining the edge pixel points;
the threshold correction module is used for distributing weight coefficients to the threshold empirical value and the threshold reference value and calculating to obtain a threshold correction value;
the double-threshold determining module is used for taking the threshold correction value as a high threshold and calculating according to a preset proportion to obtain a low threshold based on the high threshold;
and the edge enhancement module is used for detecting and connecting image edges by using a dual-threshold algorithm and enhancing the edges in the original image.
The present invention in its third aspect provides an image edge enhancement apparatus comprising: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;
the at least one processor invokes the instructions in the memory to cause the image edge enhancement apparatus to perform the Canny algorithm based edge enhancement method as in any one of the above.
A fourth aspect of the invention provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, implements the Canny algorithm based edge enhancement method according to any one of the preceding claims.
Has the advantages that: the invention provides an edge enhancement method, a device, equipment and a storage medium based on a Canny algorithm, wherein the method comprises the following steps: acquiring an original image; performing edge detection on the original image by using a Canny algorithm; before determining the edge pixel points, acquiring a threshold empirical value for determining the edge pixel points and inputting an original image into a pre-trained Enet network to acquire a threshold reference value for determining the edge pixel points; distributing weight coefficients for the threshold empirical value and the threshold reference value and calculating to obtain a threshold correction value; taking the threshold correction value as a high threshold, and calculating according to a preset proportion based on the high threshold to obtain a low threshold; and detecting and connecting image edges by using a dual-threshold algorithm, and enhancing the edges in the original image. The method combines the neural network to provide a reference value for the threshold value of each pixel point in the image and correct the threshold value, simultaneously retains the advantage of fixed threshold value, can adapt to different types of images by adjusting the weight coefficient, and has high detection precision and better image enhancement effect.
Detailed Description
The embodiment of the invention provides an edge enhancement method, an edge enhancement device, edge enhancement equipment and a storage medium based on a Canny algorithm.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For understanding, a specific flow of an embodiment of the present invention is described below, and with reference to fig. 1, a first aspect of the present invention provides an edge enhancement method based on a Canny algorithm, where the edge enhancement method includes:
s100, acquiring an original image;
s200, performing edge detection on the original image by using a Canny algorithm;
in this embodiment, the method of the present invention mainly implements edge detection on an original image based on a Canny algorithm, but of course, the method of the present invention may also be used for other edge detection algorithms;
s300, before determining the edge pixel points, obtaining a threshold empirical value for determining the edge pixel points and inputting the original image into a pre-trained Enet network to obtain a threshold reference value for determining the edge pixel points;
in this embodiment, the invention point of the method of the present invention is that a mode of selecting a threshold in a Canny algorithm is improved, on one hand, when an edge of a certain type of target is identified, an empirical value of the threshold is adopted, and a reference value given by a neural network is combined to determine whether a pixel point is an edge pixel point;
s400, distributing weight coefficients for the threshold empirical value and the threshold reference value and calculating to obtain a threshold correction value;
in this embodiment, in the process of specifically using the threshold empirical value and the threshold reference value, different weight coefficients may be assigned to the threshold empirical value and the threshold reference value, that is, the ratio of the two values is adjusted to adapt to different application scenarios, for example, sometimes for a certain type of image, a better edge recognition effect may already be obtained by using the threshold empirical value, and at this time, a greater weight coefficient may be assigned to the threshold empirical value to obtain a better edge recognition effect;
s500, taking the threshold correction value as a high threshold, and calculating according to a preset proportion to obtain a low threshold based on the high threshold;
in the embodiment, the technical scheme of the invention also adopts a double-threshold algorithm to generate the edge in the image, the high threshold and the low threshold in the invention are in a proportional relation, and the high threshold and the low threshold are mainly used for providing a reasonable edge interval, so that the phenomenon that the generated edge is too dense in some small target scenes due to the adoption of a single threshold, and the image after the later edge enhancement is difficult to be used for the identification of a target identification network is avoided;
s600, detecting and connecting image edges by using a dual-threshold algorithm, and enhancing the edges in the original image.
Specifically, the main purpose of extracting and enhancing the edges in the image is to enhance the target in the image, and then use the enhanced image for target recognition (such as pedestrian detection), so that the target contained in the image can be more prominent, thereby improving the efficiency and accuracy of pedestrian detection.
In an optional embodiment of the present invention, the performing, by using a Canny algorithm, edge detection on the original image includes:
performing Gaussian filtering processing on the original image;
in this embodiment, the method of the present invention firstly needs to perform gaussian filtering on the original image, where gaussian filtering is a convolution algorithm, but needs to select a convolution kernel that conforms to gaussian distribution, and since the filtering result is a blurring effect intuitively, the filtering result is also called gaussian blurring.
Performing derivation and gradient calculation on the original image subjected to Gaussian processing;
in this embodiment, the edge in the image can point to each direction, so the Canny algorithm uses four operators to detect the horizontal, vertical and diagonal edges in the image, and the operator for edge detection (such as Roberts, Prewitt, Sobel, etc.) returns the first derivative values in the horizontal Gx and vertical Gy directions, so that the gradient G and the direction θ of the pixel point can be determined.
θ=arctan(Gy/Gx)
Wherein G is gradient strength, theta represents gradient direction, arctan is an arctan function, and a Sobel operator is taken as an example to explain how to calculate gradient strength and direction.
The Sobel operators in the x and y directions are respectively:
wherein SxThe Sobel operator represents the x direction and is used for detecting the edge of the y direction; syThe Sobel operator, which represents the y direction, is used to detect the edges in the x direction (the edge direction is perpendicular to the gradient direction).
If a window of 3x3 in the image is a, and a pixel point to be calculated with a gradient is e, after convolution with a Sobel operator, gradient values of the pixel point e in x and y directions are respectively:
where is the sign of the convolution and sum indicates the sum of all elements in the matrix. Calculating the gradient and the direction of the pixel point e according to the G and theta formulas;
and carrying out non-maximum suppression processing on the obtained gradient amplitude.
In this embodiment, the non-maximum suppression is an edge sparse technique, the effect of the non-maximum suppression is a "thin" edge, after the gradient calculation is performed on the image, the edge extracted based on the gradient values is still very fuzzy, for the standard, there is and should be only one accurate response to the edge, but the non-maximum suppression can help suppress all gradient values except the local maximum value to 0, and the algorithm for performing the non-maximum suppression on each pixel in the gradient image is as follows: comparing the gradient strength of the current pixel with two pixels along the positive and negative gradient directions; if the gradient intensity of the current pixel is maximum compared with the other two pixels, the pixel point is reserved as an edge point, otherwise, the pixel point is restrained.
In an optional implementation manner of the present invention, the inputting the original image into a pre-trained Enet network to obtain a threshold reference value for determining an edge pixel includes:
segmenting the original image by using an Enet network, and extracting edge features in the image;
in this embodiment, the Enet network is used for performing convolution operation on a matrix of an original image, extracting features such as a contour and a form, and acquiring edge features according to the features, referring to fig. 2, in a specific manner, the Enet network specifically includes four convolution modules which are cascaded in sequence, a first convolution module is mainly used for convolving color information of the image, a first convolution module is used for outputting a matrix of 3 × 32 × H × W, a second convolution module is used for convolving shape information of the image, a second convolution module is used for outputting a matrix of 128 × 32H/2 × W/2, a third convolution module is mainly used for convolving contour information of the image, a third convolution module is used for outputting a matrix of 128 × 32 × H/2 × W/2, a fourth convolution module is mainly used for feature merging and adjusting an output size of the convolved image, and a fourth convolution module is used for obtaining edge features, and outputting 32 x 1 x H W matrix, wherein after the original image is input into the ENet network, the output threshold reference value result is the matrix with the same size as the original image, and the channel is 1, namely the result is the matrix with 1 x H W.
And acquiring the gradient amplitude of each pixel point in the edge feature, and taking the gradient amplitude as the threshold reference value.
In this embodiment, since the threshold reference value result output after the original image is input into the ENet network is a matrix with the same size as the original image, matrix correspondence of each pixel point in the original image can be realized, so that different threshold reference values can be provided for each edge pixel point in the original image, and inaccurate edge identification caused by a single threshold is avoided.
In an optional implementation manner of the present invention, before inputting the original image into a pre-trained Enet network to obtain a threshold reference value for determining an edge pixel point, the method includes:
an Enet network for extracting image edge features is constructed in advance;
training pictures with different target types are input into the Enet network to train the Enet network, so that the Enet network can meet the recognition of different types of target profiles.
In this embodiment, the Enet network may select an existing Enet network structure, and in the use process, only the last convolution module output by the Enet network needs to be replaced with a module of 32 × 1 × H × W, or a convolution module is added at the end to ensure that the size of the output matrix is the same as the size of the original image, and training pictures with different target types are input into the Enet network to train the Enet network, so as to optimize a loss function in the model, so that the Enet network can more accurately segment the edges of the targets during picture segmentation.
In an alternative embodiment of the invention, the threshold correction value R isxy=α*R1+(1-α)*RP xyWherein R is1Is a threshold empirical value, RP xyThe reference value is a threshold value, alpha is larger than zero and less than or equal to 1, and x and y are coordinates of pixel points in the image matrix. In the present embodiment, for example, α may be selected from 0.1, 0.3, 0.4, 0.5, 0.6, 0.8, etc., and in one embodiment, α is selected from 0.5 in order to be suitable for more scenarios.
In an alternative embodiment of the invention, the low threshold T isL=K*THWherein, THFor high thresholds, K is greater than zero and less than 1. In this embodiment, for example, K may be 0.3, 0.5, 0.9, and the like, and the smaller the value of K is, the smaller the low threshold is, the fewer pixel points need to be removed, the more details are retained, and meanwhile, the more false edges are, and in order to consider accuracy and better identify the edge of the target in the following, K takes a value of 0.4.
In an optional embodiment of the present invention, the detecting and connecting the image edges by using a dual-threshold algorithm, and the enhancing the edges in the original image includes:
if the gradient value of the pixel point is higher than the high threshold value, marking the pixel point as a strong edge pixel point;
if the gradient value of the pixel point is lower than the high threshold value and higher than the low threshold value, marking the pixel point as a weak edge pixel point;
if the gradient value of the pixel point is lower than the low threshold value, the pixel point is discarded;
when the image edges are connected, the strong edge pixel points are connected first, and then the weak edge pixel points are connected.
In this embodiment, after non-maxima suppression, the remaining pixels may more accurately represent the actual edges in the image, however, there are still some edge pixels due to noise and color variations, and to account for these spurious responses, the edge pixels must be filtered with weak gradient values and the edge pixels with high gradient values are retained, which can be achieved by selecting high and low thresholds. If the gradient value of the edge pixel is above the high threshold, marking it as a strong edge pixel; if the gradient value of the edge pixel is less than the high threshold and greater than the low threshold, marking it as a weak edge pixel; if the gradient values of the edge pixels are smaller than a low threshold, which is chosen depending on the content of a given input image, this is suppressed.
Referring to fig. 3, a second aspect of the present invention provides an image edge enhancement apparatus, including:
an obtaining module 10, configured to obtain an original image;
an edge detection module 20, configured to perform edge detection on the original image by using a Canny algorithm;
the threshold obtaining module 30 is configured to, before determining edge pixel points, obtain a threshold empirical value for determining the edge pixel points and input the original image into a pre-trained Enet network to obtain a threshold reference value for determining the edge pixel points;
the threshold correction module 40 is configured to assign a weight coefficient to the threshold empirical value and the threshold reference value and calculate a threshold correction value;
the double-threshold determining module 50 is configured to use the threshold correction value as a high threshold, and calculate a low threshold according to a preset ratio based on the high threshold;
and an edge enhancement module 60, configured to detect and connect image edges using a dual-threshold algorithm, and enhance the edges in the original image.
Fig. 4 is a schematic structural diagram of an image edge enhancement apparatus according to an embodiment of the present invention, which may include one or more processors 70 (CPUs) (e.g., one or more processors) and a memory 80, and one or more storage media 90 (e.g., one or more mass storage devices) for storing applications or data, based on a relatively large difference between the image edge enhancement apparatus and the image edge enhancement apparatus due to different configurations or performances. The memory and storage medium may be, among other things, transient or persistent storage. The program stored on the storage medium may include one or more modules (not shown), each of which may include a sequence of instructions operating on the image edge enhancement device. Still further, the processor may be configured to communicate with a storage medium to execute a series of instruction operations in the storage medium on the image edge enhancement device.
The image edge enhancement apparatus may also include one or more power supplies 100, one or more wired or wireless network interfaces 110, one or more input-output interfaces 120, and/or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the image edge enhancement device configuration shown in FIG. 4 does not constitute a limitation of the image edge enhancement device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the Canny algorithm-based edge enhancement method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses, and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.