CN115984105B - Hole convolution optimization method and device, computer equipment and storage medium - Google Patents
Hole convolution optimization method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115984105B CN115984105B CN202211564910.3A CN202211564910A CN115984105B CN 115984105 B CN115984105 B CN 115984105B CN 202211564910 A CN202211564910 A CN 202211564910A CN 115984105 B CN115984105 B CN 115984105B
- Authority
- CN
- China
- Prior art keywords
- interpolation
- determining
- result
- convolution
- pixel points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a cavity convolution optimization method, a cavity convolution optimization device, computer equipment and a storage medium. The method comprises the following steps: acquiring a feature map; determining an interpolation region of the feature map; performing hole convolution interpolation on the interpolation region to obtain an interpolation result; and performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value. By implementing the method provided by the embodiment of the invention, the problem of information loss of the cavity convolution can be solved, and the accuracy of the model is improved.
Description
Technical Field
The present invention relates to a computer, and more particularly, to a method and apparatus for optimizing cavity convolution, a computer device, and a storage medium.
Background
The deep learning image algorithm such as segmentation, target detection and the like is widely applied to scenes such as automatic driving, industrial crack detection and the like in society, brings various convenience to life and promotes development of various industries. People are still continuously exploring algorithms with higher accuracy and expanding the use scenes of the algorithms so as to finish tasks with higher requirements on accuracy, such as computer-aided diagnosis and the like. The receptive field is one of the directions of investigation, which means that the calculation of a certain pixel in the feature map is affected by a fixed area on a certain layer in front, i.e. the receptive field of that pixel. The larger the receptive field, the larger the range on the original image that it can feed back to, meaning that more global features may be included, which is of great importance to the accuracy improvement of the model. The multi-layer convolution and pooling operation is a method of increasing the receptive field, both of which can downsample the image resulting in reduced resolution and information loss problems. Wherein the multi-layer convolution increases the computational effort of the model.
The cavity convolution is a new convolution mode provided for the problems, and the cavity rate is introduced on the basis of the common convolution, so that the cavity rate can be increased on the premise of not reducing the size of a feature map, and meanwhile, the quantity of learnable parameters is smaller than that of the multi-layer convolution of the same cavity rate, however, the introduction of the cavity rate causes that part of information is lost, the discontinuity of image information is caused, and the cavity rate is larger or more obvious on the feature map. The above problems are generally solved by adopting a structure of mixed hole convolution, namely, adopting a plurality of convolution stacks with zigzag hole rate to achieve the purpose of covering all pixel information, but the problem of losing information of the hole convolution is still not solved.
Therefore, a new method is needed to solve the problem of information loss of the hole convolution, and improve the accuracy of the model.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a cavity convolution optimization method, a cavity convolution optimization device, computer equipment and a storage medium.
In order to achieve the above purpose, the present invention adopts the following technical scheme: the cavity convolution optimization method comprises the following steps:
acquiring a feature map;
determining an interpolation region of the feature map;
performing hole convolution interpolation on the interpolation region to obtain an interpolation result;
and performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value.
The further technical scheme is as follows: the step of performing hole convolution interpolation on the interpolation region to obtain an interpolation result comprises the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
The further technical scheme is as follows: determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining corner points of four corners in the interpolation area to obtain interpolation pixel points;
and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
The further technical scheme is as follows: determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining corner points of four corners in the interpolation area as first interpolation pixel points, and determining other pixel points in the interpolation area as second interpolation pixel points;
and carrying out hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result.
The further technical scheme is as follows: the determining the interpolation area of the feature map includes:
and sliding the sliding window on the characteristic diagram, and determining the area corresponding to the position of the sliding window as an interpolation area.
The further technical scheme is as follows: determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining pixel points of four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation area as second interpolation pixel points;
and carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and carrying out average value substitution on interpolation results in a sliding window around the second interpolation pixel point to obtain interpolation results.
The invention also provides a cavity convolution optimization device, which comprises:
the characteristic diagram acquisition unit is used for acquiring a characteristic diagram;
a region determining unit configured to determine an interpolation region of the feature map;
the interpolation unit is used for carrying out hole convolution interpolation on the interpolation area to obtain an interpolation result;
and the dot multiplication unit is used for dot multiplying the interpolation result and the set convolution kernel to obtain a target characteristic value.
The further technical scheme is as follows: the interpolation unit is used for determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
The invention also provides a computer device which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the method when executing the computer program.
The present invention also provides a storage medium storing a computer program which, when executed by a processor, performs the above-described method.
Compared with the prior art, the invention has the beneficial effects that: the interpolation processing method and the device have the advantages that the interpolation area in the feature map is determined, the interpolation processing is carried out on the interpolation area by adopting a bilinear interpolation method, the interpolation processing process can be carried out in three modes, the information and the calculated amount are weighed, other parts of the model are reserved, only the numerical value of part of pixel points is replaced, the problem of information loss of cavity convolution is solved, and the accuracy of the model is improved.
The invention is further described below with reference to the drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an application scenario of a hole convolution optimization method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a hole convolution optimization method according to an embodiment of the present invention;
FIG. 3 is a schematic sub-flowchart of a hole convolution optimization method according to an embodiment of the present invention;
FIG. 4 is a schematic sub-flowchart of a hole convolution optimization method according to an embodiment of the present invention;
FIG. 5 is a schematic sub-flowchart of a hole convolution optimization method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of hole convolution with a hole rate of 2 according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of bilinear interpolation provided by an embodiment of the present invention;
FIG. 8 is a schematic diagram of pixel points on the interpolated 3x3 feature map according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a pixel on a 5×5 feature map according to an embodiment of the present invention;
FIG. 10 is a schematic diagram I of a pixel point on a 7x7 feature map according to an embodiment of the present invention;
FIG. 11 is a second schematic diagram of a pixel on a 7×7 feature map according to an embodiment of the present invention;
FIG. 12 is a schematic diagram of pixel points on an 11x11 feature map according to an embodiment of the present invention;
FIG. 13 is a schematic view illustrating a sliding process of a sliding window in a first row and a first column according to an embodiment of the present invention;
FIG. 14 is a schematic diagram of interpolation points of a sliding window at a first row and a first column according to an embodiment of the present invention;
FIG. 15 is a schematic diagram of all interpolation points obtained by the sliding window according to the embodiment of the present invention;
FIG. 16 is a schematic block diagram of a hole convolution optimization device provided by an embodiment of the present invention;
FIG. 17 is a schematic block diagram I of an interpolation unit of a hole convolution optimization device according to an embodiment of the present invention;
FIG. 18 is a second schematic block diagram of an interpolation unit of the hole convolution optimization device according to the embodiment of the present disclosure;
FIG. 19 is a schematic block diagram III of an interpolation unit of a hole convolution optimization device provided by an embodiment of the present invention;
fig. 20 is a schematic block diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic diagram of an application scenario of a hole convolution optimization method according to an embodiment of the present invention. Fig. 2 is a schematic flow chart of a hole convolution optimization method provided by an embodiment of the present invention. The cavity convolution optimization method is applied to the server. The server and the terminal perform data interaction, so that the problem of information loss of the cavity convolution is fundamentally solved, the model accuracy is further improved, in addition, the parameter quantity is the same as that of the traditional cavity convolution, the use method is consistent, and only the part of the cavity convolution can be replaced, and the other parts of the model are reserved.
Fig. 2 is a flow chart of a hole convolution optimization method according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S140.
S110, acquiring a feature map.
In this embodiment, the feature map refers to an image extracted by a deep learning algorithm.
S120, determining an interpolation area of the feature map.
In the present embodiment, the interpolation region refers to a region where interpolation processing is required.
As shown in fig. 6, the calculation of the hole convolution with the hole ratio of 2 on the feature map with the size of 5x5 can be performed, wherein the dark color represents the pixel information extracted by the hole convolution, and the light color is the lost information. After the multi-layer convolution, the superposition of the lost information has a great influence on the model accuracy. It is therefore necessary to determine interpolation areas, interpolate the interpolation areas, i.e. interpolate the missing information.
S130, carrying out hole convolution interpolation on the interpolation region to obtain an interpolation result.
In the present embodiment, the interpolation result is a result obtained by interpolating a set interpolation area on the feature map.
Specifically, determining interpolation pixel points of the interpolation region, and performing hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
For bilinear interpolation, as shown in fig. 7, in each interpolation region, corner points of four corners serve as pixel points to be interpolated. Taking the 2x2 interpolation area adjacent to the upper left corner of the feature map as an example, a pixel value of a certain position in the image block is calculated by bilinear interpolation method is specifically described. As shown in fig. 7, P 11 ，P 12 ，P 21 ，P 22 ，P 1 ，P 2 And P is a point (x 1 ,y 1 )，(x 1 ,y 2 )，(x 2 ,y 1 )，(x 2 ,y 2 )，(x,y 1 )，(x,y 2 ) And (x, y). First, along the X-axis in reverse direction, P is used 11 And P 21 Calculation of P 1 Then use P 12 And P 22 Calculation of P 2 The calculation formula is as follows:
finally along the Y-axis direction according to the pixel value P 1 ，P 2 The pixel value P is calculated as follows:
wherein (x) 2 -x 1 ) And (y) 2 -y 1 ) The distances of the two pixels in the X-axis and the Y-axis are shown, for example, 2X2, and the values 1X and Y select the golden division point, i.e., 0.618. Based on this, the pixel information in the interpolation region of the upper left corner 2x2 of the feature map is stored by P.
Interpolation is carried out on the interpolation area of the feature map to obtain a 3x3 feature map after interpolation, and then dot multiplication is carried out on the 3x3 feature map and a convolution kernel of the 3x3 feature map to obtain a 1x1 feature value. Taking interpolation hole convolution with the 5x5 feature map and the hole rate of 2 as an example, performing interpolation hole convolution on the 5x5 feature map to form a 3x3 feature map, and performing dot multiplication on the 3x3 feature map and a 3x3 convolution kernel to obtain 1x1 pixels, wherein the pixels have a receptive field of 5x5 on the original feature map. As with the conventional hole convolution, when the hole rate is 1, that is, the feature map size is 3x3, the interpolation hole convolution is the common 3x3 convolution, and no interpolation is performed. The pixel points on the interpolated 3x3 feature map are shown in fig. 8.
In one embodiment, referring to fig. 3, the step S130 may include steps S131 to S132.
S131, determining corner points of four corners in the interpolation area to obtain interpolation pixel points.
In the present embodiment, the interpolation pixel point refers to a pixel point that needs to perform interpolation processing.
And S132, carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
In the present embodiment, P 00 、P 02 、P 20 、P 22 Instead of interpolation points, other pixels remain. The size of the region to be interpolated is d×d, where d represents the void fraction and the positions of the regions are respectively atThe top left, bottom left, top right and bottom right of the feature map, and interpolate with the four corner points of the region. According to the different positions, the relative positions of the interpolation points and the interpolation area are adjusted. As shown in fig. 9 and 10, taking 5x5 (d=2) and 7x7 (d=3) feature maps as examples, the remaining points are represented by dark colors, that is, the interpolated 3x3 feature map, the points used for interpolation are represented by light colors, the light colors represent missing points, the points of the shallow-deep-heart frame mark the interpolation area of the upper left part, and the interpolation areas of the other parts are the same.
Compared with the traditional cavity convolution, the interpolation region selection of the embodiment can retain more pixel point information, and is intuitive in principle and easy to realize.
The interpolation method of the embodiment is only used for corresponding P on the feature map in the conventional hole convolution 00 ，P 02 ，P 20 ，P 22 The four pixel points are replaced by interpolation points, and other points are reserved.
In another embodiment, referring to fig. 4, the step S130 may include steps S130a to S130b.
S130a, determining corner points of four corners in the interpolation area as first interpolation pixel points, and determining other pixel points in the interpolation area as second interpolation pixel points.
In this embodiment, interpolation points are used instead of all the pixels, and each pixel is replaced by interpolation of pixels in its surrounding area. However, since the interpolation modes for the pixels at different positions are different, it is necessary to determine the first interpolation pixel and the second interpolation pixel first. The first interpolation pixel point and the second interpolation pixel point are formed by bilinear interpolation of four pixel points obtained by corner points of four corners of the corresponding interpolation region.
The corner pixels of the four corners of the interpolation region are used for bilinear interpolation to obtain interpolation pixels. The first interpolation pixel point is obtained by bilinear interpolation of four corner points of interpolation areas of left upper part, left lower part, right upper part and right lower part of the original feature map, for example, when the original feature map is 5x5, the interpolation area of the left upper corner is 2x2 in size, and the 2x2 pixels are used for bilinear interpolation to obtain the left upper partThe first interpolation pixel of the corner, i.e. P 00 。
S130b, performing hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result.
Selecting a pixel point area to interpolate the principle: firstly, all pixel points on the feature map are contained as much as possible, and information is not lost as much as possible; and secondly, pixel areas are not intersected, so that the weight of some pixel areas is prevented from being increased due to repeated calculation, and misleading is carried out on the model. For each interpolation point position, the interpolation mode is correspondingly changed. Wherein, the first interpolation pixel point P 00 ，P 02 ，P 20 ，P 22 Following the first interpolation method of steps S131-S132, the second interpolation pixel point P 01 ，P 10 ，P 11 ，P 12 ，P 21 Interpolation mode of (a) and first interpolation pixel point P 00 ，P 02 ，P 20 ，P 22 But the second interpolation pixel point P 01 ，P 10 ，P 11 ，P 12 ，P 21 The selection of interpolation regions of (a) follows: the position of the replaced point is positioned at the center of the interpolation area as much as possible, and the specific interpolation method is related to the first interpolation pixel point, namely the four corner areas, of which the interpolation area is close to. If the interpolation area is 2x2, using the left upper corner, the left lower corner, the right upper corner and the right lower corner if the interpolation area is 3x3 and above; taking the feature maps of 7x7 (d=3) and 11x11 (d=5) as an example, as shown in fig. 11 and 12, four corner points of fig. 11 and 12 are performed in the first interpolation manner, and the other five points are P in fig. 11, for example 01 Because its interpolation area is closer to point P 00 So it follows P 00 Is described. In FIG. 12, P is calculated because the interpolation area can be equally distributed 01 ，P 10 ，P 11 ，P 12 ，P 21 No longer 0.618, instead 0.5. The characterization of the information obtained by the interpolation mode is more accurate.
In another embodiment, referring to fig. 5, the step S130 may include steps S130c to S130d.
S130c, determining pixel points of four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation region as second interpolation pixel points;
and S130d, carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and adopting average value substitution of interpolation results in a surrounding sliding window to replace the second interpolation pixel point so as to obtain an interpolation result.
For this embodiment, the step S120 includes:
and sliding the sliding window on the characteristic diagram, and determining the area corresponding to the position of the sliding window as an interpolation area.
In the previous two embodiments, some pixels are not used for calculation, but this embodiment proposes a more dense interpolation method, and all pixels in the feature map will be used for interpolation. Specifically, a sliding window with the size of d×d slides on the feature map, the step length is 1, and an interpolation method combining the first mode and the second mode is performed in the sliding window, namely, interpolation calculation is performed by using pixel points of four corner points of the sliding window, the sliding interpolation specific implementation mode is related to the position of the sliding interpolation, and the sliding window does not pass through the center point of the feature map, namely, the center point of the original feature map is reserved by the finally obtained 3×3 feature map. P (P) 00 ，P 02 ，P 20 ，P 22 The sliding window is positioned at the left upper part, the left lower part, the right upper part and the right lower part of the characteristic diagram, and the sliding window is obtained by interpolation when the sliding window is positioned at the left upper part, the left lower part and the right lower part of the characteristic diagram, namely the sliding window is obtained by interpolation when the sliding window is. P (P) 01 ，P 10 ，P 12 ，P 21 Several interpolations from its nearest sliding window are averaged. Taking a feature map of 5x5 (d=2) as an example, fig. 13 and 14 show the process of sliding the sliding window in the first row and the first column and the interpolation points thereof, respectively, fig. 15 shows all the interpolation points obtained by the sliding window, the interpolation points marked by square boxes for calculating the average value, and the resulting 3x3 feature map.
In this embodiment, the sliding windows are interpolation areas, all sliding windows perform bilinear interpolation by using pixel points of four corner points to obtain interpolation results, and then determine final interpolation pixels according to the positions of the first and second interpolation pixel pointsThe value of the dot. The feature map shown in fig. 15,5x5 is interpolated by sliding window to obtain deep color points (i.e. interpolation result), and then the upper left, lower left, upper right, lower right of these deep color points is directly used as the first interpolation pixel point, and the central deep color point is used as P 11 And other deep color points replace corresponding second interpolation pixels with average values according to the positions of the deep color points. For example, the first one of the four dark pixels in the first row is P 00 The average of the second and third (framed) is taken as P 01 Third as P 02 。
All pixel points on the feature map of the embodiment are calculated, all information is contained, and the implementation process is simpler than that of the second interpolation mode.
And S140, performing dot multiplication on the interpolation result and the set convolution kernel to obtain a target characteristic value.
In this embodiment, the target feature value refers to a result obtained by performing dot multiplication on a feature map formed by interpolation and a set convolution kernel.
According to the cavity convolution optimization method, the interpolation area in the feature map is determined, the interpolation processing is carried out on the interpolation area by adopting the bilinear interpolation method, the interpolation processing process can be carried out in three modes, the information and the calculated amount are weighed, other parts of the model are reserved, only the numerical values of part of pixel points are replaced, the problem that the cavity convolution loses information is solved, and the accuracy of the model is improved.
Fig. 16 is a schematic block diagram of a hole convolution optimization device 300 according to an embodiment of the present disclosure. As shown in fig. 16, the present invention further provides a hole convolution optimization apparatus 300 corresponding to the above hole convolution optimization method. The hole convolution optimization apparatus 300 includes a unit for performing the hole convolution optimization method described above, and may be configured in a server. Specifically, referring to fig. 16, the hole convolution optimization apparatus 300 includes a feature map acquisition unit 301, a region determination unit 302, an interpolation unit 303, and a dot multiplication unit 304.
A feature map acquisition unit 301 configured to acquire a feature map; a region determining unit 302 configured to determine an interpolation region of the feature map; an interpolation unit 303, configured to perform hole convolution interpolation on the interpolation region to obtain an interpolation result; and a dot multiplication unit 304, configured to dot multiply the interpolation result with a set convolution kernel to obtain a target feature value.
In an embodiment, the interpolation unit 303 is configured to determine an interpolation pixel of the interpolation area, and perform hole convolution interpolation on the interpolation pixel by using a bilinear interpolation method to obtain an interpolation result.
In an embodiment, the area determining unit 302 is configured to determine, by using the sliding window to slide on the feature map, an area corresponding to a position where the sliding window is located as the interpolation area.
In one embodiment, as shown in fig. 17, the interpolation unit 303 includes a first determination subunit 3031 and a first interpolation processing subunit 3032.
A first determining subunit 3031, configured to determine corner points of four corners in the interpolation area, so as to obtain an interpolation pixel point; and the first interpolation processing subunit 3032 is configured to perform hole convolution interpolation on the interpolation pixel point by using a bilinear interpolation method to obtain an interpolation result.
In an embodiment, as shown in fig. 18, the interpolation unit 303 includes a second determination subunit 3033 and a second interpolation processing subunit 3034.
A second determining subunit 3033, configured to determine corner points of four corners in the interpolation area as a first interpolation pixel point, and determine other pixel points in the interpolation area as a second interpolation pixel point; and the second interpolation processing subunit 3034 is configured to perform hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by using a bilinear interpolation method to obtain an interpolation result.
In one embodiment, as shown in fig. 19, the interpolation unit 303 includes a third determination subunit 3035 and a third interpolation processing subunit 3036.
A third determining subunit 3035, configured to determine pixel points of the four corner points of the sliding window to obtain a first interpolation pixel point, and determine other pixel points in the interpolation area as second interpolation pixel points; and a third interpolation processing subunit 3036, configured to perform hole convolution interpolation on the first interpolation pixel point by using a bilinear interpolation method, and replace the second interpolation pixel point by using an average value of interpolation results in a sliding window around to obtain an interpolation result.
It should be noted that, as will be clearly understood by those skilled in the art, the specific implementation process of the hole convolution optimization device 300 and each unit may refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, the description is omitted here.
The above-described hole convolution optimization apparatus 300 may be implemented in the form of a computer program that can run on a computer device as shown in fig. 20.
Referring to fig. 20, fig. 20 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, where the server may be a stand-alone server or may be a server cluster formed by a plurality of servers.
With reference to FIG. 20, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a hole convolution optimization method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform a hole convolution optimization method.
The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the structures shown in FIG. 20 are only block diagrams of portions of structures related to the present application and do not constitute a limitation of the computer device 500 to which the present application is applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to execute a computer program 5032 stored in a memory to implement the steps of:
acquiring a feature map; determining an interpolation region of the feature map; performing hole convolution interpolation on the interpolation region to obtain an interpolation result; and performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value.
In an embodiment, when the step of performing the hole convolution interpolation on the interpolation region to obtain the interpolation result, the processor 502 specifically performs the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
In an embodiment, when the processor 502 performs the step of determining the interpolation pixel point of the interpolation area and performing hole convolution interpolation on the interpolation pixel point by using a bilinear interpolation method to obtain an interpolation result, the following steps are specifically implemented:
determining corner points of four corners in the interpolation area to obtain interpolation pixel points; and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
In an embodiment, when the processor 502 performs the step of determining the interpolation pixel point of the interpolation area and performing hole convolution interpolation on the interpolation pixel point by using a bilinear interpolation method to obtain an interpolation result, the following steps are specifically implemented:
determining corner points of four corners in the interpolation area as first interpolation pixel points, and determining other pixel points in the interpolation area as second interpolation pixel points; and carrying out hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method so as to obtain an interpolation result.
In one embodiment, when the step of determining the interpolation area of the feature map is implemented by the processor 502, the following steps are specifically implemented:
and sliding the sliding window on the characteristic diagram, and determining the area corresponding to the position of the sliding window as an interpolation area.
In an embodiment, when the processor 502 performs the step of determining the interpolation pixel point of the interpolation area and performing hole convolution interpolation on the interpolation pixel point by using a bilinear interpolation method to obtain an interpolation result, the following steps are specifically implemented:
determining pixel points of four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation area as second interpolation pixel points; and carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and carrying out average value substitution on interpolation results in a sliding window around the second interpolation pixel point to obtain interpolation results.
It should be appreciated that in embodiments of the present application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program which, when executed by a processor, causes the processor to perform the steps of:
acquiring a feature map; determining an interpolation region of the feature map; performing hole convolution interpolation on the interpolation region to obtain an interpolation result; and performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value.
In one embodiment, when the processor executes the computer program to implement the step of performing hole convolution interpolation on the interpolation region to obtain an interpolation result, the processor specifically implements the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
In an embodiment, when the processor executes the computer program to implement the step of determining the interpolation pixel point of the interpolation area and performs hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, the specific implementation steps are as follows:
determining corner points of four corners in the interpolation area to obtain interpolation pixel points; and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
In an embodiment, when the processor executes the computer program to implement the step of determining the interpolation pixel point of the interpolation area and performs hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, the specific implementation steps are as follows:
determining corner points of four corners in the interpolation area as first interpolation pixel points, and determining other pixel points in the interpolation area as second interpolation pixel points; and carrying out hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method so as to obtain an interpolation result.
In an embodiment, the processor, when executing the computer program to implement the step of determining the interpolation region of the feature map, specifically implements the steps of:
and sliding the sliding window on the characteristic diagram, and determining the area corresponding to the position of the sliding window as an interpolation area.
In an embodiment, when the processor executes the computer program to implement the step of determining the interpolation pixel point of the interpolation area and performs hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, the specific implementation steps are as follows:
determining pixel points of four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation area as second interpolation pixel points; and carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and carrying out average value substitution on interpolation results in a sliding window around the second interpolation pixel point to obtain interpolation results.
The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (8)
1. The cavity convolution optimization method is characterized by comprising the following steps of:
acquiring a feature map;
determining an interpolation region of the feature map;
performing hole convolution interpolation on the interpolation region to obtain an interpolation result;
performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the step of performing hole convolution interpolation on the interpolation region to obtain an interpolation result comprises the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining corner points of four corners in the interpolation area to obtain interpolation pixel points;
and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result.
2. The cavity convolution optimization method is characterized by comprising the following steps of:
acquiring a feature map;
determining an interpolation region of the feature map;
performing hole convolution interpolation on the interpolation region to obtain an interpolation result;
performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the step of performing hole convolution interpolation on the interpolation region to obtain an interpolation result comprises the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining corner points of four corners in the interpolation area as first interpolation pixel points, and determining other pixel points in the interpolation area as second interpolation pixel points;
and carrying out hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result.
3. The cavity convolution optimization method is characterized by comprising the following steps of:
acquiring a feature map;
determining an interpolation region of the feature map;
performing hole convolution interpolation on the interpolation region to obtain an interpolation result;
performing dot multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the step of performing hole convolution interpolation on the interpolation region to obtain an interpolation result comprises the following steps:
determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
the determining the interpolation area of the feature map includes:
sliding the sliding window on the characteristic map, and determining the area corresponding to the position of the sliding window as an interpolation area;
determining an interpolation pixel point of the interpolation region, and performing hole convolution interpolation on the interpolation pixel point by adopting a bilinear interpolation method to obtain an interpolation result, wherein the method comprises the following steps:
determining pixel points of four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation area as second interpolation pixel points;
and carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and carrying out average value substitution on interpolation results in a sliding window around the second interpolation pixel point to obtain interpolation results.
4. The cavity convolution optimizing device is characterized by comprising:
the characteristic diagram acquisition unit is used for acquiring a characteristic diagram;
a region determining unit configured to determine an interpolation region of the feature map;
the interpolation unit is used for carrying out hole convolution interpolation on the interpolation area to obtain an interpolation result;
the point multiplication unit is used for carrying out point multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the interpolation unit is used for determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
the interpolation unit comprises a first determination subunit and a first interpolation processing subunit;
the first determining subunit is used for determining angular points of four angles in the interpolation area to obtain interpolation pixel points; and the first interpolation processing subunit is used for carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method so as to obtain an interpolation result.
5. The cavity convolution optimizing device is characterized by comprising:
the characteristic diagram acquisition unit is used for acquiring a characteristic diagram;
a region determining unit configured to determine an interpolation region of the feature map;
the interpolation unit is used for carrying out hole convolution interpolation on the interpolation area to obtain an interpolation result;
the point multiplication unit is used for carrying out point multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the interpolation unit is used for determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
the interpolation unit comprises a second determination subunit and a second interpolation processing subunit;
the second determining subunit is used for determining the corner points of the four corners in the interpolation area as first interpolation pixel points and determining other pixel points in the interpolation area as second interpolation pixel points; and the second interpolation processing subunit is used for carrying out hole convolution interpolation on the first interpolation pixel point and the second interpolation pixel point by adopting a bilinear interpolation method so as to obtain an interpolation result.
6. The cavity convolution optimizing device is characterized by comprising:
the characteristic diagram acquisition unit is used for acquiring a characteristic diagram;
a region determining unit configured to determine an interpolation region of the feature map;
the interpolation unit is used for carrying out hole convolution interpolation on the interpolation area to obtain an interpolation result;
the point multiplication unit is used for carrying out point multiplication on the interpolation result and a set convolution kernel to obtain a target characteristic value;
the region determining unit is used for sliding on the feature map by adopting a sliding window, and determining a region corresponding to the position of the sliding window as an interpolation region;
the interpolation unit is used for determining interpolation pixel points of the interpolation area, and carrying out hole convolution interpolation on the interpolation pixel points by adopting a bilinear interpolation method to obtain an interpolation result;
the interpolation unit comprises a third determination subunit and a third interpolation processing subunit;
the third determining subunit is used for determining the pixel points of the four corner points of the sliding window to obtain a first interpolation pixel point, and determining other pixel points in the interpolation area as a second interpolation pixel point; and the third interpolation processing subunit is used for carrying out hole convolution interpolation on the first interpolation pixel point by adopting a bilinear interpolation method, and carrying out average value substitution on the interpolation result in the sliding window around the second interpolation pixel point to obtain an interpolation result.
7. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-3.
8. A computer storage medium storing a computer program which, when executed by a processor, performs the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211564910.3A CN115984105B (en) | 2022-12-07 | 2022-12-07 | Hole convolution optimization method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211564910.3A CN115984105B (en) | 2022-12-07 | 2022-12-07 | Hole convolution optimization method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115984105A CN115984105A (en) | 2023-04-18 |
| CN115984105B true CN115984105B (en) | 2023-08-01 |
Family
ID=85967219
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211564910.3A Active CN115984105B (en) | 2022-12-07 | 2022-12-07 | Hole convolution optimization method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115984105B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111915615A (en) * | 2020-09-10 | 2020-11-10 | 中移（杭州）信息技术有限公司 | Image segmentation method and device, electronic equipment and computer readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9208537B1 (en) * | 2014-07-10 | 2015-12-08 | Shenzhen China Star Optoelectronics Technology Co., Ltd | Super-resolution reconstructing method for enhancing smoothness and sharpness of video image |
| CN110298346A (en) * | 2019-05-23 | 2019-10-01 | 平安科技（深圳）有限公司 | Image-recognizing method, device and computer equipment based on divisible convolutional network |
| CN113177938B (en) * | 2021-05-25 | 2023-04-07 | 深圳大学 | Method and device for segmenting brain glioma based on circular convolution kernel and related components |
| CN113392960B (en) * | 2021-06-10 | 2022-08-30 | 电子科技大学 | Target detection network and method based on mixed hole convolution pyramid |
| CN113837943A (en) * | 2021-09-28 | 2021-12-24 | 广州极飞科技股份有限公司 | Image processing method and device, electronic equipment and readable storage medium |
-
2022
- 2022-12-07 CN CN202211564910.3A patent/CN115984105B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111915615A (en) * | 2020-09-10 | 2020-11-10 | 中移（杭州）信息技术有限公司 | Image segmentation method and device, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115984105A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11244432B2 (en) | Image filtering based on image gradients | |
| EP2130175B1 (en) | Edge mapping incorporating panchromatic pixels | |
| JP3885999B2 (en) | Object detection device | |
| EP2130176B1 (en) | Edge mapping using panchromatic pixels | |
| KR20230074579A (en) | Method and system for parking space management | |
| JP2005122361A (en) | Image processor, its processing method, computer program, and recording medium | |
| CN107451976A (en) | A kind of image processing method and device | |
| CN111127543B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN103208094B (en) | For the method and system to image applications wave filter | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| US20060250407A1 (en) | Texture filtering using a programmable table filter to improve computer graphics performmance | |
| KR101937163B1 (en) | Method for directional adaptive image interpolation and an electronic device thereof | |
| CN112419372A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN115984105B (en) | Hole convolution optimization method and device, computer equipment and storage medium | |
| US20030030646A1 (en) | Trilinear texture filtering method with proper texel selection | |
| US6766281B1 (en) | Matched texture filter design for rendering multi-rate data samples | |
| US10482611B2 (en) | Determining optical flow | |
| CN115546027B (en) | Image suture line determination method, device and storage medium | |
| JP4500707B2 (en) | Image data processing device | |
| US11645793B2 (en) | Curve antialiasing based on curve-pixel intersection | |
| US7920148B2 (en) | Post-rendering anti-aliasing with a smoothing filter | |
| US20070003167A1 (en) | Interpolation of images | |
| JP4296801B2 (en) | Spatial filter, spatial filter creation method, spatial filter creation method program, and image processing apparatus and inspection apparatus using the spatial filter | |
| CN118229868A (en) | Image alignment method, image alignment device, electronic apparatus, and storage medium | |
| CN115423886A (en) | Target detection method for image scaling acceleration by convolution operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |