CN116468589A - Image processing method, apparatus, electronic device, and computer-readable storage medium - Google Patents
Image processing method, apparatus, electronic device, and computer-readable storage medium Download PDFInfo
- Publication number
- CN116468589A CN116468589A CN202210031289.8A CN202210031289A CN116468589A CN 116468589 A CN116468589 A CN 116468589A CN 202210031289 A CN202210031289 A CN 202210031289A CN 116468589 A CN116468589 A CN 116468589A
- Authority
- CN
- China
- Prior art keywords
- determining
- feature matrix
- image
- candidate
- elements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims abstract description 237
- 238000012512 characterization method Methods 0.000 claims abstract description 169
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000011176 pooling Methods 0.000 claims abstract description 54
- 238000004590 computer program Methods 0.000 claims description 20
- 238000000034 method Methods 0.000 claims description 15
- 230000009467 reduction Effects 0.000 claims description 8
- 238000005192 partition Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 6
- 238000003709 image segmentation Methods 0.000 claims description 4
- 238000003780 insertion Methods 0.000 claims description 4
- 230000037431 insertion Effects 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241000282472 Canis lupus familiaris Species 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
Abstract
The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium; in the embodiment of the application, an image to be processed is acquired; performing feature coding on the image to be processed to obtain a feature matrix; determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix; determining a target characterization value according to the candidate characterization values; and determining a pooling processing result of the image to be processed according to the target characterization value. The embodiment of the application can improve the accuracy of the pooling processing result.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a computer readable storage medium.
Background
With the development of science and technology, more and more neural network models are presented. Among the many neural network models, convolutional neural network models are commonly used.
In the convolutional neural network model, the image feature matrix is pooled. At present, the pooling processing comprises average pooling and maximum pooling, wherein the average pooling directly takes the average value of the elements in the image feature matrix as a pooling processing result, and the maximum pooling directly takes the maximum value of the elements in the image feature matrix as a pooling processing result. However, maximum pooling may miss portions of the image features and average pooling may weaken some of the more pronounced features in the image, thereby reducing the accuracy of the pooling process results.
Disclosure of Invention
The embodiment of the invention provides an image processing method, an image processing device, electronic equipment and a computer readable storage medium, which can improve the accuracy of a pooling processing result.
An image processing method, comprising:
acquiring an image to be processed;
performing feature coding on the image to be processed to obtain a feature matrix;
determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix;
determining a target characterization value according to the candidate characterization values;
and determining a pooling processing result of the image to be processed according to the target characterization value.
Optionally, the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
determining the mathematical distribution type of the elements in the feature matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix and the mathematical distribution type.
Optionally, the mathematical distribution type of the elements in the feature matrix is gaussian distribution, and the candidate feature values include a first moment and a second moment;
correspondingly, the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
determining a first moment and a second moment of the feature matrix according to the elements in the feature matrix and the Gaussian distribution;
accordingly, the determining the target characterization value according to the candidate characterization values includes:
and determining a target characterization value according to the first moment and the second moment.
Optionally, the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
stretching and transforming the feature matrix to obtain a row matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the row matrix.
Optionally, the determining the target characterization value according to the candidate characterization values includes:
taking the maximum value of the candidate characterization values as a target characterization value, or,
and taking the average value of the candidate characterization values as a target characterization value.
Optionally, the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
acquiring a preset pooling core;
dividing the feature matrix according to the preset pooling check to obtain a plurality of division matrixes;
determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix;
accordingly, the determining the target characterization value according to the candidate characterization values includes:
determining a sub-target characterization value according to the plurality of sub-candidate characterization values;
and determining the target characterization value according to the sub-target characterization value.
Optionally, the determining a plurality of sub-candidate characterization values of the partition matrix according to the elements in the partition matrix includes:
determining the sub-mathematical distribution type of the elements in the partition matrix;
and determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix and the sub-mathematical distribution types.
Optionally, after determining the pooling result of the image to be processed according to the target characterization value, the method further includes:
and carrying out image processing on the image to be processed according to the pooling processing result to obtain an image processing result, wherein the image processing comprises at least one of image classification, image detection, image segmentation, image noise reduction, image enhancement, image super-division, video noise reduction, video frame insertion, video super-division, video content understanding, video enhancement and video target tracking.
Accordingly, an embodiment of the present invention provides an image processing apparatus including:
the acquisition module is used for acquiring the image to be processed;
the coding module is used for carrying out feature coding on the image to be processed to obtain a feature matrix;
the first determining module is used for determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix, and determining a target characterization value according to the plurality of candidate characterization values;
and the second determining module is used for determining the pooling processing result of the image to be processed according to the target characterization value.
In addition, the embodiment of the invention also provides electronic equipment, which comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used for running the computer program in the memory to realize the image processing method provided by the embodiment of the invention.
In addition, the embodiment of the invention further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the steps in any one of the image processing methods provided by the embodiment of the invention.
In the embodiment of the application, an image to be processed is acquired first. And then carrying out feature coding on the image to be processed to obtain a feature matrix. And then, determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix. And determining a target characterization value according to the candidate characterization values. And finally, determining a pooling processing result of the image to be processed according to the target characterization value.
In other words, in the embodiment of the present application, the distribution information of the elements in the feature matrix may be represented by using a plurality of candidate feature values of the feature matrix, and compared with a mode that only the average value or the maximum value of the feature matrix is used to represent the feature matrix, the plurality of candidate feature values may include the distribution information of the elements in the feature matrix, so that the target feature value is determined according to the plurality of candidate feature values, and the pooling processing result of the image to be processed determined according to the target feature value is more accurate.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a feature matrix provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a partition matrix provided by an embodiment of the present invention;
fig. 4 is a schematic structural view of an image processing apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The embodiment of the invention provides an image processing method, an image processing device, electronic equipment and a computer readable storage medium. The image processing apparatus may be integrated in an electronic device, which may be a server or a terminal.
The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform.
The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.
In addition, "plurality" in the embodiments of the present application means two or more. "first" and "second" and the like in the embodiments of the present application are used for distinguishing descriptions and are not to be construed as implying relative importance.
The following will describe in detail. The following description of the embodiments is not intended to limit the preferred embodiments.
In the present embodiment, description will be made in terms of an image processing apparatus which may be integrated in a device such as a server or a terminal, and for convenience in explaining the image processing method of the present application, the image processing apparatus will be integrated in the terminal, that is, the terminal will be used as an execution subject.
Referring to fig. 1, fig. 1 is a flowchart of an image processing method according to an embodiment of the present application. The image processing method may include:
s101, acquiring an image to be processed.
The terminal can start shooting when receiving the processing instruction, so that an image to be processed is obtained. Or when the terminal receives the processing instruction, the terminal sends the acquisition instruction to other terminals, the other terminals shoot to obtain the image to be processed, and then the image to be processed is sent to the terminal, so that the terminal acquires the image to be processed. Or when the terminal receives the processing instruction, searching the image to be processed corresponding to the processing instruction from the local storage.
The method for acquiring the image to be processed may be selected by the user according to the actual situation, which is not limited in this embodiment.
For the type of the image to be processed, the user may select the image to be processed according to the actual situation, for example, the image to be processed may be a human image, an animal image, or a scenic image, which is not limited herein.
S102, performing feature coding on the image to be processed to obtain a feature matrix.
A single image may contain a lot of information, but some information need not be used during image processing, i.e. redundant information is present in the image. For example, the image to be processed contains dogs, characters and trees, image detection is required to be performed on the image to be processed, so that the identity of the characters in the image to be processed is determined, and at the moment, only the information of the characters in the image to be processed is required, and then the information of the dogs and the information of the trees in the image to be processed are redundant information.
When redundant information exists in an image, if the image is directly processed, the amount of calculation at the time of processing is large. Therefore, in order to remove redundant information in the image to be processed, the calculated amount is reduced, and after the terminal acquires the image to be processed, the terminal can perform feature encoding on the image to be processed to obtain a feature matrix, so that the feature matrix does not contain the redundant information of the image to be processed.
The mode of performing feature encoding on the image to be processed may be selected by the user according to the actual situation, for example, the terminal performs feature encoding on the image to be processed through a convolution layer in the neural network model, or the terminal may perform feature encoding on the image to be processed through a direction gradient histogram algorithm (Histogram of Oriented Gradient, HOG), which is not limited in this embodiment.
S103, determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix.
Elements in the feature matrix refer to numbers in the feature matrix, e.g., the feature matrix is as shown in fig. 2, then elements in the feature matrix may refer to 1-9 in the feature matrix.
The distribution information of the elements in the feature matrix refers to average distribution information of the elements in the feature matrix, deviation degree information between the elements in the feature matrix and the average distribution information, and the like.
The plurality of candidate characterization values may be first order moments (first order moments are also referred to as average values), second order moments (second order moments are also referred to as variances), third order moments (third order moments are also referred to as skewness), fourth order moments (fourth order moments are also referred to as kurtosis), degrees of freedom, or the like of elements in the feature matrix.
For example, when the feature matrix is shown in fig. 2, and the plurality of candidate feature values include first moments and second moments of elements in the feature matrix, the plurality of candidate feature values may be 5 and 6.67, respectively, and then 5 represents average distribution information of the elements in the feature matrix, and 6.67 represents deviation degree information between the elements in the feature matrix and the average distribution information.
S104, determining a target characterization value according to the candidate characterization values.
After obtaining the plurality of candidate characterization values, the terminal may use the maximum value of the plurality of candidate characterization values as a target characterization value, or use an average value of the plurality of candidate characterization values as the target characterization value.
For example, when the candidate feature values are 5 and 6.67, the target feature value may be 6.67, which is the larger of 5 and 6.67, or may be 5.835, which is the average of 5 and 6.67.
In this embodiment, according to the elements in the feature matrix, a plurality of candidate characterization values of the feature matrix are determined, and according to the plurality of candidate characterization values, a target characterization value is determined, thereby implementing the pooling operation.
It should be noted that if a plurality of candidate characterization values of the feature matrix are determined directly according to all the elements in the feature matrix, the target characterization value is one. If the feature matrix is divided to obtain a plurality of division matrices, and then a plurality of candidate characterization values of the feature matrix are determined according to elements in the division matrices, the target feature value can comprise a plurality of candidate characterization values.
When dividing the feature matrix to obtain a plurality of division matrices, and determining a plurality of candidate characterization values of the feature matrix according to elements in the division matrices, determining a plurality of candidate characterization values of the feature matrix according to elements in the feature matrix, including:
acquiring a preset pooling core;
dividing the feature matrix according to a preset pooling check to obtain a plurality of division matrixes;
determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix;
accordingly, determining the target token value from the plurality of candidate tokens includes:
determining a sub-target characterization value according to the plurality of sub-candidate characterization values;
and determining the target characterization value according to the sub-target characterization value.
At this time, the plurality of sub-candidate characterization values of the division matrix are used to characterize the distribution information of the elements in the division matrix.
The preset pooling core may be selected according to the actual situation, for example, the feature matrix is shown in fig. 2, and when the size of the preset pooling core is 2×2 and the sliding step length is 1, the feature matrix is divided according to the preset pooling core, and each obtained division matrix may be shown in fig. 3.
If the feature matrix is shown in fig. 2, and the first moment and the second moment are adopted as the candidate feature values, when the candidate feature values of the feature matrix are determined directly according to all the elements in the feature matrix, the candidate feature values are respectively the first moment 5 and the second moment 6.67, and the target feature value is the average value 5.835 of the 5 and the 6.67, that is, the target feature value is one.
When the feature matrix is divided to obtain four 2 x 2 division matrices, and then a plurality of candidate characterization values of the feature matrix are determined according to elements in the division matrices, the sub-candidate characterization values of the first division matrix are a first moment 3 and a second moment 2.5, the sub-candidate characterization values of the second division matrix are a first moment 4 and a second moment 2.5, the sub-candidate characterization values of the third division matrix are a first moment 6 and a second moment 2.5, and the sub-candidate characterization values of the fourth division matrix are a first moment 7 and a second moment 2.5.
The sub-target feature value of the first division matrix is 2.75 of the average value of the first moment 3 and the second moment 2.5, the sub-target feature value of the second division matrix is 3.25 of the average value of the first moment 4 and the second moment 2.5, the sub-target feature value of the third division matrix is 4.25 of the average value of the first moment 6 and the second moment 2.5, and the sub-target feature value of the fourth division matrix is 4.75 of the average value of the first moment 7 and the second moment 2.5.
And finally, forming the four sub-target characterization values into a target characterization value, namely, at the moment, the target characterization values comprise 2.75, 3.25, 4.25 and 4.75, and at the moment, the target characterization values can be expressed in a matrix form.
In some embodiments, determining a plurality of candidate token values for the feature matrix from the elements in the feature matrix comprises:
determining mathematical distribution types of elements in the feature matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix and the mathematical distribution type.
The mathematical distribution type may be gaussian distribution, chi-square distribution, t-distribution, or the like. When the mathematical distribution type of the elements in the feature matrix is gaussian distribution, the first moment and the second moment of the elements in the feature matrix may be used to represent the distribution information of the elements in the feature matrix, that is, the plurality of candidate representation values may be the first moment and the second moment in the feature matrix.
When the mathematical distribution type of the elements in the feature matrix is t distribution, the distribution information of the elements in the feature matrix can be represented by adopting the first moment, the second moment and the degree of freedom of the elements in the feature matrix, namely, the plurality of candidate representation values can be the first moment, the second moment, the degree of freedom and the like of the elements in the feature matrix, and finally, the target representation value is determined according to the first moment, the second moment and the degree of freedom.
The candidate representation values corresponding to each mathematical distribution type may include at least two, for example, when the mathematical distribution type is a gaussian distribution, the plurality of candidate representation values may be a first moment and a second moment of an element of the feature matrix, or the plurality of candidate representation values may be a first moment, a second moment, a third moment, and a fourth moment of an element of the feature matrix, respectively.
The calculation formula of the first moment of the element of the feature matrix can be as follows:
the calculation formulas of the second moment, the third moment and the fourth moment of the elements of the feature matrix can be as follows:
M(X)＝E(|X i -E(X)| k )
wherein X is i Representing elements in the feature matrix, n representing the number of elements in the feature matrix, E (X) representing a first moment, M (X) representing a second or third or fourth moment, and k representing a k-th moment.
In this embodiment, the mathematical distribution type of the elements in the feature matrix is determined first, and then, a plurality of candidate characterization values of the feature matrix are determined according to the elements in the feature matrix and the mathematical distribution type, that is, when the mathematical distribution types of the elements in the feature matrix are different, the plurality of candidate characterization values used for characterizing the distribution information of the elements in the feature matrix may also be different, so that the plurality of candidate characterization values may be changed along with the change of the mathematical distribution type of the elements in the feature matrix, thereby improving the accuracy of the plurality of candidate characterization values.
And, at present, the pooling operation includes an average pooling operation and a maximum pooling operation. However, the maximum pooling operation may miss portions of the image features, and the average pooling operation may weaken some of the more pronounced features in the image, thereby reducing the accuracy of the output results of the pooling operation.
In this embodiment, when the candidate representation values are the first moment and the second moment, since the first moment and the second moment can represent the average distribution information of the elements in the feature matrix and the deviation degree information between the element elements and the average distribution information, etc., even if a larger value of the first moment and the second moment or the average value of the first moment and the second moment is used as the target representation value, the target representation value can represent the distribution information of the elements in the feature matrix, that is, the obtained target representation value (at this time, the target representation value is equivalent to the output result of the pooling operation) is more accurate, so that the image processing is performed according to the target feature value, and the obtained processing result is more accurate.
It should be noted that, when the mathematical distribution type of the elements in the feature matrix is gaussian distribution and the plurality of candidate characterization values are first and second moments in the feature matrix, the target characterization value may be determined according to the first and second moments.
Since the feature matrix typically includes a plurality of rows and columns, the mathematical distribution type of the elements of the feature matrix may also be multidimensional. For example, when the feature matrix is a 2×2 matrix, the gaussian distribution may be a two-dimensional gaussian distribution. For convenience of calculation, in other embodiments, the feature matrix may be stretched to obtain a row matrix, and then, multiple candidate characterization values of the feature matrix are determined according to elements in the row matrix. So that a plurality of candidate token values for the feature matrix can be subsequently determined from the one-dimensional mathematical distribution and the elements in the row matrix.
For example, when the feature matrix is an h×w matrix, the stretching transformation is performed on the feature matrix, and the obtained row matrix is 1*V (v=h×w).
If the feature matrix is divided to obtain a division matrix, the division matrix can be stretched to obtain a line division matrix, and then a plurality of sub-candidate characterization values of the division matrix are determined according to elements in the line division matrix.
When determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix and the mathematical distribution type, determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix, including:
determining the sub-mathematical distribution type of the elements in the division matrix;
and determining a plurality of sub-candidate characterization values of the division matrix according to the element in the division matrix and the sub-mathematical distribution type.
In this embodiment, the feature matrix is divided to obtain a division matrix, and then the sub-mathematical distribution type of the elements in the division matrix is determined, that is, the sub-mathematical distribution type of each division matrix may be different at this time, so that the sub-candidate characterization values of each division matrix may be different, thereby further improving the accuracy of the target characterization values.
S105, determining a pooling processing result of the image to be processed according to the target characterization value.
And the terminal takes the target characterization value as a pooling processing result of the image to be processed after obtaining the target characterization value.
In some embodiments, after determining the pooled processing result of the image to be processed according to the target characterization value, further comprising:
and carrying out image processing on the image to be processed according to the pooling processing result to obtain an image processing result, wherein the image processing comprises at least one of image classification, image detection, image segmentation, image noise reduction, image enhancement, image super-division, video noise reduction, video frame insertion, video super-division, video content understanding, video enhancement and video target tracking.
It should be noted that, because the method determines the candidate characterization values of the feature matrix according to the elements in the feature matrix and determines the target characterization value according to the candidate characterization values in the application, which is equivalent to the pooling operation, the terminal may execute the subsequent operation after the pooling operation in the related art after obtaining the target characterization value. For example, in the related art, after the pooling operation, a full connection layer or a convolution layer may be connected, and after the terminal obtains the target characterization value, the terminal may also input the target characterization value into the full connection layer for identification or input the target characterization value into the convolution layer for feature extraction again.
From the above, the image to be processed is acquired first. And then carrying out feature coding on the image to be processed to obtain a feature matrix. And then, determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix. And determining a target characterization value according to the candidate characterization values. And finally, determining a pooling processing result of the image to be processed according to the target characterization value.
In other words, in the embodiment of the present application, the distribution information of the elements in the feature matrix may be represented by using a plurality of candidate feature values of the feature matrix, and compared with a mode that only the average value or the maximum value of the feature matrix is used to represent the feature matrix, the plurality of candidate feature values may include the distribution information of the elements in the feature matrix, so that the target feature value is determined according to the plurality of candidate feature values, and the pooling processing result of the image to be processed determined according to the target feature value is more accurate.
In order to better implement the above method, the embodiment of the present invention further provides an image processing apparatus, where the meaning of the term is the same as that in the above image processing method, and specific implementation details may refer to the description in the embodiment of the method.
For example, as shown in fig. 4, the image processing apparatus may include:
an acquisition module 401 is configured to acquire an image to be processed.
The encoding module 402 is configured to perform feature encoding on the image to be processed to obtain a feature matrix.
The first determining module 403 is configured to determine a plurality of candidate characterizing values of the feature matrix according to the elements in the feature matrix, where the candidate characterizing values are used to characterize distribution information of the elements in the feature matrix, and determine the target characterizing value according to the plurality of candidate characterizing values.
A second determining module 404, configured to determine a pooling result of the image to be processed according to the target characterization value.
Optionally, the first determining module 403 is specifically configured to perform:
determining mathematical distribution types of elements in the feature matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix and the mathematical distribution type.
Optionally, the mathematical distribution of the elements in the feature matrix is gaussian in type, and the plurality of candidate token values includes a first moment and a second moment.
Accordingly, the first determining module 403 is specifically configured to perform:
determining a first moment and a second moment of the feature matrix according to elements in the feature matrix and Gaussian distribution;
and determining a target characterization value according to the first moment and the second moment.
Optionally, the first determining module 403 is specifically configured to perform:
stretching and transforming the feature matrix to obtain a row matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the row matrix.
Optionally, the first determining module 403 is specifically configured to perform:
the maximum value of the plurality of candidate characterization values is taken as the target characterization value, or,
and taking the average value of the candidate characterization values as a target characterization value.
Optionally, the first determining module 403 is specifically configured to perform:
acquiring a preset pooling core;
dividing the feature matrix according to a preset pooling check to obtain a plurality of division matrixes;
determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix;
accordingly, determining the target token value from the plurality of candidate tokens includes:
determining a sub-target characterization value according to the plurality of sub-candidate characterization values;
and determining the target characterization value according to the sub-target characterization value.
Optionally, the first determining module 403 is specifically configured to perform:
determining the sub-mathematical distribution type of the elements in the division matrix;
and determining a plurality of sub-candidate characterization values of the division matrix according to the element in the division matrix and the sub-mathematical distribution type.
Optionally, the image processing apparatus further includes:
the processing module is used for carrying out image processing on the image to be processed according to the pooling processing result to obtain an image processing result, wherein the image processing comprises at least one of image classification, image detection, image segmentation, image noise reduction, image enhancement, image super-division, video noise reduction, video frame insertion, video super-division, video content understanding, video enhancement and video target tracking.
In the implementation, each module may be implemented as an independent entity, or may be combined arbitrarily, and implemented as the same entity or a plurality of entities, and the implementation and corresponding beneficial effects of each module may be referred to the foregoing method embodiments, which are not described herein again.
The embodiment of the invention also provides an electronic device, as shown in fig. 5, which shows a schematic structural diagram of the electronic device according to the embodiment of the invention, specifically:
the electronic device may include one or more processing cores 'processors 501, one or more computer-readable storage media's memory 502, a power supply 503, and an input unit 504, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 5 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:
the processor 501 is a control center of the electronic device, and connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing computer programs and/or modules stored in the memory 502, and calling data stored in the memory 502, thereby performing overall monitoring of the electronic device. Optionally, processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 501.
The memory 502 may be used to store computer programs and modules, and the processor 501 performs various functional applications and data processing by executing the computer programs and modules stored in the memory 502. The memory 502 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, a computer program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide access to the memory 502 by the processor 501.
The electronic device further comprises a power supply 503 for powering the various components, preferably the power supply 503 is logically connected to the processor 501 via a power management system, whereby the functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 503 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The electronic device may further comprise an input unit 504, which input unit 504 may be used for receiving input digital or character information and for generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.
Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 501 in the electronic device loads executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 executes the computer programs stored in the memory 502, so as to implement various functions, for example:
acquiring an image to be processed;
performing feature coding on an image to be processed to obtain a feature matrix;
determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix;
determining a target characterization value according to the plurality of candidate characterization values;
and determining a pooling processing result of the image to be processed according to the target characterization value.
The specific implementation and corresponding beneficial effects of each of the above operations can be found in the foregoing embodiments, and are not described herein.
It will be appreciated by those of ordinary skill in the art that all or part of the steps of the various methods of the above embodiments may be performed by a computer program, or by computer program control related hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present invention provides a computer readable storage medium having stored therein a computer program that can be loaded by a processor to perform the steps of any of the information classification methods provided by the embodiments of the present invention. For example, the computer program may perform the steps of:
acquiring an image to be processed;
performing feature coding on an image to be processed to obtain a feature matrix;
determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix;
determining a target characterization value according to the plurality of candidate characterization values;
and determining a pooling processing result of the image to be processed according to the target characterization value.
The specific implementation and corresponding beneficial effects of each of the above operations can be found in the foregoing embodiments, and are not described herein.
Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Since the computer program stored in the computer readable storage medium may execute the steps in any one of the image processing methods provided in the embodiments of the present invention, the beneficial effects that any one of the image processing methods provided in the embodiments of the present invention can achieve are detailed in the previous embodiments and are not described herein.
Among other things, according to one aspect of the present application, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-described image processing method.
The foregoing has described in detail the methods, apparatuses, electronic devices and computer readable storage medium for image processing according to the embodiments of the present invention, and specific examples have been applied to illustrate the principles and embodiments of the present invention, where the foregoing examples are provided to assist in understanding the methods and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.
Claims (11)
1. An image processing method, comprising:
acquiring an image to be processed;
performing feature coding on the image to be processed to obtain a feature matrix;
determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix;
determining a target characterization value according to the candidate characterization values;
and determining a pooling processing result of the image to be processed according to the target characterization value.
2. The image processing method according to claim 1, wherein the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
determining mathematical distribution types of elements in the feature matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix and the mathematical distribution type.
3. The image processing method according to claim 2, wherein the mathematical distribution type of the elements in the feature matrix is gaussian distribution, and the plurality of candidate characterization values include a first moment and a second moment;
correspondingly, the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
determining a first moment and a second moment of the feature matrix according to elements in the feature matrix and the Gaussian distribution;
accordingly, the determining the target characterization value according to the candidate characterization values includes:
and determining a target characterization value according to the first moment and the second moment.
4. The image processing method according to claim 1, wherein the determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix includes:
stretching and transforming the feature matrix to obtain a row matrix;
and determining a plurality of candidate characterization values of the feature matrix according to the elements in the row matrix.
5. The image processing method according to claim 1, wherein the determining a target characterization value from the plurality of candidate characterization values includes:
taking the maximum value of the candidate characterization values as a target characterization value, or,
and taking the average value of the candidate characterization values as a target characterization value.
6. The method according to any one of claims 1-5, wherein determining a plurality of candidate characterization values for the feature matrix from the elements in the feature matrix comprises:
acquiring a preset pooling core;
dividing the feature matrix according to the preset pooling check to obtain a plurality of division matrixes;
determining a plurality of sub-candidate characterization values of the division matrix according to elements in the division matrix;
accordingly, the determining the target characterization value according to the candidate characterization values includes:
determining a sub-target characterization value according to the plurality of sub-candidate characterization values;
and determining the target characterization value according to the sub-target characterization value.
7. The method according to claim 6, wherein determining a plurality of sub-candidate characterization values of the partition matrix from the elements in the partition matrix comprises:
determining the sub-mathematical distribution type of the elements in the partition matrix;
and determining a plurality of sub-candidate characterization values of the division matrix according to the elements in the division matrix and the sub-mathematical distribution type.
8. The image processing method according to claim 1, further comprising, after the determination of the pooling result of the image to be processed according to the target characterization value:
and carrying out image processing on the image to be processed according to the pooling processing result to obtain an image processing result, wherein the image processing comprises at least one of image classification, image detection, image segmentation, image noise reduction, image enhancement, image super-division, video noise reduction, video frame insertion, video super-division, video content understanding, video enhancement and video target tracking.
9. An image processing apparatus, comprising:
the acquisition module is used for acquiring the image to be processed;
the coding module is used for carrying out feature coding on the image to be processed to obtain a feature matrix;
the first determining module is used for determining a plurality of candidate characterization values of the feature matrix according to the elements in the feature matrix, wherein the candidate characterization values are used for characterizing the distribution information of the elements in the feature matrix, and determining a target characterization value according to the plurality of candidate characterization values;
and the second determining module is used for determining a pooling processing result of the image to be processed according to the target characterization value.
10. An electronic device comprising a processor and a memory, the memory storing a computer program, the processor being configured to execute the computer program in the memory to perform the image processing method of any one of claims 1 to 8.
11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor for performing the image processing method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210031289.8A CN116468589A (en) | 2022-01-12 | 2022-01-12 | Image processing method, apparatus, electronic device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210031289.8A CN116468589A (en) | 2022-01-12 | 2022-01-12 | Image processing method, apparatus, electronic device, and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116468589A true CN116468589A (en) | 2023-07-21 |
Family
ID=87175852
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210031289.8A Pending CN116468589A (en) | 2022-01-12 | 2022-01-12 | Image processing method, apparatus, electronic device, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116468589A (en) |
-
2022
- 2022-01-12 CN CN202210031289.8A patent/CN116468589A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111292262B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN109117940B (en) | Target detection method, device, terminal and storage medium based on convolutional neural network | |
| CN111242019B (en) | Video content detection method and device, electronic equipment and storage medium | |
| CN115344805A (en) | Material auditing method, computing equipment and storage medium | |
| CN111340213A (en) | Neural network training method, electronic device, and storage medium | |
| CN110598732B (en) | Plant health detection method and device based on image recognition | |
| CN116468589A (en) | Image processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN113705686B (en) | Image classification method, device, electronic equipment and readable storage medium | |
| CN109598190A (en) | Method, apparatus, computer equipment and storage medium for action recognition | |
| CN111652073B (en) | Video classification method, device, system, server and storage medium | |
| CN116415624A (en) | Model training method and device, and content recommendation method and device | |
| CN107295074A (en) | It is a kind of to realize the method and apparatus that cloud resource is shown | |
| CN113836005A (en) | Virtual user generation method and device, electronic equipment and storage medium | |
| CN113160942A (en) | Image data quality evaluation method and device, terminal equipment and readable storage medium | |
| CN114449701B (en) | Light type regulation and control method and device based on medical record information and electronic equipment | |
| CN111666878B (en) | Object detection method and device | |
| CN117725959A (en) | Data updating method, device, electronic equipment and computer storage medium | |
| CN113743308B (en) | Face recognition method, device, storage medium and system based on feature quality | |
| CN113139490B (en) | Image feature matching method and device, computer equipment and storage medium | |
| CN117440178A (en) | Video processing method, video processing device, electronic equipment and computer storage medium | |
| CN112086096B (en) | Data processing method, device, system and medium | |
| CN116416311A (en) | Parameter calibration method, device, electronic equipment and computer readable storage medium | |
| CN117726917A (en) | Model training method, device, electronic equipment and computer storage medium | |
| CN117036178A (en) | Image restoration method, device, computer equipment, medium and program | |
| CN117648455A (en) | Image searching method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |