CN111435544B - Picture processing method and device - Google Patents
Picture processing method and device Download PDFInfo
- Publication number
- CN111435544B CN111435544B CN201910033068.2A CN201910033068A CN111435544B CN 111435544 B CN111435544 B CN 111435544B CN 201910033068 A CN201910033068 A CN 201910033068A CN 111435544 B CN111435544 B CN 111435544B
- Authority
- CN
- China
- Prior art keywords
- picture
- sub
- region
- shape
- filling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a picture processing method and device. Wherein, the method comprises the following steps: acquiring a picture with a first shape, and acquiring picture information of the picture, wherein the picture information at least comprises; pixel information of the picture; analyzing the picture information by using a convolutional neural network model to obtain filling parameters; based on the filling parameters, the picture is filled from the first shape to the second shape. The invention solves the technical problem that in the prior art, a large number of pictures are needed for machine learning, and the image learning efficiency is low due to inconsistent sizes of the pictures.
Description
Technical Field
The invention relates to the field of data processing, in particular to a picture processing method and device.
Background
When a picture is shot by using an image acquisition device such as a camera, the picture has two different shapes of a rectangle or a square due to the resolution, and the square picture is complicated to process when algorithm processing is carried out, and needs to be converted into the square picture for processing. At present, pictures can be processed by using a model trained by machine learning, short edges of rectangular pictures are filled, and massive pictures are needed for machine learning in order to obtain the model.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a picture processing method and a picture processing device, which are used for at least solving the technical problem that in the prior art, massive pictures are needed for machine learning, and the image learning efficiency is low due to the fact that the sizes of the pictures are inconsistent.
According to an aspect of the embodiments of the present invention, there is provided an image processing method, including: acquiring a picture with a first shape, and acquiring picture information of the picture, wherein the picture information at least comprises; pixel information of the picture; analyzing the picture information by using a convolutional neural network model to obtain filling parameters; based on the filling parameters, the picture is filled from the first shape to the second shape.
Further, analyzing the picture information by using a convolutional neural network model to obtain filling parameters, including: the convolutional neural network model scans each sub-region of the picture, analyzes the pixel information of each sub-region and obtains filling parameters, wherein the filling parameters are the pixel information used for filling the picture.
Further, the convolutional neural network model scans each sub-region of the picture, analyzes the pixel information of each sub-region, and obtains filling parameters, including: each sub-region of a convolutional layer scanning picture of the convolutional neural network model is extracted, and the characteristic parameters of each sub-region are extracted, wherein the characteristic parameters comprise at least one of the following parameters: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object; determining a first sub-region for extracting the filling parameter from the plurality of sub-regions based on the characteristic parameter; the pixel information of the first sub-area is taken as a filling parameter.
Further, the method further comprises: taking a sub-region located at the edge position of the picture as a first sub-region; or, regarding a sub-region of the edge position in the predetermined direction in the picture as a first sub-region; or, taking the sub-region with the maximum average pixel value in the picture as a first sub-region; or, taking the sub-region with the minimum average pixel value in the picture as a first sub-region; or a sub-area in the picture adjacent to the area to be filled is taken as the first sub-area.
Further, the method further comprises: taking the pixel information of the background of the first sub-area or the pixel information of the object in the first sub-area as a filling parameter; or, taking the average pixel value of the first sub-area as a filling parameter; or, the pixel information of the edge position of the first sub-area is used as the filling parameter.
Further, after extracting the feature parameters of each sub-region in each sub-region of the convolutional layer scan picture of the convolutional neural network model, the method further includes: pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters; and mapping the pooled picture on a full-link layer of the convolutional neural network model, so that the position information of the pixel points of the compressed picture corresponds to the position information of the pixel points of the picture.
Further, after acquiring the picture with the first shape, the method further includes: judging whether the first shape of the picture is a second shape; if yes, directly outputting the picture; if not, performing the step of converting the picture from the first shape to the second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from the first shape to the second shape based on the filling parameters.
Further, determining whether the first shape of the picture is the second shape includes: reading the size of the picture, wherein the size comprises: the aspect ratio of the picture; and judging whether the first shape of the picture is the second shape or not based on the size of the picture.
According to another aspect of the embodiments of the present invention, there is also provided an image processing apparatus, including: the first acquisition module is used for acquiring a picture with a first shape and acquiring picture information of the picture, wherein the picture information at least comprises; pixel information of the picture; the second acquisition module is used for analyzing the picture information by using the convolutional neural network model to acquire filling parameters; and the filling module is used for filling the picture from the first shape to the second shape based on the filling parameters.
Further, the second obtaining module includes: and the analysis submodule is used for scanning each sub-region of the picture by the convolutional neural network model, analyzing the pixel information of each sub-region and acquiring filling parameters, wherein the filling parameters are the pixel information used for filling the picture.
Further, the analysis submodule includes: the extracting unit extracts the characteristic parameters of each sub-region of the convolutional layer scanning picture of the convolutional neural network model, wherein the characteristic parameters comprise at least one of the following parameters: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object; a determining unit, configured to determine a first sub-region for extracting the filling parameter from the plurality of sub-regions based on the feature parameter; and the processing unit is used for taking the pixel information of the first sub-area as the filling parameter.
Further, the above apparatus further comprises: the first processing module is used for taking a sub-region located at the edge position of the picture as a first sub-region; or, regarding a sub-region of the edge position in the predetermined direction in the picture as a first sub-region; or, taking the sub-region with the maximum average pixel value in the picture as a first sub-region; or, taking the sub-region with the minimum average pixel value in the picture as a first sub-region; or a sub-area in the picture adjacent to the area to be filled is taken as the first sub-area.
Further, the above apparatus further comprises: the second processing module is used for taking the pixel information of the background of the first sub-area or the pixel information of the object in the first sub-area as a filling parameter; or, taking the average pixel value of the first sub-area as a filling parameter; or, the pixel information of the edge position of the first sub-area is used as the filling parameter.
Further, the above apparatus further comprises: the pooling processing module is used for pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters; and the mapping module is used for mapping the pooled picture on a full connection layer of the convolutional neural network model so that the position information of the pixel points of the compressed picture corresponds to the position information of the pixel points of the picture.
Further, the above apparatus further comprises: the judging module is used for judging whether the first shape of the picture is the second shape; the third processing module is used for directly outputting the picture if the picture is the image; if not, performing the step of converting the picture from the first shape to the second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from the first shape to the second shape based on the filling parameters.
Further, the judging module comprises: a reading submodule for reading a size of the picture, wherein the size includes: the aspect ratio of the picture; and the judging submodule is used for judging whether the first shape of the picture is the second shape or not based on the size of the picture.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium, where the storage medium includes a stored program, and when the program runs, the apparatus on which the storage medium is located is controlled to execute the above-mentioned picture processing method.
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, where the program executes the above-mentioned picture processing method.
In the embodiment of the present invention, after the picture with the first shape is obtained, the picture information of the picture may be further obtained, and the filling parameter may be obtained by analyzing the picture information using the convolutional neural network model, so that the picture may be filled from the first shape to the second shape based on the filling parameter, thereby facilitating subsequent processing. The rectangular pictures are filled through the convolutional neural network model, the pictures are filled into the square, original information of the pictures is not influenced and destroyed, the unified function of processing the pictures into the square for subsequent analysis is achieved, and the technical problem that in the prior art, massive pictures are needed for machine learning, and the problem that the image learning efficiency is low due to the fact that the sizes of the pictures are inconsistent is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow chart of a method of picture processing according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating results of an alternative convolutional neural network model, according to an embodiment of the present invention;
FIG. 3 is a flow chart of an alternative picture processing method according to an embodiment of the invention; and
fig. 4 is a schematic diagram of a picture processing apparatus according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided an embodiment of a picture processing method, it should be noted that the steps shown in the flowchart of the figure may be executed in a computer system such as a set of computer executable instructions, and that while a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that here.
Fig. 1 is a flowchart of a picture processing method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, acquiring a picture with a first shape, and acquiring picture information of the picture, wherein the picture information at least comprises; pixel information of the picture.
Specifically, the first shape may be a non-square shape, for example, a rectangle, but is not limited thereto. The picture may be composed of a plurality of pixel points, and the pixel information of the picture may be information of each pixel point, for example, the pixel value of the pixel point may be included, but is not limited thereto.
And step S104, analyzing the picture information by using a convolutional neural network model to obtain filling parameters.
Specifically, the convolutional neural network model may be obtained by machine learning training using a plurality of sets of training data in advance, and each set of training data in the plurality of sets of training data may include: a picture of a non-square, and a corresponding picture of a filled square.
A suitable filling scheme can be given by the convolutional neural network model, and the scheme contains corresponding filling parameters.
And step S106, filling the picture from the first shape to the second shape based on the filling parameters.
In an optional scheme, whether an incoming picture is square or not is judged, if the incoming picture is a non-square picture, pixel information of the picture can be read, the pixel information of the picture is analyzed through a Convolutional Neural Network (CNN) (convolutional Neural networks) model, filling parameters are obtained, and a program is controlled to reasonably fill the picture, so that the square picture is obtained.
By the embodiment of the invention, after the picture with the first shape is obtained, the picture information of the picture can be further obtained, and the filling parameter can be obtained by analyzing the picture information by using the convolutional neural network model, so that the picture can be filled from the first shape to the second shape based on the filling parameter, and the subsequent processing is facilitated. The rectangular pictures are filled through the convolutional neural network model, the pictures are filled into the square, original information of the pictures is not influenced and destroyed, the unified function of processing the pictures into the square for subsequent analysis is achieved, and the technical problem that in the prior art, massive pictures are needed for machine learning, and the problem that the image learning efficiency is low due to the fact that the sizes of the pictures are inconsistent is solved.
Optionally, in the above embodiment of the present invention, analyzing the picture information by using a convolutional neural network model, and acquiring the filling parameter includes: the convolutional neural network model scans each sub-region of the picture, analyzes the pixel information of each sub-region and obtains filling parameters, wherein the filling parameters are the pixel information used for filling the picture.
In particular, each sub-region described above may be each small image in a picture.
In an optional scheme, the picture can be divided into a plurality of small images through a convolutional neural network model, characteristic parameters are extracted, the picture is compressed and parameters are reduced under the condition that the picture quality is not affected, and finally position information of pixel points in each small image corresponds to the original image to obtain final filling parameters.
Optionally, in the foregoing embodiment of the present invention, the scanning, by the convolutional neural network model, each sub-region of the picture, analyzing pixel information of each sub-region, and acquiring the filling parameter includes: each sub-region of a convolutional layer scanning picture of the convolutional neural network model is extracted, and the characteristic parameters of each sub-region are extracted, wherein the characteristic parameters comprise at least one of the following parameters: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object; determining a first sub-region for extracting the filling parameter from the plurality of sub-regions based on the characteristic parameter; the pixel information of the first sub-area is taken as a filling parameter.
Specifically, in an application scenario of electric rice cooker, the obtained picture may be a picture including rice, that is, the object in the sub-region may be rice, but is not limited thereto. The rectangular picture can be filled in a mode of edge pixels or a mode of reading edge pixel points to perform mean value calculation through a convolutional neural network model. The first sub-region may be a sub-region that can not affect and destroy original information of the picture, and can simply fill the picture, and the pixel information of the sub-region may be used as a filling parameter to fill the rectangular picture.
In an alternative, the CNN model may be composed of an input image side, a convolutional layer, and an output side. After the detailed information of the rectangular picture is obtained and completely processed, the picture is sent to a convolutional layer through an input image end to be subjected to convolution calculation, each picture is divided into a plurality of small images in the convolutional layer to be subjected to characteristic parameter extraction, the position information of the small images, the pixel value in the background, the outline of an object, the pixel value of the object and the like are extracted, a first sub-region is further selected according to needs and output to an output end, and therefore the final filling parameters can be obtained according to the pixel information of the first sub-region.
Optionally, in the above embodiment of the present invention, the method further includes: taking a sub-region located at the edge position of the picture as a first sub-region; or, regarding a sub-region of the edge position in the predetermined direction in the picture as a first sub-region; or, taking the sub-region with the maximum average pixel value in the picture as a first sub-region; or, taking the sub-region with the minimum average pixel value in the picture as a first sub-region; or a sub-area in the picture adjacent to the area to be filled is taken as the first sub-area.
In an optional scheme, the first sub-region may be selected according to pixel information of an actual picture and a filling manner, and the pixel information of the first sub-region is used as a filling parameter. The convolutional neural network may be filled in the manner of edge pixels, and thus, a sub-region at an edge position of the picture may be taken as the first sub-region. In order to convert a rectangular picture into a square picture, the short side of the picture may be filled in the manner of edge pixels, and thus, the sub-region at the edge position of the short side in the picture may be taken as the first sub-region.
Similarly, since the picture includes an object, for example, the rice picture includes rice, in order not to affect or destroy original information of the picture, a sub-region with a largest average pixel value may be used as the first sub-region, where the sub-region may be a sub-region including the object, or a sub-region with a smallest average pixel value may be used as the first region, where the sub-region may be a sub-region not including the object.
In addition, the pixel value of each pixel point in two adjacent sub-regions in the picture is not changed greatly, so that the sub-region adjacent to the region to be filled in the picture can be used as the first sub-region.
Optionally, in the above embodiment of the present invention, the method further includes: taking the pixel information of the background of the first sub-area or the pixel information of the object in the first sub-area as a filling parameter; or, taking the average pixel value of the first sub-area as a filling parameter; or, the pixel information of the edge position of the first sub-area is used as the filling parameter.
In an alternative, after the first sub-region is determined, the filling parameters may be selected according to the pixel information of the actual picture and the filling manner. The characteristic parameters of the first sub-region may include: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the contour of the object in the sub-region, the pixel information of the object, and the like, therefore, the filling parameter may be the background pixel value of the first sub-region, the pixel value of the object, the average pixel value of all the pixel points, or the pixel value of the edge position.
Optionally, in the above embodiment of the present invention, after extracting the feature parameter of each sub-region in each sub-region of the convolutional layer scan picture of the convolutional neural network model, the method further includes: pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters; and mapping the pooled picture on a full-link layer of the convolutional neural network model, so that the position information of the pixel points of the compressed picture corresponds to the position information of the pixel points of the picture.
In an alternative, as shown in fig. 2, the CNN model may be composed of an input image end, a convolutional layer, a max-pooling layer, a full link layer, and an output end, where the main purpose of the max-pooling layer is to compress a picture and reduce parameters without affecting the picture quality by means of downsampling. After the detailed information of the rectangular picture is obtained and completely processed, the picture is sent to the convolutional layer through the image input end to be subjected to convolution calculation, and each picture is divided into a plurality of small images in the convolutional layer to be subjected to characteristic parameter extraction. After the feature parameters are extracted from the convolutional layer, the convolutional layer is processed through the largest pooling layer, and finally the position information of the pixel points in the picture is corresponding to the original image through a full link layer and is output to an output end, so that the final filling parameters can be obtained.
Optionally, in the above embodiment of the present invention, after acquiring the picture with the first shape, the method further includes: judging whether the first shape of the picture is a second shape; if yes, directly outputting the picture; if not, performing the step of converting the picture from the first shape to the second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from the first shape to the second shape based on the filling parameters.
Specifically, the second shape described above may be a square shape, but is not limited thereto.
In an optional scheme, after the picture to be processed is obtained, it may be first determined whether the picture is square, and if so, the picture is directly output for a subsequent procedure; if not, reading the pixel information of the picture and analyzing the information through a convolutional neural network model, so that the non-square picture is filled into the square picture according to the obtained filling parameters.
Optionally, in the above embodiment of the present invention, determining whether the first shape of the picture is the second shape includes: reading the size of the picture, wherein the size comprises: the aspect ratio of the picture; and judging whether the first shape of the picture is the second shape or not based on the size of the picture.
In an optional scheme, after the picture needing to be processed is obtained, the picture may be preprocessed, including obtaining an aspect ratio of the picture, so that whether the picture is square or not may be determined according to the aspect ratio of the picture, and if the aspect ratio is 1, it may be determined that the picture is square, otherwise, it may be determined that the picture is non-square.
Fig. 3 is a flowchart of an optional picture processing method according to an embodiment of the present invention, and a preferred embodiment of the present invention is described in detail below with reference to fig. 3, where the method may be applied to a cooking scene of an electric rice cooker, and in the scene, the picture may be a photographed rice picture, and the picture is further processed, so that automatic cooking of the electric rice cooker may be realized, and a better cooking effect may be achieved.
The image processing method can be realized through an image analysis module, a CNN model module and a filling processing module, wherein the image analysis module can use an open source image processing framework OpenCV library. The main workflow of the method is shown in fig. 3:
when a program starts, a picture to be processed is firstly transmitted, the picture is preprocessed by using an open source image processing framework OpenCV (open source computer vision library), the length-width ratio of the picture is acquired, whether the picture is square or not is judged, and if the picture is square, the picture is directly output to enter a subsequent program; if rectangular, further processing is required.
The method comprises the steps that a program can obtain more detailed information of a picture, namely pixel information of the picture, by using an OpenCV library again, the information is arranged and then sent to a CNN model module, the CNN model module analyzes the information by using a convolutional neural network model, and analyzes the information according to the pixel information of the whole picture, so that the pixel information which can simply fill the short edge of the picture and serve as filling parameters can be obtained while original information of the picture is not influenced and damaged, the picture can be identified by a follow-up program after being filled into a square, and an analysis result of the follow-up program is not influenced.
And a proper filling scheme can be obtained through the CNN model module, relevant parameters of the scheme are transmitted to the filling processing module, and the filling processing module calls the OpenCV library of the image processing framework again to fill the image according to the filling parameters. Finally, a square picture is obtained after filling is completed, and the square picture is used for subsequent program analysis.
By the scheme, the image processing mode based on the convolutional neural network is provided and comprises an image analysis module, a CNN model module and a filling processing module respectively, the short side of the rectangular image can be expanded according to the edge pixel mode through the convolutional neural network model, the image is filled into a square, meanwhile, the information of the edge of the screen is not influenced and damaged, and the function of uniformly processing the image into the square for subsequent analysis is realized.
It should be noted that the image processing method provided by the above embodiment of the present invention may be applied to automatic control of a cooking appliance, and the accuracy of the recognition result may be improved by training the recognition model by using the processed square image and further recognizing the rice image by using the trained recognition model. And the type, embryo remaining rate and chalkiness degree of the rice can be determined by identifying the model rice image, and the cooking parameters of the cooking appliance are further determined, so that the cooking effect of the cooking appliance is improved.
Example 2
According to an embodiment of the present invention, there is provided an embodiment of a picture processing apparatus.
Fig. 4 is a schematic diagram of a picture processing apparatus according to an embodiment of the present invention, as shown in fig. 4, the apparatus includes:
a first obtaining module 42, configured to obtain a picture with a first shape and obtain picture information of the picture, where the picture information at least includes; pixel information of the picture.
Specifically, the first shape may be a non-square shape, for example, a rectangle, but is not limited thereto. The picture may be composed of a plurality of pixel points, and the pixel information of the picture may be information of each pixel point, for example, the pixel value of the pixel point may be included, but is not limited thereto.
And a second obtaining module 44, configured to analyze the picture information using a convolutional neural network model, and obtain the filling parameter.
Specifically, the convolutional neural network model may be obtained by machine learning training using a plurality of sets of training data in advance, and each set of training data in the plurality of sets of training data may include: a picture of a non-square, and a corresponding picture of a filled square.
A suitable filling scheme can be given by the convolutional neural network model, and the scheme contains corresponding filling parameters.
And a filling module 46 for filling the picture from the first shape to the second shape based on the filling parameters.
In an optional scheme, whether an incoming picture is a square or not is judged, if the incoming picture is a non-square picture, pixel information of the picture can be read, the pixel information of the picture is analyzed through a Convolutional Neural Network (CNN) model, filling parameters are obtained, a program is controlled to reasonably fill the picture, and the square picture is obtained.
By the embodiment of the invention, after the picture with the first shape is obtained, the picture information of the picture can be further obtained, and the filling parameter can be obtained by analyzing the picture information by using the convolutional neural network model, so that the picture can be filled from the first shape to the second shape based on the filling parameter, and the subsequent processing is facilitated. The rectangular pictures are filled through the convolutional neural network model, the pictures are filled into the square, original information of the pictures is not influenced and destroyed, the unified function of processing the pictures into the square for subsequent analysis is achieved, and the technical problem that in the prior art, massive pictures are needed for machine learning, and the problem that the image learning efficiency is low due to the fact that the sizes of the pictures are inconsistent is solved.
Optionally, in the foregoing embodiment of the present invention, the second obtaining module includes: and the analysis submodule is used for scanning each sub-region of the picture by the convolutional neural network model, analyzing the pixel information of each sub-region and acquiring filling parameters, wherein the filling parameters are the pixel information used for filling the picture.
In particular, each sub-region described above may be each small image in a picture.
In an optional scheme, the picture can be divided into a plurality of small images through a convolutional neural network model, characteristic parameters are extracted, the picture is compressed and parameters are reduced under the condition that the picture quality is not affected, and finally position information of pixel points in each small image corresponds to the original image to obtain final filling parameters.
Optionally, in the above embodiment of the present invention, the analysis sub-module includes: an extraction unit, configured to extract a feature parameter of each sub-region through each sub-region of a convolutional layer scan picture of a convolutional neural network model, where the feature parameter includes at least one of: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object; a determining unit, configured to determine a first sub-region for extracting the filling parameter from the plurality of sub-regions based on the feature parameter; and the processing unit is used for taking the pixel information of the first sub-area as the filling parameter.
Specifically, in an application scenario of electric rice cooker, the obtained picture may be a picture including rice, that is, the object in the sub-region may be rice, but is not limited thereto. The rectangular picture can be filled in a mode of edge pixels or a mode of reading edge pixel points to perform mean value calculation through a convolutional neural network model. The first sub-region may be a sub-region that can not affect and destroy original information of the picture, and can simply fill the picture, and the pixel information of the sub-region may be used as a filling parameter to fill the rectangular picture.
In an alternative, the CNN model may be composed of an input image side, a convolutional layer, and an output side. After the detailed information of the rectangular picture is obtained and completely processed, the picture is sent to a convolutional layer through an input image end to be subjected to convolution calculation, each picture is divided into a plurality of small images in the convolutional layer to be subjected to characteristic parameter extraction, the position information of the small images, the pixel value in the background, the outline of an object, the pixel value of the object and the like are extracted, a first sub-region is further selected according to needs and output to an output end, and therefore the final filling parameters can be obtained according to the pixel information of the first sub-region.
Optionally, in the above embodiment of the present invention, the apparatus further includes: the first processing module is used for taking a sub-region located at the edge position of the picture as a first sub-region; or, regarding a sub-region of the edge position in the predetermined direction in the picture as a first sub-region; or, taking the sub-region with the maximum average pixel value in the picture as a first sub-region; or, taking the sub-region with the minimum average pixel value in the picture as a first sub-region; or a sub-area in the picture adjacent to the area to be filled is taken as the first sub-area.
In an optional scheme, the first sub-region may be selected according to pixel information of an actual picture and a filling manner, and the pixel information of the first sub-region is used as a filling parameter. The convolutional neural network may be filled in the manner of edge pixels, and thus, a sub-region at an edge position of the picture may be taken as the first sub-region. In order to convert a rectangular picture into a square picture, the short side of the picture may be filled in the manner of edge pixels, and thus, the sub-region at the edge position of the short side in the picture may be taken as the first sub-region.
Similarly, since the picture includes an object, for example, the rice picture includes rice, in order not to affect or destroy original information of the picture, a sub-region with a largest average pixel value may be used as the first sub-region, where the sub-region may be a sub-region including the object, or a sub-region with a smallest average pixel value may be used as the first region, where the sub-region may be a sub-region not including the object.
In addition, the pixel value of each pixel point in two adjacent sub-regions in the picture is not changed greatly, so that the sub-region adjacent to the region to be filled in the picture can be used as the first sub-region.
Optionally, in the above embodiment of the present invention, the apparatus further includes: the second processing module is used for taking the pixel information of the background of the first sub-area or the pixel information of the object in the first sub-area as a filling parameter; or, taking the average pixel value of the first sub-area as a filling parameter; or, the pixel information of the edge position of the first sub-area is used as the filling parameter.
In an alternative, after the first sub-region is determined, the filling parameters may be selected according to the pixel information of the actual picture and the filling manner. The characteristic parameters of the first sub-region may include: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the contour of the object in the sub-region, the pixel information of the object, and the like, therefore, the filling parameter may be the background pixel value of the first sub-region, the pixel value of the object, the average pixel value of all the pixel points, or the pixel value of the edge position.
Optionally, in the above embodiment of the present invention, the apparatus further includes: the pooling processing module is used for pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters; and the mapping module is used for mapping the pooled picture on a full connection layer of the convolutional neural network model so that the position information of the pixel points of the compressed picture corresponds to the position information of the pixel points of the picture.
In an alternative, as shown in fig. 2, the CNN model may be composed of an input image end, a convolutional layer, a max-pooling layer, a full link layer, and an output end, where the main purpose of the max-pooling layer is to compress a picture and reduce parameters without affecting the picture quality by means of downsampling. After the detailed information of the rectangular picture is obtained and completely processed, the picture is sent to the convolutional layer through the image input end to be subjected to convolution calculation, and each picture is divided into a plurality of small images in the convolutional layer to be subjected to characteristic parameter extraction. After the feature parameters are extracted from the convolutional layer, the convolutional layer is processed through the largest pooling layer, and finally the position information of the pixel points in the picture is corresponding to the original image through a full link layer and is output to an output end, so that the final filling parameters can be obtained.
Optionally, in the above embodiment of the present invention, the apparatus further includes: the judging module is used for judging whether the first shape of the picture is the second shape; the third processing module is used for directly outputting the picture if the picture is the image; if not, performing the step of converting the picture from the first shape to the second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from the first shape to the second shape based on the filling parameters.
Specifically, the second shape described above may be a square shape, but is not limited thereto.
In an optional scheme, after the picture to be processed is obtained, it may be first determined whether the picture is square, and if so, the picture is directly output for a subsequent procedure; if not, reading the pixel information of the picture and analyzing the information through a convolutional neural network model, so that the non-square picture is filled into the square picture according to the obtained filling parameters.
Optionally, in the foregoing embodiment of the present invention, the determining module includes: a reading submodule for reading a size of the picture, wherein the size includes: the aspect ratio of the picture; and the judging submodule is used for judging whether the first shape of the picture is the second shape or not based on the size of the picture.
In an optional scheme, after the picture needing to be processed is obtained, the picture may be preprocessed, including obtaining an aspect ratio of the picture, so that whether the picture is square or not may be determined according to the aspect ratio of the picture, and if the aspect ratio is 1, it may be determined that the picture is square, otherwise, it may be determined that the picture is non-square.
Example 3
According to an embodiment of the present invention, an embodiment of a storage medium is provided, where the storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the picture processing method in embodiment 1.
Example 4
According to an embodiment of the present invention, an embodiment of a processor is provided, where the processor is configured to run a program, where the program executes the picture processing method in embodiment 1.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (14)
1. An image processing method, comprising:
acquiring a picture with a first shape, and acquiring picture information of the picture, wherein the picture information at least comprises; pixel information of the picture;
analyzing the picture information by using a convolutional neural network model to obtain filling parameters;
based on the filling parameters, filling the picture from a first shape to a second shape;
analyzing the picture information by using a convolutional neural network model to obtain filling parameters, wherein the method comprises the following steps:
the convolutional neural network model scans each sub-region of the picture, analyzes the pixel information of each sub-region and obtains the filling parameters, wherein the filling parameters are the pixel information used for filling the picture;
the convolutional neural network model scans each sub-region of the picture, analyzes the pixel information of each sub-region, and acquires the filling parameters, including:
scanning each sub-region of the picture by the convolutional layer of the convolutional neural network model, and extracting a characteristic parameter of each sub-region, wherein the characteristic parameter comprises at least one of the following parameters: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object;
determining a first sub-region for extracting a filling parameter from a plurality of sub-regions based on the characteristic parameter;
and taking the pixel information of the first sub-area as the filling parameter.
2. The method of claim 1, further comprising:
taking a sub-region located at an edge position of the picture as the first sub-region; or the like, or, alternatively,
taking a sub-region located at an edge position in a predetermined direction in the picture as the first sub-region; or the like, or, alternatively,
taking a sub-region with the largest average pixel value in the picture as the first sub-region; or the like, or, alternatively,
taking a sub-region with the minimum average pixel value in the picture as the first sub-region; or
And taking a sub-region adjacent to a region to be filled in the picture as the first sub-region.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
taking pixel information of the background of the first sub-area or pixel information of an object in the first sub-area as the filling parameter; or the like, or, alternatively,
taking the average pixel value of the first sub-region as the filling parameter; or the like, or, alternatively,
and taking the pixel information of the edge position of the first sub-area as the filling parameter.
4. The method of claim 1, wherein after the convolutional layer of the convolutional neural network model scans each sub-region of the picture and extracts the characteristic parameters of each sub-region, the method further comprises:
pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters;
and mapping the image subjected to pooling on a full-link layer of the convolutional neural network model, so that the position information of the pixel points of the compressed image corresponds to the position information of the pixel points of the image.
5. The method of claim 1, wherein after obtaining the picture having the first shape, the method further comprises:
judging whether the first shape of the picture is the second shape or not;
if yes, directly outputting the picture;
if not, performing a step of converting the picture from a first shape to a second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from a first shape to a second shape based on the filling parameters.
6. The method of claim 5, wherein determining whether the first shape of the picture is the second shape comprises:
reading a size of the picture, wherein the size comprises: an aspect ratio of the picture;
and judging whether the first shape of the picture is the second shape or not based on the size of the picture.
7. A picture processing apparatus, comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a picture with a first shape and acquiring picture information of the picture, and the picture information at least comprises; pixel information of the picture;
the second acquisition module is used for analyzing the picture information by using a convolutional neural network model to acquire filling parameters;
a filling module for filling the picture from a first shape to a second shape based on the filling parameters;
the second acquisition module includes:
the analysis submodule is used for scanning each sub-region of the picture through the convolutional neural network model, analyzing the pixel information of each sub-region and acquiring the filling parameters, wherein the filling parameters are the pixel information used for filling the picture;
the analysis submodule includes:
an extraction unit, wherein the convolutional layer of the convolutional neural network model scans each sub-region of the picture, and extracts a characteristic parameter of each sub-region, wherein the characteristic parameter includes at least one of the following parameters: the position of the sub-region in the picture, the pixel information of the background in the sub-region, the outline of the object in the sub-region and the pixel information of the object;
a determination unit configured to determine a first sub-region for extracting a filling parameter from a plurality of sub-regions based on the feature parameter;
and the processing unit is used for taking the pixel information of the first sub-area as the filling parameter.
8. The apparatus of claim 7, further comprising:
the first processing module is used for taking a sub-region located at the edge position of the picture as the first sub-region; or the like, or, alternatively,
taking a sub-region located at an edge position in a predetermined direction in the picture as the first sub-region; or, taking the sub-region with the largest average pixel value in the picture as the first sub-region; or, taking a sub-region with the minimum average pixel value in the picture as the first sub-region; or
And taking a sub-region adjacent to a region to be filled in the picture as the first sub-region.
9. The apparatus of claim 7 or 8, further comprising:
the second processing module is used for taking the pixel information of the background of the first sub-area or the pixel information of the object in the first sub-area as the filling parameter; or the like, or, alternatively,
taking the average pixel value of the first sub-region as the filling parameter; or the like, or, alternatively,
and taking the pixel information of the edge position of the first sub-area as the filling parameter.
10. The apparatus of claim 7, further comprising:
the pooling processing module is used for pooling the characteristic parameters of each sub-region through a pooling layer of the convolutional neural network model to obtain a compressed picture and reduced characteristic parameters;
and the mapping module is used for mapping the pooled picture on the full-connection layer of the convolutional neural network model so that the position information of the pixel points of the compressed picture corresponds to the position information of the pixel points of the picture.
11. The apparatus of claim 7, further comprising:
the judging module is used for judging whether the first shape of the picture is the second shape;
the third processing module is used for directly outputting the picture if the picture is the original picture; if not, performing a step of converting the picture from a first shape to a second shape, wherein the step of converting the picture from the first shape to the second shape comprises at least: and analyzing the picture information by using a convolutional neural network model to obtain filling parameters, and filling the picture from a first shape to a second shape based on the filling parameters.
12. The apparatus of claim 11, wherein the determining module comprises:
a reading sub-module for reading the size of the picture, wherein the size includes: an aspect ratio of the picture;
and the judging submodule is used for judging whether the first shape of the picture is the second shape or not based on the size of the picture.
13. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, a device where the storage medium is located is controlled to execute the picture processing method according to any one of claims 1 to 6.
14. A processor, configured to execute a program, wherein the program executes the picture processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910033068.2A CN111435544B (en) | 2019-01-14 | 2019-01-14 | Picture processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910033068.2A CN111435544B (en) | 2019-01-14 | 2019-01-14 | Picture processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111435544A CN111435544A (en) | 2020-07-21 |
| CN111435544B true CN111435544B (en) | 2021-11-05 |
Family
ID=71580886
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910033068.2A Active CN111435544B (en) | 2019-01-14 | 2019-01-14 | Picture processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111435544B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108109124A (en) * | 2017-12-27 | 2018-06-01 | 北京诸葛找房信息技术有限公司 | Indefinite position picture watermark restorative procedure based on deep learning |
| CN108510485A (en) * | 2018-03-27 | 2018-09-07 | 福州大学 | It is a kind of based on convolutional neural networks without reference image method for evaluating quality |
| CN109087283A (en) * | 2018-07-03 | 2018-12-25 | 怀光智能科技（武汉）有限公司 | Cervical cell pathological section sick cell recognition methods and system based on cell mass |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1988002158A1 (en) * | 1986-09-18 | 1988-03-24 | Violet Frances Leavers | Shape detection |
| US20040136570A1 (en) * | 2002-04-30 | 2004-07-15 | Shimon Ullman | Method and apparatus for image enhancement for the visually impaired |
| PL1934860T3 (en) * | 2005-10-12 | 2014-11-28 | Intelligent Virus Imaging Inc | Identification and classification of virus particles in textured electron micrographs |
| JP6489800B2 (en) * | 2014-01-16 | 2019-03-27 | キヤノン株式会社 | Image processing apparatus, image diagnostic system, image processing method, and program |
| CN104899854B (en) * | 2014-03-05 | 2018-01-16 | 航天信息股份有限公司 | The detection method and device of heap grain altitude line |
| US10331975B2 (en) * | 2016-11-29 | 2019-06-25 | Google Llc | Training and/or using neural network models to generate intermediary output of a spectral image |
| CN108171244A (en) * | 2016-12-07 | 2018-06-15 | 北京深鉴科技有限公司 | Object identifying method and system |
| CN107945158A (en) * | 2017-11-15 | 2018-04-20 | 上海摩软通讯技术有限公司 | A kind of dirty method and device of detector lens |
-
2019
- 2019-01-14 CN CN201910033068.2A patent/CN111435544B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108109124A (en) * | 2017-12-27 | 2018-06-01 | 北京诸葛找房信息技术有限公司 | Indefinite position picture watermark restorative procedure based on deep learning |
| CN108510485A (en) * | 2018-03-27 | 2018-09-07 | 福州大学 | It is a kind of based on convolutional neural networks without reference image method for evaluating quality |
| CN109087283A (en) * | 2018-07-03 | 2018-12-25 | 怀光智能科技（武汉）有限公司 | Cervical cell pathological section sick cell recognition methods and system based on cell mass |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111435544A (en) | 2020-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112102204B (en) | Image enhancement method and device and electronic equipment | |
| CN111881913A (en) | Image recognition method and device, storage medium and processor | |
| JP5456159B2 (en) | Method and apparatus for separating the top of the foreground from the background | |
| US20040021779A1 (en) | Image processing apparatus, image processing method, recording medium thereof, and program thereof | |
| CN110366001B (en) | Method and device for determining video definition, storage medium and electronic device | |
| CN106875408B (en) | Screenshot method and device and terminal equipment | |
| CN107346546B (en) | Image processing method and device | |
| CN110691226B (en) | Image processing method, device, terminal and computer readable storage medium | |
| CN110276831B (en) | Method and device for constructing three-dimensional model, equipment and computer-readable storage medium | |
| CN110674759A (en) | Monocular face in-vivo detection method, device and equipment based on depth map | |
| KR20200039043A (en) | Object recognition device and operating method for the same | |
| CN110365897B (en) | Image correction method and device, electronic equipment and computer readable storage medium | |
| CN113052754A (en) | Method and device for blurring picture background | |
| JP6294524B1 (en) | Image processing method and computer program | |
| CN111435544B (en) | Picture processing method and device | |
| JP5969105B1 (en) | Imaging apparatus and imaging method | |
| CN116977190A (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN111147693B (en) | Noise reduction method and device for full-size photographed image | |
| JP7369562B2 (en) | Image enhancement method and imaging system | |
| Sun et al. | No-reference image quality assessment through sift intensity | |
| CN112839167A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN112862653A (en) | Data display system based on image processing | |
| CN114119376A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112801932A (en) | Image display method, image display device, electronic equipment and storage medium | |
| CN113837020B (en) | Cosmetic progress detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |