WO2022121701A1

WO2022121701A1 - Image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022121701A1
Application number: PCT/CN2021/133246
Authority: WO
Inventors: 吴佳涛; 郭彦东; 李亚乾; 杨林
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-12-08
Filing date: 2021-11-25
Publication date: 2022-06-16
Also published as: CN112487223A

Abstract

Disclosed in embodiments of the present application are an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring an image to be processed; searching, on the basis of a feature of the image to be processed, a specified storage area for an image similar to the image to be processed to obtain a similar image, wherein images in the specified storage area have corresponding image processing parameters; acquiring an image processing parameter corresponding to the similar image; and performing, on the basis of the image processing parameter, image processing on the image to be processed. Therefore, during a process of performing image processing on an image to beautify the image, an image processing parameter of an image similar to an image to be processed may be acquired by means of querying, so that image processing can be performed on the image to be processed directly on the basis of the image processing parameter of the image similar to the image to be processed, thereby improving the degree of intelligence during an image processing process, and also improving image processing efficiency.

Description

Image processing method, device, electronic device, and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Application No. 202011446215.8 filed on December 8, 2020, which is hereby incorporated by reference in its entirety for all purposes.

technical field

The present application relates to the field of computer technology, and more particularly, to an image processing method, apparatus, electronic device, and storage medium.

Background technique

With more and more image sources, more and more images can be obtained in the electronic device. For some images, the electronic device can perform further beautification processing, so that the visual effect of the image can be improved.

SUMMARY OF THE INVENTION

In view of the above problems, the present application proposes an image processing method, apparatus, electronic device and storage medium to improve the above problems.

In a first aspect, the present application provides an image processing method, which is applied to an electronic device, the method comprising: acquiring an image to be processed; searching for a similar image to the image to be processed in a specified storage area based on the characteristics of the image to be processed to obtain similar images, wherein the images in the designated storage area correspond to image processing parameters; acquire image processing parameters corresponding to the similar images; and perform image processing on the to-be-processed images based on the image processing parameters.

In a second aspect, the present application provides an image processing method, which is applied to a server. The method includes: receiving an image to be processed sent by an electronic device; Process images with similar images to obtain similar images, wherein the images in the designated storage area correspond to image processing parameters; obtain image processing parameters corresponding to the similar images; perform processing on the to-be-processed images based on the image processing parameters Image processing to obtain a processed image; and returning the processed image to the electronic device.

In a third aspect, the present application provides an image processing apparatus, which runs on an electronic device, and the apparatus includes: an image acquisition unit for acquiring an image to be processed; an image search unit for Searching for an image similar to the image to be processed in the designated storage area, and obtaining a similar image, wherein, the image in the designated storage area corresponds to an image processing parameter; a parameter obtaining unit is used to obtain the image processing corresponding to the similar image. parameters; an image processing unit configured to perform image processing on the to-be-processed image based on the image processing parameters.

In a fourth aspect, the present application provides an image processing apparatus, running on a server, the apparatus comprising: a data communication unit for receiving an image to be processed sent by an electronic device; an image search unit for based on the image to be processed The feature of the specified storage area is to search for an image similar to the image to be processed to obtain a similar image, wherein the image in the specified storage area corresponds to an image processing parameter; a parameter acquisition unit is used to obtain the corresponding image of the similar image. the image processing parameters; the image processing unit is used to perform image processing on the to-be-processed image based on the image processing parameters to obtain the processed image; the data communication unit is also used to return the processed image to to the electronic device.

In a fifth aspect, the present application provides an electronic device including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

In a sixth aspect, the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, wherein the above-mentioned method is executed when the program code is executed by a startup controller.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a schematic diagram of an application scenario of an image processing method proposed by an embodiment of the present application;

FIG. 2 shows a schematic diagram of an application scenario of an image processing method proposed by an embodiment of the present application;

FIG. 3 shows a flowchart of an image processing method proposed by an embodiment of the present application;

4 shows a schematic diagram of an image being selected in an image management program according to an embodiment of the present application;

5 shows a schematic diagram of an editing interface in an embodiment of the present application;

FIG. 6 shows a schematic diagram of determining similar images in an embodiment of the present application;

FIG. 7 shows a flowchart of another image processing method proposed by an embodiment of the present application;

FIG. 8 shows a flowchart of interface jumping based on link information proposed by an embodiment of the present application;

FIG. 9 shows a schematic diagram of triggering interface jumping proposed by an embodiment of the present application;

FIG. 10 shows a flowchart of still another image processing method proposed by an embodiment of the present application;

11 shows a flowchart of obtaining a global feature and a local feature map in an embodiment of the present application;

FIG. 12 shows a schematic diagram of obtaining a global feature in an embodiment of the present application;

FIG. 13 shows a schematic diagram of a feature set corresponding to a position in an initial feature map in an embodiment of the present application;

FIG. 14 shows a schematic diagram of obtaining a local feature map in an embodiment of the present application;

FIG. 15 shows a schematic diagram of position correspondence in an embodiment of the present application;

FIG. 16 shows a flowchart of an image processing method proposed by another embodiment of the present application;

FIG. 17 shows a structural block diagram of an image processing apparatus proposed by an embodiment of the present application;

FIG. 18 shows a structural block diagram of an image processing apparatus proposed by another embodiment of the present application;

FIG. 19 shows a structural block diagram of an image processing apparatus proposed by still another embodiment of the present application;

FIG. 20 shows a structural block diagram of an electronic device of the present application for executing the image processing method according to an embodiment of the present application;

FIG. 21 is a storage unit for storing or carrying a program code for implementing the image processing method according to the embodiment of the present application according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

As the number of images increases, there are more applications for images. For example, image processing can be performed on an image to enhance the visual effect of the image. However, the inventor found in the research that the related image beautification processing methods still have the problems of low degree of intelligence and poor beautification efficiency.

Specifically, in the relevant image processing process, the parameters to be processed are usually selected manually by the user, and because of the differences in the user's own image processing experience, the parameter selection process needs to consume a lot of Time, resulting in low processing efficiency, and does not have better intelligence.

Therefore, the inventor proposes an image processing method, device, electronic device, and storage medium in the present application that can improve the above-mentioned problems. Describe the image similar to the image to be processed, and obtain the similar image, wherein, the image in the designated storage area corresponds to the image processing parameter, and then the similar image can also be obtained while obtaining the image processing parameter corresponding to the similar image, and then based on the similar image Perform image processing on the to-be-processed image with the image processing parameters.

Therefore, through the above method, in the process of performing image processing on the image to beautify the image, the method of acquiring the image processing parameters of the image similar to the image to be processed can be obtained by query, so that the image processing parameters of the image similar to the image to be processed can be directly obtained based on the similarity of the image to be processed. The image processing parameters of the image to be processed are used to process the image to be processed, thereby improving the degree of intelligence in the image processing process. Moreover, because the image to be processed is directly processed based on the image processing parameters of the image similar to the image to be processed, and the user does not need to perform the processing by himself, the efficiency of image processing is also improved.

It should be noted that, the image processing method provided by the embodiments of the present application may be executed in electronic devices such as mobile phones and tablet computers, and may also be executed in a server. Among them, the server can be a single physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud services, cloud computing, cloud storage, CDN (Content Delivery Network, Content Delivery Network), And cloud servers for basic cloud computing services such as artificial intelligence platforms.

Wherein, the image processing method provided by the embodiment of the present application is performed by an electronic device alone, or can be performed by an electronic device and a server cooperatively, or can be performed cooperatively by a server cluster or a distributed system composed of multiple physical servers. Optionally, as shown in FIG. 1 , in the case where the image processing method provided by the embodiment of the present application is completed by an electronic device and a server in cooperation, the acquisition of the to-be-processed image may be completed by the client 110 in the electronic device, and The subsequent search for a similar image corresponding to the to-be-processed image and an image processing unit corresponding to the similar image, and subsequent image processing on the to-be-processed image based on the image processing parameters may be performed by the server 120 .

In the case where different steps in the image processing method may be executed by different physical servers respectively, or may be executed in a distributed manner by a server constructed based on a distributed system. For example, as shown in FIG. 2 , the acquisition of the to-be-processed image may be performed by the server 130 alone, and the subsequent search for a similar image corresponding to the to-be-processed image and an image processing unit corresponding to the similar image, and based on the image processing parameters Image processing on the to-be-processed image may be performed by the server 140 .

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Referring to FIG. 3, an image processing method provided by an embodiment of the present application is applied to an electronic device, and the method includes:

S110: Acquire an image to be processed.

Wherein, the image processing method provided in this embodiment may operate in various scenarios, and correspondingly, the images to be processed obtained in different scenarios are different.

Optionally, the image processing method provided in this embodiment may be executed in an image management program (eg, a photo album). In this scenario, an image selected by the user from the image management program may be used as an image to be processed. As shown in FIG. 4 , an image a, an image b, an image c, an image d, an image e, an image f, an image g, an image h, and an image i are displayed on the image display interface 10 . If it is detected that the image a is selected, the image enlargement display interface 11 corresponding to the image a on the right side in FIG. 4 is triggered to be displayed. In the image magnification display interface 11, the image a will be enlarged and displayed, and an editing control will be displayed. If it is detected that the editing control is touched, the editing interface 12 shown in FIG. 5 will be displayed by triggering. In the editing interface 12 The beautification operation control is displayed in the . If it is detected that the beautification operation control is touched, it will enter the beautification editing mode, and at the same time, the image a will be used as the image to be processed, and the subsequent image will be directly executed after the image a is used as the image to be processed. , and then realize one-click beautification of the image to be processed.

It should be noted that the manner of determining the image to be processed as shown in FIG. 4 and FIG. 5 is only an exemplary manner, and there may be other manners to trigger the determination of the image to be processed. For example, the editing interface 12 shown in FIG. 5 can also be directly triggered by long-pressing the image in the image display interface 10 . Furthermore, entry into the corresponding beautification editing mode in the editing interface 12 can also be directly triggered by long-pressing the image in the image display interface 10 .

In addition to the foregoing scenarios, optionally, the image processing method provided in this embodiment may also be run in a network information search scenario. In this scenario, the user can input an image, and then search for images similar to the input image. Correspondingly, the image input by the user can be used as the image to be processed.

S120: Search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter.

Among them, the features of the image can be used to characterize the content of the image. In this embodiment, the content of the image to be processed can be characterized based on the feature vector. Optionally, the feature vector may be inputting the image to be processed into the target neural network model, and then acquiring the feature vector output by the neural network model as a feature representing the content of the image to be processed. Correspondingly, the images in the designated storage area also have their own characteristics, so in the process of determining similar images, the characteristics of the images to be processed can be compared with the characteristics of the images in the designated storage area, and then obtain to similar images.

Taking the feature vector as an example, as shown in Figure 6, for the image to be processed and the image currently being compared with the object to be processed in the specified storage area, it can be input into the CNN (Convolutional Neural Networks) model respectively to obtain the respective features. vector, and then calculate the distance between the respective feature vectors of the image to be processed and the image currently being compared. If the distance is less than the distance threshold, it is determined that the image to be processed is similar to the image currently being compared with the object to be processed. , and then determined to be similar images.

Optionally, the similar images in the embodiments of the present application may be IND (identical ND images) similar images, or may be NIND (non-identical ND images) similar images

S130: Acquire image processing parameters corresponding to the similar images.

The image in the designated storage area may correspondingly store image processing parameters. Optionally, the image processing parameter is an image processing parameter that enables the image of the designated storage area to exhibit the current visual effect. Exemplarily, if saturation enhancement is performed on image j and gamma correction is also performed on color cast, the image processing parameters corresponding to image j include the gamma value when performing gamma correction and the saturation value when performing saturation enhancement. degree enhancement parameter. Then, if it is determined that the image j in the specified storage area is a similar image corresponding to the image to be processed, the acquired image processing parameters will include the gamma value corresponding to the image j for gamma correction, and the gamma value used for saturation enhancement. The saturation enhancement parameter of .

S140: Perform image processing on the to-be-processed image based on the image processing parameter.

After the image processing parameters corresponding to the similar images are acquired, the to-be-processed images may be processed based on the image processing parameters corresponding to the similar images. Exemplarily, the aforementioned image j is determined to be a similar image, and the image processing parameters corresponding to the image j include a gamma value for gamma correction and a saturation enhancement parameter for saturation enhancement, and then the object to be processed is processed. , the gamma correction will be performed on the image to be processed based on the gamma value, and the saturation enhancement will be performed on the image to be processed based on the saturation enhancement parameter.

In an image processing method provided by this embodiment, after acquiring an image to be processed, an image similar to the image to be processed is searched in a designated storage area based on the characteristics of the image to be processed, so as to obtain a similar image, wherein, in The images in the designated storage area correspond to image processing parameters, and then the image processing parameters corresponding to the similar images can be obtained while the similar images are obtained, and then image processing is performed on the to-be-processed images based on the image processing parameters of the similar images.

Referring to FIG. 7 , an image processing method provided by an embodiment of the present application is applied to an electronic device, and the method includes:

S210: Acquire an image to be processed.

S220: Search for an image similar to the to-be-processed image in a designated storage area based on the feature of the to-be-processed image, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter.

It should be noted that the designated storage area in this embodiment may be a local storage area of the electronic device, or may be a storage area in a server. If the storage area is a storage area in the server, the electronic device may implement S120 by sending a search instruction to the server where the storage area is located. Optionally, an image to be processed may be carried in the sending search instruction, and after receiving the image to be processed, the server needs to first acquire the features of the image to be processed, and then search for similar images based on the acquired features. Furthermore, optionally, the electronic device can also obtain the feature (for example, a feature vector) of the image to be processed locally, and directly carry the feature vector in the search instruction, so that the server can directly obtain the feature of the image to be processed. vector, and then search for similar images faster.

It can be understood that if the electronic device locally executes the feature process of extracting the picture to be processed, the processing resources of the electronic device will be consumed to a certain extent, and if the electronic device directly uploads the picture to be processed to the designated storage area. The server performs the feature extraction process by the server, which consumes more network resources because of the need to transmit pictures. Then, in this embodiment of the present application, in order to utilize the current resources more reasonably, the electronic device may determine, in real time, the data content carried in the search instruction according to the currently available processing resources or network resources.

Optionally, if the electronic device detects that the current remaining available processing resources meet the resource threshold condition, it will locally acquire the characteristics of the image to be processed, and send a search instruction that carries the characteristics of the image to be processed (that is, when searching for the image to be processed). The instruction does not carry the image to be processed), so that the server that receives the search instruction directly searches for images similar to the image to be processed in the specified storage area according to the features in the search instruction to obtain similar images. Thus, in this way, when the available processing resources do not meet the resource threshold condition, the electronic device is still stuck locally, which is caused by acquiring the feature.

The resource threshold condition may include that the remaining available processing resources are greater than the specified resource threshold. The processing resource may be CPU availability. The CPU availability rate may be 1 minus the current load rate of the CPU. The load rate of the CPU can be the ratio of the CPU's currently running threads to the total supported threads.

If the electronic device detects that the current network resource meets the specified network conditions, it can add the image to be processed to the search instruction, and then directly send the image to be processed to the server through the search instruction, so that the server can first obtain the characteristics of the image to be processed and then , and then search for images similar to the to-be-processed image in the designated storage area based on the characteristics of the to-be-processed image to obtain a similar image. In this way, the current network resources can be more effectively utilized, and the effective utilization rate of the network resources can be improved.

The specified network conditions may include at least one of the following conditions: the Internet is currently accessed based on WIFI communication; and the network transmission rate satisfies the specified efficiency. The network transmission efficiency can be understood as the number of bits of binary numbers transmitted per second.

It should be noted that, for the first two conditions (resource threshold condition and specified network condition), detection can be triggered based on different ways.

As one approach, the electronic device can trigger the detection of whether the resource threshold condition and the specified network condition are met at the same time. It should be noted that whether the resource threshold conditions are met and the specified network conditions need to obtain certain data for calculation, and the calculation will consume time, then trigger the detection of whether the resource threshold conditions and the specified network conditions are met at the same time. Under the following conditions, the results of the detection of the two conditions may be returned in sequence, and in this embodiment of the present application, the generation method of the search instruction may be determined based on the detection result returned first.

Specifically, in this way, the electronic device can trigger the simultaneous detection of whether the resource threshold condition and the specified network condition are satisfied. The corresponding steps when the resource threshold conditions are met, and the subsequent detection results of whether the specified network conditions are met are ignored. If the detection result for the resource threshold condition is obtained first, but it is detected that the resource threshold condition is not met, it will wait for the subsequent detection result for the specified network condition, and execute the corresponding steps according to the subsequent detection result for the specified network condition .

If the detection result for the specified network condition is obtained first, and it is detected that the specified network condition is met, the aforementioned steps corresponding to when the specified network condition is met are performed, and the subsequent detection of whether the resource threshold condition is met is ignored. If the detection result for the specified network condition is obtained first, and it is detected that the specified network condition is not satisfied, the subsequent detection result for the resource threshold condition is waited for, and corresponding steps are performed according to the detection result for the resource threshold condition.

As another method, it is possible to first detect whether the resource threshold condition is met. If the resource threshold condition is met, the aforementioned steps corresponding to the resource threshold condition are directly executed. If the resource threshold condition is not met, then the detection is triggered The specified network conditions are met.

It should be noted that, in this embodiment, if it is detected that neither the resource threshold condition nor the specified network condition is met, the electronic device will provide randomly acquired image processing parameters, so as to process images based on the randomly acquired image processing parameters. Parameters to process the image to be processed. The randomly acquired image processing parameters may be randomly acquired from locally pre-stored image processing parameters, or may be randomly acquired from the aforementioned image processing parameters in the designated storage area.

S230: Display the multiple similar images found.

It should be noted that there may be multiple similar images corresponding to the image to be processed in the designated storage area, and although they are all images similar to the image to be processed, their corresponding image processing parameters will be different. Furthermore, the corresponding visual effects will be different. In order to facilitate the user to select the desired effect, the found multiple similar images may be displayed. The similar images have been processed by their corresponding image processing parameters, and when the similar images are displayed, the user can perceive the visual effects processed by the corresponding image processing parameters.

S240: Acquire a target similar image determined from the plurality of similar images.

S250: Acquire image processing parameters corresponding to the target similar image.

S260: Perform image processing on the to-be-processed image based on the image processing parameter corresponding to the target similar image.

In one manner, the method further includes: acquiring a reference similar image, where the reference similar image is a similar image detected to be subjected to a specified touch operation among the plurality of similar images. During the action of the designated touch operation, the preview effect of the image to be processed after image processing based on the image processing parameters corresponding to the reference similar image is displayed. If it is detected that the designated touch operation ends, the preview effect is canceled. Optionally, the specified touch operation may include a pressing operation whose duration satisfies the specified duration or a pressing operation whose pressing area satisfies the specified area.

As one way, as shown in FIG. 8 , the image in the designated storage area also corresponds to link information of the display interface; the search for the image to be processed in the designated storage area based on the characteristics of the image to be processed is related to the image to be processed. Similar images, after obtaining similar images, also include:

S270: Acquire the link information corresponding to the similar images.

Optionally, the link information in this embodiment may be a link address of a display interface corresponding to the similar image.

S280: Jump to the display interface corresponding to the link information based on the link information.

It should be noted that the aforementioned designated storage area may be the resource storage area corresponding to the image download platform or the theme download platform, then in this case, after the similar image is obtained, the similar image can also be displayed by triggering the electronic device. The similar images are displayed on the display interface in the image download platform or the theme download platform, thereby facilitating traffic flow for the image download platform or the theme download platform. Wherein, after the electronic device displays the similar image on the display interface in the image download platform or theme download platform, it can search for other types of images or themes in the image download platform or theme download platform.

Optionally, the electronic device may jump to the display interface corresponding to the link information based on the link information in response to the user's touch operation. Exemplarily, as shown in FIG. 9 , a similar image k and a similar image m are displayed in the interface shown in FIG. 9 . If it is detected that the similar image k is clicked, the menu interface 13 in the right image in FIG. 9 can be displayed. The menu interface 13 includes the option of using image processing parameters and the option of jumping to the corresponding platform. If it is detected that the option of jumping to the corresponding platform is selected, the electronic device can trigger jumping to the display interface in the image download platform or the theme download platform to display the similar image k.

In the image processing method provided by this embodiment, in the above manner, in the process of performing image processing on an image to beautify the image, the method of obtaining image processing parameters of an image similar to the image to be processed can be obtained by querying , so that the image to be processed can be directly processed based on the image processing parameters of the image similar to the image to be processed, thereby improving the degree of intelligence in the image processing process. Moreover, because the image to be processed is directly processed based on the image processing parameters of the image similar to the image to be processed, and the user does not need to perform the processing by himself, the efficiency of image processing is also improved. In addition, in this embodiment, the preview effect of the corresponding image processing can also be triggered by triggering a designated touch operation on a plurality of similar images, which is beneficial for the user to select the image processing parameters required by himself, and further The convenience of image processing is improved.

Referring to FIG. 10, an image processing method provided by an embodiment of the present application is applied to an electronic device, and the method includes:

S310: Acquire the image to be processed.

S320: Based on the global feature and the local feature map of the image to be processed, search for an image similar to the image to be processed in a designated storage area, and obtain a similar image, wherein the image in the designated storage area corresponds to image processing parameter.

Among them, the global feature can be understood as the feature that characterizes the image content as a whole, and correspondingly, the local feature can be understood as the feature that characterizes the image content from the part of the image.

As one way, as shown in FIG. 11 , the obtaining of the global feature and the local feature map includes:

S321: Input the image to be processed into a target neural network, and obtain a feature map output by the target neural network as an initial feature map. Optionally, the target neural network can be a CNN (Convolutional Neural Networks) network. The target neural network in this embodiment may be a model that has been trained in other tasks, such as a classification model, a segmentation model, and the like.

S322: Perform global average pooling processing on the initial feature map to obtain global features.

Optionally, global average pooling can be performed based on the following formula, which is:

Among them, F(i,j) represents the feature vector at position (i,j), w represents the width of the initial feature map, and h represents the height of the initial feature map. Take the initial feature map F of size w*h*d as an example, where w represents the width of the initial feature map, h represents the height of the initial feature map, and d represents the number of channels of the initial feature map. As shown in Figure 12, the initial feature map F with size w*h*d can be processed by global average pooling to obtain global image feature G with size 1*d as global feature.

S323: Acquire feature sets corresponding to multiple positions in the initial feature map, where the feature sets corresponding to each position include features of each position and features of adjacent positions of each position.

As one approach, the plurality of locations may be each location in the initial feature map. It should be noted that the initial feature map may correspond to a channel dimension, then the information of each position (i, j) includes the color information of each channel. Exemplarily, as shown in FIG. 13 , the initial feature map 20 shown in FIG. 13 is exemplarily shown as position a to position i (the initial feature map 20 may also include positions other than position a to position i) other positions), wherein, for position a, adjacent positions include position b, position e and position d, for example, for position e, adjacent positions include position a, position b, position c, position f , position i, position h, position g, position d.

S324: Perform averaging processing on the feature sets corresponding to each of the positions to obtain local features corresponding to each of the positions.

S325: Obtain the local feature map based on the local features corresponding to each of the positions.

As shown in Figure 14, if the size of the image to be processed is W*H*3, after processing by the target neural network, an initial feature map F with a size of w*h*d can be obtained, and then the position (i, j ) of the feature set, and then average the local domain of the feature set at the position (i, j) (that is, average processing), you can obtain a local feature of size 1*d, and then the size corresponding to each position is 1 When the local features of *d are aggregated, the local feature map corresponding to the to-be-processed image of size W*H*3 can be obtained.

It should be noted that the local feature corresponding to each position can also be understood as the local feature vector of each position. Optionally, the local feature vector of each position can be calculated by the following formula. The formula is: :

Among them, F(u, v) represents the feature vector at the position (u, v), and M represents the number of adjacent positions corresponding to the position (u, v).

As one way, based on the global feature and local feature map of the to-be-processed image, searching for an image similar to the to-be-processed image in a designated storage area to obtain a similar image, including:

Compare the global feature of the image to be processed with the global feature of the image in the designated storage area, and compare the local feature map of the image to be processed with the local feature map of the image in the designated storage area make similar comparisons;

In the designated storage area, the corresponding global feature and the global feature of the image to be processed satisfy the first similarity condition, and the corresponding local feature map and the local feature map of the image to be processed satisfy the second similarity condition images as similar images.

In one way, the global feature of the image to be processed may be used as the first global feature, and the local feature map of the image to be processed may be used as the first local feature map. The global feature of the image that is currently compared with the image to be processed in the specified storage area can be used as the second global feature, and the local feature map of the image that is currently compared with the image to be processed in the specified storage area can be used as the second local feature. picture.

Wherein, optionally, if the distance between the first global feature and the second global feature is less than a first distance threshold, it is determined that the first global feature and the second global feature satisfy a first similarity condition. Wherein, as an approach, the distance between the first global feature and the second global feature may be calculated based on the Euclidean distance or the cosine distance. In this embodiment, the distance between the first global feature and the second global feature can be calculated based on the following formula, where the formula is:

D _G =||G ¹ -G ² || ₂

Wherein, D _G represents the calculated distance between the first global feature and the second global feature, G ¹ represents the first global feature, and G ² represents the second global feature.

Optionally, calculating whether the first local feature map and the second local feature map satisfy the second similarity condition may include:

Calculate the feature distance between the corresponding first local feature and the second local feature, the first local feature is the local feature included in the first local feature map, and the second local feature is the second local feature The local features included in the feature map.

It should be noted that the first local feature map is obtained by combining local features corresponding to multiple positions in the aforementioned first initial feature map. Correspondingly, the second local feature map is obtained by combining local features corresponding to multiple positions in the aforementioned second initial feature map.

Optionally, the positions in the first initial feature map and the second initial feature map may be identified based on coordinates, and then positions with the same corresponding coordinates may be used as mutually corresponding positions. For example, the coordinates of the position (i ₁ , j ₁ ) of the first local feature map and the position (i ₁ , j ₁ ) of the second local feature map are both (i ₁ , j ₁ ), then the first local The position (i ₁ , j ₁ ) in the feature map and the position (i ₁ , j ₁ ) in the second local feature map are two positions whose positions correspond to each other. Correspondingly, the local features corresponding to the two positions corresponding to each other are the corresponding local features. Then, the first local feature and the second local feature corresponding to each other can be understood as the position corresponding to the first local feature, and the position corresponding to the second local feature is corresponding to each other.

As shown in FIG. 15 , the first initial feature map 30 and the second initial feature map 40 are shown, wherein the coordinates of the position 31 of the first initial feature map 30 are (1, 1), and the coordinates of the second initial feature map 40 are The coordinates corresponding to the position 41 are also (1, 1), because the coordinates of the position 31 in the first initial feature map 30 and the coordinates of the position 41 in the second initial feature map 40 are the same, then the position 31 and the position feature 31 is the corresponding position, and the first local feature corresponding to the position 31 and the first local feature corresponding to the position 41 are the corresponding first local feature and the second local feature.

Then, as a way, in the case where the sizes of the first image and the second image are the same, the sizes of the first initial feature map and the second initial feature map are also the same. Then, for each position in the first initial feature map, there will be a corresponding position in the second initial feature map.

As a way, the feature distance between the first local feature and the second local feature corresponding to each other can be calculated based on the following formula, which is:

D _L (i, j)=||L ¹ (i, j)-L ² (i, j)|| ₂

Wherein, L ¹ (i, j) represents the local feature corresponding to the position (i, j) in the first initial feature map. L ² (i, j) represents the local feature corresponding to the position (i, j) in the second initial feature map.

The first local feature whose corresponding feature distance is smaller than the second distance threshold is used as the target local feature.

As shown in the foregoing, in the case where each position in the first initial feature map has a corresponding position in the second initial feature map, the first partial corresponding to each position in the first initial feature map can be obtained by calculation The feature distance corresponding to the feature. The feature distance corresponding to the first local feature may be understood as the feature distance between the first local feature and the second local feature corresponding to the position.

If the ratio of the number of target local features to the number of local features included in the first local feature map is greater than a scale threshold, it is determined that the first local feature map and the second local feature map satisfy a second similarity condition.

where the ratio can be calculated based on the following formula, which is:

Among them, _NL represents the number of target local features, w represents the width of the first initial feature map, and h is the height of the first initial feature map. Optionally, the R can be 0.6.

It should be noted that the product of the width of the first initial feature map and the height of the first initial feature map represents the number of positions in the first initial feature map. In the case that each position corresponds to a local feature, The number of local features included in the first local feature map is the number of positions in the first initial feature map.

S330: Acquire image processing parameters corresponding to the similar images.

S340: Perform image processing on the to-be-processed image based on the image processing parameter.

In the image processing method provided by this embodiment, in the above manner, in the process of performing image processing on an image to beautify the image, the method of obtaining image processing parameters of an image similar to the image to be processed can be obtained by querying , so that the image to be processed can be directly processed based on the image processing parameters of the image similar to the image to be processed, thereby improving the degree of intelligence in the image processing process. In addition, in this embodiment, in the process of determining similar images, the global features and local features of the images can be combined to perform similarity detection, so that a more comprehensive detection can be performed in the process of image similarity detection, which improves the performance of the image similarity detection process. Accuracy of similarity detection.

Referring to FIG. 16, an image processing method provided by an embodiment of the present application is applied to a server, and the method includes:

S410: Receive the to-be-processed image sent by the electronic device.

S420: Search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter.

S430: Acquire image processing parameters corresponding to the similar images.

S440: Perform image processing on the to-be-processed image based on the image processing parameter to obtain a processed image.

S450: Return the processed image to the electronic device.

Therefore, through the above method, in the process of performing image processing on the image to beautify the image, the method of acquiring the image processing parameters of the image similar to the image to be processed can be obtained by query, so that the image processing parameters of the image similar to the image to be processed can be directly obtained based on the The image processing parameters of the image to be processed are used to process the image to be processed, thereby improving the degree of intelligence in the image processing process. Moreover, because the image to be processed is directly processed based on the image processing parameters of the image similar to the image to be processed, and the user does not need to perform the processing by himself, the efficiency of image processing is also improved.

Referring to FIG. 17 , an image processing apparatus 500 provided by an embodiment of the present application operates on an electronic device, and the apparatus 500 includes:

an image acquisition unit 510, configured to acquire an image to be processed;

The image search unit 520 is configured to search for an image similar to the image to be processed in a designated storage area based on the characteristics of the image to be processed, and obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter .

The parameter obtaining unit 530 is configured to obtain image processing parameters corresponding to the similar images.

The image processing unit 540 is configured to perform image processing on the to-be-processed image based on the image processing parameter.

As an approach, there are multiples of the similar images. The parameter obtaining unit 530 is specifically configured to display a plurality of similar images found; obtain a target similar image determined from the plurality of similar images; and obtain image processing parameters corresponding to the target similar images. The image processing unit 540 is specifically configured to perform image processing on the to-be-processed image based on image processing parameters corresponding to the target similar image.

As shown in FIG. 18 , the apparatus 500 further includes: a processing effect preview unit 550 and a flow drainage unit 560 .

Among them, the processing effect preview unit 550 is used to obtain a reference similar image, where the reference similar image is a similar image that is detected with a designated touch operation in the plurality of similar images; during the action of the designated touch operation, Display the preview effect of the image to be processed based on the image processing parameters corresponding to the reference similar image after image processing; if it is detected that the specified touch operation ends, cancel the display of the preview effect.

In one way, the images in the designated storage area also correspond to link information of the display interface, and the traffic diversion unit 560 is configured to obtain the link information corresponding to the similar images; jump to the link information based on the link information corresponding display interface.

In one way, the features include global features and local feature maps, and the image search unit 520 is specifically configured to search for images related to the to-be-processed image in the designated storage area based on the global features and local feature maps of the to-be-processed image. Similar images, get similar images. In this manner, optionally, the image search unit 520 is specifically configured to perform similarity comparison between the global feature of the image to be processed and the global feature of the image in the designated storage area, and compare the global feature of the image to be processed with The local feature map of the processed image and the local feature map of the image in the designated storage area are similarly compared; in the designated storage area, the corresponding global feature and the global feature of the image to be processed satisfy the first similarity condition , and an image satisfying the second similarity condition between the corresponding local feature map and the local feature map of the image to be processed is regarded as a similar image.

Referring to FIG. 19 , an image processing apparatus 600 provided by an embodiment of the present application operates on an electronic device, and the apparatus 600 includes:

The data communication unit 610 is configured to receive the to-be-processed image sent by the electronic device.

The image search unit 620 is configured to search for an image similar to the image to be processed in a designated storage area based on the characteristics of the image to be processed, and obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter .

The parameter obtaining unit 630 is configured to obtain image processing parameters corresponding to the similar images.

The image processing unit 640 is configured to perform image processing on the to-be-processed image based on the image processing parameter to obtain a processed image.

The data communication unit 610 is further configured to return the processed image to the electronic device.

An image processing apparatus provided by this embodiment obtains a similar image by searching for an image similar to the to-be-processed image in a designated storage area based on the characteristics of the to-be-processed image after acquiring the to-be-processed image, wherein: The images in the designated storage area correspond to image processing parameters, and then the image processing parameters corresponding to the similar images can be obtained while obtaining the similar images, and then image processing is performed on the to-be-processed images based on the image processing parameters of the similar images. Therefore, through the above method, in the process of performing image processing on the image to beautify the image, the method of acquiring the image processing parameters of the image similar to the image to be processed can be obtained by query, so that the image processing parameters of the image similar to the image to be processed can be directly obtained based on the The image processing parameters of the image to be processed are used to process the image to be processed, thereby improving the degree of intelligence in the image processing process. Moreover, because the image to be processed is directly processed based on the image processing parameters of the image similar to the image to be processed, and the user does not need to perform the processing by himself, the efficiency of image processing is also improved.

It should be noted that the apparatus embodiments in the present application correspond to the foregoing method embodiments, and the specific principles in the apparatus embodiments may refer to the content in the foregoing method embodiments, which will not be repeated here.

An electronic device provided by the present application will be described below with reference to FIG. 20 .

Referring to FIG. 20 , based on the above-mentioned image processing method and apparatus, another electronic device 200 that can execute the above-mentioned image processing method is further provided by the embodiments of the present application. The electronic device 200 includes one or more (only one shown in the figure) a processor 102, a memory 104, and a network module 106 that are coupled to each other. Wherein, the memory 104 stores a program that can execute the content in the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104 .

The processor 102 may include one or more cores for processing data. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 104, and calling the data stored in the memory 104. Various functions of the electronic device 200 and processing data. Optionally, the processor 102 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). A hardware form is implemented. The processor 102 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 102, and is implemented by a communication chip alone.

The memory 104 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 104 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 104 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing the operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. For example, the memory 104 may store means for image processing. The image processing apparatus may be the aforementioned apparatus 500 or the aforementioned apparatus 600 . The storage data area may also store data created by the terminal 100 during use (such as phone book, audio and video data, chat record data) and the like.

The network module 106 is used for receiving and sending electromagnetic waves, realizing mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, for example, communicate with an audio playback device. The network module 106 may include various existing circuit elements for performing these functions, eg, antennas, radio frequency transceivers, digital signal processors, encryption/decryption chips, subscriber identity module (SIM) cards, memory, etc. . The network module 106 can communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other devices through a wireless network. The aforementioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 106 may interact with the base station for information.

Please refer to FIG. 21 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 1100 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 1100 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 1100 has storage space for program code 1110 that performs any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 1110 may be compressed, for example, in a suitable form.

To sum up, an image processing method, device, electronic device and storage medium provided by the present application, after acquiring an image to be processed, and then based on the characteristics of the to-be-processed image in a designated storage area to search for and the to-be-processed image Images with similar images are obtained to obtain similar images, wherein the images in the designated storage area correspond to image processing parameters, and then the image processing parameters corresponding to similar images can be obtained while obtaining similar images, and then the image processing parameters based on similar images can be obtained. The parameter performs image processing on the to-be-processed image. Therefore, through the above method, in the process of performing image processing on the image to beautify the image, the method of acquiring the image processing parameters of the image similar to the image to be processed can be obtained by query, so that the image processing parameters of the image similar to the image to be processed can be directly obtained based on the The image processing parameters of the image to be processed are used to process the image to be processed, thereby improving the degree of intelligence in the image processing process. Moreover, because the image to be processed is directly processed based on the image processing parameters of the image similar to the image to be processed, and the user does not need to perform the processing by himself, the efficiency of image processing is also improved.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements to some of the technical feature diagrams; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of the present application .

Claims

An image processing method, characterized in that, applied to an electronic device, the method comprising:

Get the image to be processed;

Search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter;

obtaining image processing parameters corresponding to the similar images;

Image processing is performed on the to-be-processed image based on the image processing parameter.
The method according to claim 1, wherein there are multiple similar images, and the acquiring image processing parameters corresponding to the similar images comprises:

Display multiple similar images found;

obtaining a target similar image determined from the plurality of similar images;

obtaining image processing parameters corresponding to the target similar images;

The performing image processing on the to-be-processed image based on the image processing parameter includes:

Image processing is performed on the to-be-processed image based on image processing parameters corresponding to the target similar image.
The method according to claim 2, wherein the method further comprises:

Acquiring a reference similar image, where the reference similar image is a similar image detected with a specified touch operation in the plurality of similar images;

During the action of the designated touch operation, displaying the preview effect of the image to be processed after image processing based on the image processing parameters corresponding to the reference similar image;

If it is detected that the designated touch operation ends, the preview effect is canceled.
The method according to claim 3, wherein the specified touch operation comprises a pressing operation whose duration meets the specified duration or a pressing operation whose pressing area satisfies the specified area.
The method according to any one of claims 1-4, wherein the image in the designated storage area also corresponds to link information of the display interface;

The searching for an image similar to the image to be processed in the designated storage area based on the feature of the image to be processed, and after obtaining the similar image, further includes:

obtaining the link information corresponding to the similar images;

Jump to the display interface corresponding to the link information based on the link information.
The method according to any one of claims 1-4, wherein the features include a global feature and a local feature map, and the feature of the to-be-processed image is searched in a specified storage area for the corresponding image to be processed. Similar images to get similar images include:

Based on the global feature and the local feature map of the to-be-processed image, an image similar to the to-be-processed image is searched in a designated storage area to obtain a similar image.
The method according to claim 6, wherein, based on the global feature and local feature map of the image to be processed, searching for an image similar to the image to be processed in a designated storage area to obtain a similar image, comprising: :

Compare the global feature of the image to be processed with the global feature of the image in the designated storage area, and compare the local feature map of the image to be processed with the local feature map of the image in the designated storage area make similar comparisons;

In the designated storage area, the corresponding global feature and the global feature of the image to be processed satisfy the first similarity condition, and the corresponding local feature map and the local feature map of the image to be processed satisfy the second similarity condition images as similar images.
The method according to claim 7, characterized in that the global feature of the image to be processed is similarly compared with the global feature of the image in the designated storage area, and the local feature of the image to be processed is compared After the feature map is similarly compared with the local feature map of the image in the designated storage area, it also includes:

Taking the global feature of the image to be processed as the first global feature, and taking the local feature map of the image to be processed as the first local feature map;

Taking the global feature of the image currently being similarly compared with the to-be-processed image in the designated storage area as the second global feature, and taking the local part of the image currently being similarly compared with the to-be-processed image in the designated storage area The feature map is used as the second local feature map;

If the distance between the first global feature and the second global feature is less than a first distance threshold, determining that the first global feature and the second global feature satisfy a first similarity condition;

Calculate the feature distance between the corresponding first local feature and the second local feature, the first local feature is the local feature included in the first local feature map, and the second local feature is the second local feature The local features included in the feature map;

Taking the first local feature whose corresponding feature distance is less than the second distance threshold as the target local feature;

If the ratio of the number of target local features to the number of local features included in the first local feature map is greater than a scale threshold, it is determined that the first local feature map and the second local feature map satisfy a second similarity condition.
The method according to any one of claims 6-8, wherein, based on the global feature and local feature map of the to-be-processed image, searching for an image similar to the to-be-processed image in a designated storage area, and obtaining Similar images also include:

Input the image to be processed into the target neural network, and obtain the feature map output by the target neural network as the initial feature map;

Performing global average pooling on the initial feature map to obtain the global feature;

Obtaining feature sets corresponding to multiple positions in the initial feature map, where the feature sets corresponding to each position include features of each position and features of adjacent positions of each position;

Average processing is performed on the respective feature sets corresponding to each of the positions to obtain local features corresponding to each of the positions;

The local feature map is obtained based on the local features corresponding to each of the positions.
The method according to any one of claims 1-9, wherein the designated storage area is a storage area in a server, and the search and the to-be-processed image are searched in the designated storage area based on the characteristics of the to-be-processed image. Images with similar images, obtaining similar images also includes:

If the electronic device detects that the currently remaining available processing resources meet the resource preset conditions, the characteristics of the to-be-processed image are acquired locally;

sending a search instruction carrying the characteristics of the image to be processed to the server, so that the server searches for an image similar to the image to be processed in the designated storage area according to the characteristics in the search instruction, get similar images.
The method according to claim 10, wherein the resource preset condition comprises that the remaining available processing resources are greater than a specified resource threshold, wherein the processing resources are CPU availability.
The method according to any one of claims 1-9, wherein the designated storage area is a storage area in a server, and the search is performed in the designated storage area based on the characteristics of the to-be-processed image in the designated storage area. For images similar to the images to be processed, obtaining similar images also includes:

If the electronic device detects that the current network resource satisfies the specified network condition, the to-be-processed image is added to the search instruction;

Send the image to be processed to the server through the search instruction, so that the server first obtains the feature of the image to be processed, and then searches the specified storage area based on the feature of the image to be processed Similar images to the images to be processed are obtained.
The method according to claim 12, wherein the specified network condition comprises at least one of the following conditions: the Internet is currently accessed based on WIFI communication; and the network transmission rate satisfies the specified efficiency.
The method according to any one of claims 10-13, wherein the method further comprises:

Trigger to detect whether resource threshold conditions and specified network conditions are met at the same time;

If the detection result for the resource threshold condition is obtained first, and it is detected that the resource threshold condition is satisfied, execute a search instruction that carries the feature of the image to be processed to the server;

If the detection result for the specified network condition is obtained first, and it is detected that the specified network condition is satisfied, then the image to be processed is sent to the server through the search instruction.
The method according to claim 14, wherein the triggering, after simultaneously detecting whether the resource threshold condition and the specified network condition are satisfied, further comprises:

If it is detected that neither the resource threshold condition nor the specified network condition is met, the image processing parameters obtained randomly are used to process the image to be processed based on the randomly obtained image processing parameters.
An image processing method, characterized in that, applied to a server, the method comprising:

Receive images to be processed sent by electronic devices;

Search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter;

obtaining image processing parameters corresponding to the similar images;

Perform image processing on the to-be-processed image based on the image processing parameters to obtain a processed image;

The processed image is returned to the electronic device.
An image processing device, characterized in that it runs on electronic equipment, and the device includes:

an image acquisition unit, used to acquire an image to be processed;

an image search unit, configured to search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter;

a parameter obtaining unit, used for obtaining image processing parameters corresponding to the similar images;

An image processing unit, configured to perform image processing on the to-be-processed image based on the image processing parameter.
An image processing device, characterized in that it runs on a server, and the device includes:

A data communication unit, used for receiving the image to be processed sent by the electronic device;

an image search unit, configured to search for an image similar to the image to be processed in a designated storage area based on the feature of the image to be processed, to obtain a similar image, wherein the image in the designated storage area corresponds to an image processing parameter;

a parameter obtaining unit, used for obtaining image processing parameters corresponding to the similar images;

an image processing unit, configured to perform image processing on the to-be-processed image based on the image processing parameter to obtain a processed image;

The data communication unit is further configured to return the processed image to the electronic device.
An electronic device, comprising a processor and a memory;

One or more programs are stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-15.
A computer-readable storage medium, characterized in that a program code is stored in the computer-readable storage medium, wherein the method of any one of claims 1-15 is executed when the program code is executed by a processor.