CN114898122A

CN114898122A - Image processing method, storage medium, and computer terminal

Info

Publication number: CN114898122A
Application number: CN202210504218.5A
Authority: CN
Inventors: 纪德益; 赵一儒; 陶明渊; 黄建强; 华先胜
Original assignee: Alibaba Damo Institute Hangzhou Technology Co Ltd
Current assignee: Hangzhou Alibaba Cloud Feitian Information Technology Co ltd
Priority date: 2022-05-10
Filing date: 2022-05-10
Publication date: 2022-08-12

Abstract

The invention discloses an image processing method, a storage medium and a computer terminal. Wherein, the method comprises the following steps: acquiring a target image; and processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by extracting the feature of the training image through the first processing model, the second texture feature is obtained by extracting the feature of the training image through the second processing model, and the second processing model is a trained processing model. The invention solves the technical problem that the accuracy of a processing result obtained by processing an image by using a processing model is lower under the condition of identifying or detecting the image in the related technology.

Description

Image processing method, storage medium, and computer terminal

Technical Field

The present invention relates to the field of image processing, and in particular, to an image processing method, a storage medium, and a computer terminal.

Background

At present, in the field of image processing, a large model is difficult to deploy due to overlarge parameter quantity and excessive resource consumption, and therefore knowledge distillation from the large model to a small model is mostly adopted to optimize and improve the performance of the small model. In the existing method, high-level deep features (high-level deep features) information is mostly adopted as knowledge to perform distillation training on the small model. The high-level depth features generally originate from a relatively large receptive field and lack many local details, which causes the problem that edges are not clear in processing results obtained when images are identified, detected and the like.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the invention provides an image processing method, a storage medium and a computer terminal, which are used for at least solving the technical problem that the accuracy of a processing result obtained by processing an image by using a processing model is lower under the condition of identifying or detecting the image in the related art.

According to an aspect of an embodiment of the present invention, there is provided an image processing method including: acquiring a target image; and processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

According to an aspect of an embodiment of the present invention, there is provided a model training method, including: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

According to an aspect of an embodiment of the present invention, there is provided an image processing method including: the cloud server acquires a target image; the cloud server processes the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

According to an aspect of an embodiment of the present invention, there is provided an image processing method including: displaying a target image on an interactive interface; the method comprises the steps that under the condition that target touch operation is induced in an interactive interface, a processing result corresponding to a target image is displayed in the interactive interface, wherein the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained by extracting features of a training image through the first processing model, the second texture features are obtained by extracting features of the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

According to an aspect of the embodiments of the present invention, there is provided a storage medium including a stored program, wherein when the program is executed, a device on which the storage medium is located is controlled to perform an image processing method according to any one of claims and/or a model training method.

A processor according to an embodiment of the invention is configured to run a program, wherein the program performs an image processing method according to any one of the claims and/or a model training method during the running.

In the embodiment of the invention, a target image is obtained firstly; and then, processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by extracting the features of the training image through the first processing model, the second texture feature is obtained by extracting the features of the training image through the second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models, so that the accuracy of processing the image is improved. It is easy to notice that in the process of obtaining the first processing model through training, the trained processing model can be used for supervising the training process of the first processing model, extraction of texture knowledge in the training process is more comprehensive through extraction of texture features, so that the first processing model with higher accuracy can be obtained, a target image is processed through the first processing model, and the technical problem that in the related technology, under the condition of identifying or detecting the image, the accuracy of a processing result obtained by processing the image through the processing model is lower is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

fig. 1 shows a block diagram of a hardware configuration of a computer terminal (or mobile device) for implementing an image processing method;

FIG. 2 is a flow chart of a method of image processing according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a profile wave decomposition module according to an embodiment of the invention;

FIG. 4 is a block diagram of a noise reduction based texture intensity equalization module according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a training student network according to an embodiment of the invention;

FIG. 6 is a flow chart of a method of model training according to an embodiment of the present invention;

FIG. 7 is a flow diagram of another image processing method according to an embodiment of the invention;

FIG. 8 is a flow chart of another image processing method according to an embodiment of the present invention;

fig. 9 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of a model training apparatus according to an embodiment of the present invention;

fig. 11 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention;

FIG. 13 is a block diagram of a computer terminal according to an embodiment of the present invention;

FIG. 14 is a flow chart of a method of image rendering according to an embodiment of the present application;

FIG. 15 is a diagram of a hardware environment for implementing an image rendering method according to an embodiment of the present application;

FIG. 16 is a schematic diagram of another hardware environment for implementing an image rendering method according to an embodiment of the present application.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms or terms appearing in the description of the embodiments of the present invention are applicable to the following explanations:

texture is a description mode of the digital image processing field about an image, and the structured texture generally describes local features of the image, including image boundaries, fuzziness and the like; the statistical texture generally describes the distribution change of the whole image, such as a light and shade histogram.

Semantic segmentation (semantic segmentation): the class of each pixel of the input image is predicted.

Knowledge distillation (knowledge distillation): and (4) knowledge distillation, which guides and promotes the training process of the small model by using the knowledge of the large model. Where knowledge is generally derived from characteristic information at some stage of the large model. The distillation process utilizes feature computation loss functions to enhance the learning of supervised small models.

Example 1

There is also provided, in accordance with an embodiment of the present invention, an image processing method embodiment, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.

The method provided by the first embodiment of the present invention may be executed in a mobile terminal, a computer terminal, or a similar computing device. Fig. 1 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing an image processing method. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more processors (shown as 102a, 102b, … …, 102n in the figures) which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the invention, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the image processing method in the embodiment of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the above-mentioned image processing method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

It should be noted here that in some alternative embodiments, the computer device (or mobile device) shown in fig. 1 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that fig. 1 is only one example of a particular specific example and is intended to illustrate the types of components that may be present in the computer device (or mobile device) described above.

Under the above operating environment, the present application provides an image processing method as shown in fig. 2. Fig. 2 is a flowchart of an image processing method according to an embodiment of the present invention.

Step S202, a target image is acquired.

The target image may be an image to be subjected to semantic segmentation, and the semantic segmentation is performed on the target image.

In an alternative embodiment, the target image may be acquired, and then semantic segmentation is performed on the target image to obtain the category of the target image.

And step S204, processing the target image by using the first processing model to obtain a processing result.

The first processing model is obtained through training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained through feature extraction of a training image through the first processing model, the second texture features are obtained through feature extraction of the training image through the second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

The first processing model may be a convolutional neural network-based semantic segmentation small model (PSPNet small model of ResNet 18), and the second processing model may be a convolutional neural network-based semantic segmentation large model (PSPNet large model of ResNet 101) that has been trained in advance.

The first texture feature and the second texture feature may include a structured texture feature and a statistical texture feature, where the structured texture feature may be used to describe local information details of a boundary, sharpness, ambiguity, and the like of a target image, and the statistical texture feature may be used to describe overall brightness and color change information of the image.

In an optional embodiment, in the process of training the first processing model, feature extraction may be performed on the training image through the first processing model to obtain a structured texture feature and a statistical texture feature corresponding to the first texture feature, and feature extraction may be performed on the training image by using the second processing model to obtain a structured texture feature and a statistical texture feature corresponding to the second texture feature, and a target loss function may be constructed according to the structured texture feature and the statistical texture feature corresponding to the first texture feature and the structured texture feature and the statistical texture feature corresponding to the second texture feature, so as to adjust a model parameter of the first processing model according to the target loss function, and obtain the trained first processing model.

In another alternative embodiment, the first processing model may be used to perform semantic segmentation on the target image to obtain a semantic segmentation result, and the category of the target image may be identified according to the semantic segmentation result.

In the embodiment of the invention, a target image is obtained firstly; and then, processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by extracting the feature of the training image through the first processing model, the second texture feature is obtained by extracting the feature of the training image through the second processing model, and the second processing model is a trained processing model, so that the accuracy of processing the image is improved. It is easy to notice that in the process of obtaining the first processing model through training, the trained processing model can be used for supervising the training process of the first processing model, extraction of texture knowledge in the training process is more comprehensive through extraction of texture features, so that the first processing model with higher accuracy can be obtained, a target image is processed through the first processing model, and the technical problem that in the related technology, under the condition of identifying or detecting the image, the accuracy of a processing result obtained by processing the image through the processing model is lower is solved.

In the above embodiment of the present application, the method further includes: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

The objective loss function described above may be a cross entropy loss function.

In an alternative embodiment, the first processing model and the second processing model may respectively extract the structured texture features and the statistical texture features of the training image using a contourlet decomposition module and a noise reduction-based texture intensity equalization module, respectively calculate the loss of the first processing model and the second processing model with respect to the knowledge of the two textures by using a mean-square loss function, and jointly train the first processing model by combining with a conventional cross entropy loss function, during which process, the model parameters of the second processing model are unchanged. When the training effect is tested, only the training effect of the first processing model is tested, and the model parameters of the first processing model can be further updated according to the test result, so that the accuracy of the first processing model is improved.

In the above embodiments of the present application, the first texture feature includes at least one of: the first structured texture feature and the first statistical texture feature are used for extracting features of the training image by using a first processing model to obtain a first texture feature, wherein the first texture feature comprises at least one of the following characteristics: performing feature extraction on the training image by using a first feature extraction module in the first processing model to obtain a first structured textural feature, wherein the first feature extraction module is used for extracting local textural features related to contour information in the training image; and performing feature extraction on the training image by using a second feature extraction module in the first processing model to obtain a first statistical texture feature, wherein the second feature extraction module is used for extracting the global texture feature related to the display parameter in the training image.

The display parameters are the overall brightness and color change information of the image.

In the above embodiments of the present application, the first feature extraction module is a contour wave decomposition module, and the second feature extraction module is an intensity equalization module, where the intensity equalization module may be a noise reduction texture based intensity equalization module.

The first structural texture features are extracted by the first processing model and used for describing local detail information of image boundaries, definition, blur and the like of the training image. The first statistical texture feature is a feature extracted by the first processing model and used for describing global information such as overall brightness and color change information of the training image.

The first feature extraction Module is a Contourlet Decomposition Module (CDM for short), wherein the Contourlet Decomposition Module uses Contourlet changes in conventional digital image processing, and as shown in fig. 3, is a schematic diagram of the Contourlet Decomposition Module, first performing feature extraction on a training image to obtain an initial feature, then filtering the initial feature by a Laplacian Pyramid (LP for short) to obtain a high-pass sub-band (high-pass) and a low-pass sub-band (low-pass sub-band), performing Directional filtering on the high-pass sub-band (Directional filter) to obtain a band-pass Directional sub-band (band-pass Directional sub-band), and performing downsampling on the low-pass sub-band by 2 ds, 2) for the low-pass sub-band, so that a sampling result is input 1/2, which can reduce a resolution of the low-pass part band, and obtaining the intermediate features, continuing to filter the intermediate features to obtain band-pass directional subbands, and obtaining the first structural texture features according to the obtained multiple band-pass directional subbands, wherein the band-pass directional subbands are local texture features used for describing contour information in the image.

The second feature extraction Module is a de-noising Texture Intensity Equalization Module (DTIEM), wherein the DTIEM is similar to histogram Equalization in digital image processing, and as shown in fig. 4, the second feature extraction Module is a schematic diagram based on the de-noising Texture Intensity Equalization Module, and can quantize input features into a discrete space, then perform statistics, and finally use a graph convolution network for further iterative upgrade, optionally, an importance sampling algorithm can be used for sampling an image, a quantization series can be selected in a self-adaptive manner, and then a peak in the histogram is suppressed, so as to improve the accuracy of the obtained structured Texture features.

In an alternative embodiment, a first feature extraction module in the first processing model may be used to extract a local texture feature related to the contour information in the training image, so as to obtain a first structured texture feature, and the first texture feature may be determined to be the first structured texture feature.

In another alternative embodiment, the second extraction module in the first processing model may be used to extract the global texture features related to the display parameters in the training image to obtain the first statistical texture feature, and the first texture feature may be determined to be the first statistical texture feature.

In another alternative embodiment, the first texture feature may be determined to be the first structured texture feature and the first statistical texture feature.

In the above embodiments of the present application, the second texture feature includes at least one of: and the second structured texture feature and the second statistical texture feature utilize a second processing model to perform feature extraction on the training image to obtain a second texture feature, and the method comprises the following steps: performing feature extraction on the training image by using a first feature extraction module in the second processing model to obtain a second structured texture feature; and performing feature extraction on the training image by using a second feature extraction module in the second processing model to obtain a second statistical texture feature.

In an alternative embodiment, a second feature extraction module in the second processing model may be used to extract local texture features related to the contour information in the training image to obtain a second structured texture feature, and the second texture feature may be determined to be the second structured texture feature.

In another alternative embodiment, a second extraction module in the second processing model may be used to extract the global texture features related to the display parameters in the training image to obtain the first statistical texture feature, and the second texture feature may be determined to be the second statistical texture feature.

In another alternative embodiment, the second texture feature may be determined to be a second structured texture feature and a second statistical texture feature.

In the above embodiments of the present application, the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: under the condition of a second structured texture feature and a second statistical texture feature, constructing an objective loss function based on the first texture feature and the second texture feature, wherein the method comprises the following steps: determining a first loss function based on the first structured textural features and the second structured textural features; determining a second loss function based on the first statistical texture feature and the second statistical texture feature; based on the first loss function and the second loss function, a target loss function is constructed.

The first and second loss functions described above may be mean square loss functions.

In an alternative embodiment, a first loss function may be constructed according to the first structured texture feature and the second structured texture feature, where the first loss function may be used to represent a part of the first processing model that needs to be learned to the second processing model in processing the local information related to the contour, and a second loss function may be constructed according to the first statistical texture feature and the second statistical texture feature, where the second loss function may be used to represent a part of the second processing model that needs to be learned to the second processing model in processing the global texture feature related to the display parameter. The cross entropy loss function, that is, the above-mentioned target loss function, may be constructed according to the first loss function and the second loss function, so as to update the model parameters of the first processing model according to the target loss function, so that the processing precision of the first processing model is higher.

In the above embodiments of the present application, the feature extraction is performed on the training image by using a second feature extraction module to obtain a first statistical texture feature or a second statistical texture feature, including: sampling the training images by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model; and performing feature extraction on the first image to obtain a first statistical texture feature, or performing feature extraction on the second image to obtain a second statistical texture feature.

The above target sampling algorithm may be an importance-based sampling algorithm based on an object-based box (anchor-based).

In an optional embodiment, the first processing model may sample the training image by using a target sampling algorithm to obtain a first image, and the second processing model may sample the training image by using a target sampling algorithm to obtain a second image, so as to improve the sampling precision of the training image, reduce the amount of computation, and improve the processing precision of the subsequently obtained processing result.

In the above embodiments of the present application, performing feature extraction on the first image to obtain a first statistical texture feature, or performing feature extraction on the second image to obtain a second statistical texture feature includes: quantizing the first image based on the target quantization series number to obtain a third image corresponding to the first image; extracting the features of the third image to obtain a first statistical texture; or, quantizing the second image based on the target quantization progression to obtain a fourth image corresponding to the second image; and performing feature extraction on the fourth image to obtain a second statistical texture feature.

The target quantization technique stage number described above may not be a uniformly distributed stage number.

In an optional embodiment, the first image may be quantized according to a adaptively selected target quantization level of the training image to obtain a third image corresponding to the first image, so as to reduce the occurrence of an unbalanced phenomenon in the third image, and the first statistical texture with higher accuracy may be obtained by performing feature extraction on the third image. Or, the second image may be quantized according to a quantization level number of a target selected by the training image in a self-adaptive manner to obtain a fourth image corresponding to the second image, so as to reduce the occurrence of an unbalanced phenomenon in the fourth image, and a second statistical texture with higher accuracy may be obtained by performing feature extraction on the fourth image.

In the above embodiments of the present application, the target noise in the first image is suppressed during quantization of the first image based on the target quantization level number, or the target noise in the second image is suppressed during quantization of the second image based on the target quantization technique.

The target noise described above may be caused by spikes.

In an alternative embodiment, in the process of quantizing the first image based on the target quantization level, a spike in the first image may be suppressed, so as to reduce target noise in the first image and improve accuracy of a quantization result. The peak in the second image can be suppressed in the process of quantizing the second image based on the target quantization level number, so that the target noise in the second image is reduced, and the accuracy of the quantization result is improved.

As shown in fig. 5, a schematic diagram of a training student network is shown, first, a training image is input into a student network and a teacher network, wherein the student network is the first processing model mentioned above, the teacher network is the second processing model mentioned above, the teacher network is trained first to make the teacher network achieve a good performance, then distillation is started, optionally, the training image can be input into the teacher network and the student network respectively, wherein the teacher network and the student network respectively use CDM and DTIEM to extract structured texture features and statistical texture features to obtain first texture features corresponding to the student network and second texture features corresponding to the teacher network, and loss of the teacher network and the student network about the structured texture features and the statistical texture features is calculated respectively according to the first texture features and the second texture features by using a mean square loss function, the student network can be trained by combining a traditional cross entropy loss function, in the process, the network parameters of the teacher network are unchanged, and after the training is finished, the performance of the student network can be tested, so that the student network with higher accuracy can be obtained. According to the scheme, the extraction of texture knowledge in the distillation process can be more comprehensive and fine, and the performance of a student network is improved.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Example 2

There is also provided, in accordance with an embodiment of the present invention, an embodiment of a model training method, to note that the flow chart of the accompanying figures illustrates steps that may be performed, for example, in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flow chart, in some cases the steps illustrated or described may be performed in a different order herein.

FIG. 6 is a flow chart of a model training method according to an embodiment of the present invention, as shown in FIG. 6, the method may include the following steps:

step S602, acquiring a training image;

step S604, feature extraction is carried out on the training image by respectively utilizing a first processing model and a second processing model to obtain a first texture feature and a second texture feature, wherein the first processing model and the second processing model are machine learning models;

step S606, constructing a target loss function based on the first texture feature and the second texture feature;

step S608, updating the model parameters of the first processing model by using the target loss function, and obtaining an update result.

In the above embodiments of the present application, the first texture feature includes at least one of: the first structured texture feature and the first statistical texture feature are used for extracting features of the training image by using a first processing model to obtain a first texture feature, wherein the first texture feature comprises at least one of the following characteristics: performing feature extraction on the training image by using a first feature extraction module in the first processing model to obtain a first structured texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to the contour information in the training image; and performing feature extraction on the training image by using a second feature extraction module in the first processing model to obtain a first statistical texture feature, wherein the second feature extraction module is used for extracting the global texture feature related to the display parameter in the training image.

In the above embodiments of the present application, the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: under the condition of the second structured textural feature and the second statistical textural feature, constructing an object loss function based on the first textural feature and the second textural feature, comprising: determining a first loss function based on the first structured textural features and the second structured textural features; determining a second loss function based on the first statistical texture feature and the second statistical texture feature; based on the first loss function and the second loss function, a target loss function is constructed.

In the above embodiments of the present application, the target noise in the first image is suppressed during the quantization of the first image based on the target quantization level, or the target noise in the second image is suppressed during the quantization of the second image based on the target quantization technique.

In the above embodiments of the present application, the first feature extraction module is a contour wave decomposition module, and the second feature extraction module is an intensity equalization module.

It should be noted that the preferred embodiments described in the above examples of the present application are the same as the schemes, application scenarios, and implementation procedures provided in example 1, but are not limited to the schemes provided in example 1.

Example 3

There is also provided, in accordance with an embodiment of the present invention, an embodiment of an image processing method, to note that the flowcharts of the figures illustrate steps that may be performed, for example, in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in a different order than presented herein.

Fig. 7 is a flowchart of an image processing method according to an embodiment of the present invention, and as shown in fig. 7, the method may include the steps of:

step S702, a cloud server acquires a target image;

step S704, the cloud server processes the target image by using a first processing model to obtain a processing result, where the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

In an alternative embodiment, the client may provide a target image, the cloud server may obtain the target image provided by the client, and then the cloud server may process the target image by using a first processing model to obtain a processing result, where the first processing model may be obtained by training a third-party server, the third-party server may obtain a first texture feature by performing feature extraction on the training image through the first processing model, the third-party server may obtain a second texture feature by performing feature extraction on the training image through the second processing model, the third-party server may construct a target loss function of the first texture feature and the second texture feature, train the first processing model according to the target loss function, upload the finally trained first processing model to the cloud server, so that the cloud server processes the target image by using the first processing model, and obtaining a processing result.

In the above embodiment of the present application, the method further includes: the cloud server acquires a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; the cloud server constructs a target loss function based on the first texture feature and the second texture feature; and the cloud server updates the model parameters of the first processing model by using the target loss function to obtain an update result.

In the above embodiments of the present application, the first texture feature includes at least one of: the cloud server performs feature extraction on the training image by using a first processing model to obtain a first texture feature, wherein the first texture feature comprises at least one of the following characteristics: the cloud server utilizes a first feature extraction module in the first processing model to perform feature extraction on the training image to obtain a first structural texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to the contour information in the training image; the cloud server utilizes a second feature extraction module in the first processing model to extract features of the training image to obtain first statistical texture features, wherein the second feature extraction module is used for extracting global texture features related to the display parameters in the training image.

In the above embodiments of the present application, the second texture feature includes at least one of: the second structured texture feature and the second statistical texture feature, the cloud server utilizes the second processing model to perform feature extraction on the training image to obtain a second texture feature, and the method comprises the following steps: the cloud server utilizes a first feature extraction module in the second processing model to extract features of the training image to obtain second structured texture features; and the cloud server utilizes a second feature extraction module in the second processing model to extract features of the training image to obtain a second statistical texture feature.

In the above embodiments of the present application, the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: under the condition of the second structured texture feature and the second statistical texture feature, the cloud server constructs an objective loss function based on the first texture feature and the second texture feature, and the method comprises the following steps: the cloud server determines a first loss function based on the first structured textural features and the second structured textural features; the cloud server determines a second loss function based on the first statistical texture feature and the second statistical texture feature; the cloud server constructs a target loss function based on the first loss function and the second loss function.

In the above embodiment of the present application, the cloud server utilizes the second feature extraction module to perform feature extraction on the training image, to obtain the first statistical texture feature or the second statistical texture feature, including: the cloud server samples the training images by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model; and the cloud server extracts the features of the first image to obtain a first statistical textural feature, or extracts the features of the second image to obtain a second statistical textural feature.

In the above embodiment of the present application, the cloud server performs feature extraction on the first image to obtain a first statistical texture feature, or performs feature extraction on the second image to obtain a second statistical texture feature, including: the cloud server quantizes the first image based on the target quantization progression to obtain a third image corresponding to the first image; the cloud server extracts the features of the third image to obtain a first statistical texture; or the cloud server quantizes the second image based on the target quantization progression to obtain a fourth image corresponding to the second image; and the cloud server extracts the features of the fourth image to obtain a second statistical texture feature.

In the above embodiments of the present application, the cloud server suppresses the target noise in the first image in the process of quantizing the first image based on the target quantization level, or the cloud server suppresses the target noise in the second image in the process of quantizing the second image based on the target quantization technology.

Example 4

There is also provided, in accordance with an embodiment of the present invention, an embodiment of an image processing method, it being noted that the flow charts in the accompanying drawings illustrate that the steps may be implemented in a computer system such as a set of computer-executable instructions and that, although a logical order is depicted in the flow charts, in some cases, the steps shown or described may be performed in a different order than here.

Fig. 8 is a flowchart of an image processing method according to an embodiment of the present invention, and as shown in fig. 8, the method may include the steps of:

step S802, displaying a target image on an interactive interface;

step S804, displaying a processing result corresponding to the target image in the interactive interface under the condition that the target touch operation is sensed in the interactive interface, where the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by extracting features of the training image through the first processing model, the second texture feature is obtained by extracting features of the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

The target touch operation may be an operation of performing touch on the interactive interface by a user, where the target touch operation may be used to process a target image to obtain a processing result, and display the processing result in the interactive interface for the user to view.

In the above embodiment of the present application, the method further includes: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing an objective loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

Example 5

According to an embodiment of the present invention, there is also provided an image processing apparatus for implementing the above-described image processing method, and fig. 9 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention, as shown in fig. 9, the apparatus including: an acquisition module 902 and a processing module 904.

The acquisition module is used for acquiring a target image; the processing module is used for processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

Here, it should be noted that the obtaining module 902 and the processing module 904 correspond to step S202 to step S204 in embodiment 1, and the implementation examples and application scenarios of the two modules and the corresponding steps are the same, but are not limited to what is disclosed in embodiment 1 above, and it should be noted that the modules described above may be operated in the computer terminal 10 provided in embodiment 1 as a part of the tool.

In the above embodiment of the present application, the method further includes: the device comprises an extraction module, a construction module and an updating module.

The acquisition module is further used for acquiring a training image; the extraction module is used for extracting the features of the training image by respectively utilizing the first processing model and the second processing model to obtain a first texture feature and a second texture feature; the construction module is used for constructing a target loss function based on the first texture feature and the second texture feature; the updating module is used for updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

In the above embodiments of the present application, the first texture feature includes at least one of: the first structured textural features and the first statistical textural features, the extraction module comprising: the device comprises a first extraction unit and a second extraction unit.

The first extraction unit is used for extracting features of a training image by using a first feature extraction module in a first processing model to obtain a first structural texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to contour information in the training image; the second extraction unit is used for extracting features of the training image by using a second feature extraction module in the first processing model to obtain a first statistical texture feature, wherein the second feature extraction module is used for extracting a global texture feature related to the display parameter in the training image.

In the above embodiments of the present application, the second texture feature includes at least one of: the first extraction unit is used for extracting the features of the training image by using a first feature extraction module in the second processing model to obtain second structured texture features; the second extraction unit is used for extracting the features of the training image by using a second feature extraction module in the second processing model to obtain a second statistical texture feature.

In the above embodiments of the present application, the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: building a module under the condition of the second structured texture features and the second statistical texture features, including: a determining unit and a constructing unit.

Wherein the determining unit is configured to determine a first loss function based on the first structured textural features and the second structured textural features; the determining unit is further configured to determine a second loss function based on the first statistical texture feature and the second statistical texture feature; the construction unit is used for constructing a target loss function based on the first loss function and the second loss function.

In the above embodiment of the present application, the second extraction unit includes: a sampling subunit and a feature extraction subunit.

The sampling subunit is used for sampling the training image by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model; the feature extraction subunit is configured to perform feature extraction on the first image to obtain a first statistical texture feature, or perform feature extraction on the second image to obtain a second statistical texture feature.

In the above embodiment of the present application, the feature extraction subunit is further configured to quantize the first image based on the target quantization level number, so as to obtain a third image corresponding to the first image; the feature extraction subunit is further configured to perform feature extraction on the third image to obtain a first statistical texture; or, the feature extraction subunit is further configured to quantize the second image based on the target quantization progression to obtain a fourth image corresponding to the second image; the feature extraction subunit is further configured to perform feature extraction on the fourth image to obtain a second statistical texture feature.

In the above embodiment of the present application, the feature extraction subunit is further configured to suppress the target noise in the first image during the quantization of the first image based on the target quantization level number, or suppress the target noise in the second image during the quantization of the second image based on the target quantization technology.

The preferred embodiment of the present application in the above example 1 is the same as the scheme, application scenario and implementation process provided in example 1, but is not limited to the scheme provided in example 1.

Example 6

According to an embodiment of the present invention, there is also provided a model training apparatus for implementing the above model training method, and fig. 10 is a schematic diagram of a model training apparatus according to an embodiment of the present invention, as shown in fig. 10, the apparatus includes: an acquisition module 1002, a processing module 1004, a construction module 1006, and an update module 1008.

The acquisition module is used for acquiring a training image; the processing module is used for extracting features of the training image by using the first processing model and the second processing model respectively to obtain a first texture feature and a second texture feature; a construction module 1006 is configured to construct an objective loss function based on the first texture feature and the second texture feature; the updating module is used for updating the model parameters of the first processing model by using the target loss function to obtain an updating result, and the first processing model and the second processing model are machine learning models.

It should be noted here that the acquiring module 1002, the processing module 1004, the constructing module 1006, and the updating module 1008 are corresponding to steps S602 to S608 in embodiment 2, and the four modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 2. It should be noted that the modules described above may be operated in the computer terminal 10 provided in embodiment 1 as a part of the tool.

Example 7

According to an embodiment of the present invention, there is also provided an image processing apparatus for implementing the above-described image processing method, and fig. 11 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention, as shown in fig. 11, the apparatus including: an acquisition module 1102 and a processing module 1104.

The acquisition module is used for the cloud server to acquire a target image; the processing module is used for processing a target image by the cloud server through a first processing model to obtain a processing result, wherein the first processing model is obtained through training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained through feature extraction of the training image through the first processing model, the second texture feature is obtained through feature extraction of the training image through the second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

It should be noted here that the above-mentioned acquiring module 1102 and processing module 1104 correspond to steps S702 to S704 in embodiment 3, and the two modules are the same as the example and application scenarios realized by the corresponding steps, but are not limited to the disclosure of embodiment 2. It should be noted that the modules described above may be operated in the computer terminal 10 provided in embodiment 1 as a part of the tool.

Example 8

According to an embodiment of the present invention, there is also provided an image processing apparatus for implementing the above-described image processing method, and fig. 12 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention, as shown in fig. 12, the apparatus including: a display module 1202 and a processing module 1204.

The display module is used for displaying a target image on the interactive interface; the processing module is used for displaying a processing result corresponding to a target image in the interactive interface under the condition that target touch operation is induced in the interactive interface, wherein the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained by extracting features of a training image through the first processing model, the second texture features are obtained by extracting features of the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

It should be noted that the display module 1202 and the processing module 1204 described above correspond to steps S802 to S804 in embodiment 4, and the two modules are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the disclosure of embodiment 4. It should be noted that the modules described above may be operated in the computer terminal 10 provided in embodiment 1 as a part of the tool.

It should be noted that the preferred embodiment related to example 1 of the present application is the same as the scheme, application scenario and implementation process provided in example 1, but is not limited to the scheme provided in example 1.

Example 9

The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.

In this embodiment, the computer terminal may be located in program code of the following steps in the computer image processing method: acquiring a target image; and processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model.

Alternatively, fig. 13 is a block diagram of a computer terminal according to an embodiment of the present invention. As shown in fig. 13, the computer terminal a may include: one or more (only one shown) processors 102, memory 104.

The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the image processing method and apparatus in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the image processing method. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory remotely located from the processor, and these remote memories may be connected to terminal a through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring a target image; and processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model.

Optionally, the processor may further execute the program code of the following steps: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

Optionally, the processor may further execute the program code of the following steps: the first texture feature comprises at least one of: the first structured texture feature and the first statistical texture feature are used for extracting features of the training image by using a first processing model to obtain a first texture feature, wherein the first texture feature comprises at least one of the following characteristics: performing feature extraction on the training image by using a first feature extraction module in the first processing model to obtain a first structured texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to the contour information in the training image; and performing feature extraction on the training image by using a second feature extraction module in the first processing model to obtain a first statistical texture feature, wherein the second feature extraction module is used for extracting the global texture feature related to the display parameter in the training image.

Optionally, the processor may further execute the program code of the following steps: the second texture feature comprises at least one of: and the second structured texture feature and the second statistical texture feature utilize a second processing model to perform feature extraction on the training image to obtain a second texture feature, and the method comprises the following steps: performing feature extraction on the training image by using a first feature extraction module in the second processing model to obtain a second structured texture feature; and performing feature extraction on the training image by using a second feature extraction module in the second processing model to obtain a second statistical texture feature.

Optionally, the processor may further execute the program code of the following steps: the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: under the condition of a second structured texture feature and a second statistical texture feature, constructing an objective loss function based on the first texture feature and the second texture feature, wherein the method comprises the following steps: determining a first loss function based on the first structured textural features and the second structured textural features; determining a second loss function based on the first statistical texture feature and the second statistical texture feature; based on the first loss function and the second loss function, a target loss function is constructed.

Optionally, the processor may further execute the program code of the following steps: and performing feature extraction on the training image by using a second feature extraction module to obtain a first statistical texture feature or a second statistical texture feature, wherein the method comprises the following steps: sampling the training images by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model; and performing feature extraction on the first image to obtain a first statistical texture feature, or performing feature extraction on the second image to obtain a second statistical texture feature.

Optionally, the processor may further execute the program code of the following steps: quantizing the first image based on the target quantization series number to obtain a third image corresponding to the first image; extracting the features of the third image to obtain a first statistical texture; or, quantizing the second image based on the target quantization progression to obtain a fourth image corresponding to the second image; and performing feature extraction on the fourth image to obtain a second statistical texture feature.

Optionally, the processor may further execute the program code of the following steps: the target noise in the first image is suppressed during quantization of the first image based on a target quantization level or the target noise in the second image is suppressed during quantization of the second image based on a target quantization technique.

Optionally, the first feature extraction module is a contour wave decomposition module, and the second feature extraction module is an intensity equalization module.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: the cloud server acquires a target image; the cloud server processes the target image by using the first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: displaying a target image on an interactive interface; the method comprises the steps that under the condition that target touch operation is induced in an interactive interface, a processing result corresponding to a target image is displayed in the interactive interface, wherein the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained by extracting features of a training image through the first processing model, the second texture features are obtained by extracting features of the training image through a second processing model, and the second processing model is a trained processing model.

By adopting the embodiment of the invention, the target image is firstly obtained; and then, processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model, so that the accuracy of processing the image is improved. It is easy to notice that in the process of obtaining the first processing model through training, the trained processing model can be used for supervising the training process of the first processing model, extraction of texture knowledge in the training process is more comprehensive through extraction of texture features, so that the first processing model with higher accuracy can be obtained, a target image is processed through the first processing model, and the technical problem that in the related technology, under the condition of identifying or detecting the image, the accuracy of a processing result obtained by processing the image through the processing model is lower is solved.

Example 10

The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the image processing method provided in the first embodiment.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

It can be understood by those skilled in the art that the structure shown in fig. 13 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 13 is a diagram illustrating a structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 13, or have a different configuration than shown in FIG. 13.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a target image; and processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model.

Optionally, the storage medium is further configured to store program code for performing the following steps: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

Optionally, the storage medium is further configured to store program code for performing the following steps: the first texture feature comprises at least one of: the first structured texture feature and the first statistical texture feature are used for extracting features of the training image by using a first processing model to obtain a first texture feature, wherein the first texture feature comprises at least one of the following characteristics: performing feature extraction on the training image by using a first feature extraction module in the first processing model to obtain a first structured texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to the contour information in the training image; and performing feature extraction on the training image by using a second feature extraction module in the first processing model to obtain a first statistical texture feature, wherein the second feature extraction module is used for extracting the global texture feature related to the display parameter in the training image.

Optionally, the storage medium is further configured to store program code for performing the following steps: the second texture feature comprises at least one of: and the second structured texture feature and the second statistical texture feature utilize a second processing model to perform feature extraction on the training image to obtain a second texture feature, and the method comprises the following steps: performing feature extraction on the training image by using a first feature extraction module in the second processing model to obtain a second structured texture feature; and performing feature extraction on the training image by using a second feature extraction module in the second processing model to obtain a second statistical texture feature.

Optionally, the storage medium is further configured to store program code for performing the following steps: the first texture feature includes: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: under the condition of a second structured texture feature and a second statistical texture feature, constructing an objective loss function based on the first texture feature and the second texture feature, wherein the method comprises the following steps: determining a first loss function based on the first structured textural features and the second structured textural features; determining a second loss function based on the first statistical texture feature and the second statistical texture feature; based on the first loss function and the second loss function, a target loss function is constructed.

Optionally, the storage medium is further configured to store program code for performing the following steps: and performing feature extraction on the training image by using a second feature extraction module to obtain a first statistical texture feature or a second statistical texture feature, wherein the method comprises the following steps: sampling the training images by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model; and performing feature extraction on the first image to obtain a first statistical texture feature, or performing feature extraction on the second image to obtain a second statistical texture feature.

Optionally, the storage medium is further configured to store program code for performing the following steps: quantizing the first image based on the target quantization series number to obtain a third image corresponding to the first image; extracting the features of the third image to obtain a first statistical texture; or, quantizing the second image based on the target quantization level number to obtain a fourth image corresponding to the second image; and performing feature extraction on the fourth image to obtain a second statistical texture feature.

Optionally, the storage medium is further configured to store program code for performing the following steps: the target noise in the first image is suppressed during quantization of the first image based on a target quantization level or the target noise in the second image is suppressed during quantization of the second image based on a target quantization technique.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a training image; respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature; constructing a target loss function based on the first texture features and the second texture features; and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the cloud server acquires a target image; the cloud server processes the target image by using the first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through the second processing model, and the second processing model is a trained processing model.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: displaying a target image on an interactive interface; the method comprises the steps that under the condition that target touch operation is induced in an interactive interface, a processing result corresponding to a target image is displayed in the interactive interface, wherein the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained by extracting features of a training image through the first processing model, the second texture features are obtained by extracting features of the training image through a second processing model, and the second processing model is a trained processing model.

Example 11

According to an embodiment of the present invention, there is also provided an image rendering method, and fig. 14 is a flowchart of an image rendering method according to the present application, where the method includes:

step S1402, displaying a target image on a presentation screen of a virtual reality VR device or an augmented reality AR device;

step S1404, processing the target image by using the first processing model to obtain a processing result;

Step S1406, renders the processing result to a display screen of the virtual reality VR device or the augmented reality AR device.

The image rendering method can be applied to a hardware environment formed by the server 102 and the AR-VR device 104 as shown in fig. 15, and fig. 15 is a schematic diagram of the hardware environment of an image rendering method according to an embodiment of the present application. As shown in fig. 15, the server 102 is connected to the AR-VR device 104 through a network, which may be a server corresponding to a media file operator, including but not limited to: the AR-VR device 104 may be a virtual reality VR device or an augmented reality AR device, where the virtual reality VR device is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines and the like.

Optionally, the AR-VR device 104 includes: memory, processor, and transmission means. The memory is used for storing an application program, and the application program can be used for executing: displaying a target image on a presentation screen of the virtual reality VR device or the augmented reality AR device; processing a target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models; and rendering the processing result to a display screen of the virtual reality VR device or the augmented reality AR device.

The processor of this embodiment may invoke the application stored in the memory via the transmission device to perform the steps described above. The transmission device may receive the target image sent by the server through the network, and may also be used for data transmission between the processor and the memory.

Alternatively, in the AR-VR device 104, a Head Mounted Display (HMD) with eye tracking function, a screen in the HMD Head Display for displaying real-time pictures, an eye tracking module in the HMD for acquiring a real-time movement locus of the user's eyes, a tracking system for tracking the position information and movement information of the user in the real three-dimensional space, and a calculation processing unit for acquiring the real-time position and movement information of the user from the tracking system and calculating the three-dimensional coordinates of the user's Head in the virtual three-dimensional space, the orientation of the user's field of view in the virtual three-dimensional space, and the like are provided.

Fig. 16 is a schematic diagram of a hardware environment of another method for delivering a media file according to an embodiment of the present invention. As shown in fig. 16, the AR-VR device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 via a network, and the AR-VR device 104 is not limited to: the terminal 104 is not limited to a PC, a mobile phone, a tablet computer, etc., and the server 102 may be a server corresponding to a media file operator, where the network includes but is not limited to: a wide area network, a metropolitan area network, or a local area network.

Optionally, the AR-VR device 104 of this embodiment functions as in the above-described embodiment, and the terminal of this embodiment may be configured to perform: displaying a target image on a presentation screen of the virtual reality VR device or the augmented reality AR device; processing a target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on the training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models; and sends the processing result to the AR-VR device 104, which the AR-VR device 104 displays after receiving the processing result.

Optionally, the AR-VR device 104 of this embodiment has an eye tracking HMD display and an eye tracking module that function the same as those in the above-described embodiments, that is, a screen in the HMD display is used for displaying real-time images, and the eye tracking module in the HMD is used for obtaining a real-time movement track of the user's eyes. The terminal of the embodiment acquires the position information and the motion information of the user in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the head of the user in the virtual three-dimensional space and the visual field orientation of the user in the virtual three-dimensional space.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An image processing method, comprising:

acquiring a target image;

the method comprises the steps of processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on a training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

2. The method of claim 1, further comprising:

acquiring a training image;

respectively utilizing the first processing model and the second processing model to perform feature extraction on the training image to obtain the first texture feature and the second texture feature;

constructing the target loss function based on the first texture feature and the second texture feature;

and updating the model parameters of the first processing model by using the target loss function to obtain an updating result.

3. The method of claim 2, wherein the first texture feature comprises at least one of: the first structural texture feature and the first statistical texture feature are used for extracting features of the training image by using the first processing model to obtain the first texture feature, wherein the first texture feature comprises at least one of the following characteristics:

performing feature extraction on the training image by using a first feature extraction module in the first processing model to obtain the first structured texture feature, wherein the first feature extraction module is used for extracting a local texture feature related to contour information in the training image;

and performing feature extraction on the training image by using a second feature extraction module in the first processing model to obtain the first statistical texture feature, wherein the second feature extraction module is used for extracting the global texture feature related to the display parameter in the training image.

4. The method of claim 3, wherein the second texture feature comprises at least one of: the second structured texture feature and the second statistical texture feature are used for extracting the features of the training image by using the second processing model to obtain the second texture feature, and the method comprises the following steps:

performing feature extraction on the training image by using a first feature extraction module in the second processing model to obtain the second structured texture feature;

and performing feature extraction on the training image by using a second feature extraction module in the second processing model to obtain the second statistical texture feature.

5. The method of claim 2, wherein the first texture feature comprises: a first structured textural feature and a first statistical textural feature, the second textural feature comprising: in the case of a second structured textural feature and a second statistical textural feature, constructing an objective loss function based on the first textural feature and the second textural feature, comprising:

determining a first loss function based on the first structured textural features and the second structured textural features;

determining a second loss function based on the first statistical texture feature and the second statistical texture feature;

constructing the target loss function based on the first loss function and the second loss function.

6. The method of claim 4, wherein performing feature extraction on the training image by using the second feature extraction module to obtain the first statistical texture feature or the second statistical texture feature comprises:

sampling the training images by using a target sampling algorithm to obtain a first image corresponding to the first processing model or a second image corresponding to the second processing model;

and performing feature extraction on the first image to obtain the first statistical textural feature, or performing feature extraction on the second image to obtain the second statistical textural feature.

7. The method of claim 6, wherein performing feature extraction on the first image to obtain the first statistical texture feature, or performing feature extraction on the second image to obtain the second statistical texture feature comprises:

quantizing the first image based on a target quantization level number to obtain a third image corresponding to the first image;

extracting features of the third image to obtain the first statistical texture;

or the like, or, alternatively,

quantizing the second image based on the target quantization series number to obtain a fourth image corresponding to the second image;

and performing feature extraction on the fourth image to obtain the second statistical texture feature.

8. The method of claim 7, wherein the target noise in the first image is suppressed during quantization of the first image based on a target quantization level or the target noise in the second image is suppressed during quantization of the second image based on the target quantization technique.

9. A method of model training, comprising:

acquiring a training image;

respectively utilizing a first processing model and a second processing model to perform feature extraction on the training image to obtain a first texture feature and a second texture feature, wherein the first processing model and the second processing model are machine learning models;

constructing an objective loss function based on the first texture features and the second texture features;

10. An image processing method, comprising:

the cloud server acquires a target image;

the cloud server processes the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on a training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

11. An image processing method, comprising:

displaying a target image on an interactive interface;

the method comprises the steps that under the condition that target touch operation is induced in the interactive interface, a processing result corresponding to a target image is displayed in the interactive interface, wherein the processing result is obtained by processing the target image through a first processing model, the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on first texture features and second texture features, the first texture features are obtained by extracting features of a training image through the first processing model, the second texture features are obtained by extracting features of the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models.

12. An image rendering method, comprising:

displaying a target image on a presentation screen of the virtual reality VR device or the augmented reality AR device;

processing the target image by using a first processing model to obtain a processing result, wherein the first processing model is obtained by training based on a target loss function, the target loss function is constructed based on a first texture feature and a second texture feature, the first texture feature is obtained by performing feature extraction on a training image through the first processing model, the second texture feature is obtained by performing feature extraction on the training image through a second processing model, the second processing model is a trained processing model, and the first processing model and the second processing model are machine learning models;

rendering the processing result to a display screen of the virtual reality VR device or the augmented reality AR device.

13. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program is run, a device on which the storage medium is located is controlled to execute the image processing method and/or the model training method of any one of claims 1 to 12.

14. A computer terminal, comprising: a processor and a memory, the processor being configured to execute a program stored in the memory, wherein the program is configured to perform the image processing method of any of claims 1 to 12, and/or the model training method when executed.