CN117857928A

CN117857928A - Image processing method and electronic device

Info

Publication number: CN117857928A
Application number: CN202410218993.3A
Authority: CN
Inventors: 程志华; 王博
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2024-02-28
Filing date: 2024-02-28
Publication date: 2024-04-09
Anticipated expiration: 2044-02-28
Also published as: CN117857928B

Abstract

The embodiment of the application provides an image processing method and electronic equipment, wherein the method comprises the following steps: receiving a first operation of a camera application by a user; responding to a first operation, and acquiring a first image through a camera; under the condition that the shooting scene is determined to be a target scene, loading a target algorithm corresponding to the target scene, wherein the target algorithm comprises a target floating point model and a target fixed point model; if the richness of the texture details of the first region meets the preset condition, the first region is processed through the target floating point model to obtain a second region, and the first region is all or part of the first image; if the richness of the texture details of the first region does not meet the preset condition, the first region is processed through the target fixed-point model, and a second region is obtained; the second image is displayed based on the second region. The method can give consideration to algorithm instantaneity and graph effect.

Description

Image processing method and electronic device

Technical Field

The present application relates to the field of electronic technologies, and in particular, to an image processing method and an electronic device.

Background

Today, photographing functions of electronic devices such as mobile phones and tablet computers are very mature. In the photographing process of the electronic equipment, the method has high requirements on the real-time performance of the algorithm and the image effect of the image. However, algorithm real-time and graph effects are in most cases contradictory and conflicting. The selection of the algorithm model is one reason for contradiction between real-time performance and graph effect.

Specifically, the model may be classified into a floating point model and a fixed point model according to the difference in the form of operand values. The calculation accuracy of the floating point model is higher, the graph effect can be ensured, but the occupied memory is larger, and the reasoning time is longer. The fixed-point model occupies less memory, the reasoning speed is faster, but the precision is lower, and the plotting effect can be influenced.

Thus, how to select a model is a problem faced in image processing.

Disclosure of Invention

The application provides an image processing method and electronic equipment, which can give consideration to both algorithm resource performance and accuracy, and further give consideration to real-time performance and graph effect.

In a first aspect, the present application provides an image processing method, the method being performed by an electronic device, the electronic device including a camera, the method comprising: receiving a first operation of a camera application by a user; responding to a first operation, and acquiring a first image through a camera; under the condition that the shooting scene is determined to be a target scene, loading a target algorithm corresponding to the target scene, wherein the target algorithm comprises a target floating point model and a target fixed point model; if the richness of the texture details of the first region meets the preset condition, the first region is processed through the target floating point model to obtain a second region, and the first region is all or part of the first image; if the richness of the texture details of the first region does not meet the preset condition, the first region is processed through the target fixed-point model, and a second region is obtained; the second image is displayed based on the second region.

The richness of the texture details of the first area meets the preset condition, the texture details of the first area are characterized, the first area is processed through the target floating point model, the processing precision of the first area can be guaranteed, the texture details of the first area are better kept, and the drawing effect is guaranteed. The richness of the texture details of the first region does not meet the preset condition, the texture details representing the first region are not rich, namely the texture details are lack, and the first region is processed through the target fixed-point model. For images lacking in texture details, the precision loss caused by the quantization of the model is very small, and the influence on the images is very small, so that the first area is processed by the target fixed-point model, the influence on the precision of the first area is very small, the occupied memory can be reduced by the fixed-point model, the reasoning speed is greatly improved, and the waste of computing resources and the equipment performance are saved.

In summary, according to the method provided by the embodiment, different models are selected to process the image according to the difference of the abundance of the texture details, so that the processing precision can be ensured, the image drawing effect of the image is ensured, the reasoning speed is not reduced, the resource waste and the performance loss are not caused, the balance of the resource performance and the precision is realized, and the balance of the real-time performance and the image drawing effect is further realized.

In a possible implementation manner, the method further includes: performing Fourier transform on the first region to obtain a spectrogram; counting the duty ratio of a target frequency component in the spectrogram; the target frequency component is a frequency component greater than a preset frequency threshold; if the duty ratio of the target frequency component is larger than the preset duty ratio, determining that the richness of the texture details of the first area meets the preset condition; if the duty ratio of the target frequency component is smaller than or equal to the preset duty ratio, determining that the richness of the texture details of the first area does not meet the preset condition.

In the implementation mode, the spectrogram is obtained through Fourier transformation, the duty ratio of the high-frequency component is counted based on the spectrogram, the degree of richness of the texture details of the image is judged according to the duty ratio of the high-frequency component, the method is simple, the calculation result is accurate, and the accuracy of fixed-point model and floating-point model selection is further improved.

In a possible implementation manner, the statistics of the duty ratio of the target frequency component in the spectrogram includes: centering the spectrogram to obtain a centering spectrogram; based on the centering spectrogram, the duty ratio of the target frequency component is counted.

In the implementation mode, through the centering processing, points with lower frequency components in the spectrogram are moved to a central area, and points with higher frequency components are moved to four corner parts of the spectrogram. Therefore, frequency statistics is convenient, subsequent statistics of high-frequency components based on a centralized frequency spectrum is convenient, the algorithm operation process is simplified, and the algorithm operation efficiency is improved.

In a possible implementation manner, based on the centralized spectrogram, counting the duty ratio of the target frequency component includes: counting the amplitude sum of points corresponding to the target frequency components in the centralized spectrogram to obtain a first amplitude sum; counting the sum of the amplitudes of all points in the centralized spectrogram to obtain a second amplitude sum; and calculating the ratio of the first amplitude sum to the second amplitude sum to obtain the duty ratio of the target frequency component.

In the implementation mode, the target frequency component duty ratio is represented by the amplitude duty ratio of the point of the target frequency component, and the amplitude of the midpoint of the spectrogram is known and accurate, so that the target frequency component duty ratio can be conveniently and accurately determined without counting, and the calculation accuracy and calculation efficiency are improved.

In one possible implementation, the camera comprises a tele camera; acquiring a first image through a camera, including: under the condition that the zoom multiple is larger than a preset value, acquiring a first image through a long-focus camera; the target scene is a long-focus scenery scene, the long-focus scenery scene is a scene of shooting scenery by using a long-focus camera, and the target algorithm is a long Jiao Fengjing super-resolution algorithm.

In a tele landscape scene, the uncertainty of the richness of the image texture details is large. The image processing method provided by the embodiment of the application is applied to a long-focus landscape scene, so that both the resource performance and the precision can be realized more effectively, the instantaneity and the plotting effect of the long Jiao Fengjing super-resolution algorithm can be balanced more effectively, and the user experience is improved.

In a possible implementation manner, the method further includes: and under the condition that the zoom multiple is larger than a preset value and the shooting environment is an outdoor environment, determining that the shooting scene is a tele landscape scene.

In the implementation mode, the long-focus scenery scene can be simply and quickly identified through the zoom multiple and the shooting environment.

In a possible implementation manner, the first area is all of the first image, and the second image is displayed based on the second area, including: and displaying the second image by taking the second area as the second image.

That is, in the tele landscape scene, the full texture detail degree of the first image may be determined, and then, based on the determination result, the full first image is input into the target floating point model or the target fixed point model, to obtain the final second image.

In a possible implementation manner, the target scene is a face scene, the face scene refers to a scene that a shooting object contains a face, and the target algorithm is a face super-resolution algorithm.

In the face scene, the face area needs to be enhanced, so that the display effect is improved. Different scenes, different characters, different faces and greater differences in the abundance of texture details. The image processing method provided by the embodiment of the invention is applied to a face scene, can more effectively realize the balance of resource performance and precision, more effectively balance the instantaneity and the plotting effect of the face super-resolution algorithm, and improve the user experience.

In a possible implementation manner, the first region is a face region in the first image, and the method further includes: image segmentation is carried out on the first image, and a region containing a human face in the first image is obtained to obtain a first region; displaying a second image based on the second region, comprising: splicing the second region with other regions except the first region in the first image to obtain a second image; the second image is displayed.

In the implementation manner, the image processing process provided by the embodiment is effectively fused with the processing process of the face super-resolution algorithm, and the resource performance and the accuracy are both realized on the basis that the operation of the face super-resolution algorithm is not affected, and the real-time performance and the drawing effect are both realized.

In a possible implementation manner, the first operation is used for indicating to display a preview interface, or indicating to take a picture, or indicating to record a video.

The first operation may be an operation to trigger the display of a photo preview interface (may be referred to as a photo preview operation), for example, an operation to launch a camera application at a user.

The first operation may also be an operation of triggering the display of a video preview interface (may be referred to as a video preview operation), for example, an operation of clicking a video mode control.

The first operation may also be an operation to trigger photographing (may be referred to as a photographing operation), such as an operation to click a photographing control in a photographing preview interface.

The first operation may also be an operation that triggers video recording (may be referred to as a video recording operation), such as an operation of clicking a start video control in a video recording preview interface.

The embodiment of the present application does not make any limitation on the first operation.

In a second aspect, the present application provides an apparatus, which is included in an electronic device, and which has a function of implementing the electronic device behavior in the first aspect and possible implementations of the first aspect. The functions may be realized by hardware, or may be realized by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the functions described above. Such as a receiving module or unit, a processing module or unit, etc.

In a third aspect, the present application provides an electronic device, the electronic device comprising: a processor, a memory, and an interface; the processor, the memory and the interface cooperate with each other such that the electronic device performs any one of the methods of the technical solutions of the first aspect.

In a fourth aspect, the present application provides a system-on-chip comprising a processor. The processor is configured to read and execute a computer program stored in the memory to perform the method of the first aspect and any possible implementation thereof.

Optionally, the chip system further comprises a memory, and the memory is connected with the processor through a circuit or a wire.

Further optionally, the chip system further comprises a communication interface.

In a fifth aspect, the present application provides a computer readable storage medium, in which a computer program is stored, which when executed by a processor causes the processor to perform any one of the methods of the first aspect.

In a sixth aspect, the present application provides a computer program product comprising: computer program code which, when run on an electronic device, causes the electronic device to perform any one of the methods of the solutions of the first aspect.

Drawings

FIG. 1 is a schematic diagram illustrating an example of texture detail comparison provided in an embodiment of the present application;

fig. 2 is a schematic structural diagram of an example of an electronic device according to an embodiment of the present application;

FIG. 3 is a block diagram of a software architecture of an example electronic device according to an embodiment of the present application;

FIG. 4 is a schematic block diagram of an exemplary long-focus landscape oversubstance algorithm according to an embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating an example of an image processing method according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an interface of a photo preview according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating another exemplary image processing method according to an embodiment of the present disclosure;

FIG. 8 is a block diagram of an exemplary face superdivision algorithm according to an embodiment of the present disclosure;

fig. 9 is a flowchart of an example of an image processing method according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, "/" means or is meant unless otherwise indicated, for example, a/B may represent a or B; "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

The terms "first," "second," "third," and the like, are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", or a third "may explicitly or implicitly include one or more such feature.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more, but not all, embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

For a better understanding of the embodiments of the present application, terms or concepts that may be referred to in the embodiments are explained below.

1. Floating-point model (floating-point model)

The floating point model refers to a model in which the weight value and/or the input value is a floating point number. The floating point model occupies larger computing resources and has higher output precision.

2. Fixed point model (fixed-point model)

The fixed-point model refers to a model in which weight values are integer numbers, and may also be referred to as a quantization model. The fixed-point model firstly quantizes the input of the floating point number into integer numbers, calculates by utilizing the quantized integer numbers, and the quantized integer numbers can comprise 1 bit (bit), 2bit, 4bit, 8bit or 16bit and the like. The fixed-point model occupies less computing resources, has great potential in the aspects of compression model size and acceleration reasoning time, but has lower output precision.

The fixed-point model can be obtained by model quantization from a floating-point model.

3. Texture (texture)

Texture in computer graphics includes both texture of the surface of an object in the general sense, even if the surface of the object exhibits rugged grooves, and also includes colored patterns on the smooth surface of the object, which are often more often referred to as motifs. As for the pattern, a color pattern or a pattern is drawn on the surface of the object, and the surface of the object after the texture is generated is still smooth. In practice, grooves are also required to be colored or patterned on the surface, and a visual uneven feeling is required. The rugged pattern is generally irregular. In computer graphics, the generation methods of the two types of textures are completely consistent, which is why they are collectively called textures in computer graphics.

4. Texture details (texture detail)

Texture detail, also referred to as detail texture, refers to the fine structure in an image that constitutes the texture.

How much texture detail can be described as rich texture detail and lack of texture detail. The rich texture details of the image means that the image contains more texture details, namely more fine structure information. The lack of texture details of the image means that the image contains less texture details, i.e. less fine structure information.

As described in the background, in the image processing process, the fixed-point model and the floating-point model have advantages in terms of reasoning speed and precision. Therefore, in practical application, whether the algorithm adopts a fixed-point model or a floating-point model is a problem faced by the research personnel. If a floating point model is selected, the output precision is higher, the graph effect is good, but more calculation resources are occupied, the reasoning time is long, and the algorithm instantaneity is poor. If a fixed-point model is selected, the occupied computing resource is less, the reasoning speed is high, but in the process of converting a floating-point model into the fixed-point model, the precision loss caused by model quantization is unavoidable, so that the graph effect is necessarily retracted to a certain extent.

In view of this, the inventors of the present application have found that the degree of richness of texture details contained in an image varies from scene to scene. The precision requirements of the algorithm are different when images with different degrees of texture detail are processed. The image with rich texture details has higher requirements on algorithm precision, and the image with lack of texture details has lower requirements on algorithm precision.

Referring specifically to fig. 1, in comparison, the texture details of the (a) graph of fig. 1 are relatively lacking, and the texture details of the (b) graph of fig. 1 are relatively rich. If both images are processed by a high-precision floating point model, the image with lack of texture details shown in the (a) diagram in fig. 1 will cause slow reasoning speed, poor real-time performance, and waste of computing resources and loss of device performance. If both images are processed using a low-precision fixed-point model, then for the rich-texture-detail image shown in the (b) diagram of fig. 1, the poor precision may result in poor image effect, affecting the user experience. If a compromise scheme is adopted, the computing resource, the device performance (resource performance for short) and the precision are balanced by reducing the weight of the floating point model, and the (a) diagram and the (b) diagram in fig. 1 have different degrees of sacrifice in terms of both the resource performance and the precision. Therefore, how to select the fixed-point model and the floating-point model to consider the resource performance and the precision, and further consider the real-time performance and the drawing effect is a difficult problem faced by the person skilled in the art.

In view of this, the embodiment of the present application provides an image processing method, in which during image processing, the abundance of texture details of an image is determined, and if the texture details of the image are richer, a floating point model is selected for processing. If the image texture detail is relatively lacking, a fixed-point model is selected for processing. Therefore, images with different texture detail richness are selected and used in different types of models, so that the model precision can be ensured, the image drawing effect of the images is ensured, the reasoning speed is not reduced, the resource waste and the performance loss are not caused, the balance of the resource performance and the precision is realized, and the balance of the real-time performance and the image drawing effect is further realized.

The image processing method provided by the embodiment of the application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (augmented reality, AR)/Virtual Reality (VR) devices, notebook computers, ultra-mobile personal computer (UMPC), netbooks, personal digital assistants (personal digital assistant, PDA) and the like, and Application (APP) can be installed on the electronic devices, and the specific types of the electronic devices are not limited.

Fig. 2 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller may be a neural hub and a command center of the electronic device 100, among others. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it may be called directly from memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

The ISP is used to process data fed back by the camera 193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, so that the electrical signal is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The touch sensor 180K, also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is for detecting a touch operation acting thereon or thereabout. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to touch operations may be provided through the display 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a different location than the display 194.

The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is described below. The software system may adopt a layered architecture, an event driven architecture, a microkernel architecture, a micro-service architecture or a cloud architecture, and in the embodiment of the present application, the software system of the electronic device 100 is exemplarily described by taking an Android system of the layered architecture as an example.

As shown in fig. 3, the software system using the layered architecture is divided into several layers, each of which has a clear role and division. The layers communicate with each other through a software interface. In some embodiments, the software system may be divided into four layers, from top to bottom, an application layer 310, an application framework layer 320, a hardware abstraction layer 330, and a driver layer 340, respectively. In addition, for ease of understanding, fig. 3 also shows the hardware layer 350 of the electronic device 100. The respective layers are described below.

The application layer 310 may include cameras, gallery applications, and may also include calendar, conversation, map, navigation, WLAN, bluetooth, music, video, short message applications, and the like.

The application framework layer 320 provides an application access interface and programming framework for the applications of the application layer 310.

For example, the application framework layer 320 includes a camera access interface for providing a photographing service of a camera through camera management and a camera device.

Camera management in the application framework layer 320 is used to manage cameras. The camera management may obtain parameters of the camera, for example, determine an operating state of the camera, and the like.

The camera devices in the application framework layer 320 are used to provide a data access interface between the different camera devices and camera management.

The hardware abstraction layer 330 is used to abstract the hardware. For example, the hardware abstraction layer 330 may include a camera hardware abstraction layer and other hardware device abstraction layers; the camera hardware abstract layer may include a camera device 1, a camera device 2, and the like; the camera hardware abstraction layer may be coupled to a camera algorithm library, and the camera hardware abstraction layer may invoke algorithms in the camera algorithm library.

The camera algorithm library may include preset algorithms, and one or more of the algorithms may be implemented by a fixed point model or a floating point model, for example, super-resolution (SR) algorithms, super-score (portrait SR) algorithms, or the like. For convenience of explanation, in this application, an algorithm that can be implemented by a fixed-point model or a floating-point model is referred to as a target algorithm, and a scene that needs to call the target algorithm to process an image is referred to as a target scene. For example, when the target scene is a tele landscape scene, the target algorithm is a tele landscape oversubstance algorithm. The long-focus scenery scene refers to a scene of shooting scenery by using a long-focus camera; when the target scene is a face scene, the target algorithm is a face superscore algorithm. The face scene refers to a scene containing a face in a shooting object.

As shown in FIG. 3, in an embodiment of the present application, for a target algorithm, a target fixed-point model, a target floating-point model, and a texture determination module may be included. The texture judging module is used for judging the degree of richness of the texture details of the image.

The driver layer 340 is used to provide drivers for different hardware devices. For example, the drive layer may include a camera drive; a digital signal processor driver and a graphics processor driver.

Hardware layer 350 may include sensors, image signal processors, digital signal processors, graphics processors, and other hardware devices. The sensor may include a tele camera, a mid-focus camera, a wide-angle camera, and the like, and may also include a depth sensor (TOF) and a multispectral sensor.

The workflow of the software system of the electronic device 100 is illustrated in connection with displaying a photo scene.

When a user performs a click operation on the touch sensor 180K, after the camera APP is awakened by the click operation, each camera device of the camera hardware abstraction layer is invoked through the camera access interface. The camera hardware abstraction layer judges the current zoom multiple, and calls the corresponding camera through the camera device driver according to the zoom multiple. Meanwhile, the camera hardware abstraction module acquires current environment parameters, and determines the current shooting scene according to zoom times, environment parameters and the like. And according to the current shooting scene, calling an algorithm library to start loading a corresponding algorithm.

For example, the camera hardware abstraction layer determines that the current zoom factor is greater than a preset value (e.g., 10), and thus, the tele camera may be invoked by issuing an instruction to the camera device driver to invoke the tele camera. Meanwhile, the camera hardware abstraction module acquires current environment parameters, and determines that the current shooting environment is an outdoor environment according to the environment parameters. In this way, by combining the zoom multiple, determining the long-focus scenery scene of the current shooting scene, and calling a camera algorithm library to start loading the long-focus scenery oversubstance algorithm.

When the sensor of the hardware layer is called, corresponding data are acquired. For example, after calling a tele camera to acquire an original image. The long-focus camera sends the obtained original image to image signal processing for preliminary processing such as registering and decoding, and an initial image is generated. The image signal processor drives the processed initial image data to return to the hardware abstraction layer through the camera equipment, and then processes the initial image data by utilizing an algorithm loaded in a camera algorithm library, for example, a long-focus landscape super-division algorithm in the camera algorithm library is utilized, and the initial image data are processed according to the related processing steps provided by the embodiment of the application, so that a target image is obtained. The algorithm in the camera algorithm library can be driven by the digital signal processor to call the digital signal processor, and the image processor is driven by the image processor to call the image processor, so that the algorithm operation is realized.

And sending the obtained target image back to the camera application for display and storage through the camera hardware abstraction layer and the camera access interface.

For easy understanding, the following embodiments of the present application will take an electronic device having a structure shown in fig. 2 and fig. 3 as an example, and specifically describe an image processing method provided in the embodiments of the present application with reference to the accompanying drawings and application scenarios.

Embodiment one:

in this embodiment, a tele landscape scene is described as an example. Exemplary, fig. 4 is a schematic block diagram of an exemplary long-focus landscape oversubstance algorithm according to an embodiment of the present application. As described above, the target algorithm may include a target fixed-point model, a target floating-point model, and a texture determination module, and for a long-focus landscape scene, the target algorithm is a long-focus landscape super-division algorithm, the target floating-point model is a long-focus landscape super-division floating-point model (hereinafter referred to as long Jiao Fudian model), and the target fixed-point model is a long-focus landscape super-division fixed-point model (hereinafter referred to as long-focus fixed-point model), as shown in fig. 4.

Fig. 5 is a flowchart of an example of an image processing method provided in an embodiment of the present application, please refer to fig. 4 and fig. 5 together, and the method includes:

s101, the camera application responds to shooting preview operation, video preview operation, shooting operation or video operation of a user, invokes a camera hardware abstraction layer of the hardware abstraction layer, and the camera hardware layer identifies that the current scene is a tele landscape scene.

The photo preview refers to a screen preview in a shooting mode (non-recording mode). The video preview refers to a picture preview in a video mode. The photographing operation refers to image acquisition in a photographing mode. Video recording refers to video recording in a video recording mode. The photographing preview operation, the video preview operation, the photographing operation, and the video operation are also collectively referred to as a first operation. It should be understood that whether taking a preview, recording a preview, taking a photograph, or recording a video, the camera application is required to invoke the underlying module to acquire and process the image, and display the processed image in the interface, and/or save it to memory.

The specific implementation of the photographing preview operation, the video preview operation, the photographing operation and the video operation is not limited in this application. The following describes a process of recognizing a tele landscape scene using a photographing preview as an example.

Exemplary, fig. 6 is an interface schematic diagram of an example of a photo preview according to an embodiment of the present application. Taking an electronic device as a mobile phone, as shown in fig. 6 (a), an icon 601 of a camera application is included in a desktop of the mobile phone, and in response to a user clicking on the icon 601 of the camera application, the mobile phone displays a photographing preview interface 602, as shown in fig. 6 (b). The zooming multiple information 603 is displayed in the photographing preview interface 602, and the current zooming multiple information 603 is displayed as' 1 ", indicates that the current zoom magnification is 1.

The user can adjust the zoom multiple according to a preset operation, for example, the zoom multiple can be reduced through a double-finger pinch gesture, and the zoom multiple can be increased through a gesture of sliding outwards in two directions. For another example, the zoom magnification may be reduced by performing a right drag operation at the zoom magnification information 603, and the zoom magnification may be increased by performing a left drag operation at the zoom magnification information 603. And when the zoom multiple exceeds a preset value, the camera hardware abstraction layer judges that the current scene is a tele scene. As shown in fig. 6 (c), it is assumed that the user is inThe operation of dragging to the left is performed at the zoom magnification information 603, when the zoom magnification is increased to 12 times, the user lifts his/her hand, and after lifting his/her hand, the zoom magnification information 603 displayed in the interface is "12"", as shown in fig. 6 (d). Taking the preset value of 10 times as an example, the camera hardware abstraction layer determines that the current zoom multiple 12 is greater than the preset value of 10, so that the current scene is a tele scene.

In addition, the camera hardware abstraction layer can judge whether the current shooting environment is an indoor environment or an outdoor environment according to the environmental light reported by the ambient light sensor and the like, the depth information (namely the distance between the camera and the TOF or the preset depth algorithm module) of the shooting object, and one or more information of the temperature detected by the temperature sensor, the GPS signal and the like. If the current scene is a tele scene and the shooting environment is an outdoor environment, the current tele landscape scene can be determined.

S102, the camera hardware abstraction layer calls a long-focus camera to acquire an initial image (also called a first image), and calls a long-focus landscape superdivision algorithm in an algorithm library.

After the camera hardware abstraction layer determines that the current scene is a long-focus scene, calling a long-focus camera to shoot an image, acquiring an original image by the long-focus camera, performing preliminary processing such as registration on the original image by an image signal processor, and returning the original image to the camera hardware abstraction layer. Alternatively, the initial image may be in the. Raw format.

Meanwhile, after the camera hardware abstraction layer determines that the current scene is a long-focus landscape scene, a camera algorithm library is called to load a long-focus landscape superdistribution algorithm. After the initial image is returned to the camera hardware abstraction layer, the camera hardware abstraction layer calls a long-focus landscape super-division algorithm in a camera algorithm library to process the initial image. Specifically, the camera hardware abstraction layer may send a call message to the texture judgment module in the long-focus landscape super-classification algorithm, so as to trigger the texture judgment module to execute the judgment of the richness of the texture, that is, execute step S103, where the call message may carry the initial image.

S103, a texture judging module in the long-focus landscape super-classification algorithm judges whether the richness of the texture details of the initial image meets the preset condition; if the degree of richness of the texture details meets the preset condition, executing step S104; if the degree of richness of the texture details does not meet the preset condition, step S107 is performed.

Alternatively, the texture judging module may judge the abundance of the texture details of the image by various methods. For example, method one: in one embodiment, the texture determining module may calculate a pixel gray level gradient through a gray level histogram corresponding to the initial image, and further determine the abundance of texture details according to the pixel gray level gradient.

The second method is as follows: in another embodiment, the level of texture detail richness may also be determined by calculating the number of boundaries per unit area in the initial image. The image with lacking texture details has similar gray scale in local neighborhood, little change and smaller boundary number in unit area. The image with rich texture details has quicker gray level change in local neighborhood, so the number of boundaries in unit area is more.

Of course, in some other embodiments, the richness of the texture details of the initial image may be determined by other methods, which are not limited in this embodiment, and the following embodiments are further described.

It should be understood that different methods are used to determine the richness of texture details, the parameters required to perform the determination are different, and the corresponding preset conditions are also different. For example, by determining the richness of the texture details by the above method, the required parameter is a pixel gray-scale gradient, and the preset condition may be that the pixel gray-scale gradient is greater than a preset pixel gray-scale gradient threshold. By the method II, the richness of the texture details is judged, the required parameters are the number of boundaries in each unit area, and the preset condition can be that the number of target unit areas is larger than the preset number. The target unit area refers to a unit area with a number of boundaries greater than a preset threshold of the number of boundaries (i.e., a larger number of boundaries).

And S104, under the condition that the abundance of the texture details meets the preset condition, the texture judgment module inputs the initial image into the long-focus super-resolution floating point model.

The degree of the texture detail is satisfied with the preset condition, the characteristic texture detail is abundant, the image is input into the floating point model, and the processing precision of the image can be ensured, so that the texture detail information of the initial image is better kept, and the image effect is ensured.

S105, the long-focus super-resolution floating point model processes the initial image to obtain a target image (also called a second image).

And S106, returning the target image to the camera application by the long-focus super-resolution floating point model.

Specifically, the obtained target image is sent back to the camera application through the camera hardware abstraction layer and the camera access interface, and the camera application displays and stores the target image, which is not described herein.

And S107, under the condition that the richness of the texture details does not meet the preset condition, the texture judgment module inputs the initial image into the long-focus super-resolution fixed-point model.

The degree of richness of the texture details does not meet the preset condition, the characteristic of the richness of the texture details is that the texture details are lacking, and the image is input into the fixed-point model. It should be understood that the fixed-point model is adopted to process the image lacking in texture details, the precision loss caused by the quantization of the model is very small, the influence on the image is very small, the fixed-point model can reduce occupied memory, greatly improve the reasoning speed, and save the waste of computing resources and the equipment performance.

S108, processing the initial image by the long-focus super-resolution fixed point model to obtain a target image.

And S109, returning the target image to the camera application by the long-focus super-resolution fixed point model.

According to the image processing method provided by the embodiment, the richness of the image texture details is judged, and the image is input into the floating point model for processing under the condition that the richness of the image texture details meets the preset condition; and under the condition that the image texture details are rich and do not meet the preset conditions, inputting the image into the fixed-point model for processing. Therefore, the processing precision can be ensured, the image drawing effect is ensured, the reasoning speed is not reduced, the resource waste and the performance loss are not caused, the resource performance and the precision are both realized, and the real-time performance and the image drawing effect are both realized.

The process of determining the richness of the texture details is further described below.

Fig. 7 is a schematic flow chart of another image processing method according to an embodiment of the present application. As shown in fig. 7, the above-described step "S103, the texture judgment module in the long-focus landscape super-classification algorithm judges whether the richness of the texture details of the initial image satisfies the preset condition" may include the following steps S1031 to S1034. The execution main bodies of the following steps are texture judgment modules, and are not described in detail.

S1031, performing Fourier transform on the initial image to obtain a spectrogram.

Specifically, fourier transformation is performed on the initial image, and it is essential to convert the initial image from the spatial domain to the frequency domain (abbreviated as frequency domain). Alternatively, in the calculation of the fourier transform, a discrete fourier transform (discrete fourier transform, DFT) may be used, a fast fourier transform (fast fourier transform, FFT) may be used, and the embodiment of the present application is not limited in this respect.

Taking discrete fourier transform as an example, the initial image may be processed by the following formula (1) to obtain a spectrogram:

（1）

wherein,complex value representing the kth frequency in the frequency domain, < >>Is the complex value of the nth sample in the spatial domain, N being the number of samples. The formula will spatial domain signal +.>Conversion to a frequency-domain signal>。

S1032, carrying out centering treatment on the spectrogram to obtain a centering spectrogram.

It will be appreciated that the spectrogram includes a plurality of points, each representing a frequency component.

Taking a conventional rectangular image as an example, before fourier transform processing, the frequency components gradually rise from the edge to the central region of the spectrogram, that is, the points at which the frequency components are lower are typically located at the four corner portions of the spectrogram, and the points at which the frequency components are higher are located at the central region of the spectrogram. Through the centering treatment, the point with lower frequency component is moved to the center area of the spectrogram, and the point with higher frequency component is moved to the four corner parts of the spectrogram. That is, the frequency components of each point gradually decrease from the four corners of the centered spectrogram to the center region. Therefore, frequency statistics is convenient, subsequent statistics of high-frequency components based on a centralized frequency spectrum is convenient, the algorithm operation process is simplified, and the algorithm operation efficiency is improved.

It should be appreciated that step S1032 is an optional step, and in some embodiments, the centering process may not be performed.

S1033, counting the duty ratio of frequency components (hereinafter referred to as high-frequency components, also referred to as target frequency components) larger than a preset frequency domain threshold in the centralized spectrogram.

That is, whether the frequency component of one point in the centered spectrogram is a high frequency component or a low frequency component may be defined by a preset frequency threshold. The preset frequency threshold may be expressed as. Greater than the preset frequency threshold->Is considered as a high frequency component; less than or equal to the preset threshold +.>Is referred to as a low frequency component. Wherein the frequency component is a point of the high frequency component, calledHigh frequency component points.

Optionally, shooting scenes are different, algorithms are different, and corresponding preset frequency thresholds are usedMay be different. Taking a long-focus landscape scene and a long-focus landscape oversubstance algorithm as an example, a corresponding preset frequency threshold value +.>For example, 2.5. Taking a face scene and a face super-division algorithm as an example, a corresponding preset frequency threshold value +.>For example, 3.7.

As one possible implementation, the duty cycle of the high frequency component may be characterized by the number of high frequency component points duty cycle, i.e. the duty cycle of the high frequency component = the number of high frequency component points/the total number of points in the centralised spectrogram.

As another possible implementation, the duty cycle of the high frequency component may also be characterized by the amplitude duty cycle of the high frequency component points, i.e. the amplitude of the high frequency component points and/or the total amplitude of the mid-points of the centralised spectrogram. Wherein the magnitude of the mid-point of the spectrogram is used to characterize the intensity of the frequency. The sum of the magnitudes of the high frequency component points is also referred to as a first magnitude sum, and the total magnitude of the points in the centered spectrogram is also referred to as a second magnitude sum. In the implementation mode, the high-frequency component duty ratio is represented by the amplitude duty ratio of the high-frequency component point, and the amplitude is known and accurate, so that the high-frequency component duty ratio can be conveniently and accurately determined without counting, and the calculation accuracy and calculation efficiency are improved.

S1034, judging whether the duty ratio of the high-frequency component is larger than a preset duty ratio; if yes, determining that the texture detail richness of the initial image meets the preset condition, and obtaining an image with rich texture detail; if not, determining that the richness of the texture details of the initial image does not meet the preset condition, and determining the image with the lacking texture details.

That is, in the present embodiment, the preset conditions are: the duty cycle of the high frequency component is greater than a preset duty cycle. The preset duty cycle may be expressed as。

Optionally, a preset duty cycle For example, 70%, 60% and the like can be adopted, and the specific setting can be realized according to actual requirements. In addition, the preset duty ratio can be different in different shooting scenes and different algorithms.

The duty cycle of the high frequency component is greater than a preset duty cycleThe content of the high-frequency component in the initial image is more, and the texture details of the image are rich. The duty ratio of the high frequency component is less than or equal to the preset duty ratio +.>The content of the high frequency component in the initial image is less, and the lack of texture details of the image is indicated.

In this embodiment, the content including the high-frequency component is considered to be more in the image with abundant texture details. For example, the graph (b) in fig. 1 has a larger content of high-frequency components than the graph (a) in fig. 1. Therefore, after Fourier transformation, the duty ratio of the high-frequency component is counted based on the spectrogram, the richness of the texture details of the image is judged according to the duty ratio of the high-frequency component, the method is simple, the calculation result is accurate, and the accuracy of fixed-point model and floating-point model selection is further improved.

The image processing method provided by the embodiment of the application is further described below by taking a face super-division algorithm in a face scene as an example. It can be understood that in a face scene, after the image is segmented, the face super-segmentation algorithm model is adopted to enhance the region containing the face, so that the resolution of the face is improved, and the visual experience of the user is improved. The face hyper-score algorithm model can be a fixed-point model or a floating-point model. The inventor finds that the abundance degree of the texture details of the region containing the face is greatly different under different shooting objects or shooting scenes. For example, some areas containing faces have hair that is larger in that area, and thus the texture details are more abundant. While some areas containing faces contain hats, which block hair, thus lacking texture details. For another example, some areas containing the face contain glasses or ornaments such as earrings, and the texture details are also rich. Therefore, in the embodiment, different kinds of detail of textures adopt different models, so that the balance of resource performance and precision is realized, and further the balance of instantaneity and graph effect is realized.

Fig. 8 is a schematic block diagram of an exemplary face super-score algorithm according to an embodiment of the present application. As described above, the target algorithm may include a target fixed-point model, a target floating-point model, and a texture determination module, and for a face scene, the target algorithm is a face hyper-score algorithm, the target floating-point model is a face hyper-score floating-point model, and the target fixed-point model is a face hyper-score fixed-point model, as shown in fig. 8. In addition, the face super-segmentation algorithm also comprises an image segmentation module and an image stitching module. The image segmentation module is used for segmenting the area containing the human face from the initial image based on an image segmentation technology. The image stitching module is used for stitching the plurality of areas to obtain a target image.

Fig. 9 is a flowchart of an example of an image processing method provided in an embodiment of the present application, please refer to fig. 8 and fig. 9 together, and the method includes:

s201, the camera application responds to shooting preview operation, video preview operation, shooting operation or video operation of a user, invokes a camera hardware abstraction layer of the hardware abstraction layer, and the camera hardware layer recognizes that the current scene is a face scene.

Specifically, the camera hardware layer may call a face recognition algorithm, recognize that the current scene contains a face, and determine that the current scene is a face scene.

S202, the camera hardware abstraction layer calls a corresponding camera (taking a middle-focus camera as an example) to acquire an initial image, and calls a face super-resolution algorithm in an algorithm library.

Specifically, the camera hardware abstraction layer may send a call instruction to the image segmentation module in the face super-segmentation algorithm, so as to trigger the image segmentation module to segment the image, that is, execute step S203, where the call message may carry the initial image.

And S203, an image segmentation module in the face super-segmentation algorithm performs image segmentation on the initial image to obtain a region containing a face (called a face region and also called a first region).

S204, the image segmentation module sends the face area to the texture judgment module.

S205, a texture judging module judges whether the richness of the texture details of the face area meets the preset condition; if the degree of richness of the texture details meets the preset condition, steps S206 to S208 are executed, and then S212 and S213 are executed; if the degree of richness of the texture details does not meet the preset condition, steps S209 to S211 are performed, followed by S212 and S213.

This step is similar to step S103 in the above embodiment, and will not be described again.

S206, under the condition that the abundance of the texture details meets the preset condition, the texture judging module inputs the face region into the face super-dividing floating point model.

S207, the face super-resolution floating point model processes the face region to obtain a face enhancement region (also called a second region).

S208, the face super-resolution floating point model sends the face enhancement region to an image stitching module.

S209, under the condition that the richness of the texture details does not meet the preset condition, the texture judging module inputs the face area into the face super-division fixed point model.

S210, the face super-division fixed point model processes the face area to obtain a face enhancement area.

S211, the face super-division fixed point model sends the face enhancement area to an image stitching module.

And S212, the image stitching module is used for stitching the face enhancement region with other regions of the initial image to obtain a target image (also called a second image).

It can be understood that while the face super-resolution floating point model or the face super-resolution fixed point model processes the face region, other algorithms or modules can process other regions in the initial image, and return the processed result to the image stitching module, and finally the image stitching module performs stitching to obtain the target image.

S213, the image stitching module returns the target image to the camera application.

According to the image processing method provided by the embodiment, the richness of the image texture details of the face area is judged, and the face area is input into the floating point model for processing under the condition that the richness of the texture details meets the preset condition; and under the condition that the texture details are rich and do not meet the preset conditions, inputting the face area into the fixed-point model for processing. Therefore, the processing precision of the final face area can be ensured, the drawing effect is ensured, the precision reasoning speed is avoided, the resource waste and the performance loss are caused, the resource performance and the drawing effect are both realized, and the instantaneity and the drawing effect are both realized.

It should be noted that, in the above embodiment, the image processing method is described by taking a long-focus landscape scene and a human face scene as examples, but the method is not limited to being applied to the two scenes, and other shooting scenes and any other image processing algorithm involving a fixed-point model or a floating-point model may be applied, which is not limited in this embodiment of the present application. Moreover, in some embodiments, where multiple image processing algorithms are involved in some scenarios, these algorithms may be implemented simultaneously by the present method. In addition, when various image processing algorithms are implemented by the present method, the step of determining the degree of richness of texture details (i.e., the above-described step S103 or S205) may be performed only once. Therefore, the algorithm operation process can be simplified, the algorithm operation efficiency can be improved, and the resource performance can be further saved.

Examples of the image processing method provided in the embodiments of the present application are described above in detail. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application in conjunction with the embodiments, but such implementation is not to be considered as outside the scope of this application.

The embodiment of the present application may divide the functional modules of the electronic device according to the above method examples, for example, may divide each function into each functional module corresponding to each function, for example, a detection unit, a processing unit, a display unit, or the like, or may integrate two or more functions into one module. The integrated modules may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present application, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

It should be noted that, all relevant contents of each step related to the above method embodiment may be cited to the functional description of the corresponding functional module, which is not described herein.

The electronic device provided in this embodiment is configured to execute the above-described image processing method, and therefore the same effects as those of the above-described implementation method can be achieved.

In case an integrated unit is employed, the electronic device may further comprise a processing module, a storage module and a communication module. The processing module can be used for controlling and managing the actions of the electronic equipment. The memory module may be used to support the electronic device to execute stored program code, data, etc. And the communication module can be used for supporting the communication between the electronic device and other devices.

Wherein the processing module may be a processor or a controller. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. A processor may also be a combination that performs computing functions, e.g., including one or more microprocessors, digital signal processing (digital signal processing, DSP) and microprocessor combinations, and the like. The memory module may be a memory. The communication module can be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip and other equipment which interact with other electronic equipment.

In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device according to this embodiment may be a device having the structure shown in fig. 2.

The present application also provides a computer-readable storage medium in which a computer program is stored, which when executed by a processor, causes the processor to execute the image processing method of any one of the above embodiments.

The present application also provides a computer program product which, when run on a computer, causes the computer to perform the above-mentioned related steps to implement the image processing method in the above-mentioned embodiments.

In addition, embodiments of the present application also provide an apparatus, which may be specifically a chip, a component, or a module, and may include a processor and a memory connected to each other; the memory is used for storing computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory, so that the chip executes the image processing method in each method embodiment.

The electronic device, the computer readable storage medium, the computer program product or the chip provided in this embodiment are used to execute the corresponding method provided above, so that the beneficial effects thereof can be referred to the beneficial effects in the corresponding method provided above, and will not be described herein.

It will be appreciated by those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts shown as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method performed by an electronic device including a camera, the method comprising:

receiving a first operation of a camera application by a user;

responding to the first operation, and acquiring a first image through the camera;

under the condition that the shooting scene is determined to be a target scene, loading a target algorithm corresponding to the target scene, wherein the target algorithm comprises a target floating point model and a target fixed point model;

if the richness of the texture details of the first region meets the preset condition, the first region is processed through the target floating point model to obtain a second region, and the first region is all or part of the first image;

if the richness of the texture details of the first region does not meet the preset condition, the first region is processed through the target fixed-point model, and a second region is obtained;

And displaying a second image based on the second region.

2. The method according to claim 1, wherein the method further comprises:

performing Fourier transform on the first region to obtain a spectrogram;

counting the duty ratio of a target frequency component in the spectrogram; the target frequency component is a frequency component larger than a preset frequency threshold;

if the duty ratio of the target frequency component is larger than the preset duty ratio, determining that the richness of the texture details of the first area meets the preset condition;

if the duty ratio of the target frequency component is smaller than or equal to the preset duty ratio, determining that the richness of the texture details of the first area does not meet the preset condition.

3. The method of claim 2, wherein said counting the duty cycle of the target frequency components in the spectrogram comprises:

carrying out centering treatment on the spectrogram to obtain a centering spectrogram;

based on the centering spectrogram, the duty ratio of the target frequency component is counted.

4. A method according to claim 3, wherein said counting the duty cycle of the target frequency component based on the centralised spectrogram comprises:

counting the amplitude sum of points corresponding to the target frequency component in the centering spectrogram to obtain a first amplitude sum;

Counting the sum of the amplitudes of all points in the centralized spectrogram to obtain a second sum of the amplitudes;

and calculating the ratio of the first amplitude sum to the second amplitude sum to obtain the duty ratio of the target frequency component.

5. The method of claim 1, wherein the camera comprises a tele camera; the obtaining the first image through the camera includes:

under the condition that the zoom multiple is larger than a preset value, acquiring the first image through the long-focus camera;

the target scene is a tele landscape scene, the tele landscape scene is a scene of shooting a landscape by using the tele camera, and the target algorithm is a long Jiao Fengjing super-resolution algorithm.

6. The method of claim 5, wherein the method further comprises:

and under the condition that the zoom multiple is larger than the preset value and the shooting environment is the outdoor environment, determining that the shooting scene is the tele landscape scene.

7. The method of claim 5, wherein the first region is an entirety of the first image, and wherein displaying a second image based on the second region comprises:

and taking the second area as the second image, and displaying the second image.

8. The method according to claim 1, wherein the target scene is a face scene, the face scene is a scene in which a photographed object contains a face, and the target algorithm is a face super-resolution algorithm.

9. The method of claim 8, wherein the first region is a face region in the first image, the method further comprising:

image segmentation is carried out on the first image, and a region containing a human face in the first image is obtained to obtain the first region;

the displaying a second image based on the second region includes:

splicing the second region with other regions except the first region in the first image to obtain the second image;

and displaying the second image.

10. The method of any one of claims 1 to 9, wherein the first operation is for indicating to display a preview interface, or to take a photograph, or to record a video.

11. An electronic device, the electronic device comprising: one or more processors, cameras, and memory;

the memory is coupled with the one or more processors, the memory for storing computer program code comprising computer instructions that the one or more processors invoke to cause the electronic device to perform the method of any of claims 1-10.

12. A chip system for application to an electronic device, the chip system comprising one or more processors to invoke computer instructions to cause the electronic device to perform the method of any of claims 1 to 10.

13. A computer readable storage medium comprising instructions that, when run on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 10.