CN113852751A

CN113852751A - Image processing method, device, terminal and storage medium

Info

Publication number: CN113852751A
Application number: CN202010597481.4A
Authority: CN
Inventors: 蔡振锋
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-06-28
Filing date: 2020-06-28
Publication date: 2021-12-28
Anticipated expiration: 2040-06-28
Also published as: CN113852751B

Abstract

The embodiment of the application discloses an image processing method, an image processing device, a terminal and a storage medium, wherein the application can acquire initial preview image data in a first color coding format in a kernel layer of a terminal system, the first color coding format is the color coding format suitable for the kernel layer, then a priority index of the initial preview image data processing is determined in an application layer of the terminal system, then a target image conversion mode is determined in a plurality of candidate image conversion modes of the kernel layer based on the priority index, then the initial preview image data is converted into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, the second color coding format is the color coding format suitable for the application layer, and then the preview image data is displayed in the application layer; the scheme can effectively improve the efficiency of terminal image processing.

Description

Image processing method, device, terminal and storage medium

Technical Field

The present application relates to the field of communications technologies, and in particular, to an image processing method, an image processing apparatus, a terminal, and a storage medium.

Background

When the Android Application uses a Camera to take a picture, if a photo provided by a bottom layer needs to be acquired, the Android Application can be implemented by executing a capture operation in a Camera API (Camera Application Programming Interface), and then processing the callback data.

However, when the Camera API is used, due to different requirements of the application layer, there may be operations of post-processing and format conversion on the final image, whereas the existing scheme for image post-processing on the application layer mainly uses a Bitmap processing algorithm to rotate, crop, mirror, and the like a Jpeg (one image file format) format file carrying image data, and the image generation speed of the scheme is slow and takes a long time.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, a terminal and a storage medium, which can effectively improve the image processing efficiency of the terminal.

An embodiment of the present application provides an image processing method, including:

acquiring initial preview image data in a first color coding format in a kernel layer of a terminal system, wherein the first color coding format is a color coding format suitable for the kernel layer;

determining a priority index of the initial preview image data processing at an application layer of a terminal system;

determining a target image conversion mode in a plurality of candidate image conversion modes of the kernel layer based on the priority index;

based on the target image conversion mode, converting the initial preview image data into preview image data in a second color coding format in the kernel layer, wherein the second color coding format is a color coding format suitable for the application layer;

and displaying the preview image data in the application layer.

Correspondingly, an embodiment of the present application further provides an image processing apparatus, including:

the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring initial preview image data in a first color coding format in a kernel layer of a terminal system, and the first color coding format is a color coding format suitable for the kernel layer;

a first determining unit, configured to determine, at an application layer of a terminal system, a priority indicator of the initial preview image data processing;

a second determination unit configured to determine a target image conversion manner among a plurality of candidate image conversion manners of the kernel layer based on the priority index;

a conversion unit, configured to convert, in the kernel layer, the initial preview image data into preview image data in a second color coding format based on the target image conversion manner, where the second color coding format is a color coding format suitable for the application layer;

and the display unit is used for displaying the preview image data on the application layer.

Optionally, in some embodiments, in the image processing apparatus, the priority index includes an image processing speed and an image processing accuracy, and the second determining unit includes a first determining subunit and a second determining subunit, as follows:

the first determining subunit is configured to determine, when the priority indicator is an image processing speed, a target image conversion manner as a first image conversion manner from among a plurality of candidate image conversion manners;

and the second determining subunit is configured to determine, when the priority indicator is the image processing accuracy, that the target image conversion mode is the second image conversion mode from among the plurality of candidate image conversion modes.

Optionally, in some embodiments, in the image processing apparatus, the conversion unit includes a first conversion subunit and a second conversion subunit, as follows:

the first conversion subunit is configured to, when the target image conversion mode is a first image conversion mode, convert the initial preview image data in the kernel layer by using a system conversion algorithm of the terminal system to obtain preview image data in a second color coding format, where the system conversion algorithm is a format conversion algorithm of the terminal system;

and the second conversion subunit is configured to, when the target image conversion mode is a second image conversion mode, convert the initial preview image data in the kernel layer by using a custom conversion algorithm to obtain preview image data in a second color coding format, where the custom conversion algorithm is an algorithm that customizes conversion accuracy in advance and stores the conversion accuracy in the terminal system.

Optionally, in some embodiments, in the image processing apparatus, the system conversion algorithm includes a call conversion script and a call target interface function, and the first conversion subunit is specifically configured to, when the initial preview image data does not need to be cropped, call the conversion script in the kernel layer to convert the initial preview image data, so as to obtain preview image data in a second color coding format; when the initial preview image data needs to be cut, calling a target interface function in the kernel layer to convert the initial preview image data to obtain preview image data in a second color coding format, wherein the target interface function is an interface function obtained by modifying an interface function of a terminal system.

Optionally, in some embodiments, in the image processing apparatus, the first conversion subunit is specifically configured to, when the target image conversion mode is a first image conversion mode, call a target interface function at the kernel layer to separate parameters in the initial preview image data in the first color coding format, so as to obtain a luminance component, a saturation component, and a chrominance component; carrying out format conversion on the brightness component, the saturation component and the chrominance component to obtain target preview image data in a second color coding format; and cutting the target preview image data in the second color coding format according to a preset offset to obtain the preview image data in the second color coding format.

Optionally, in some embodiments, in the image processing apparatus, the second converting subunit is specifically configured to, when the target image conversion mode is the second image conversion mode, call a target interface function at the kernel layer to separate parameters in the initial preview image data in the first color coding format, so as to obtain a luminance component, a saturation component, and a chrominance component; carrying out format conversion on the brightness component, the saturation component and the chrominance component by using a custom conversion mode to obtain target preview image data in a second color coding format, wherein the custom conversion mode is a conversion mode which carries out custom-defining on the precision of format conversion in advance and stores the conversion mode in a terminal system; and cutting the target preview image data in the second color coding format according to a preset offset to obtain the preview image data in the second color coding format.

Optionally, in some embodiments, in the image processing apparatus, the first determining unit is specifically configured to determine, at an application layer of a terminal system, a display requirement of the initial preview image data; and determining a priority index of the initial preview image data processing according to the display requirement.

Optionally, in some embodiments, the image processing apparatus further includes a spatial transform unit, as follows:

the space transformation unit is used for acquiring the coordinates of each pixel point in the preview image data when the preview image data needs to be subjected to space transformation; carrying out coordinate transformation on the coordinates of each pixel point in the preview image data to obtain transformed preview image data;

the display unit is configured to display the transformed preview image data on the application layer.

Optionally, in some embodiments, the image processing apparatus further includes a setting unit, as follows:

the setting unit is used for configuring a shooting interface of the terminal system; setting a shooting imaging format of the terminal system to be a first color coding format;

the obtaining unit is specifically configured to obtain, in a kernel layer of the terminal system, initial preview image data in a first color coding format when a shooting operation of a user for the terminal is detected.

In addition, the embodiment of the present application further provides a terminal, where the terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in any one of the image processing methods provided in the embodiment of the present application.

In addition, a computer-readable storage medium is provided, where the computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to perform the steps in any one of the image processing methods provided in the embodiments of the present application.

According to the embodiment of the application, initial preview image data in a first color coding format can be obtained in a kernel layer of a terminal system, the first color coding format is a color coding format suitable for the kernel layer, then a priority index for processing the initial preview image data is determined in an application layer of the terminal system, then a target image conversion mode is determined in a plurality of candidate image conversion modes of the kernel layer based on the priority index, then the initial preview image data is converted into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, the second color coding format is a color coding format suitable for the application layer, and then the preview image data is displayed in the application layer; the scheme can effectively improve the efficiency of terminal image processing.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1a is a scene schematic diagram of an image processing method provided in an embodiment of the present application;

FIG. 1b is a schematic diagram of a first process of an image processing method according to an embodiment of the present disclosure;

fig. 2a is a second flowchart of an image processing method according to an embodiment of the present application;

FIG. 2b is a schematic diagram of a third flow chart of an image processing method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description that follows, specific embodiments of the present application will be described with reference to steps and symbols executed by one or more computers, unless otherwise indicated. Accordingly, these steps and operations will be referred to, several times, as being performed by a computer, the computer performing operations involving a processing unit of the computer in electronic signals representing data in a structured form. This operation transforms the data or maintains it at locations in the computer's memory system, which may be reconfigured or otherwise altered in a manner well known to those skilled in the art. The data maintains a data structure that is a physical location of the memory that has particular characteristics defined by the data format. However, while the principles of the application have been described in language specific to above, it is not intended to be limited to the specific form set forth herein, and it will be recognized by those of ordinary skill in the art that various of the steps and operations described below may be implemented in hardware.

The principles of the present application may be employed in numerous other general-purpose or special-purpose computing, communication environments or configurations. Examples of well known computing systems, environments, and configurations that may be suitable for use with the application include, but are not limited to, hand-held telephones, personal computers, servers, multiprocessor systems, microcomputer-based systems, mainframe-based computers, and distributed computing environments that include any of the above systems or devices.

The terms "first", "second", and "third", etc. in this application are used to distinguish between different objects and not to describe a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions.

The embodiment of the application provides an image processing method, an image processing device, a terminal and a storage medium. The image processing apparatus may be specifically integrated in a terminal capable of implementing image processing, such as a mobile phone, a tablet Computer, a Personal Digital Assistant (PDA), a notebook Computer, and a Personal Computer (PC).

For example, referring to fig. 1a, when a user uses a terminal to perform shooting, the terminal sends a shooting request to the terminal, after receiving the shooting request, the terminal obtains initial preview image data in a first color coding format in a kernel layer of a self system (i.e., the terminal system), the first color coding format being a color coding format suitable for the kernel layer, then determines a priority index of processing the initial preview image data in an application layer of the self system (i.e., the terminal system), then determines a target image conversion mode among a plurality of candidate image conversion modes of the kernel layer based on the priority index, then converts the initial preview image data into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, the second color coding format being a color coding format suitable for the application layer, and then, and displaying the preview image data at the application layer.

According to the scheme, the target image conversion mode can be selected in the plurality of candidate image conversion modes according to different priority indexes of the initial preview image data processing, and the format of the initial preview image data is converted in the kernel layer of the terminal system, so that the efficiency of the terminal image processing is effectively improved.

The following are detailed below. The order of the following examples is not intended to limit the preferred order of the examples.

In the present embodiment, description will be made from the viewpoint of an image processing apparatus that can be integrated in a terminal that can be a mobile phone, a tablet computer, a palmtop computer, a notebook computer, a personal computer, or the like that can implement image processing.

An embodiment of the present application provides an image processing method, including: the method comprises the steps of obtaining initial preview image data in a first color coding format in a kernel layer of a terminal system, determining a priority index of processing the initial preview image data in an application layer of the terminal system, determining a target image conversion mode in a plurality of candidate image conversion modes of the kernel layer based on the priority index, converting the initial preview image data into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, and displaying the preview image data in the application layer.

Referring to fig. 1b, fig. 1b is a first flowchart of an image processing method according to an embodiment of the present disclosure. The specific process of the method can be as follows:

101. the method comprises the steps of obtaining initial preview image data in a first color coding format in a kernel layer of a terminal system, wherein the first color coding format is a color coding format suitable for the kernel layer.

When a camera of the terminal is used for shooting, the terminal system can be subjected to environment building before shooting, for example, a shooting interface of the terminal system can be configured, and a shooting imaging format of the terminal system is set to be a first color coding format.

For example, the first color coding format may be a YUV format. Among them, YUV is a color coding method. Are often used in various video processing components. YUV allows for reduced bandwidth of chrominance in view of human perception when encoding photos or videos. YUV is a kind of compiled true-color space (color space), and the proper terms such as Y' UV, YUV, YCbCr, YPbPr, etc. may be called YUV, overlapping with each other. "Y" represents brightness (Luma) or gray scale value, and "U" and "V" represent Chroma (Chroma or Chroma) and are used to describe the color and saturation of the image for specifying the color of the pixel.

For example, when the user executes the shooting function, a shooting request may be sent to the terminal, and the terminal receives the shooting request and obtains the current initial preview image data in the first color coding format in the kernel layer of the terminal system according to the shooting request. That is, when a photographing operation of a user for a terminal is detected, initial preview image data in a first color coding format is acquired in a kernel layer of the terminal system.

Because the algorithm for processing the image is based on the YUV format, and all display panels receive the RGB format data, after the image data is acquired, the data stream in the YUV format is generally output, and then other steps such as compression coding and the like are performed to transmit or store the data. The final display is presented on the screen, again typically in RGB format. Therefore, it is necessary to convert the acquired YUV format data into RGB format data for display on the terminal screen, i.e., convert the initial preview image data in the first color coding format into the preview image data in the second color coding format.

102. And determining the priority index of the initial preview image data processing at the application layer of the terminal system.

The priority index refers to a processing standard that is preferentially selected by the terminal system when processing the initial preview image data, and may include an image processing speed and an image processing accuracy, for example.

For example, the display requirement of the initial preview image data may be specifically determined at an application layer of the terminal system, and the priority index for processing the initial preview image data may be determined according to the display requirement.

The display requirement refers to a requirement that the preview image data is displayed on the terminal screen, for example, whether the preview image data only needs to be converted into a format, or needs to retain precision, and the like. For example, when the format of the initial preview image data only needs to be converted, the priority index for processing the initial preview image data is determined as the image processing speed, or when the accuracy of the initial preview image data needs to be maintained, the priority index for processing the initial preview image data is determined as the image processing accuracy, and so on.

103. Based on the priority index, a target image conversion mode is determined among a plurality of candidate image conversion modes of the kernel layer.

For example, when the priority index is the image processing speed, the target image conversion mode may be determined to be the first image conversion mode from the plurality of candidate image conversion modes; when the priority index is the image processing precision, the target image conversion mode is determined to be the second image conversion mode from the candidate image conversion modes.

The candidate image conversion mode refers to a preset image conversion mode, and the setting mode of the candidate image conversion mode may be various, for example, the candidate image conversion mode may be flexibly set according to the requirement of the actual application, and may also be preset and stored in the terminal. In addition, the candidate image conversion mode may be built in the terminal, or may be stored in a memory and transmitted to the terminal, and the like.

For example, the first image conversion mode may specifically be a system conversion algorithm of the terminal system, and the second image conversion mode may specifically be a conversion algorithm which is customized in advance and stored in the terminal system, which is referred to as a customized conversion algorithm for short.

104. And converting the initial preview image data into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, wherein the second color coding format is a color coding format suitable for the application layer.

For example, when the target image conversion mode is the first image conversion mode, the system conversion algorithm of the terminal system is used in the kernel layer to convert the initial preview image data to obtain preview image data in the second color coding format, and the system conversion algorithm is a format conversion algorithm carried by the terminal system; and when the target image conversion mode is a second image conversion mode, converting the initial preview image data in the kernel layer by using a custom conversion algorithm to obtain preview image data in a second color coding format, wherein the custom conversion algorithm is an algorithm which customizes conversion precision in advance and stores the conversion precision in a terminal system.

The system conversion algorithm may include calling a conversion script and calling a target interface function, where the target interface function is an interface function obtained by modifying an interface function of the terminal system. For example, when the optimization index of the initial preview image data is the image processing speed, it can be determined whether the initial preview image data needs to be cropped. When the initial preview image data does not need to be cut, a conversion script of the system can be directly called to convert the format of the initial preview image data. When the initial preview image data needs to be cut, the target interface function modified by the system interface function can be called to perform format conversion on the initial preview image data.

For example, when the initial preview image data does not need to be cut, a conversion script is called in the kernel layer to convert the initial preview image data, so as to obtain preview image data in a second color coding format; when the initial preview image data needs to be cut, a target interface function is called in the kernel layer to convert the initial preview image data to obtain preview image data in a second color coding format, wherein the target interface function is an interface function obtained after an interface function of a terminal system is modified.

The method includes the steps of calling a target interface function to convert initial preview image data, converting parameters of the initial preview image data in a first color coding format into parameter values, converting the format in an interface provided by an execution system, and controlling a cropping area by inputting offset values of xOffset and yOffset so as to obtain preview image data in a second color coding format. For example, when the initial preview image data in the first color encoding format is YUV format data, Y, U, V three components in the YUV data may be separated and converted into RGB format data.

For example, when the target image conversion mode is the first image conversion mode, a target interface function is called at the kernel layer to separate parameters in the initial preview image data in the first color coding format, so as to obtain a luminance component, a saturation component and a chrominance component; carrying out format conversion on the brightness component, the saturation component and the chrominance component to obtain target preview image data in a second color coding format; and cutting the target preview image data in the second color coding format according to the preset offset to obtain the preview image data in the second color coding format.

The preset offset may be set in various ways, for example, the preset offset may be flexibly set according to the requirements of the actual application, or may be preset and stored in the terminal. In addition, the preset offset may be built in the terminal, or may be stored in a memory and transmitted to the terminal, and so on.

For example, when the optimization index of the initial preview image data is the image processing precision, a custom conversion mode, for example, a custom conversion formula, may be used to achieve preservation of the image precision. Wherein, the custom conversion formula can be as follows:

r＝y+(1.370705*v)

g＝y-(0.337633*u)–(0.698001*v)

b＝y+(1.732446*u)

for example, when the target image conversion mode is the second image conversion mode, a target interface function is called at the kernel layer to separate parameters in the initial preview image data in the first color coding format, so as to obtain a luminance component, a saturation component and a chrominance component; carrying out format conversion on the brightness component, the saturation component and the chrominance component by using a user-defined conversion mode to obtain target preview image data in a second color coding format, wherein the user-defined conversion mode is a conversion mode which carries out user definition on the precision of format conversion in advance and stores the target preview image data in a terminal system; and cutting the target preview image data in the second color coding format according to the preset offset to obtain the preview image data in the second color coding format.

105. And displaying the preview image data at the application layer.

Since the photographed scene may have mirror images and rotation, the RGB data obtained after the preprocessing needs to be spatially transformed. Because the rotation and mirror image principle of the RGB array is to transform the coordinates of the RGB data, the transformation of the coordinates is realized, and the data processing operation can be well realized. The spatial transformation refers to processing operations such as mirroring and rotation performed on the preview image data.

For example, it may be specifically determined whether the preview image data needs to be spatially transformed, when the preview image data needs to be spatially transformed, the coordinates of each pixel point in the preview image data are obtained, and the coordinates of each pixel point in the preview image data are subjected to coordinate transformation to obtain transformed preview image data, and at this time, the preview image data is displayed on the application layer, and specifically, the transformed preview image data may be displayed on the application layer. When the preview image data does not need to be subjected to spatial transformation, the preview image data is directly displayed on the application layer.

As can be seen from the above, in this embodiment, the initial preview image data in the first color coding format is obtained in the kernel layer of the terminal system, where the first color coding format is a color coding format suitable for the kernel layer, then the application layer of the terminal system determines the priority index for processing the initial preview image data, then, based on the priority index, determines the target image conversion manner among a plurality of candidate image conversion manners of the kernel layer, and then, based on the target image conversion manner, converts the initial preview image data in the kernel layer into the preview image data in the second color coding format, where the second color coding format is a color coding format suitable for the application layer, and then, displays the preview image data in the application layer. According to the scheme, the target image conversion mode can be selected in the plurality of candidate image conversion modes according to different priority indexes of the initial preview image data processing, and the format of the initial preview image data is converted in the kernel layer of the terminal system, so that the efficiency of the terminal image processing is effectively improved.

According to the method described in the foregoing embodiment, the image processing apparatus is specifically integrated in a terminal, and the terminal system is an Android system, which is described in further detail below.

The image processing method provided by the embodiment of the application can comprise the following steps:

firstly, an environment building before photographing is needed, which specifically comprises the following steps:

the Camera Api is configured as Camera Api2, and the format of the last imaging carrier YUV _420_888 is selected to replace the JPEG format in the conventional manner, that is, the initial preview image data acquired when the user shoots is in YUV _420_888 format.

The acquired initial image data is loaded by YUV data, and the operations of image processing and format conversion are realized by using a RenderScript technology of an Android system, so that the optimization of the post-processing capability and the generation speed of the image can be realized. Among them, RenderScript is an efficient computing platform provided by android. The method has the obvious characteristics that various cores including a CPU, a GPU, a DSP and the like can be automatically utilized to perform parallel computation, the high-efficiency computing capability provided in the fields of picture processing, mathematical models and the like can be greatly improved, different codes do not need to be written aiming at different core platforms, and the RenderScript is compiled when running on equipment.

Secondly, processing the image data of the terminal through the established environment, which can be seen in fig. 2a and 2 b.

Referring to fig. 2a, fig. 2a is a second flow chart of an image processing method according to an embodiment of the present disclosure. The specific process can be as follows:

201. the terminal receives a shooting request triggered by a user.

For example, when a user triggers a photographing function of a terminal camera, the terminal may generate a photographing request. When the terminal detects that the user triggers the shooting function, executing step 202; if the terminal does not detect the user's trigger operation, the terminal does not execute step 202 for the moment.

202. After receiving the shooting request, the terminal obtains the initial preview image data in the first color coding format in the kernel layer of its own system (i.e., the terminal system), and then executes step 203.

The first color coding format is a color coding format suitable for the kernel layer, and for example, the first color coding format may be YUV _420_888 format.

For example, after receiving the shooting request, the terminal receives the initial preview image data in the YUV _420_888 format in the kernel layer of the own system (i.e., the terminal system). The terminal can be divided into a data preprocessing stage and a data processing stage according to the service situation.

In the data preprocessing stage, data in YUV _420_888 format in Image (Image) format is acquired through an ImageReader, and corresponding Y, U, V component data are filled into a parameter of render, and the logic can select 3 modes according to business conditions, namely a pure API, a half API and a full-custom mode. For example, the service only needs to convert the format, and a pure API manner (i.e. a call conversion script in a system conversion algorithm) can be used; the service needs to convert the format and also needs to execute operations such as cutting, rotation and the like, and a semi-API mode (namely calling a target interface function in a system conversion algorithm) can be used; the service needs to convert the format and also needs to perform operations such as clipping and rotation, and a full-custom mode (i.e., a custom conversion algorithm) can be used when the precision needs to be maintained, and a specific flow can be shown in fig. 2 b.

The ImageReader is a class in Android, and allows an application program to directly access image data of the presentation surface.

203. The terminal determines the priority index of the initial preview image data processing at the application layer of the system (namely, the terminal system).

For example, the terminal may specifically determine a display requirement of the initial preview image data at an application layer of its own system (i.e., a terminal system), and determine a priority index for processing the initial preview image data according to the display requirement.

For example, the service only needs to convert the format, for example, the preview image data is displayed in full screen in the application layer, and the priority index of the initial preview image data processing is determined as the image processing speed under the condition that the preview image data does not need to be cut; for example, the service needs to convert the format and also needs to perform operations such as cropping and rotation, for example, adapting to the size of screen display, and when the image needs to be cropped, the priority index of the initial preview image data processing is determined as the image processing speed; for another example, when the initial preview image data needs to maintain the precision, if the precision cannot be lost while the format is converted, the priority index of the initial preview image data processing is determined as the image processing precision, and so on.

204. Based on the priority index, the terminal determines a target image conversion mode among a plurality of candidate image conversion modes of the kernel layer.

For example, when the priority index is the image processing speed, the terminal may specifically determine that the target image conversion mode is the first image conversion mode from the multiple candidate image conversion modes, that is, perform conversion by using a system conversion algorithm of its own system; when the priority index is the image processing precision, the target image conversion mode is determined to be the second image conversion mode from the candidate image conversion modes, namely, the conversion is carried out by using a conversion algorithm which is customized in advance and stored in the terminal system.

205. Based on the target image conversion mode, the terminal converts the initial preview image data into the preview image data in the second color coding format in the kernel layer.

For example, when the target image conversion mode is the first image conversion mode, the terminal may specifically convert the initial preview image data in the kernel layer by using a system conversion algorithm of the terminal system to obtain the preview image data in the second color coding format.

The system conversion algorithm may include calling a conversion script and calling a target interface function, where the target interface function is an interface function obtained by modifying an interface function of the terminal system.

For example, when the target image conversion mode is the first image conversion mode, the terminal may determine whether the initial preview image data needs to be cropped.

(1) When the initial preview image data does not need to be cut, a conversion script of the system can be directly called to convert the format of the initial preview image data. For example, the service only needs to convert the format, and can operate using the conversion algorithm of the render itself, and the conversion can be performed by referring to the conversion algorithm of the system itself, and the following part is referred to:

(2) when the initial preview image data needs to be cut, the target interface function modified by the system interface function can be called to perform format conversion on the initial preview image data. For example, when the target image conversion mode is the first image conversion mode and the initial preview image data needs to be cut, the terminal may specifically call a target interface function at the kernel layer to separate parameters in the initial preview image data in the first color coding format, so as to obtain a luminance component, a saturation component, and a chrominance component; carrying out format conversion on the brightness component, the saturation component and the chrominance component to obtain target preview image data in a second color coding format; and cutting the target preview image data in the second color coding format according to the preset offset to obtain the preview image data in the second color coding format.

For example, when a service needs to convert a format and also needs to perform operations such as clipping and rotation, an API of the system may be modified to customize a set of algorithms suitable for the service scenario, the parameters are converted into corresponding parameter values by rsGetElementAtYuv _ uchar _ Y and rsGetElementAt _ uchar of rendercript, the format is converted by executing an interface rsyuvttorba _ uchar provided by the system, and then a clipping region is controlled by an offset of input xOffset and yOffset. Wherein, the partial code can be as follows:

(II) for example, when the target image conversion mode is the second image conversion mode, the terminal may specifically convert the initial preview image data in the kernel layer by using a custom conversion algorithm to obtain the preview image data in the second color coding format.

For example, when the optimization index of the initial preview image data is image processing precision, a custom conversion mode may be used, for example, when a service needs to convert a format and also needs to perform operations such as cropping and rotation, and precision is lost, a full-custom algorithm mode is used, Renderscript is used as an operation platform, and the conversion formulas of customized Yuv and Rgb are used to replace an important implementation part of a mode of calling a target interface function, so as to implement precision reservation, and other cropping is implemented according to the same offset mode using the calling target interface function. Wherein, the custom conversion formula can be as follows:

206. and when the preview image data needs to be subjected to space transformation, the terminal performs coordinate transformation on the coordinates of each pixel point in the preview image data to obtain transformed preview image data.

The operation of the data processing stage is mainly defined according to business requirements, and since the photographed scene has the conditions of mirroring and rotation, the preprocessed RGB data needs to be converted. Because the rotation and mirror image principle of the RGB array is to transform the coordinates of the RGB data, the coordinates can be transformed by using rsGetElementAt in rendercript, and the processing operation on the data can be well realized. The spatial transformation refers to processing operations such as mirroring and rotation performed on the preview image data.

For example, the terminal may specifically determine whether the preview image data needs to be subjected to spatial transformation, and when the preview image data needs to be subjected to spatial transformation, obtain coordinates of each pixel point in the preview image data, and perform coordinate transformation on the coordinates of each pixel point in the preview image data to obtain transformed preview image data. For example, the partial implementation code of the preview image data that needs to be mirrored may be as follows:

207. and the terminal displays the converted preview image data on the application layer.

For example, when the preview image data needs to be spatially transformed, the transformed preview image data is displayed on the application layer; when the preview image data does not need to be subjected to spatial transformation, the preview image data is directly displayed on an application layer.

The image processing process is accelerated by using the render technology (algorithm five), and the acceleration effect can be about 3-4 times that of the traditional Bitmap conversion algorithm (algorithm one), as shown in the following table 1:

TABLE 1

	Algorithm 1	Algorithm two	Algorithm III	Algorithm four	Algorithm five
						Terminal 1	272.5	276.5	169.5	147	97
Terminal 2	350.5	280.5	192.5	172.5	99.5

The second algorithm is a mode of only using a call conversion script in the system conversion algorithm, the third algorithm is a mode of only using a call target interface function in the system conversion algorithm, and the fourth algorithm is a mode of only using a self-defined conversion algorithm.

In detail, by clipping and optimizing scenes such as data acquisition, format conversion and the like, the time consumption of partial nodes can be further reduced, and an acceleration effect close to 1 time that of an initial algorithm can be realized, as shown in table 2 below:

TABLE 2

At present, the operation of reading and outputting intermediate data can be reduced by integrating a plurality of algorithms in rendercript, and the optimization effect is more obvious than that of the existing scheme along with the increase of image operation business. When operations such as image format conversion, space conversion and the like are performed by using render, multithread processing can be performed on each pixel point in the initial preview image data and the preview image data, namely, all the pixel points are processed simultaneously, so that the image processing speed can be greatly increased, and the image processing efficiency is improved.

As can be seen from the above, the terminal of this embodiment may obtain, in a kernel layer of a self system (i.e., a terminal system), initial preview image data in a first color coding format, where the first color coding format is a color coding format suitable for the kernel layer, then determine, in an application layer of the self system (i.e., the terminal system), a priority index for processing the initial preview image data, then determine, based on the priority index, a target image conversion manner among a plurality of candidate image conversion manners in the kernel layer, then convert, in the kernel layer, the initial preview image data into preview image data in a second color coding format, where the second color coding format is a color coding format suitable for the application layer, based on the target image conversion manner, and then display the preview image data in the application layer. According to the scheme, the target image conversion mode can be selected in the plurality of candidate image conversion modes according to different priority indexes of the initial preview image data processing, and the format of the initial preview image data is converted in the kernel layer of the terminal system, so that the efficiency of the terminal image processing is effectively improved.

In order to better implement the image processing method provided by the embodiment of the present application, an embodiment of the present application further provides an image processing apparatus, where the image processing apparatus may be integrated in a terminal, and the terminal may be a device capable of implementing image processing, such as a mobile phone, a tablet computer, a palmtop computer, a notebook computer, and a personal computer. The terms are the same as those in the image processing method, and details of implementation can be referred to the description in the method embodiment.

For example, as shown in fig. 3, the interface calling apparatus may include an acquisition unit 301, a first determination unit 302, a second determination unit 303, a conversion unit 304, and a display unit 305, as follows:

(1) an acquisition unit 301;

an obtaining unit 301, configured to obtain, in a kernel layer of a terminal system, initial preview image data in a first color coding format, where the first color coding format is a color coding format suitable for the kernel layer.

Optionally, in some embodiments, in the image processing apparatus, a setting unit 306 is further included, as follows:

the setting unit 306 is configured to configure a shooting interface of the terminal system; setting a shooting imaging format of the terminal system to be a first color coding format;

the obtaining unit 301 is specifically configured to obtain, in a kernel layer of the terminal system, initial preview image data in a first color coding format when a shooting operation of a user for the terminal is detected.

(2) A first determination unit 302;

a first determining unit 302, configured to determine a priority indicator of the initial preview image data processing at an application layer of a terminal system.

Optionally, in some embodiments, in the image processing apparatus, the first determining unit 302 is specifically configured to determine, at an application layer of a terminal system, a display requirement of the initial preview image data; and determining a priority index of the initial preview image data processing according to the display requirement.

(3) A second determination unit 303;

a second determining unit 303, configured to determine a target image conversion manner among a plurality of candidate image conversion manners of the kernel layer based on the priority index.

Optionally, in some embodiments, in the image processing apparatus, the priority index includes an image processing speed and an image processing precision, and the second determining unit 303 includes a first determining subunit and a second determining subunit, as follows:

(4) A conversion unit 304;

a converting unit 304, configured to convert, in the kernel layer, the initial preview image data into preview image data in a second color coding format based on the target image conversion manner, where the second color coding format is a color coding format suitable for the application layer.

(5) A display unit 305;

a display unit 305, configured to display the preview image data in the application layer.

Optionally, in some embodiments, in the image processing apparatus, a spatial transform unit 307 is further included, as follows:

the space transformation unit 307 is configured to obtain coordinates of each pixel point in the preview image data when the preview image data needs to be subjected to space transformation; carrying out coordinate transformation on the coordinates of each pixel point in the preview image data to obtain transformed preview image data;

the display unit 305 is configured to display the transformed preview image data at the application layer.

In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.

As can be seen from the above, the image processing apparatus of the present embodiment can acquire, by the acquisition unit 301, the initial preview image data in the core layer of the terminal system in the first color coding format, which is a color coding format suitable for the core layer, then, a priority index of the initial preview image data processing is determined at the application layer of the terminal system by the first determining unit 302, and then, based on the priority index, a target image transformation mode is determined among the plurality of candidate image transformation modes of the kernel layer, and then the transformation unit 304 transforms the image data based on the target image transformation mode, converting the initial preview image data into preview image data in a second color-coded format in the kernel layer, the second color-coded format is a color-coded format suitable for the application layer, and then the preview image data is displayed on the application layer by the display unit 305. According to the scheme, the target image conversion mode can be selected in the plurality of candidate image conversion modes according to different priority indexes of the initial preview image data processing, and the format of the initial preview image data is converted in the kernel layer of the terminal system, so that the efficiency of the terminal image processing is effectively improved.

Correspondingly, an embodiment of the present application further provides a terminal, as shown in fig. 4, where the terminal may be used to implement the image processing method and apparatus provided in the foregoing embodiments. The terminal may include Radio Frequency (RF) circuitry 401, memory 402 including one or more computer-readable storage media, input unit 403, display unit 404, sensor 405, audio circuitry 406, Wireless Fidelity (WiFi) module 407, processor 408 including one or more processing cores, and power supply 409. Those skilled in the art will appreciate that the terminal configuration shown in fig. 4 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:

the RF circuit 401 may be used for receiving and transmitting signals during a message transmission or communication process, and in particular, for receiving downlink information of a base station and then sending the received downlink information to the one or more processors 408 for processing; in addition, data relating to uplink is transmitted to the base station. In general, the RF circuitry 401 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 401 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.

The memory 402 may be used to store software programs and modules, and the processor 408 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the terminal, etc. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 408 and the input unit 403 access to the memory 402.

The input unit 403 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, in a particular embodiment, the input unit 403 may include a touch-sensitive surface as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations by a user (e.g., operations by a user on or near the touch-sensitive surface using a finger, a stylus, or any other suitable object or attachment) thereon or nearby, and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts it to touch point coordinates, and sends the touch point coordinates to the processor 408, and can receive and execute commands from the processor 408. In addition, touch sensitive surfaces may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 403 may include other input devices in addition to the touch-sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 404 may be used to display information input by or provided to the user and various graphical user interfaces of the terminal, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 404 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch-sensitive surface may overlay the display panel, and when a touch operation is detected on or near the touch-sensitive surface, the touch operation is transmitted to the processor 408 to determine the type of touch event, and then the processor 408 provides a corresponding visual output on the display panel according to the type of touch event. Although in FIG. 4 the touch-sensitive surface and the display panel are shown as two separate components to implement input and output functions, in some embodiments the touch-sensitive surface may be integrated with the display panel to implement input and output functions.

The terminal may also include at least one sensor 405, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or the backlight when the terminal is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured in the terminal, detailed description is omitted here.

Audio circuitry 406, a speaker, and a microphone may provide an audio interface between the user and the terminal. The audio circuit 406 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electric signal, which is received by the audio circuit 406 and converted into audio data, which is then processed by the audio data output processor 408, and then transmitted to, for example, another terminal via the RF circuit 401, or the audio data is output to the memory 402 for further processing. The audio circuitry 406 may also include an earbud jack to provide peripheral headset communication with the terminal.

WiFi belongs to short distance wireless transmission technology, and the terminal can help the user to send and receive e-mail, browse web page and access streaming media etc. through WiFi module 407, it provides wireless broadband internet access for the user. Although fig. 4 shows the WiFi module 407, it is understood that it does not belong to the essential constitution of the terminal, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 408 is a control center of the terminal, connects various parts of the entire handset using various interfaces and lines, and performs various functions of the terminal and processes data by operating or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby integrally monitoring the handset. Optionally, processor 408 may include one or more processing cores; preferably, the processor 408 may integrate an application processor, which handles primarily the operating system, user interface, applications, etc., and a modem processor, which handles primarily the wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 408.

The terminal also includes a power source 409 (e.g., a battery) for powering the various components, which may preferably be logically coupled to the processor 408 via a power management system to manage charging, discharging, and power consumption via the power management system. The power supply 409 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

Although not shown, the terminal may further include a camera, a bluetooth module, and the like, which will not be described herein. Specifically, in this embodiment, the processor 408 in the terminal loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 408 runs the application programs stored in the memory 402, thereby implementing various functions:

the method comprises the steps of obtaining initial preview image data in a first color coding format in a kernel layer of a terminal system, determining a priority index of processing the initial preview image data in an application layer of the terminal system, determining a target image conversion mode in a plurality of candidate image conversion modes of the kernel layer based on the priority index, converting the initial preview image data into preview image data in a second color coding format in the kernel layer based on the target image conversion mode, and displaying the preview image data in the application layer.

The above operations can be referred to the previous embodiments specifically, and are not described herein again.

As can be seen from the above, the terminal of this embodiment may obtain, in a kernel layer of a self system (i.e., a terminal system), initial preview image data in a first color coding format, where the first color coding format is a color coding format suitable for the kernel layer, then determine, in an application layer of the self system (i.e., the terminal system), a priority index for processing the initial preview image data, then determine, based on the priority index, a target image conversion manner among a plurality of candidate image conversion manners of the kernel layer, and then convert, based on the target image conversion manner, the initial preview image data in the kernel layer into preview image data in a second color coding format, where the second color coding format is a color coding format suitable for the application layer, and then display the preview image data in the application layer. According to the scheme, the target image conversion mode can be selected in the plurality of candidate image conversion modes according to different priority indexes of the initial preview image data processing, and the format of the initial preview image data is converted in the kernel layer of the terminal system, so that the efficiency of the terminal image processing is effectively improved.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a storage medium, where a plurality of instructions are stored, and the instructions can be loaded by a processor to execute any step of the method for sharing network traffic resources by a sharing terminal provided in the embodiment of the present application. For example, the instructions may perform the steps of:

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the storage medium can execute any step applied to the image processing method provided in the embodiments of the present application, the beneficial effects that any applied image processing method provided in the embodiments of the present application can achieve can be achieved, for details, see the foregoing embodiments, and are not described herein again.

The foregoing detailed description has provided an image processing method, an image processing apparatus, a terminal and a storage medium according to embodiments of the present application, and specific examples are applied herein to explain the principles and implementations of the present application, and the description of the foregoing embodiments is only used to help understand the method and the core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, comprising:

and displaying the preview image data in the application layer.

2. The method of claim 1, wherein the priority index includes an image processing speed and an image processing accuracy, and wherein determining a target image transformation mode among a plurality of candidate image transformation modes of the kernel layer based on the priority index comprises:

when the priority index is the image processing speed, determining a target image conversion mode as a first image conversion mode from a plurality of candidate image conversion modes;

and when the priority index is the image processing precision, determining the target image conversion mode as a second image conversion mode from a plurality of candidate image conversion modes.

3. The method according to claim 2, wherein said converting the initial preview image data into the preview image data in the second color-coded format in the kernel layer based on the target image conversion manner comprises:

when the target image conversion mode is a first image conversion mode, converting the initial preview image data in the kernel layer by using a system conversion algorithm of a terminal system to obtain preview image data in a second color coding format, wherein the system conversion algorithm is a format conversion algorithm carried by the terminal system;

and when the target image conversion mode is a second image conversion mode, converting the initial preview image data in the kernel layer by using a custom conversion algorithm to obtain preview image data in a second color coding format, wherein the custom conversion algorithm is an algorithm which customizes conversion precision in advance and stores the conversion precision in a terminal system.

4. The method of claim 3, wherein the system conversion algorithm comprises calling a conversion script and calling a target interface function, and the converting the initial preview image data in the kernel layer by using the conversion algorithm of the terminal system to obtain the preview image data in the second color-coded format comprises:

when the initial preview image data does not need to be cut, calling a conversion script in the kernel layer to convert the initial preview image data to obtain preview image data in a second color coding format;

when the initial preview image data needs to be cut, calling a target interface function in the kernel layer to convert the initial preview image data to obtain preview image data in a second color coding format, wherein the target interface function is an interface function obtained by modifying an interface function of a terminal system.

5. The method according to claim 4, wherein when the initial preview image data needs to be cropped, calling a target interface function in the kernel layer to convert the initial preview image data to obtain preview image data in a second color coding format, including:

when the target image conversion mode is a first image conversion mode, calling a target interface function at the kernel layer to separate parameters in the initial preview image data in the first color coding format to obtain a brightness component, a saturation component and a chrominance component;

carrying out format conversion on the brightness component, the saturation component and the chrominance component to obtain target preview image data in a second color coding format;

and cutting the target preview image data in the second color coding format according to a preset offset to obtain the preview image data in the second color coding format.

6. The method of claim 3, wherein the custom conversion algorithm comprises invoking a target interface function and a custom conversion mode, and when the target image conversion mode is a second image conversion mode, converting the initial preview image data in the kernel layer using the custom conversion algorithm to obtain the preview image data in the second color-coded format comprises:

when the target image conversion mode is a second image conversion mode, calling a target interface function at the kernel layer to separate parameters in the initial preview image data in the first color coding format to obtain a brightness component, a saturation component and a chrominance component;

carrying out format conversion on the brightness component, the saturation component and the chrominance component by using a custom conversion mode to obtain target preview image data in a second color coding format, wherein the custom conversion mode is a conversion mode which carries out custom-defining on the precision of format conversion in advance and stores the conversion mode in a terminal system;

7. The method of claim 1, wherein determining the priority indicator of the initial preview image data processing at an application layer of a terminal system comprises:

determining the display requirement of the initial preview image data at an application layer of a terminal system;

and determining a priority index of the initial preview image data processing according to the display requirement.

8. The method according to claim 1, wherein after converting the initial preview image data into the preview image data in the second color-coding format in the kernel layer based on the target image conversion manner, further comprising:

when the preview image data needs to be subjected to space transformation, acquiring coordinates of each pixel point in the preview image data;

carrying out coordinate transformation on the coordinates of each pixel point in the preview image data to obtain transformed preview image data;

the displaying the preview image data at the application layer includes: and displaying the transformed preview image data on the application layer.

9. The method of claim 1, wherein prior to obtaining the initial preview image data in the first color-coded format in a kernel layer of the terminal system, further comprising:

configuring a shooting interface of a terminal system; and

setting a shooting imaging format of the terminal system to be a first color coding format;

the acquiring of the initial preview image data in the first color coding format in the kernel layer of the terminal system includes: when the shooting operation of a user for a terminal is detected, acquiring initial preview image data in a first color coding format in a kernel layer of the terminal system.

10. An image processing apparatus characterized by comprising:

11. A terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method according to any of claims 1 to 9 are implemented when the program is executed by the processor.

12. A computer-readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the image processing method according to any one of claims 1 to 9.