CN110475065B

CN110475065B - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN110475065B
Application number: CN201910763721.0A
Authority: CN
Inventors: 张兴华; 王学智; 黎文学
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Douyin Vision Co Ltd; Douyin Vision Beijing Co Ltd
Priority date: 2019-08-19
Filing date: 2019-08-19
Publication date: 2021-03-16
Anticipated expiration: 2039-08-19
Also published as: WO2021031850A1; CN110475065A

Abstract

The embodiment of the disclosure provides an image processing method, an image processing device, an electronic device and a storage medium; the method comprises the following steps: acquiring an original image frame acquired in an image previewing process; performing special effect rendering operation on the original image frame to obtain a special effect image frame, and performing drawing operation of the special effect image frame in a display area; detecting a preview frame rate for presenting the original image frame in an image preview process and a drawing frame rate for presenting the special effect image frame in a display area; and controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on the difference value between the drawing frame rate and the previewing frame rate. The method and the device can intelligently balance the resolution and the frame rate in the special effect processing process.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.

Background

The system camera usually draws the acquired image data directly on a screen without intermediate processing, so that the overall performance loss is less, the frame rate is guaranteed to a certain extent in the resolution range supported by the system, and the problems of blocking and the like caused by frame loss are less.

The beauty camera generally has the functions of face recognition and special effect addition, which consumes a high amount of CPU, and because the processes are composed of complex algorithms or graphic processing, delay is inevitably generated in the special effect processing stage, so that a user has obvious click feeling when using the beauty camera.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

acquiring an original image frame acquired in an image previewing process;

performing special effect rendering operation on the original image frame to obtain a special effect image frame, and performing drawing operation of the special effect image frame in a display area;

detecting a preview frame rate for presenting the original image frame in an image preview process and a drawing frame rate for presenting the special effect image frame in a display area;

and controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on the difference value between the drawing frame rate and the previewing frame rate.

In the above scheme, the acquiring of the original image frame acquired in the image preview process includes:

triggering a camera application to start an image preview window by calling an application program interface of the camera application;

and acquiring the original image frame acquired by the camera application in the image preview window through the application program interface.

In the above scheme, the performing a special effect rendering operation on the original image frame to obtain a special effect image frame includes:

detecting a target object in the original image frame;

and in the area of the original image frame corresponding to the target object, performing special effect rendering operation of at least one of a filter, a beauty and a sticker to obtain the special effect image frame.

In the above solution, the detecting a target object in the original image frame includes:

acquiring a detection frame rate in the process of detecting the target object;

when the detection frame rate is smaller than the preview frame rate, based on the difference value between the detection frame rate and the preview frame rate, performing frame loss processing on an original image frame acquired in the image preview process;

and detecting a target object in the original image frame subjected to frame loss processing.

In the above solution, the processing of frame loss on the collected original image frame based on the difference between the detection frame rate and the preview frame rate includes:

carrying out interval frame loss processing on the original image frames acquired every second, wherein the number of the original image frames subjected to the frame loss processing every second is greater than or equal to the difference value;

alternatively, the first and second electrodes may be,

and continuously dropping the frames of the acquired original image frames per second, wherein the number of the original image frames per second processed by dropping the frames is more than or equal to the difference value.

In the foregoing solution, the controlling, based on the difference between the rendering frame rate and the preview frame rate, the resolution for acquiring a new original image frame in the image preview process includes:

and controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on different conditions of the value of the rendering frame rate, the difference value between the rendering frame rate and the previewing frame rate and the value of the resolution ratio for acquiring the original image frame.

In the foregoing solution, the controlling, based on different situations of the value of the rendering frame rate, the difference between the rendering frame rate and the preview frame rate, and the value of the resolution of the collected original image frame, the resolution of the collected new original image frame in the process of image preview includes:

when a first preset condition is met, controlling the resolution ratio for acquiring a new original image frame to be unchanged;

wherein the first preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the collected original image frame is greater than or equal to the screen resolution;

when a second preset condition is met, improving the resolution ratio for acquiring a new original image frame;

wherein the second preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is smaller than a second preset threshold, and the resolution of the collected original image frame is smaller than the screen resolution.

In the above scheme, the method further comprises:

when the original image frames captured during the image preview are used for video recording,

and, when a second preset condition is satisfied;

and the resolution ratio for acquiring the new original image frame is improved, and the code rate for recording the video is reduced.

In the foregoing solution, the controlling, based on different situations of the value of the rendering frame rate, the difference between the rendering frame rate and the preview frame rate, and the value of the resolution of the collected original image frame, the resolution of the collected new original image frame in the image preview process includes:

when a third preset condition is met, controlling the resolution ratio for acquiring a new original image frame to be unchanged;

wherein the third preset condition comprises: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution and less than the screen resolution;

when a fourth preset condition is met, reducing the resolution ratio for acquiring a new original image frame;

wherein the fourth preset condition includes: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution or greater than or equal to the screen resolution.

when a fifth preset condition is met, controlling the resolution ratio for acquiring a new original image frame to be unchanged;

wherein the fifth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution;

when a sixth preset condition is met, reducing the resolution ratio for acquiring a new original image frame;

wherein the sixth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution.

In a second aspect, an embodiment of the present disclosure provides an apparatus for image processing, including:

the acquisition unit is used for acquiring an original image frame acquired in the image preview process;

the execution unit is used for executing special effect rendering operation on the original image frame to obtain a special effect image frame and executing drawing operation of the special effect image frame in a display area;

the detection unit is used for detecting a preview frame rate for presenting the original image frame in the image previewing process and a drawing frame rate for presenting the special effect image frame in a display area;

and the control unit is used for controlling the resolution ratio for acquiring a new original image frame in the image previewing process according to the difference value between the drawing frame rate and the previewing frame rate.

In the above scheme, the obtaining unit is further configured to trigger the camera application to start an image preview window by calling an application program interface of the camera application; and acquiring the original image frame acquired by the camera application in the image preview window through the application program interface.

In the above solution, the execution unit is further configured to detect a target object in the original image frame; and in the area of the original image frame corresponding to the target object, performing special effect rendering operation of at least one of a filter, a beauty and a sticker to obtain the special effect image frame.

In the above solution, the execution unit is further configured to obtain a detection frame rate in the process of detecting the target object; when the detection frame rate is smaller than the preview frame rate, based on the difference value between the detection frame rate and the preview frame rate, performing frame loss processing on an original image frame acquired in the image preview process; and detecting a target object in the original image frame subjected to frame loss processing.

In the above scheme, the execution unit is further configured to perform frame dropping processing on the original image frames collected every second at intervals, and the number of the original image frames processed by frame dropping per second is greater than or equal to the difference; or continuously dropping the frames of the acquired original image frames per second, wherein the number of the original image frames per second dropped is greater than or equal to the difference.

In the foregoing solution, the control unit is further configured to control the resolution of the new original image frame to be acquired in the image previewing process based on different conditions of the value of the rendering frame rate, the difference between the rendering frame rate and the preview frame rate, and the value of the resolution of the acquired original image frame.

In the above scheme, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a first preset condition is met; wherein the first preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the collected original image frame is greater than or equal to the screen resolution; when a second preset condition is met, improving the resolution ratio for acquiring a new original image frame; wherein the second preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is smaller than a second preset threshold, and the resolution of the collected original image frame is smaller than the screen resolution.

In the above scheme, the control unit is further configured to, when an original image frame acquired in an image preview process is used for video recording, and when a second preset condition is met; and the resolution ratio for acquiring the new original image frame is improved, and the code rate for recording the video is reduced.

In the above scheme, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a third preset condition is met; wherein the third preset condition comprises: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution and less than the screen resolution; when a fourth preset condition is met, reducing the resolution ratio for acquiring a new original image frame; wherein the fourth preset condition includes: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution or greater than or equal to the screen resolution.

In the above scheme, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a fifth preset condition is met; wherein the fifth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution; when a sixth preset condition is met, reducing the resolution ratio for acquiring a new original image frame; wherein the sixth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

a memory for storing executable instructions;

and the processor is used for realizing the image processing method provided by the embodiment of the disclosure when executing the executable instruction.

In a fourth aspect, the present disclosure provides a storage medium storing executable instructions, which when executed, are configured to implement the method for image processing provided by the embodiments of the present disclosure.

The embodiment of the disclosure has the following beneficial effects:

the method comprises the steps of performing special effect rendering operation on an original image frame acquired in an image previewing process to obtain a special effect image frame, performing drawing operation on the special effect image frame in a display area, and controlling the resolution ratio of a new original image frame acquired in the image previewing process based on the difference between the drawing frame rate of the special effect image frame and the previewing frame rate of the original image frame, so that the resolution ratio and the frame rate in the special effect processing process can be intelligently balanced, the satisfaction degree of a user on a camera previewing effect and video recording is relatively optimal, and better use experience is obtained.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

FIG. 1 is a schematic structural diagram of an electronic device implementing an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a component structure of an image processing apparatus implementing an embodiment of the present disclosure;

FIG. 3 is an alternative flow diagram of an image processing method implementing an embodiment of the present disclosure;

FIG. 4 is an alternative flow diagram of an image processing method implementing an embodiment of the present disclosure;

FIG. 5 is an alternative flow diagram of an image processing method implementing an embodiment of the present disclosure;

FIG. 6A is an alternative flow diagram of an image processing method implementing an embodiment of the present disclosure;

FIG. 6B is a schematic flow chart diagram of an alternative image processing method for implementing the disclosed embodiment;

fig. 6C is an alternative flow diagram of an image processing method implementing an embodiment of the disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

Before further detailed description of the embodiments of the present disclosure, terms and expressions referred to in the embodiments of the present disclosure are explained, and the terms and expressions referred to in the embodiments of the present disclosure are applied to the following explanations.

1) The preview frame rate is the number of original image frames (i.e., image frames that do not undergo special effect processing) displayed per second in the image preview window.

2) A drawing frame rate, which is the number of image frames of a special effect (i.e., image frames including a special effect formed by adding a special effect to an original image frame) to be displayed on a screen, drawn by a display component of an operating system every second.

3) And the detection frame rate is the number of original image frames for completing the detection of the target object per second.

Referring now to fig. 1, fig. 1 is a schematic diagram of an electronic device implementing an embodiment of the present disclosure. The electronic device may be various terminals including a mobile terminal such as a mobile phone, a notebook computer, a Digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a Portable Multimedia Player (PMP), a vehicle mounted terminal (e.g., a car navigation terminal), etc., and a fixed terminal such as a Digital Television (TV), a desktop computer, etc. The electronic device shown in fig. 1 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 1, the electronic device may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 110, which may perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) 120 or a program loaded from a storage device 180 into a Random Access Memory (RAM) 130. In the RAM130, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing device 110, the ROM 120, and the RAM130 are connected to each other through a bus 140. An Input/Output (I/O) interface 150 is also connected to bus 140.

Generally, the following devices may be connected to the I/O interface 150: input devices 160 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 170 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; a storage device 180 including, for example, a magnetic tape, a hard disk, or the like; and a communication device 190. The communication device 190 may allow the electronic device to communicate wirelessly or by wire with other devices to exchange data.

In particular, the processes described by the provided flowcharts may be implemented as computer software programs according to embodiments of the present disclosure. For example, the disclosed embodiments include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network through the communication device 190, or installed from the storage device 180, or installed from the ROM 120. The functions in the image processing method of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 110.

It should be noted that the computer readable medium described above in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the disclosed embodiments, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the disclosed embodiments, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including over electrical wiring, fiber optics, Radio Frequency (RF), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may be present alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to perform the image processing method provided by the embodiments of the present disclosure.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) and a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams provided by the embodiments of the present disclosure illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".

The functions described in the embodiments of the present disclosure may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field-Programmable Gate arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs)), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of embodiments of the present disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The following describes units and/or modules in an image processing apparatus implementing the embodiments of the present disclosure. It is understood that the units or modules in the image processing apparatus can be implemented in the electronic device shown in fig. 2 by software (for example, a computer program stored in the above computer software program), and also can be implemented in the electronic device shown in fig. 2 by the above hardware logic components (for example, FPGA, ASIC, SOC, and CPLD).

Referring to fig. 2, fig. 2 is an alternative structural schematic diagram of an image processing apparatus 300 implementing an embodiment of the present disclosure, showing the following units: an acquisition unit 310, an execution unit 320, a detection unit 330, and a control unit 340, the functions of each of which will be described below.

It should be noted that the above-mentioned classification of units does not constitute a limitation of the electronic device itself, for example, some units may be split into two or more sub-units, or some units may be combined into a new unit.

It should also be noted that the names of the units do not in some cases form a limitation on the units themselves, and for example, the acquiring unit 310 may also be described as a unit for "acquiring an original image frame captured during image preview".

For the same reason, units and/or modules in the electronic device, which are not described in detail, do not represent defaults of the corresponding units and/or modules, and all operations performed by the electronic device may be implemented by the corresponding units and/or modules in the electronic device.

With continuing reference to fig. 3, fig. 3 is an optional flowchart illustrating the steps shown in fig. 3, by taking an electronic device running an operating system and installed with a system camera application and an image special effect application to execute the image processing method provided by the embodiment of the present disclosure as an example, where the camera application may be a camera application used by default in the operating system, such as a native camera application (i.e., a system camera), and the image special effect application is an application capable of adding a special effect, such as a short video APP (add a special effect to a photo and a video and share a user space of a short video platform) and a beauty camera.

In step S101, the image special effect application calls an application program interface of the camera application to start a preview function of the camera application, and in step S102, the image special effect application acquires an original image frame acquired by the camera application in the process of image preview.

In some embodiments, an application program interface of the camera application is first called to trigger the camera application to start an image preview window, and raw image frames acquired by the camera application in the image preview window are acquired through the application program interface.

By way of example, referring to fig. 4, fig. 4 is an optional schematic diagram for implementing an image processing method according to an embodiment of the present disclosure, when an image special effect application installed in an electronic device is started, the image special effect application calls an application program interface of a camera application to trigger the camera application, so as to start an image preview window to display a real-time acquired original image frame, and acquire the original image frame from the application program interface of the camera application and store the original image frame in a Buffer (Buffer) of a graphic image special effect application.

In some embodiments, as an alternative to step S101 and step S102, the image special effects application may obtain the read authority of the camera from the operating system, so that the preview process may be started by directly using the camera (i.e., without calling the camera application), and the original image frame is collected during the preview process and stored in the cache of the graphic image special effects application.

As an example, the image special effects application may have a read permission of a camera of the electronic device, and bypass the native camera application to implement an image preview function, that is, directly acquire data of the original image frame from an interface of the camera in the operating system.

In step S103, the image special effect application performs a special effect rendering operation on the original image frame to obtain a special effect image frame.

In some embodiments, the image special effects application may invoke the GPU to complete a rendering operation of special effects on the raw image frames without performing target object detection on the raw image frames. For example, the special effects are uniformly added at all positions of the original image frame, including changing the style (old film), adding picture frames and the like.

In other embodiments, the image special effect application may invoke the CPU to detect a target object in the raw image frame and invoke the GPU to perform a special effect rendering operation of at least one of a filter, a beauty and a sticker in a region of the raw image frame corresponding to the target object to obtain a special effect image frame.

As an example, referring to fig. 4, the image special effect application acquires an original image frame acquired by the camera application from the camera application, stores the original image frame in a cache of the image special effect application, and calls a drawing interface of an operating system to draw the original image frame in the cache in a display area to implement a real-time preview function; when a user marks a face in a previewed original image frame, the image special effect application calls a CPU to read the original image frame from a cache and carries out face recognition, and the original image frame comprising a face recognition result is stored in a corresponding cache; then the image special effect application calls a CPU to read the original image frame from the corresponding cache, and renders the original image frame by combining the recognized face and special effects of selecting a filter, beautifying the face, pasting a paper and the like by a user.

In step S104, the image special effect application sends the rendered special effect image frame to a rendering interface of the operating system to cause the operating system to perform a rendering operation of the special effect image frame in the display area.

As an example, the display area may be an image preview area, which is an area where the original image frame is displayed when image preview is started, and when rendering of the special effect image frame is completed, the original image frame covering the image preview area is used to display the special effect image frame. The display area can also be an area independent from the image preview area, such as a preview image and a special effect image are displayed in a window, so that the acquired original image frame and the special effect image frame added with special effects can be conveniently and simultaneously viewed.

In step S105, the image special effect application detects a preview frame rate of the original image frame in the process of image preview, and detects a rendering frame rate at which the operating system renders the special effect image frame in the display area.

In some embodiments, the special effect rendered in step S103 may be performed for a target object detected in the original image frame. In the process of implementing the embodiment of the present disclosure, the applicant finds that if the CPU is called to perform target detection on an original image frame to perform special effect rendering, if the detection frame rate of the CPU is smaller than the preview frame rate, the subsequent special effect rendering and the drawing of the special effect image frame may be delayed inevitably.

In view of the above situation, referring to fig. 5, fig. 5 is an optional flowchart of the image processing method provided in the embodiment of the present application, and step S103 may be implemented by step S1031 to step S1035, which can solve the technical problem of delay caused when the CPU performs target object detection, and will be described with reference to each step.

In step S1031, the image special effect application acquires a detection frame rate in detecting the target object.

Step S1032, when the detection frame rate is smaller than the preview frame rate, based on the difference between the detection frame rate and the preview frame rate, the image special effect application performs frame dropping processing on the original image frame collected in the image preview process.

In some embodiments, when the detection frame rate is less than the preview frame rate, based on a difference between the detection frame rate and the preview frame rate, performing frame dropping processing on the original image frames acquired every second at intervals, wherein the number of the original image frames processed every second is greater than or equal to the difference; or continuously dropping the frames of the acquired original image frames per second, wherein the number of the original image frames per second dropped is greater than or equal to the difference.

As an example, referring to fig. 4, in the process of image preview, when the preview frame rate is greater than the detection frame rate, the process of the detection unit recognizing the target object from the original image frame is time-consuming. At this time, the image special effect application can call the CPU to read the original image frame from the cache of the original image frame collected by the camera application, and perform interval frame dropping processing or continuous frame dropping processing, so that the number of frames of the original image frame obtained after frame dropping per second is not more than the preview frame rate, and then the original image frame after frame dropping processing is stored in the cache of the image special effect application.

As an example, when the preview frame rate is 30 frames and the detection frame rate is 20 frames, the detection unit processes 20 frames of original images at most per second, and at this time, 10 frames of frames transmitted to the detection unit per second by the camera program may be discarded, so that the number of frames of the original image frames obtained after the frames are discarded in each second is 20 and is equal to the preview frame rate; alternatively, 12 frames per second of frames transmitted by the camera program to the detection unit may be dropped, so that the number of frames of the original image frame obtained after dropping frames per second is 18, which is smaller than the preview frame rate. The frame loss processing mode can be continuous frame loss, namely 10 continuous original image frames are discarded once in every second; the frame loss processing mode can also be frame loss at intervals, that is, 1 original image frame is lost every 2 frames in each second.

The above-mentioned selection of the determination of the difference between the rendering frame rate and the preview frame rate to perform the subsequent frame loss processing, but the selection of the determination of the difference between the rendering frame rate and the preview frame rate in fig. 4 is not to perform the determination of the difference between the rendering frame rate and the preview frame rate, because the selection of the difference between the rendering frame rate and the preview frame rate can better embody the delay that may occur between the screen display process and the process of identifying the target object, the pre-determination process of the processing apparatus 110 before performing the frame loss processing can be more reasonable and accurate by selecting the difference between the rendering frame rate and the preview frame rate. The rendering frame rate refers to the number of frames of an original image frame output per second after the special effect rendering operation is performed.

The above-mentioned method for performing frame dropping processing on the original image frame acquired in the image previewing process not only can reduce the pressure of the detection unit for identifying the target object from the original image frame, but also can improve the speed of the detection unit for identifying the target object from the original image frame.

Step S1033, the image special effect is applied to detect the target object in the original image frame after the frame loss processing.

As an example, referring to fig. 4, the image special effect application reads the original image frame after the frame dropping process from the buffer of the image special effect application, and performs face recognition in the original image frame to detect the target object.

As an example, when the preview frame rate is 30 and the detection frame rate is 20, 10 frames may be dropped from frames transferred to a Buffer (Buffer) per second by the camera program, so that the number of frames of the original image frame obtained after dropping frames per second is 20, and then the beauty camera may detect the target object in the remaining 20 frames of the original image frame by calling the CPU.

The method for detecting the target object in the original image frame after the frame loss processing not only reduces the number of frames of the original image frame which needs to be detected by the electronic equipment, further reduces the pressure of the electronic equipment for identifying the target object from the original image frame, but also can improve the speed of the electronic equipment for identifying the target object from the original image frame, thereby improving the experience of a user in the shooting process.

As an example, the way of detecting the target object may be automatic detection, i.e. automatically identifying the features of the target object, without manual operation by the user. Or, the detection may be performed according to user calibration, that is, at least one object of the special effect to be added is calibrated in the preview window according to the user, and then the object calibrated by the user is continuously detected in the subsequent acquired original image frame to render the special effect.

As an example, the type of target object may be a whole or a part of a human, an animal, and an object. Such as a human face.

Step S1034, the image special effect application calls the CPU to execute the special effect rendering operation of at least one of the filter, the beauty and the sticker in the area corresponding to the target object in the original image frame to obtain the special effect image frame.

In some embodiments, after the detection of the target object is completed, a special effect rendering operation of at least one of a filter, a beauty filter and a sticker may be performed on a region of the target corresponding to the original image frame, so as to obtain a special effect image frame after the special effect rendering operation.

As an example, after completing face recognition, operations such as skin polishing and whitening on the face of a person, or operations such as adding decorations around the face of a person may be selected according to the position of the face in the original image frame, for example: when a user opens a facial camera program to take a self-timer, a camera is started, the program automatically identifies the position of the face of the user, and then the user performs operations such as skin grinding or whitening or decoration adding according to self requirements.

In step S1035, the image special effect application calls a rendering interface of the operating system, and performs a rendering operation of the special effect image frame in the display area.

In step S106, the image special effect application controls the resolution for acquiring a new original image frame in the process of image preview based on the difference between the rendering frame rate and the preview frame rate.

In the process of implementing the application, the applicant finds that in the process of image preview and adding special effects, due to the limitation of the performance of the CPU and the GPU of the electronic device, the real-time property of displaying special effect image frames depends on the difference between the preview frame rate and the resolution of the acquired original image frames, and the larger the difference is, the worse the real-time property of displaying special effect image frames is, so that the resolution of acquiring new original image frames in the process of image preview can be controlled according to the difference between the rendering frame rate and the preview frame rate, so that the resolution is adapted to the performance of the CPU and the GPU of the electronic device to reduce the difference, and the real-time property of displaying special effect image frames is ensured.

In some embodiments, step S106 may also be implemented in such a way: and controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on different conditions of the value of the rendering frame rate, the difference value between the rendering frame rate and the previewing frame rate and the value of the resolution ratio for acquiring the original image frame (step S1061).

As an example, according to different situations of a value of a rendering frame rate, a difference between the rendering frame rate and a preview frame rate, and a value of a resolution of an acquired original image frame, a resolution for acquiring a new original image frame in the process of controlling image preview is shown in table 1.

TABLE 1 specific control strategy

Wherein, the high, medium and low values of the drawing frame rate respectively represent: a first value space, a second value space, and a third value space.

High, medium and low of the difference between the rendering frame rate and the preview frame rate respectively represent: a fourth value space, a fifth value space, and a sixth value space.

Acquiring high, medium and low values of the resolution of the original image frame respectively: a seventh value space, an eighth value space, and a ninth value space.

The relationship between the value spaces may be interval or continuous.

Referring to fig. 6A, fig. 6A is an optional flowchart of the image processing method according to the embodiment of the present application, and fig. 6A illustrates that step S1061 may be implemented by step S10611 to step S10612, which will be described with reference to the steps.

Step S10611, controlling the resolution for acquiring a new original image frame to be unchanged when a first preset condition is met; wherein, the first preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the collected original image frame is greater than or equal to the screen resolution.

In some embodiments, when the rendering frame rate is greater than a first preset threshold, the difference between the rendering frame rate and the preview frame rate is less than a second preset threshold, and the resolution for acquiring the original image frame is greater than or equal to the screen resolution, the resolution for acquiring the new original image frame is controlled to be unchanged.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the preview frame rate is 20 frames, the rendering frame rate is 19 frames, the screen resolution is 92 ten thousand pixels, the resolution of the acquired original image frame is 100 ten thousand pixels, the rendering frame rate 19 is greater than the first preset threshold 18, the difference between the rendering frame rate and the preview frame rate is 1 and less than the second preset threshold 2, and the resolution of the acquired original image frame is 100 ten thousand pixels and more than the screen resolution is 92 ten thousand pixels, at this time, the condition of acquiring the original image frame is the best state, and the resolution of acquiring the new original image frame is not adjusted.

Step S10612, when a second preset condition is met, increasing the resolution for acquiring a new original image frame; wherein the second preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is smaller than a second preset threshold, and the resolution of the collected original image frame is smaller than the screen resolution.

In some embodiments, when the rendering frame rate is greater than a first preset threshold, the difference between the rendering frame rate and the preview frame rate is less than a second preset threshold, and the resolution for acquiring the original image frame is less than the screen resolution, the resolution for acquiring the new original image frame is increased, wherein the resolution may be increased proportionally or by a certain step size.

In some embodiments, when the rendering frame rate is greater than a first preset threshold, and the difference between the rendering frame rate and the preview frame rate is less than a second preset threshold, and the resolution for acquiring the original image frame is greater than or equal to one-half of the screen resolution and less than the screen resolution, the resolution for acquiring the new original image frame is controlled to be unchanged or increased.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the preview frame rate is 20 frames, the rendering frame rate is 19 frames, the screen resolution is 92 ten thousand pixels, the resolution of the acquired original image frame is 80 ten thousand pixels, the rendering frame rate 19 is greater than the first preset threshold 18, the difference between the rendering frame rate and the preview frame rate is 1 and less than the second preset threshold 2, and the resolution of the acquired original image frame is 80 ten thousand pixels and less than the screen resolution of 92 ten thousand pixels and greater than one-half screen resolution of 46 ten thousand pixels, the resolution of acquiring the new original image frame may not be adjusted or the resolution of the acquired original image frame is increased to 100 ten thousand pixels for acquiring the new original image frame.

In some embodiments, when the rendering frame rate is greater than a first preset threshold, and the difference between the rendering frame rate and the preview frame rate is less than a second preset threshold, and the resolution for acquiring the original image frame is less than one-half of the screen resolution, the resolution for acquiring the new original image frame is increased.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the preview frame rate is 20 frames, the rendering frame rate is 19 frames, the screen resolution is 92 ten thousand pixels, the resolution of the acquired original image frame is 30 ten thousand pixels, the rendering frame rate 19 is greater than the first preset threshold 18, the difference between the rendering frame rate and the preview frame rate is 1 and less than the second preset threshold 2, and the resolution of the acquired original image frame is 30 ten thousand pixels and less than 46 ten thousand pixels of screen resolution of one half, at this time, the resolution of the acquired original image frame may be increased to 100 ten thousand pixels for acquiring a new original image frame.

In some embodiments, when the original image frames acquired during the image preview process are used for video recording, and when the rendering frame rate is greater than a first preset threshold, the difference between the rendering frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the acquired original image frames is less than the screen resolution, the resolution for acquiring new original image frames is increased, and the code rate for recording the video is reduced. The code rate may be increased proportionally or by a certain step size.

Here, the code rate of the recorded video is the number of data bits transmitted per second when the video transmission data is recorded, and the unit is kbps, that is, kilobits per second.

In some embodiments, when a video is recorded, when the drawing frame rate is greater than a first preset threshold, the difference between the drawing frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the acquired original image frame is less than the screen resolution, the resolution for acquiring a new original image frame may be increased, and when the resolution for acquiring a new original image frame is increased, the resolution may be too high to cause an unclear video recording, and at this time, the code rate for recording a video may be reduced, so that the video is clearer when recorded, and user experience is improved.

Referring to fig. 6B, fig. 6B is an optional flowchart of the image processing method according to the embodiment of the present disclosure, and fig. 6B illustrates that step S1061 may be implemented by step S10613 to step S10614, which will be described with reference to the steps.

Step S10613, controlling the resolution ratio for acquiring a new original image frame to be unchanged when a third preset condition is met; wherein the third preset condition comprises: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one-half screen resolution and less than screen resolution.

In some embodiments, when the rendering frame rate is equal to or less than a first preset threshold and equal to or greater than a third preset threshold, and a difference between the rendering frame rate and the preview frame rate is equal to or greater than a second preset threshold and equal to or less than a fourth preset threshold, and the resolution for acquiring the original image frame is equal to or greater than one-half of the screen resolution and less than the screen resolution, the resolution for acquiring the new original image frame is controlled to be unchanged.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 17 frames, the screen resolution is 92 ten thousand pixels, and the resolution of the acquired original image frame is 80 thousand pixels, the rendering frame rate 17 is less than the first preset threshold 18 and greater than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 3 and equal to the fourth preset threshold 3, and the resolution of the acquired original image frame is 80 thousand pixels less than the screen resolution of 92 thousand pixels and greater than one half of the screen resolution and 46 thousand pixels, at this time, the condition of acquiring the original image frame is the best condition, and the resolution of acquiring a new original image frame is not adjusted.

Step S10614, when a fourth preset condition is met, reducing the resolution for acquiring a new original image frame; wherein the fourth preset condition includes: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is less than one-half of the screen resolution or greater than or equal to the screen resolution.

In some embodiments, when the rendering frame rate is equal to or less than the first preset threshold and equal to or greater than the third preset threshold, the difference between the rendering frame rate and the preview frame rate is equal to or greater than the second preset threshold and equal to or less than the fourth preset threshold, and the resolution for acquiring the original image frame is less than one-half of the screen resolution or equal to or greater than the screen resolution, the resolution for acquiring the new original image frame is reduced, wherein the resolution may be reduced proportionally or by a certain step size.

In some embodiments, when the rendering frame rate is equal to or less than a first preset threshold and equal to or greater than a third preset threshold, and a difference between the rendering frame rate and the preview frame rate is equal to or greater than a second preset threshold and equal to or less than a fourth preset threshold, and the resolution at which the original image frame is acquired is equal to or greater than the screen resolution, the resolution at which the new original image frame is acquired is controlled to be unchanged or the resolution at which the new original image frame is acquired is reduced.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 17 frames, the screen resolution is 92 ten thousand pixels, and the resolution of the acquired original image frame is 100 ten thousand pixels, the rendering frame rate 17 is less than the first preset threshold 18 and greater than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 3 equal to the fourth preset threshold 3, and the resolution of the acquired original image frame is 100 ten thousand pixels greater than the screen resolution and is 92 ten thousand pixels, at this time, the resolution of the acquired new original image frame may not be adjusted or the resolution of the acquired original image frame is reduced to 80 ten thousand pixels for acquiring the new original image frame.

In some embodiments, when the rendering frame rate is equal to or less than the first preset threshold and equal to or greater than the third preset threshold, and the difference between the rendering frame rate and the preview frame rate is equal to or greater than the second preset threshold and equal to or less than the fourth preset threshold, and the resolution at which the original image frame is acquired is less than one-half of the screen resolution, the resolution at which the new original image frame is acquired is decreased.

As an example, when the first preset threshold is 18, the second preset threshold is 2, the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 17 frames, the screen resolution is 92 ten thousand pixels, and the resolution of the acquired original image frame is 30 ten thousand pixels, the rendering frame rate 17 is less than the first preset threshold 18 and greater than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 3 equal to the fourth preset threshold 3, and the resolution of the acquired original image frame is 30 ten thousand pixels and less than 46 ten thousand pixels of the screen resolution of one-half, at this time, the resolution of the acquired original image frame may be reduced to 20 ten thousand pixels for acquiring a new original image frame.

Referring to fig. 6C, fig. 6C is an optional flowchart of the image processing method according to the embodiment of the present application, and fig. 6C shows that step S1061 can be implemented by step S10615 to step S10616, which will be described with reference to the steps.

Step S10615, controlling the resolution for acquiring a new original image frame to be unchanged when a fifth preset condition is met; wherein the fifth preset condition includes: the drawing frame rate is smaller than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is larger than a fourth preset threshold, and the resolution of the collected original image frame is smaller than one half of the screen resolution.

In some embodiments, when the rendering frame rate is less than a third preset threshold, and the difference between the rendering frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of acquiring the original image frame is less than one-half of the screen resolution, the resolution for acquiring the new original image frame is controlled to be unchanged.

As an example, when the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 14 frames, the screen resolution is 92 ten thousand pixels, and the resolution of the acquired original image frame is 30 ten thousand pixels, the rendering frame rate 14 is smaller than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 6 and larger than the fourth preset threshold 2, and the resolution of the acquired original image frame is 30 ten thousand pixels and smaller than 46 ten thousand pixels of one-half screen resolution, at this time, the condition of acquiring the original image frame is the best state, and the resolution of acquiring the new original image frame is not adjusted.

Step S10616, when a sixth preset condition is met, reducing the resolution for acquiring a new original image frame; wherein the sixth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one-half of the screen resolution.

In some embodiments, when the rendering frame rate is less than a third preset threshold, the difference between the rendering frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution for acquiring the original image frame is greater than or equal to one-half of the screen resolution, the resolution for acquiring the new original image frame is reduced, wherein the resolution may be reduced proportionally or by a certain step size.

In some embodiments, when the rendering frame rate is less than a third preset threshold, and the difference between the rendering frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution for acquiring the original image frame is greater than or equal to one-half of the screen resolution and less than the screen resolution, the resolution for acquiring the new original image frame is controlled to be unchanged or reduced.

As an example, when the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 14 frames, the screen resolution is 92 ten thousand pixels, and the resolution of the acquired original image frame is 80 ten thousand pixels, the rendering frame rate 14 is less than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 6 greater than the fourth preset threshold 2, and the resolution of the acquired original image frame is 80 ten thousand pixels less than the screen resolution of 92 ten thousand pixels and greater than one-half screen resolution of 46 ten thousand pixels, at this time, the resolution of acquiring a new original image frame may not be adjusted or the resolution of acquiring an original image frame is reduced to 30 ten thousand pixels for acquiring a new original image frame.

In some embodiments, when the rendering frame rate is less than a third preset threshold, and the difference between the rendering frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of acquiring the original image frame is greater than or equal to the screen resolution, the resolution for acquiring the new original image frame is decreased.

As an example, when the third preset threshold is 15, the fourth preset threshold is 3, the preview frame rate is 20 frames, the rendering frame rate is 14 frames, the screen resolution is 92 ten thousand pixels, the resolution of the acquired original image frame is 100 ten thousand pixels, the rendering frame rate 14 is less than the third preset threshold 15, the difference between the rendering frame rate and the preview frame rate is 6 greater than the fourth preset threshold 2, and the resolution of the acquired original image frame is 100 ten thousand pixels greater than the screen resolution and is 92 ten thousand pixels, at this time, the resolution of the acquired original image frame may be reduced to 30 ten thousand pixels for acquiring a new original image frame.

In some embodiments, the electronic device may locally store or upload the finally adjusted optimal value of the acquired new original image frame to the server, so that when the camera is started next time, the optimal value obtained by the last adjustment may be called locally or acquired from the server to be used as the initial value of the resolution of the acquired original image frame.

As an example, when electronic devices of the same model or similar models take photos or videos in the same or similar application scenes, the optimal resolution value in a period of time is stable, so that when the camera is started next time, the optimal value obtained by the last adjustment can be called locally or obtained from the server to be used as the initial value of the resolution for collecting the original image frames

The method for controlling the resolution of the image for acquiring the new original image frame in the image previewing process based on the difference between the drawing frame rate and the previewing frame rate can intelligently balance the resolution and the frame rate in the special effect processing process, so that the satisfaction of a user during image previewing is optimal, and better use experience is obtained.

Fig. 2 is a schematic composition diagram of an image processing apparatus 300 according to an embodiment of the present disclosure, and referring to fig. 2, the image processing apparatus 300 according to the embodiment of the present disclosure includes:

In some embodiments, the obtaining unit is further configured to trigger the camera application to start an image preview window by calling an application program interface of the camera application; and acquiring the original image frame acquired by the camera application in the image preview window through the application program interface.

In some embodiments, the execution unit is further configured to detect a target object in the raw image frame; and in the area of the original image frame corresponding to the target object, performing special effect rendering operation of at least one of a filter, a beauty and a sticker to obtain the special effect image frame.

In some embodiments, the execution unit is further configured to obtain a detection frame rate in detecting the target object; when the detection frame rate is smaller than the preview frame rate, based on the difference value between the detection frame rate and the preview frame rate, performing frame loss processing on an original image frame acquired in the image preview process; and detecting a target object in the original image frame subjected to frame loss processing.

In some embodiments, the execution unit is further configured to perform frame dropping processing on the acquired raw image frames every second at intervals, and the number of the raw image frames processed by frame dropping per second is greater than or equal to the difference; or continuously dropping the frames of the acquired original image frames per second, wherein the number of the original image frames per second dropped is greater than or equal to the difference.

In some embodiments, the control unit is further configured to control a resolution for acquiring a new original image frame in the image previewing process based on different conditions of a value of the rendering frame rate, a difference between the rendering frame rate and the preview frame rate, and a value of a resolution for acquiring the original image frame.

In some embodiments, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a first preset condition is met; wherein the first preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is less than a second preset threshold, and the resolution of the collected original image frame is greater than or equal to the screen resolution; when a second preset condition is met, improving the resolution ratio for acquiring a new original image frame; wherein the second preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is smaller than a second preset threshold, and the resolution of the collected original image frame is smaller than the screen resolution.

In some embodiments, the control unit is further configured to, when an original image frame captured during image preview is used for video recording, and when a second preset condition is satisfied; and the resolution ratio for acquiring the new original image frame is improved, and the code rate for recording the video is reduced.

In some embodiments, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a third preset condition is met; wherein the third preset condition comprises: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution and less than the screen resolution; when a fourth preset condition is met, reducing the resolution ratio for acquiring a new original image frame; wherein the fourth preset condition includes: the drawing frame rate is less than or equal to a first preset threshold and greater than or equal to a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than or equal to a second preset threshold and less than or equal to a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution or greater than or equal to the screen resolution.

In some embodiments, the control unit is further configured to control the resolution for acquiring a new original image frame to be unchanged when a fifth preset condition is met; wherein the fifth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is less than half of the screen resolution; when a sixth preset condition is met, reducing the resolution ratio for acquiring a new original image frame; wherein the sixth preset condition includes: the drawing frame rate is less than a third preset threshold, the difference value between the drawing frame rate and the preview frame rate is greater than a fourth preset threshold, and the resolution of the collected original image frame is greater than or equal to one half of the screen resolution.

An embodiment of the present disclosure provides an electronic device, including:

a memory for storing executable instructions;

The embodiment of the disclosure provides a storage medium, which stores executable instructions, and the executable instructions are executed to realize the image processing method provided by the embodiment of the disclosure.

According to one or more embodiments of the present disclosure, an embodiment of the present disclosure provides an image processing method, including:

acquiring an original image frame acquired in an image previewing process;

According to one or more embodiments of the present disclosure, an embodiment of the present disclosure provides an image processing method, further including:

the acquiring of the original image frame acquired in the image preview process includes:

the performing a special effect rendering operation on the original image frame to obtain a special effect image frame includes:

detecting a target object in the original image frame;

the detecting a target object in the original image frame comprises:

acquiring a detection frame rate in the process of detecting the target object;

the processing of frame loss of the collected original image frame based on the difference between the detection frame rate and the preview frame rate includes:

alternatively, the first and second electrodes may be,

the controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on the difference value between the drawing frame rate and the previewing frame rate includes:

controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on different conditions of the value of the rendering frame rate, the difference value between the rendering frame rate and the previewing frame rate and the value of the resolution ratio for acquiring the original image frame, wherein the control comprises the following steps:

wherein the first preset condition comprises: the drawing frame rate is greater than a first preset threshold, the difference value between the drawing frame rate and the preview frame rate is smaller than a second preset threshold, and the resolution of the collected original image frame is greater than or equal to the screen resolution;

and, when a second preset condition is satisfied;

According to one or more embodiments of the present disclosure, an embodiment of the present disclosure provides an image processing apparatus, further including:

The above description is only an example of the present disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of image processing, comprising:

acquiring an original image frame acquired in an image previewing process;

and controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on a comparison result between the rendering frame rate and a first preset threshold value and/or a third preset threshold value, a comparison result between a difference value between the rendering frame rate and the previewing frame rate and a second preset threshold value and/or a fourth preset threshold value, and a comparison result between the resolution ratio for acquiring the original image frame and the screen resolution ratio.

2. The method of claim 1, wherein said acquiring raw image frames acquired during image preview comprises:

3. The method of claim 1, wherein performing a special effect rendering operation on the original image frame to obtain a special effect image frame comprises:

detecting a target object in the original image frame;

4. The method of claim 3, the detecting a target object in the raw image frame comprising:

acquiring a detection frame rate in the process of detecting the target object;

5. The method of claim 4, wherein the performing frame loss processing on the original image frames collected in the image preview process based on the difference between the detection frame rate and the preview frame rate comprises:

alternatively, the first and second electrodes may be,

6. The method according to claim 1, wherein the controlling the resolution for acquiring a new original image frame in the process of image preview based on the comparison result between the rendering frame rate and the first preset threshold and/or the third preset threshold, the comparison result between the difference between the rendering frame rate and the preview frame rate and the second preset threshold and/or the fourth preset threshold, and the comparison result between the resolution for acquiring the original image frame and the screen resolution comprises:

7. The method of claim 6, further comprising:

and when the second preset condition is satisfied;

8. The method according to claim 1, wherein the controlling the resolution for acquiring a new original image frame in the process of image preview based on the comparison result between the rendering frame rate and the first preset threshold and/or the third preset threshold, the comparison result between the difference between the rendering frame rate and the preview frame rate and the second preset threshold and/or the fourth preset threshold, and the comparison result between the resolution for acquiring the original image frame and the screen resolution comprises:

9. The method according to claim 1, wherein the controlling the resolution for acquiring a new original image frame in the process of image preview based on the comparison result between the rendering frame rate and the first preset threshold and/or the third preset threshold, the comparison result between the difference between the rendering frame rate and the preview frame rate and the second preset threshold and/or the fourth preset threshold, and the comparison result between the resolution for acquiring the original image frame and the screen resolution comprises:

10. An apparatus for image processing, comprising:

and the control unit is used for controlling the resolution ratio for acquiring a new original image frame in the image previewing process based on a comparison result between the rendering frame rate and a first preset threshold value and/or a third preset threshold value, a comparison result between a difference value between the rendering frame rate and the previewing frame rate and a second preset threshold value and/or a fourth preset threshold value, and a comparison result between the resolution ratio for acquiring the original image frame and the screen resolution ratio.

11. An electronic device, comprising:

a memory for storing executable instructions;

a processor for implementing the method of image processing as claimed in any one of claims 1 to 9 when executing the executable instructions.

12. A computer-readable storage medium having stored thereon executable instructions for, when executed, implementing the method of image processing according to any one of claims 1 to 9.