CN112017137A

CN112017137A - Image processing method, image processing device, electronic equipment and computer readable storage medium

Info

Publication number: CN112017137A
Application number: CN202010838952.6A
Authority: CN
Inventors: 邱情
Original assignee: Realme Mobile Telecommunications Shenzhen Co Ltd
Current assignee: Realme Mobile Telecommunications Shenzhen Co Ltd
Priority date: 2020-08-19
Filing date: 2020-08-19
Publication date: 2020-12-01
Anticipated expiration: 2040-08-19
Also published as: CN112017137B

Abstract

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring a shot image shot by a camera module, wherein the camera module comprises at least two cameras; editing the shot image according to the detected editing operation to obtain a first image; respectively editing N frames of original images corresponding to the shot images according to editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one camera in the camera module, and N is a positive integer greater than or equal to 1; and selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image. The image processing method, the image processing device, the electronic equipment and the computer readable storage medium can improve the image quality presented after the image is edited and enhance the visual effect presented by the edited image.

Description

Image processing method, image processing device, electronic equipment and computer readable storage medium

Technical Field

The present application relates to the field of image technologies, and in particular, to an image processing method and apparatus, an electronic device, and a computer-readable storage medium.

Background

With the rapid development of electronic technology, more and more users like to take various photos with electronic equipment and record all times in life. After the electronic device takes a picture, the user often edits the picture, and adjusts the content of the picture to obtain a desired image. After editing a photo, the quality of the photo is often degraded.

Disclosure of Invention

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium, which can improve the image quality presented after image editing and enhance the visual effect presented by the edited image.

The embodiment of the application discloses an image processing method, which comprises the following steps:

acquiring a shot image shot by a camera module, wherein the camera module comprises at least two cameras;

editing the shot image according to the detected editing operation to obtain a first image;

respectively editing N frames of original images corresponding to the shot images according to the editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one camera in the camera module, and N is a positive integer greater than or equal to 1;

and selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

An embodiment of the application discloses an image processing apparatus, including:

the camera module is used for shooting a camera module to obtain a shot image;

the first editing module is used for editing the shot image according to the detected editing operation to obtain a first image;

the second editing module is used for respectively editing and processing N frames of original images corresponding to the shot images according to the editing operation to obtain N frames of second images, each frame of original image is acquired through one camera in the camera module, and N is a positive integer greater than or equal to 1;

and the selecting module is used for selecting a target image from the N frames of second images and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

The embodiment of the application discloses an electronic device, which comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is enabled to realize the method.

An embodiment of the application discloses a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method as described above.

The image processing method, the image processing device, the electronic device and the computer-readable storage medium, which are disclosed by the embodiment of the application, acquire a shot image shot through a camera module, edit the shot image according to a detected editing operation to obtain a first image, edit N frames of original images corresponding to the shot image according to the editing operation to obtain N frames of second images, select a target image with higher image quality than the first image from the N frames of second images and output the target image, and select an image with better image quality to present to a user after editing the shot image, so that the problem that the image quality slides seriously under the condition of editing the image is avoided, the image quality presented after editing the shot image can be improved, and the visual effect presented by the edited image is enhanced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a block diagram of an image processing circuit according to an embodiment;

FIG. 2 is a flow diagram of a method of image processing in one embodiment;

FIG. 3 is a diagram illustrating an embodiment of editing a captured image and a corresponding original image;

FIG. 4 is a flow diagram of an embodiment of obtaining N frames of original images for editing;

FIG. 5 is a flowchart of an image processing method in another embodiment;

FIG. 6 is a block diagram of an image processing apparatus in one embodiment;

fig. 7 is a block diagram of an electronic device in one embodiment.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the examples and figures of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image, without departing from the scope of the present application. Both the first image and the second image are edited images, but they are not the same image.

The embodiment of the application provides electronic equipment. The electronic device includes therein an Image Processing circuit, which may be implemented using hardware and/or software components, and may include various Processing units defining an ISP (Image Signal Processing) pipeline. FIG. 1 is a block diagram of an image processing circuit in one embodiment. For ease of illustration, FIG. 1 illustrates only aspects of image processing techniques related to embodiments of the present application.

As shown in fig. 1, the image processing circuit includes an ISP processor 140 and control logic 150. The image data captured by the imaging device 110 is first processed by the ISP processor 140, and the ISP processor 140 analyzes the image data to capture image statistics that may be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. Image sensor 114 may include an array of color filters (e.g., Bayer filters), and image sensor 114 may acquire light intensity and wavelength information captured by each imaging pixel and provide a set of raw image data that may be processed by ISP processor 140. The attitude sensor 120 (e.g., a three-axis gyroscope, hall sensor, accelerometer, etc.) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 140 based on the type of interface of the attitude sensor 120. The attitude sensor 120 interface may employ an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination thereof.

It should be noted that, although only one imaging device 110 is shown in fig. 1, in the embodiment of the present application, at least two imaging devices 110 may be included, each imaging device 110 may respectively correspond to one image sensor 114, or a plurality of imaging devices 110 may correspond to one image sensor 114, which is not limited herein. The operation of each image forming apparatus 110 can refer to the above description.

In addition, the image sensor 114 may also transmit raw image data to the attitude sensor 120, the attitude sensor 120 may provide the raw image data to the ISP processor 140 based on the type of interface of the attitude sensor 120, or the attitude sensor 120 may store the raw image data in the image memory 130.

The ISP processor 140 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.

The ISP processor 140 may also receive image data from the image memory 130. For example, the attitude sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image Memory 130 may be a portion of a Memory device, a storage device, or a separate dedicated Memory within an electronic device, and may include a DMA (Direct Memory Access) feature.

Upon receiving raw image data from the image sensor 114 interface or from the attitude sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. ISP processor 140 receives the processed data from image memory 130 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed by ISP processor 140 may be output to display 160 for viewing by a user and/or further processed by a Graphics Processing Unit (GPU). Further, the output of the ISP processor 140 may also be sent to the image memory 130, and the display 160 may read image data from the image memory 130. In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

The statistics determined by the ISP processor 140 may be sent to the control logic 150. For example, the statistical data may include image sensor 114 statistics such as gyroscope vibration frequency, auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 112 shading correction, and the like. The control logic 150 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters of the imaging device 110 and control parameters of the ISP processor 140 based on the received statistical data. For example, the control parameters of the imaging device 110 may include attitude sensor 120 control parameters (e.g., gain, integration time of exposure control, anti-shake parameters, etc.), camera flash control parameters, camera anti-shake displacement parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), as well as lens 112 shading correction parameters.

The image processing method provided by the embodiment of the present application is exemplarily described with reference to the image processing circuit of fig. 1. The electronic device may first acquire multiple frames of original images through the lens 112 and the image sensor 114 in each imaging device (camera) 110, and each imaging device may acquire one or multiple frames of original images and transmit the acquired multiple frames of original images to the ISP processor 140. The ISP processor 140 may perform fusion processing on multiple frames of original images to obtain a captured image, and send the captured image and the multiple frames of original images to the image memory 130. The image memory 130 may store the photographed image and the plurality of frames of original images transmitted from the ISP processor 140. The ISP processor 140 may also transmit the photographed image to the display 160 for display. When the electronic device detects that the user performs an editing operation on the captured image displayed on the display 160, the ISP processor 140 may perform an editing process on the captured image according to the editing operation to obtain the first image. The ISP processor 140 may obtain N frames of original images corresponding to the captured image from the image memory 130, edit the N frames of original images according to the editing operation to obtain N frames of second images, select a target image with an image quality higher than that of the first image from the N frames of second images, output the target image to the display 160, and display the target image to the user through the display 160.

As shown in fig. 2, in an embodiment, an image processing method is provided, where the image processing method is applicable to electronic devices such as a mobile phone, a smart wearable device, a tablet computer, and a digital camera, and the embodiment of the present application is not limited thereto. The operating system of the electronic device may include, but is not limited to, an Android operating system, an IOS operating system, a Symbian operating system, a Windows operating system, and the like, which is not limited in the embodiment of the present application. The method comprises the following steps:

and step 210, acquiring a shot image shot by the camera module.

In this embodiment, the camera module of the electronic device may include at least two cameras, and each camera may be of a different type, or may be a camera of the same type but of a different model (e.g., having different pixels). Optionally, the camera included in the camera module may be any combination of various cameras such as a wide-angle camera, a telephoto camera, a standard camera, an ultra-wide-angle camera, and a depth camera. The standard camera usually has a focal length of about 50mm (millimeter), and the wide-angle camera, the telephoto camera and the ultra-wide-angle camera have different focal lengths and viewing angles, for example, the wide-angle camera may have a focal length of 38mm to 24mm, a viewing angle of 60 degrees to 84 degrees, the ultra-wide-angle camera may have a focal length of 20mm to 13mm, a viewing angle of 94 degrees to 118 degrees, and a focal length of the telephoto camera may have a focal length of 60mm to 400 mm. The embodiment of the application does not specifically limit the camera contained in the camera module.

When the electronic equipment receives a shooting instruction, all cameras contained in the camera module can be controlled to synchronize, one frame or multiple frames of images are respectively collected, and at least the original images collected by all the cameras at the same moment can be obtained. After the ISP processor obtains the original images respectively collected by the cameras, the captured images can be obtained based on the original images.

Alternatively, the captured image may be an image obtained by fusing a plurality of frames of original images. For example, the camera module includes a wide-angle camera and a telephoto camera, so that the wide-angle camera can collect the original image a, the telephoto camera can collect the original image B, and the long-range part in the original image a and the shooting subject in the original image B can be fused to obtain a better shot image.

Alternatively, the shot image may also be one frame of original image, for example, the camera module includes a standard camera and a depth camera, and the shot image may be an original image captured by the standard camera, but is not limited thereto. In some embodiments, for different shooting scenes, one frame of original image corresponding to the shooting scene may also be selected from the original images captured by the respective cameras as the shooting image, for example, the camera module includes a standard camera, a wide-angle camera and a telephoto camera, if the distance from the camera to the shot subject object is short, the original image captured by the standard camera may be selected as the shooting image, if the distance from the camera to the shot subject object is long, the original image captured by the telephoto camera may be selected as the shooting image, and if the shooting range is wide, the original image captured by the wide-angle camera may be selected as the shooting image, and the like, but is not limited thereto.

After the electronic equipment obtains the shot image, the shot image and the original image collected by each camera can be stored, only the shot image is displayed, and when a user accesses an application program such as an electronic photo album, the user can only see the shot image but not the original image. The situation that a plurality of frames of images are displayed simultaneously, confusion is caused to a user, and operation such as mistaken deletion is caused can be prevented.

Step 220, editing the shot image according to the detected editing operation to obtain a first image.

The user may perform editing operations on the displayed photographed image as desired, which may include, but is not limited to, zooming in, zooming out, cropping, adding image content, beautification processing, and the like, wherein the beautification processing may include, but is not limited to, background virtualization, portrait processing, brightness adjustment, white balance processing, and the like.

In some embodiments, when the electronic device detects an editing operation for the captured image triggered by the user, it may be determined whether the editing operation changes the image size of the captured image or changes the image content size in the captured image, and if the editing operation changes the image size of the captured image or changes the image content size in the captured image, it indicates that the quality of the captured image may be degraded after the editing process is performed on the captured image, so the electronic device may perform step 230 after the editing process is performed on the captured image to obtain the first image. If the editing operation does not change the image size and the image content size of the captured image, which indicates that the quality of the captured image does not slip down after the captured image is edited, the electronic device may directly edit the captured image according to the detected editing operation, and output the first image obtained after the editing process.

And step 230, respectively editing the N frames of original images corresponding to the shot images according to the editing operation to obtain N frames of second images.

After the captured image is subjected to the editing process, the electronic device may acquire N frames of original images corresponding to the captured image from the memory, the N frames of original images being distinguishable from the captured image. Wherein N may be a positive integer greater than or equal to 1. Optionally, the acquired N frames of original images may include at least one frame of original image acquired by each camera, for example, if the camera module includes a standard camera, a wide-angle camera and a telephoto camera, at least one frame of original image acquired by each standard camera, each wide-angle camera and each telephoto camera may be acquired. Optionally, the acquired N frames of original images may also be one or more frames of original images respectively acquired by one or more cameras in the camera module. For example, the camera module includes a standard camera, a wide-angle camera and a telephoto camera, and only one or more frames of original images respectively collected by the wide-angle camera and the telephoto camera may be acquired.

In some embodiments, after obtaining the captured image, the electronic device may assign the same image label to the captured image and each frame of original image corresponding to the captured image, respectively, where the label may be used to mark a correspondence between the captured image and the original image. For example, when a captured image is available and stored in an electronic album, a sequence number of the captured image in the electronic album may be generated, the sequence number may be edited as an image tag added to each frame of the original image corresponding to the captured image, and the like. After the electronic device detects the editing operation, the electronic device can acquire the image tag of the shot image for the editing operation, search the corresponding original image in the memory according to the image tag, and then select N frames from the original image carrying the image tag for editing processing. The efficiency of data searching and processing can be improved.

After acquiring the N frames of original images, the same type of editing process may be performed on the N frames of original images according to an editing operation, for example, an editing process of enlarging and cropping the captured image, or an editing process of enlarging and cropping each acquired frame of original image. The image content of the N frames of second images obtained after the editing process needs to be consistent with that of the first image, that is, the pixel coordinates of each pixel point in the second image and the pixel point matched with the pixel point in the first image in the image are consistent, the image content contained in each frame of second image and the first image should be the same, and the positions of the image content in the image are also the same.

As an optional implementation manner, after the captured image is edited to obtain the first image, the obtained N frames of original images may also be edited according to the first image, so as to ensure that the obtained N frames of second images are consistent with the image content of the first image. For example, each feature point in the first image may be extracted, and each matched feature point may be found in the original image, and then the original image may be edited according to the position of each feature point in the first image, so as to ensure that the obtained second image and the matched feature point in the first image have the same position in the image.

And 240, selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

Because each frame of original image is acquired by one camera in the camera module, the acquired N frames of original images are different from the shot image (for example, the size of the image acquired by the shot object is different due to the difference of the focal length and the field angle of the camera), and after the shot image and the N frames of original images are edited and processed in the same type, the image quality of the acquired N frames of second images is different from that of the first image. The image quality of each frame of the second image and the first image can be analyzed, and the second image with the image quality higher than that of the first image is selected from the N frames of the second image to serve as a target image. In some embodiments, all second images with higher image quality than the first image can be used as target images, and any second image with higher image quality than the first image can be selected as target images. Optionally, the second image with the highest image quality may also be directly selected as the target image, and the selection manner is not limited in the embodiment of the present application.

In the embodiment of the present application, since the editing operation mainly changes the size of the image or the size of the image content, the sharpness of the image is affected. Therefore, the image quality may include an evaluation index such as sharpness. For example, the electronic device may measure the sharpness of the second image and the first image of each frame, and the manner of measuring the sharpness of the images may be various, for example, obtained by measuring Spatial Frequency Response (SFR), or obtained by measuring sharpness of the images, and the like, and is not limited herein. The definition of each frame of the second image may be compared with the definition of the first image one by one, and the second image having a definition greater than that of the first image may be determined. The image quality can be evaluated more intuitively and simply by utilizing the definition, and the image processing efficiency is improved.

It should be noted that the image quality may be evaluated from other dimensions, and is not limited to the above definition, and the evaluation parameters and modes of the image quality are not limited in the present application.

FIG. 3 is a diagram illustrating an embodiment of editing a captured image and a corresponding original image. As shown in fig. 3, the electronic device may display a captured image 310, and when an enlarged cropping operation for the captured image 310 is detected, the captured image 310 is subjected to an enlarged cropping process, resulting in a first image 312. The electronic device can acquire the stored original image a320 and original image B330 corresponding to the captured image 310, wherein the original image a320 and original image B330 are different from the captured image 310 (different capturing angles). The original image a320 and the original image B330 may be respectively subjected to an enlarging and cropping process to obtain a second image a322 and a second image B332, where image contents of the second image a322, the second image B332, and the first image 312 are consistent. The image quality of the second image a322, the second image B332 and the first image 312 may be obtained, respectively, where the image quality of the second image B332 is higher than that of the second image a322, and the image quality of the second image a322 is higher than that of the first image 312. The second image a322 and the second image B332 may be output as the target images at the same time, or either one of them may be selected to be output as the target image, or the second image B332 with the highest image quality may be directly output.

In some embodiments, outputting the target image may be directly replacing the target image with the first image, and only presenting the target image with better quality to the user, or displaying the first image and the target image at the same time, and prompting the user that the image quality of the target image is higher than that of the first image, and the user selects to save the target image or the first image, thereby assisting the user in selecting the image with better quality for saving.

In the embodiment of the application, after the shot image is edited, the image with better image quality is selected and presented to the user, so that the problem of serious image quality slide caused by editing the image is avoided, the image quality presented after the shot image is edited can be improved, and the visual effect presented by the edited image is enhanced.

In one embodiment, the step of selecting the target image from the N second images comprises: and selecting a second image with image quality arranged in the previous M frames as a target image from second images with image quality higher than that of the first image according to the arrangement sequence of the image quality from high to low.

After the electronic device edits the N frames of original images, the image quality of each frame of second image can be obtained, and the N frames of second images are arranged in sequence from high to low according to the image quality. After the arrangement, a second image having a higher image quality than the first image may be determined, and the second image having the image quality arranged in the previous M frames may be selected as the target image in the arrangement order. Wherein, M can be a positive integer greater than or equal to 1 and less than or equal to N. Alternatively, M may be a preset number, e.g., 1, 2, etc.

For example, if 4 frames of second images are obtained and the second images a-c-b-d are sequentially arranged according to the order of image quality from high to low, where the image quality of the second image a, the second image c, and the second image b is higher than that of the first image, and the image quality of the second image d is lower than that of the first image, the second image a and the second image c with the image quality arranged in the top 2 may be obtained and output as target images.

Optionally, the second images with higher image quality than the first images in the N frames of second images may be determined, and then the second images with higher image quality than the first images are arranged according to the sequence of the image quality from high to low, so as to select the second image with image quality arranged in the previous M frames as the target image. If the number of frames of the second image with higher image quality than the first image is less than the set M, all the second image with higher image quality than the first image can be taken as the target image and output.

In one embodiment, M may be 1, and the electronic device may select the second image with the highest image quality as the target image directly from the second images with higher image quality than the first image. After the image is edited, the image with the best image quality is presented to the user, so that the visual effect presented by the edited image is enhanced.

In other embodiments, the target image may be selected according to other selection rules, for example, M frames of second images with higher image quality than the first image may be randomly selected as the target image, and the like, which is not limited herein.

In the embodiment of the application, the target image can be selected according to the image quality of the second image of each frame and output, so that the image quality presented after the shot image is edited can be further improved, and the visual effect presented by the edited image is enhanced.

As shown in fig. 4, in an embodiment, the step of editing the N frames of original images corresponding to the captured images according to the editing operation to obtain N frames of second images may include the following steps:

step 402, determining a camera corresponding to the operation type of the editing operation.

The electronic equipment can acquire the original image acquired by each camera for editing processing, and can also acquire only part of the original images acquired by the cameras. In the embodiment of the present application, the camera module may include at least two of a telephoto camera, a wide-angle camera, a standard camera, and the like. After detecting the editing operation on the shot image, the electronic device may acquire an operation type of the editing operation, optionally, the editing operation may include one or more of an enlarging operation, a reducing operation, a cropping operation, and the like, and the corresponding operation types may be an image enlarging operation, an image reducing operation, an image cropping, and the like.

When editing operations of different operation types are carried out, an original image collected by a fixed camera can exist, and the image quality of a second image obtained after editing processing is higher than that of a first image with high probability. The camera corresponding to the operation type is the fixed camera of the finger. For example, when performing an enlarging and cutting operation, the image quality of a second image obtained after an original image acquired by a tele camera is edited is higher than that of a first image in a high probability, and then the camera corresponding to the image enlarging type can be a tele camera; when the zoom-out operation is performed, the image quality of a second image obtained by editing an original image acquired by the wide-angle camera is higher than that of the first image, and the camera corresponding to the image zoom-out type can be the wide-angle camera.

And step 404, acquiring N frames of original images corresponding to the shot images and acquired by the corresponding camera, and editing the acquired N frames of original images according to editing operation to obtain N frames of second images.

In some embodiments, when the electronic device stores the original images acquired by the cameras, the electronic device may record the camera identifiers corresponding to the original images of the frames, where the camera identifiers may be used to characterize the cameras acquiring the original images. For example, the camera identification of the wide-angle camera may be "001", the camera identification of the standard camera may be "002", the camera identification of the tele camera may be "003", and the like, but is not limited thereto. The method can acquire the camera identification of the camera corresponding to the operation type of the editing operation, and acquire one or more frames of original images acquired by the camera according to the camera identification.

In some embodiments, the electronic device may also store the original images from different cameras in different storage partitions respectively, and after determining a camera corresponding to the operation type of the editing operation, may obtain one or more frames of original images corresponding to the captured image from the storage partition corresponding to the camera.

Optionally, after the N frames of original images collected by the cameras corresponding to the operation types of the editing operation are edited, if the image quality of the obtained N frames of second images is lower than that of the first image, the original images collected by other cameras may be obtained again for editing, so as to output a target image with an image quality higher than that of the first image to be presented to the user.

In the embodiment of the application, the camera corresponding to the operation type of the editing operation can be determined, and the N frames of original images acquired by the corresponding camera are acquired for editing, and after the original images acquired by the camera corresponding to the operation type of the editing operation are edited, the image quality of the acquired second image is higher than that of the acquired first image, so that the edited images with higher image quality can be acquired, the processed image data is less, and the processing speed is increased.

As shown in fig. 5, in an embodiment, another image processing method is provided, which is applicable to the electronic device described above, and the method may include the following steps:

step 502, acquiring a shot image shot by the camera module.

Step 504, editing the shot image according to the detected editing operation to obtain a first image.

The descriptions of steps 502-504 can refer to the related descriptions in the above embodiments, and are not repeated herein.

Step 506, identifying the main content in the first image, and acquiring the image information of the main content in the first image.

The subject content may refer to the subject captured in the first image, and in some embodiments, the manner of identifying the subject content in the first image may include, but is not limited to, the following:

the first method is as follows: the subject content may be identified based on depth information of the first image. The electronic equipment can obtain the depth information of each pixel point in the first image, the depth information can be used for representing the distance between the physical point corresponding to the pixel point and the camera in the real shooting scene, and the larger the depth information of the pixel point is, the farther the distance between the physical point corresponding to the pixel point and the camera can be represented. The foreground region and the background region in the first image may be distinguished by depth information, where the depth information of each pixel point in the foreground region may be smaller than a first depth value, and the depth information of each pixel point in the background region may be larger than a second depth value, where the first depth value may be smaller than the second depth value. The image content in the foreground region may be taken as the subject content.

The second method comprises the following steps: the subject content may be identified based on the image proportions occupied by the respective photographic subjects in the first image. The electronic device may extract an edge feature in the first image, and divide the first image into a plurality of image regions according to the edge feature, and optionally, the edge feature may be a pixel having a relatively high contrast with an adjacent pixel. Whether the adjacent image areas describe the same shot object or not can be judged, the image areas describing the same shot object are combined, and the proportion of each shot object in the first image is determined according to the occupied area of the image area contained in each shot object in the first image. The image area included in the subject occupying the largest proportion of the image can be used as the subject content.

The third method comprises the following steps: the subject content of the first image can be identified by using a subject identification model obtained by pre-training. The subject recognition model can be obtained by training a large number of image samples, subject content can be labeled in each image sample, and the subject recognition model can be trained by learning the characteristics of the image samples.

After identifying the subject content in the first image, image information of the subject content in the first image may be obtained. Optionally, the image information may include one or more of position information of the subject content in the first image, occupied image proportion, corresponding field angle, and the like. The position information of the main content in the first image can be represented by the pixel coordinates of each pixel point contained in the main content in the first image. The corresponding Field angle of the subject content in the first image refers to a Field of view (FOV) that is required to obtain the first image by directly using the camera, and the larger the proportion of the subject content in the first image is, the smaller the corresponding Field angle can be, and the smaller the proportion of the subject content in the first image is, the larger the corresponding Field angle can be.

And step 508, respectively editing the N frames of original images corresponding to the shot images according to the editing operation and the image information to obtain N frames of second images.

The electronic equipment can perform editing processing on each frame of original image according to the image information of the main content in the first image, so that the image information of the main content in the N frames of second images obtained through the editing processing is the same as the image information of the main content in the first image. The consistency of the image information of the main content of the image and the main content of the image is ensured, and the image presented to the user can be ensured to be the image required by the user and to be the image with better quality.

Alternatively, the editing process performed on the original image may be the same type of editing process as the captured image, or may be a different type of editing process, for example, a user reduces the captured image to obtain a first image reduced from the captured image, and for the original image captured by the wide-angle camera, the proportion of the main content occupied in the original image may be smaller than the proportion occupied by the main content of the first image, and the original image captured by the wide-angle camera needs to be enlarged to obtain a second image whose main content keeps the same image information as the main content of the first image.

And step 510, selecting a second image with image quality arranged in the previous M frames as a target image from second images with image quality higher than that of the first image according to the arrangement sequence of the image quality from high to low.

And step 512, outputting the target image.

In one embodiment, the captured image for the user to edit may be an image subjected to an image beautification process, which may be a process performed by the ISP processor when the captured image is obtained, such as a background blurring process, a white balance process, or the like, or a process performed by the ISP processor for beautification of the captured image at a later stage, such as a portrait beautifying process, a brightness adjustment process, or the like. After the electronic device selects the target image, it may obtain the beautification processing parameter of the captured image, where the beautification processing parameter refers to a parameter of beautification processing performed on the captured image, for example, if the captured image is subjected to brightness adjustment processing, parameters such as a brightness adjusted area position and an adjusted brightness value may be obtained, and if the captured image is subjected to background blurring processing, parameters such as a blurring degree may be obtained, which is not limited herein.

The electronic equipment can beautify the target image according to the beautification processing parameters and output the beautified target image. The target image after the image beautification processing can keep the same image effect as the first image, so that the output target image meets the requirement of a user on the image effect. After the image is edited, the image with higher image quality is presented, and the image effect of the original image is not influenced.

In the embodiment of the application, the original image can be edited according to the image information of the main content in the first image, so that the main content of each frame of second image obtained by editing can be kept consistent with the first image, the presented target image can be ensured to meet the editing operation requirement of a user, and the presented image has higher quality.

As shown in fig. 6, in an embodiment, an image processing apparatus 600 is provided, which is applicable to the electronic device described above, and the image processing apparatus 600 may include an image obtaining module 610, a first editing module 620, a second editing module 630, and a selecting module 640.

The image acquisition module 610 is configured to acquire a captured image captured by a camera module, where the camera module includes at least two cameras.

The first editing module 620 is configured to perform editing processing on the captured image according to the detected editing operation to obtain a first image.

The second editing module 630 is configured to edit N frames of original images corresponding to the captured images according to an editing operation, so as to obtain N frames of second images, where each frame of original image is acquired by one of the cameras in the camera module, and N is a positive integer greater than or equal to 1.

And a selecting module 640, configured to select a target image from the N frames of second images, and output the target image, where an image quality of the target image is higher than an image quality of the first image.

In one embodiment, the selecting module 640 is further configured to select, as the target image, a second image with image quality arranged in M previous frames from among second images with image quality higher than that of the first image, in an order from high image quality to low image quality, and output the target image, where M is a positive integer greater than or equal to 1 and less than or equal to N.

In one embodiment, the selecting module 640 is further configured to select, as the target image, a second image with the highest image quality from among second images with higher image quality than the first image, and output the target image.

In an embodiment, the second editing module 630 is further configured to determine a camera corresponding to an operation type of the editing operation, acquire N frames of original images corresponding to the captured image and collected by the corresponding camera, and edit the acquired N frames of original images according to the editing operation to obtain N frames of second images.

Optionally, the camera module includes at least two of a telephoto camera, a wide-angle camera and a standard camera; the editing operation includes one or more of a zoom-in operation, a zoom-out operation, and a clipping operation.

In one embodiment, the second editing module 630 includes a subject recognition unit and an editing unit.

And the main body identification unit is used for identifying the main body content in the first image and acquiring the image information of the main body content in the first image.

Optionally, the image information includes one or more of position information of the main content in the first image, occupied image proportion and corresponding field angle.

And the editing unit is used for respectively editing the N frames of original images corresponding to the shot images according to the editing operation and the image information to obtain N frames of second images.

In one embodiment, the image processing apparatus 600 further includes a beautification processing module in addition to the image acquiring module 610, the first editing module 620, the second editing module 630 and the selecting module 640.

And the beautification processing module is used for acquiring beautification processing parameters of the shot image after the target image is selected by the selection module 640, and beautifying the image of the target image according to the beautification processing parameters.

The selecting module 640 is further configured to output the beautified target image.

In the embodiment of the application, after the image is edited, the image with higher image quality is presented, and meanwhile, the image effect of the original image is not influenced.

Fig. 7 is a block diagram of an electronic device in one embodiment. The electronic equipment can be mobile phones, tablet computers, intelligent wearable equipment, PCs, notebook computers and other equipment. As shown in fig. 7, electronic device 700 may include one or more of the following components: a processor 710, a memory 720 coupled to the processor 710, and a camera module 730 coupled to the processor. The camera module 730 can include at least two cameras, and can be used to collect multiple frames of original images. Memory 720 may store one or more applications that may be configured to be executed by the one or more processors 710, the one or more programs configured to perform the methods described in the embodiments above.

Processor 710 may include one or more processing cores. The processor 710 interfaces with various components throughout the electronic device 700 using various interfaces and circuitry to perform various functions of the electronic device 700 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 720 and invoking data stored in the memory 720. Alternatively, the processor 710 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 710 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 710, but may be implemented by a communication chip.

The Memory 720 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 720 may be used to store instructions, programs, code sets, or instruction sets. The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The storage data area may also store data created during use by the electronic device 700, and the like.

It is understood that the electronic device 700 may include more or less structural elements than those shown in the above structural block diagrams, for example, a power supply, an input button, a speaker, a screen, an RF (Radio Frequency) circuit, a Wi-Fi (Wireless Fidelity) module, a bluetooth module, a sensor, etc., and is not limited thereto.

The embodiment of the application discloses a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to realize the method described in the embodiments.

Embodiments of the present application disclose a computer program product comprising a non-transitory computer readable storage medium storing a computer program, and the computer program, when executed by a processor, implements the method as described in the embodiments above.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.

Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable non-volatile memory can include ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), and Direct Rambus DRAM (DRDRAM).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also appreciate that the embodiments described in this specification are all alternative embodiments and that the acts and modules involved are not necessarily required for this application.

In various embodiments of the present application, it should be understood that the size of the serial number of each process described above does not mean that the execution sequence is necessarily sequential, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present application, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, may be embodied in the form of a software product, stored in a memory, including several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of the embodiments of the present application.

The foregoing detailed description has provided a detailed description of an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium, which are disclosed in the embodiments of the present application, and the detailed description has been provided to explain the principles and implementations of the present application, and the description of the embodiments is only provided to help understanding the method and the core idea of the present application. Meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, comprising:

2. The method of claim 1, wherein said selecting a target image from said N second images comprises:

and selecting a second image with image quality arranged in the previous M frames as a target image from second images with image quality higher than the first image according to the arrangement sequence of the image quality from high to low, wherein M is a positive integer which is greater than or equal to 1 and less than or equal to N.

3. The method according to claim 2, wherein the selecting a second image with image quality arranged in the previous M frames as the target image comprises:

and selecting the second image with the highest image quality as the target image.

4. The method according to claim 1, wherein the editing the N frames of original images corresponding to the captured images according to the editing operation to obtain N frames of second images comprises:

identifying the main content in the first image and acquiring the image information of the main content in the first image;

and respectively editing the N frames of original images corresponding to the shot images according to the editing operation and the image information to obtain N frames of second images.

5. The method of claim 4, wherein the image information comprises one or more of position information, occupied image proportion and corresponding field angle of the subject content in the first image.

6. The method according to any one of claims 1 to 5, wherein the editing the N frames of original images corresponding to the captured images according to the editing operation to obtain N frames of second images comprises:

determining a camera corresponding to the operation type of the editing operation;

and acquiring N frames of original images which are acquired by the corresponding cameras and correspond to the shot images, and editing the acquired N frames of original images according to the editing operation to obtain N frames of second images.

7. The method of any one of claims 1 to 5, wherein the camera module comprises at least two of a tele camera, a wide camera, and a standard camera; the editing operation comprises one or more of a zooming-in operation, a zooming-out operation and a cutting-out operation.

8. The method according to claim 1, wherein the captured image is an image subjected to an image beautification process;

after the selecting a target image from the N second images, the method further comprises:

acquiring beautification processing parameters of the shot image;

carrying out image beautification processing on the target image according to the beautification processing parameter;

the outputting the target image includes:

and outputting the beautified target image.

9. An image processing apparatus characterized by comprising:

the camera module is used for shooting a camera module to obtain a shot image;

10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program that, when executed by the processor, causes the processor to carry out the method of any one of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 8.