CN112017137B

CN112017137B - Image processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN112017137B
Application number: CN202010838952.6A
Authority: CN
Inventors: 邱情
Original assignee: Realme Mobile Telecommunications Shenzhen Co Ltd
Current assignee: Realme Mobile Telecommunications Shenzhen Co Ltd
Priority date: 2020-08-19
Filing date: 2020-08-19
Publication date: 2024-02-27
Anticipated expiration: 2040-08-19
Also published as: CN112017137A

Abstract

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring a shooting image shot by a camera module, wherein the camera module comprises at least two cameras; editing the photographed image according to the detected editing operation to obtain a first image; respectively editing N frames of original images corresponding to the photographed images according to editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one of cameras in a camera module, and N is a positive integer greater than or equal to 1; and selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image. The image processing method, the device, the electronic equipment and the computer readable storage medium can improve the quality of the image displayed after the image is edited and enhance the visual effect displayed by the edited image.

Description

Image processing method, device, electronic equipment and computer readable storage medium

Technical Field

The present invention relates to the field of image technology, and in particular, to an image processing method, an image processing device, an electronic device, and a computer readable storage medium.

Background

With the rapid development of electronic technology, more and more users like to take various photos by using electronic equipment, and record various moments in life. After the electronic device takes the photo, the user often edits the photo and adjusts the content of the photo to obtain the desired image. After editing the photo, there is a problem that the quality of the photo is often degraded.

Disclosure of Invention

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a computer readable storage medium, which can improve the quality of an image displayed after image editing and enhance the visual effect displayed by the edited image.

The embodiment of the application discloses an image processing method, which comprises the following steps:

acquiring a shooting image shot by a camera module, wherein the camera module comprises at least two cameras;

editing the photographed image according to the detected editing operation to obtain a first image;

respectively editing N frames of original images corresponding to the photographed images according to the editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one of cameras in the camera module, and N is a positive integer greater than or equal to 1;

And selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

The embodiment of the application discloses an image processing device, including:

the image acquisition module is used for acquiring a shooting image shot by the camera module, and the camera module comprises at least two cameras;

the first editing module is used for editing the photographed image according to the detected editing operation to obtain a first image;

the second editing module is used for respectively editing N frames of original images corresponding to the photographed images according to the editing operation to obtain N frames of second images, each frame of original image is acquired through one of the cameras in the camera module, and N is a positive integer greater than or equal to 1;

and the selecting module is used for selecting a target image from the N frames of second images and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

The embodiment of the application discloses electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to enable the processor to realize the method.

The present embodiments disclose a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described above.

According to the image processing method, the device, the electronic equipment and the computer readable storage medium disclosed by the embodiment of the application, a shooting image obtained through shooting by the camera module is obtained, the shooting image is edited according to the detected editing operation to obtain a first image, N frames of original images corresponding to the shooting image are edited according to the editing operation to obtain N frames of second images, then a target image with higher image quality than the first image is selected from the N frames of second images to be output, after the shooting image is edited, an image with better image quality is selected to be presented to a user, the problem that the image quality is seriously slipped under the condition that the image is edited is avoided, the image quality presented after the shooting image is edited can be improved, and the visual effect presented by the edited image is enhanced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a block diagram of an image processing circuit in one embodiment;

FIG. 2 is a flow chart of an image processing method in one embodiment;

FIG. 3 is a schematic diagram of editing a captured image and a corresponding original image according to an embodiment;

FIG. 4 is a flowchart of an embodiment of editing processing performed to obtain N frames of original images;

FIG. 5 is a flowchart of an image processing method in another embodiment;

FIG. 6 is a block diagram of an image processing apparatus in one embodiment;

fig. 7 is a block diagram of an electronic device in one embodiment.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments and figures herein are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

It will be understood that the terms "first," "second," and the like, as used herein, may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image, without departing from the scope of the present application. Both the first image and the second image are edited images, but they are not the same image.

The embodiment of the application provides electronic equipment. The electronic device includes image processing circuitry, which may be implemented using hardware and/or software components, and may include various processing units defining an ISP (Image Signal Processing ) pipeline. FIG. 1 is a block diagram of an image processing circuit in one embodiment. For ease of illustration, fig. 1 illustrates only aspects of image processing techniques related to embodiments of the present application.

As shown in fig. 1, the image processing circuit includes an ISP processor 140 and a control logic 150. Image data captured by imaging device 110 is first processed by ISP processor 140, where ISP processor 140 analyzes the image data to capture image statistics that may be used to determine one or more control parameters of imaging device 110. Imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (e.g., bayer filters), and the image sensor 114 may acquire light intensity and wavelength information captured by each imaging pixel and provide a set of raw image data that may be processed by the ISP processor 140. The attitude sensor 120 (e.g., tri-axis gyroscope, hall sensor, accelerometer, etc.) may provide acquired image processing parameters (e.g., anti-shake parameters) to the ISP processor 140 based on the type of attitude sensor 120 interface. The attitude sensor 120 interface may employ an SMIA (Standard Mobile Imaging Architecture ) interface, other serial or parallel camera interfaces, or a combination of the above.

It should be noted that, although only one imaging device 110 is shown in fig. 1, in the embodiment of the present application, at least two imaging devices 110 may be included, where each imaging device 110 may correspond to one image sensor 114, or a plurality of imaging devices 110 may correspond to one image sensor 114, which is not limited herein. The operation of each imaging device 110 may be as described above.

In addition, the image sensor 114 may also send raw image data to the gesture sensor 120, the gesture sensor 120 may provide raw image data to the ISP processor 140 based on the gesture sensor 120 interface type, or the gesture sensor 120 may store raw image data in the image memory 130.

The ISP processor 140 processes the raw image data on a pixel-by-pixel basis in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and ISP processor 140 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.

ISP processor 140 may also receive image data from image memory 130. For example, the gesture sensor 120 interface sends the raw image data to the image memory 130, where the raw image data in the image memory 130 is provided to the ISP processor 140 for processing. Image memory 130 may be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and may include DMA (Direct Memory Access ) features.

Upon receiving raw image data from the image sensor 114 interface or from the pose sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. The ISP processor 140 receives the processing data from the image memory 130 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processing data. The image data processed by ISP processor 140 may be output to display 160 for viewing by a user and/or further processing by a graphics engine or GPU (Graphics Processing Unit, graphics processor). In addition, the output of ISP processor 140 may also be sent to image memory 130, and display 160 may read image data from image memory 130. In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

The statistics determined by ISP processor 140 may be sent to control logic 150. For example, the statistics may include image sensor 114 statistics such as vibration frequency of gyroscope, auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 112 shading correction, etc. Control logic 150 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters of imaging device 110 and control parameters of ISP processor 140 based on the received statistics. For example, the control parameters of the imaging device 110 may include attitude sensor 120 control parameters (e.g., gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, camera anti-shake displacement parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balancing and color adjustment (e.g., during RGB processing), as well as lens 112 shading correction parameters.

The image processing method provided in the embodiment of the present application will be described with reference to the image processing circuit of fig. 1. The electronic device may acquire multiple frames of original images through the lens 112 and the image sensor 114 in each imaging device (camera) 110, and each imaging device may acquire one or more frames of original images, and transmit the acquired multiple frames of original images to the ISP processor 140. The ISP processor 140 may perform fusion processing on the multiple frames of original images to obtain a photographed image, and send the photographed image and the multiple frames of original images to the image memory 130. The image memory 130 may store the photographed image and the multi-frame original image transmitted from the ISP processor 140. ISP processor 140 may also transmit the captured image to display 160 for display. When the electronic device detects that the user performs an editing operation on the photographed image displayed on the display 160, the ISP processor 140 may perform an editing process on the photographed image according to the editing operation, resulting in a first image. The ISP processor 140 may obtain N frames of original images corresponding to the photographed image from the image memory 130, edit the N frames of original images according to the editing operation, obtain N frames of second images, select a target image with higher image quality than the first image from the N frames of second images, output the target image to the display 160, and present the target image to the user through the display 160.

As shown in fig. 2, in one embodiment, an image processing method is provided, and the image processing method may be applicable to electronic devices such as a mobile phone, an intelligent wearable device, a tablet computer, a digital camera, and the like. The operating system of the electronic device may include, but is not limited to, an Android operating system, an IOS operating system, a Symbian operating system, a Windows operating system, etc., which are not limited in the embodiments of the present application. The method comprises the following steps:

step 210, acquiring a shooting image shot by the camera module.

In the embodiment of the application, the camera module of the electronic device may include at least two cameras, and each camera may be different in type, or may be cameras of the same type but different models (for example, with different pixels). Optionally, the camera module includes any combination of various cameras such as a wide-angle camera, a telephoto camera, a standard camera, an ultra-wide-angle camera, and a depth camera. The focal length of the standard camera is usually about 50mm (millimeter), the focal length and the visual angle of the wide-angle camera, the long-focus camera and the ultra-wide-angle camera are different, for example, the focal length of the wide-angle camera can be 38 mm-24 mm, the visual angle is 60-84 degrees, the focal length of the ultra-wide-angle camera can be 20-13 mm, the visual angle can be 94-118 degrees, the focal length of the long-focus camera can be 60-400 mm, and the like. The embodiment of the application does not specifically limit the camera contained in the camera module.

When the electronic equipment receives a shooting instruction, each camera contained in the camera module can be controlled to synchronize, one or more frames of images are respectively acquired, and at least an original image acquired by each camera at the same moment can be obtained. After the ISP processor acquires the original images respectively acquired by the cameras, a shooting image can be obtained based on the original images.

Alternatively, the captured image may be an image obtained after fusion processing of a plurality of frames of original images. For example, the camera module includes a wide-angle camera and a telephoto camera, and then the wide-angle camera can collect an original image a, and the telephoto camera can collect an original image B, so that a distant view part in the original image a and a shooting subject in the original image B can be fused, so as to obtain a shooting image with a better effect, and the like.

Alternatively, the photographed image may be an original image of one frame, for example, the camera module includes a standard camera and a depth camera, and the photographed image may be an original image collected by the standard camera, but is not limited thereto. In some embodiments, for different shooting scenes, an original image corresponding to the shooting scene may be selected from the original images acquired by each camera, for example, the camera module includes a standard camera, a wide-angle camera, and a telephoto camera, if the distance between the subject to be shot and the camera is relatively close, the original image captured by the standard camera may be selected as the shooting image, if the distance between the subject to be shot and the camera is relatively far, the original image captured by the telephoto camera may be selected as the shooting image, and if the subject to be shot is a subject with a relatively wide range, the original image captured by the wide-angle camera may be selected as the shooting image.

After the electronic device obtains the photographed image, the photographed image and the original images collected by each camera can be stored, only the photographed image is displayed, and when a user accesses an application program such as an electronic album, the user can only see the photographed image, but not see the original images. The multi-frame images can be prevented from being displayed simultaneously, which causes confusion to users and causes operations such as erroneous deletion.

And 220, editing the photographed image according to the detected editing operation to obtain a first image.

The user may perform editing operations on the displayed photographed image according to the need, which may include, but is not limited to, enlarging, reducing, cropping, adding image content, beautifying processes, etc., wherein the beautifying processes may include, but are not limited to, background virtualization, portrait processes, brightness adjustment, white balance processes, etc., but are not limited thereto.

In some embodiments, when the electronic device detects the user-triggered editing operation for the captured image, it may be determined whether the editing operation will change the image size of the captured image or change the image content size in the captured image, if the editing operation will change the image size of the captured image or change the image content size in the captured image, it indicates that the quality of the captured image will likely be degraded after the editing process is performed on the captured image, so after the electronic device performs the editing process on the captured image, the step 230 may be executed after the first image is obtained. If the editing operation does not change the image size and the image content size of the photographed image, it can be stated that the quality of the photographed image does not slip down after the editing process is performed on the photographed image, the electronic device may directly perform the editing process on the photographed image according to the detected editing operation, and output the first image obtained after the editing process.

And 230, respectively editing N frames of original images corresponding to the photographed images according to the editing operation to obtain N frames of second images.

After editing the photographed image, the electronic device may acquire N frames of original images corresponding to the photographed image from the memory, the N frames of original images being distinguishable from the photographed image. Wherein N may be a positive integer greater than or equal to 1. Optionally, the N frames of original images may include at least one frame of original image acquired by each camera, for example, the camera module includes a standard camera, a wide-angle camera, and a telephoto camera, and then at least one frame of original image acquired by each of the standard camera, the wide-angle camera, and the telephoto camera may be acquired. Optionally, the obtained N frames of original images may also be one or more frames of original images respectively collected by one or more cameras in the camera module. For example, the camera module includes a standard camera, a wide-angle camera and a telephoto camera, so that only one or more frames of original images respectively acquired by the wide-angle camera and the telephoto camera can be acquired.

In some embodiments, after obtaining the captured image, the electronic device may assign the same image tag to each captured image and its corresponding original image frame, and the tag may be used to mark the correspondence between the captured image and the original image. For example, when a captured image is available and stored in an electronic album, a sequence number of the captured image in the electronic album may be generated, the sequence edit may be added as an image tag to each frame of original image corresponding to the captured image, or the like. After the electronic equipment detects the editing operation, the image tag of the shooting image aimed at by the editing operation can be obtained, the corresponding original image is searched in the memory according to the image tag, and then N frames are selected from the original image carrying the image tag for editing processing. The efficiency of data searching and processing can be improved.

After the N frames of original images are acquired, the N frames of original images may be subjected to the same type of editing processing, for example, editing processing of enlarging and then cutting out the photographed image, or editing processing of enlarging and then cutting out the acquired frames of original images, respectively, according to the editing operation. The N frames of second images obtained after editing processing are required to keep the image content consistent with the first images, namely, the pixel coordinates of each pixel point in the second images and the pixel point matched with the pixel point in the first images in the images are consistent, the image content contained in each frame of second images and the image content contained in each frame of first images are required to be identical, and the positions of each image content in the images are also identical.

As an optional implementation manner, after the captured image is edited to obtain the first image, the obtained N frames of original images may be edited according to the first image, so as to ensure that the image content of the obtained N frames of second images is consistent with that of the first image. For example, each feature point in the first image may be extracted, each matched feature point may be found in the original image, and then the original image may be edited according to the position of each feature point in the first image, so as to ensure that the obtained second image is identical to the position of the matched feature point in the first image in the image.

Step 240, selecting a target image from the N frames of second images, and outputting the target image, wherein the image quality of the target image is higher than that of the first image.

Because each frame of original image is acquired through one camera in the camera module, the acquired N frames of original images are different from the shot image (such as different sizes in the image acquired by the shot object due to different focal lengths and angles of view of the cameras), and after the shot image and the N frames of original images are subjected to the same type of editing processing, the image quality of the acquired N frames of second images is different from that of the first image. The image quality of each frame of the second image and the first image can be analyzed, and the second image with higher image quality than the first image is selected from N frames of the second images as a target image. In some embodiments, all the second images with higher quality than the first image may be used as the target images, or any one of the second images with higher quality than the first image may be selected as the target images. Alternatively, the second image with the highest image quality may be directly selected as the target image, and the selected manner is not limited in the embodiment of the present application.

In the embodiment of the present application, since the editing operation mainly changes the size of the image or the size of the image content, the sharpness of the image is affected. Therefore, the image quality may include evaluation indexes such as sharpness. For example, the electronic device may measure the sharpness of the second image and the first image of each frame separately, and the image sharpness may be measured in various ways, for example, by measuring the spatial frequency response (Spatial Frequency Response, SFR), or by measuring the sharpness of the image, which is not limited herein. The sharpness of the second image per frame may be compared with the sharpness of the first image one by one and a second image having a sharpness greater than the sharpness of the first image may be determined. The image quality can be more visual and simpler by utilizing the definition evaluation, and the image processing efficiency is improved.

It should be noted that, the image quality may be evaluated from other dimensions, and is not limited to the definition described above, and the evaluation parameters and the manner of implementing the image quality are not limited in the present application.

Fig. 3 is a schematic diagram of editing a captured image and a corresponding original image according to an embodiment. As shown in fig. 3, the electronic device may display a captured image 310, and upon detecting an enlarged cropping operation for the captured image 310, the captured image 310 is subjected to an enlarged cropping process, resulting in a first image 312. The electronic device may obtain the stored original image a320 and the original image B330 corresponding to the photographed image 310, where the original image a320 and the original image B330 are different from the photographed image 310 (the photographing angles are different). The original image a320 and the original image B330 may be subjected to the magnification clipping process, so as to obtain a second image a322 and a second image B332, where the image contents of the second image a322, the second image B332 and the first image 312 remain consistent. The image quality of the second image a322, the second image B332, and the first image 312 may be respectively acquired, wherein the image quality of the second image B332 is higher than the image quality of the second image a322, and the image quality of the second image a322 is higher than the image quality of the first image 312. The second image a322 and the second image B332 may be simultaneously outputted as target images, or one of them may be selected to be outputted as target images, or the second image B332 having the highest image quality may be directly outputted.

In some embodiments, the outputting the target image may be directly replacing the target image with the first image, and only presenting the target image with better quality to the user, or may also be displaying the first image and the target image simultaneously, prompting the user that the image quality of the target image is higher than that of the first image, and selecting to save the target image or the first image by the user, so as to assist the user in selecting the image with better quality for saving.

In the embodiment of the application, after the shot image is edited, the image with better image quality is selected and presented to the user, so that the problem that the image quality slides down seriously due to the fact that the image is edited is avoided, the image quality presented after the shot image is edited can be improved, and the visual effect presented by the edited image is enhanced.

In one embodiment, the step of selecting the target image from the N frames of the second image comprises: and selecting a second image with the image quality arranged in the previous M frames from the second images with the image quality higher than the first image as a target image according to the arrangement sequence of the image quality from high to low.

After the electronic device edits the N frames of original images, the image quality of each frame of second images can be obtained, and the N frames of second images are sequentially arranged according to the order of the image quality from high to low. After the arrangement, a second image having higher image quality than the first image may be determined, and the second image having the image quality arranged in the previous M frames is selected as the target image in the arrangement order. Wherein M may be a positive integer greater than or equal to 1 and less than or equal to N. Alternatively, M may be a preset value, for example, 1, 2, etc.

For example, if 4 frames of second images are obtained, the images are sequentially arranged into a second image a-a second image c-a second image b-a second image d in order from high to low, wherein the image quality of the second image a, the image quality of the second image c and the image quality of the second image b are higher than those of the first image, the image quality of the second image d is not higher than that of the first image, and the second image a and the second image c with the image quality arranged in the previous 2 can be obtained and output as target images.

Alternatively, the second images with higher image quality than the first image in the N frames of second images may be determined first, and then the second images with higher image quality than the first image may be arranged according to the order of the images from high to low, so as to select the second image with the image quality arranged in the previous M frames as the target image. If the number of frames of the second image having higher image quality than the first image is smaller than the set M, all the second images having higher image quality than the first image may be taken as target images and output.

In one embodiment, M may be 1, and the electronic device may directly select, from the second images having higher image quality than the first image, the second image having the highest image quality as the target image. After the image is edited, the image with the best image quality is displayed to the user, so that the visual effect of the edited image is enhanced.

In other embodiments, the target image may be selected according to other selection rules, for example, the M frames of the second image may be selected randomly from the second images with higher image quality than the first image as the target image, which is not limited herein.

In the embodiment of the application, the target image can be selected and output according to the image quality of the second image of each frame, so that the image quality presented after the shot image is edited can be further improved, and the visual effect presented by the edited image is enhanced.

As shown in fig. 4, in one embodiment, the step of performing editing processing on N frames of original images corresponding to the captured image according to an editing operation to obtain N frames of second images may include the following steps:

step 402, determining a camera corresponding to an operation type of the editing operation.

The electronic equipment can acquire the original image acquired by each camera for editing processing, and can also acquire only partial original images acquired by the cameras. In the embodiment of the application, the camera module may include at least two of a tele camera, a wide camera, a standard camera, and the like. After detecting the editing operation for the shot image, the electronic device may acquire an operation type of the editing operation, and optionally, the editing operation may include one or more of an enlarging operation, a reducing operation, a cropping operation, and the like, and the corresponding operation types may be respectively an image enlarging type, an image reducing type, an image cropping type, and the like.

When editing operations of different operation types are carried out, original images acquired by a fixed camera can exist, and the image quality of a second image obtained after editing processing is high in probability than that of the first image. The camera corresponding to the operation type is the fixed camera. For example, when the enlarging and cutting operation is performed, the image quality of the second image obtained by editing the original image acquired by the tele camera is higher than that of the first image, and the camera corresponding to the image enlarging type can be the tele camera; when the zoom-out and cutting operation is performed, the image quality of the second image obtained by editing the original image acquired by the wide-angle camera is higher than that of the first image, and the camera corresponding to the zoom-out type of the image can be the wide-angle camera.

Step 404, acquiring N frames of original images corresponding to the photographed images acquired by the corresponding cameras, and performing editing processing on the acquired N frames of original images according to the editing operation to obtain N frames of second images.

In some embodiments, when the electronic device stores the original images acquired by each camera, the camera identifiers corresponding to the original images of each frame may be recorded respectively, where the camera identifiers may be used to characterize the camera that acquired the original images. For example, the camera identification of the wide-angle camera may be "001", the camera identification of the standard camera may be "002", the camera identification of the telephoto camera may be "003", etc., but is not limited thereto. The method can acquire the camera identification of the camera corresponding to the operation type of the editing operation, and acquire one or more frames of original images acquired by the camera according to the camera identification.

In some embodiments, the electronic device may also store the original images from different cameras in different memory partitions, and after determining the camera corresponding to the operation type of the editing operation, may obtain one or more frames of original images corresponding to the captured image from the memory partition corresponding to the camera.

Optionally, after the N frames of original images collected by the cameras corresponding to the operation type of the editing operation are edited, if the image quality of the obtained N frames of second images is lower than that of the first image, the original images collected by other cameras can be obtained again for editing, so that a target image with the image quality higher than that of the first image is output and presented to the user.

In the embodiment of the application, the camera corresponding to the operation type of the editing operation can be determined, the N frames of original images acquired by the corresponding camera are acquired for editing, and the image quality probability of the obtained second image is higher than that of the first image after the original images acquired by the camera corresponding to the operation type of the editing operation are subjected to editing, so that the image with higher image quality after editing processing can be obtained, the processed image data are fewer, and the processing speed is accelerated.

As shown in fig. 5, in one embodiment, another image processing method is provided, which is applicable to the electronic device described above, and the method may include the following steps:

step 502, obtaining a shooting image obtained by shooting through a camera module.

And step 504, performing editing processing on the shot image according to the detected editing operation to obtain a first image.

The descriptions of steps 502 to 504 may refer to the related descriptions in the above embodiments, and are not repeated here.

In step 506, the subject content in the first image is identified, and image information of the subject content in the first image is acquired.

The subject content may refer to a subject of the first image, and in some embodiments, the manner in which the subject content in the first image is identified may include, but is not limited to, the following:

mode one: the subject content may be identified based on depth information of the first image. The electronic device may obtain depth information of each pixel point in the first image, where the depth information may be used to characterize a distance between a physical point corresponding to the pixel point in the real shooting scene and the camera, and the greater the depth information of the pixel point, the farther the distance between the corresponding physical point and the camera may be indicated. The foreground region and the background region in the first image can be distinguished through the depth information, wherein the depth information of each pixel point in the foreground region can be smaller than a first depth value, and the depth information of each pixel point in the background region can be larger than a second depth value, and the first depth value can be smaller than the second depth value. The image content in the foreground region may be regarded as subject content.

Mode two: the subject content can be identified based on the image scale occupied by each photographic subject in the first image. The electronic device may first extract an edge feature in the first image, and divide the first image into a plurality of image areas according to the edge feature, where the edge feature may alternatively be a pixel point with a larger contrast with an adjacent pixel point. It may be determined whether adjacent image areas describe the same photographic subject, and the image areas describing the same photographic subject are combined, and then the proportion of each photographic subject in the first image is determined according to the area occupied by the image area included in each photographic subject in the first image. The image area included in the subject having the largest occupied image proportion may be used as the subject content.

Mode three: the subject content of the first image may be identified using a pre-trained subject identification model. The main body recognition model can be obtained through training a large number of image samples, main body content can be marked in each image sample, and the main body recognition model can be trained through learning the characteristics of the image samples.

After identifying the subject content in the first image, image information of the subject content in the first image may be acquired. Alternatively, the image information may include one or more of position information of the subject content in the first image, a ratio of the occupied image, and a corresponding angle of view, etc. The position information of the main body content in the first image can be represented by pixel coordinates of each pixel point contained in the main body content in the first image. The corresponding Field angle of view (FOV) of the subject content in the first image refers to the Field angle (FOV) required to be adopted if the first image is acquired directly by using the camera, and the larger the proportion of the subject content occupied in the first image, the smaller the corresponding Field angle may be, and the smaller the proportion of the subject content occupied in the first image, the larger the corresponding Field angle may be.

And step 508, respectively editing N frames of original images corresponding to the shot images according to the editing operation and the image information to obtain N frames of second images.

The electronic device may edit the original image of each frame according to the image information of the main content in the first image, so that the image information of the main content in the N frames of the second image obtained by the editing process is the same as the image information of the main content in the first image. The image information of the main content of the two main contents is ensured to be consistent, and the image presented to the user can be ensured to be the image required by the user and the image with better quality.

Alternatively, the editing process performed on the original image may be the same type of editing process as the captured image, or may be a different type of editing process, for example, the user reduces the captured image to obtain a first image reduced with respect to the captured image, and for the original image collected by the wide-angle camera, the proportion of the main content in the original image may be smaller than the proportion of the main content in the first image, so that the original image collected by the wide-angle camera needs to be enlarged to obtain a second image consistent with the main content of the first image in image information.

Step 510, selecting the second image with the image quality arranged in the previous M frames from the second images with the image quality higher than the first image as the target image according to the arrangement sequence of the image quality from high to low.

Step 512, outputting the target image.

In one embodiment, the captured image subjected to the editing operation by the user may be an image subjected to an image beautifying process, which may be a process performed by the ISP processor when the captured image is obtained, such as a background blurring process, a white balance process, etc., or may be a process performed later on the captured image, such as a portrait beautifying process, a brightness adjusting process, etc. After selecting the target image, the electronic device may obtain a beautification parameter of the shot image, where the beautification parameter refers to a parameter of beautification performed on the shot image, for example, the shot image may obtain a parameter such as a brightness-adjusted region position and an adjusted brightness value after brightness adjustment, and the shot image may obtain a parameter such as a blurring degree after background blurring.

The electronic equipment can carry out image beautification processing on the target image according to the beautification processing parameters and output the beautified target image. The target image after the image beautifying treatment can keep the same image effect as the first image, so that the output target image meets the requirement of a user on the image effect. After the editing processing is carried out on the image, the image with higher image quality is presented, and meanwhile, the image effect of the original image is not influenced.

In the embodiment of the application, the original image can be edited according to the image information of the main body content in the first image, so that the main body content of each frame of the second image obtained by the editing processing is consistent with the first image, the presented target image can be ensured to meet the editing operation requirement of a user, and the presented image has higher image quality.

As shown in fig. 6, in one embodiment, an image processing apparatus 600 is provided, which is applicable to the above-mentioned electronic device, and the image processing apparatus 600 may include an image acquisition module 610, a first editing module 620, a second editing module 630, and a selection module 640.

The image acquisition module 610 is configured to acquire a captured image captured by a camera module, where the camera module includes at least two cameras.

The first editing module 620 is configured to perform editing processing on the captured image according to the detected editing operation, so as to obtain a first image.

The second editing module 630 is configured to perform editing processing on N frames of original images corresponding to the captured images according to an editing operation, so as to obtain N frames of second images, where each frame of original image is acquired by one of the cameras in the camera module, and N is a positive integer greater than or equal to 1.

And a selecting module 640, configured to select a target image from the N frames of second images, and output the target image, where the image quality of the target image is higher than that of the first image.

In one embodiment, the selecting module 640 is further configured to select, from the second images with higher image quality than the first image, the second images with the image quality arranged in the previous M frames as the target images according to the order of the image quality from high to low, and output the target images, where M is a positive integer greater than or equal to 1 and less than or equal to N.

In one embodiment, the selecting module 640 is further configured to select, from the second images with higher image quality than the first image, the second image with the highest image quality as the target image, and output the target image.

In one embodiment, the second editing module 630 is further configured to determine a camera corresponding to an operation type of the editing operation, acquire N frames of original images corresponding to the captured image acquired by the corresponding camera, and edit the acquired N frames of original images according to the editing operation to obtain N frames of second images.

Optionally, the camera module comprises at least two of a tele camera, a wide-angle camera and a standard camera; the editing operation includes one or more of a zoom-in operation, a zoom-out operation, and a crop operation.

In one embodiment, the second editing module 630 includes a body recognition unit and an editing unit.

And the main body identification unit is used for identifying main body content in the first image and acquiring image information of the main body content in the first image.

Optionally, the image information includes one or more of position information of the subject content in the first image, a ratio of the occupied image, and a corresponding angle of view.

And the editing unit is used for respectively editing N frames of original images corresponding to the photographed images according to the editing operation and the image information to obtain N frames of second images.

In one embodiment, the image processing apparatus 600 further includes a beautification processing module in addition to the image acquisition module 610, the first editing module 620, the second editing module 630, and the selection module 640.

The beautifying processing module is used for acquiring beautifying processing parameters of the shot image after the selecting module 640 selects the target image, and carrying out image beautifying processing on the target image according to the beautifying processing parameters.

The selecting module 640 is further configured to output the target image after the beautifying process.

In the embodiment of the application, after the image is edited, the image with higher image quality is displayed, and meanwhile, the image effect of the original image is not influenced.

Fig. 7 is a block diagram of an electronic device in one embodiment. The electronic device may be a mobile phone, a tablet computer, an intelligent wearable device, a PC, a notebook computer, or the like. As shown in fig. 7, the electronic device 700 may include one or more of the following components: a processor 710, a memory 720 coupled to the processor 710, and a camera module 730 coupled to the processor. The camera module 730 may include at least two cameras, and may be configured to acquire multiple frames of original images. Memory 720 may store one or more application programs that may be configured to be executed by one or more processors 710, the one or more program(s) configured to perform the methods as described in the various embodiments above.

Processor 710 may include one or more processing cores. The processor 710 utilizes various interfaces and lines to connect various portions of the overall electronic device 700, perform various functions of the electronic device 700, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 720, and invoking data stored in the memory 720. Alternatively, the processor 710 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 710 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 710 and may be implemented solely by a single communication chip.

The Memory 720 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Memory 720 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 720 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The storage data area may also store data created by the electronic device 700 in use, and the like.

It is to be appreciated that electronic device 700 may include more or fewer structural elements than those described in the above-described structural block diagrams, including, for example, a power source, input keys, speakers, a screen, an RF (Radio Frequency) circuit, a Wi-Fi (Wireless Fidelity) module, a bluetooth module, a sensor, etc., and may not be limited herein.

The present embodiments disclose a computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method as described in the above embodiments.

The present embodiments disclose a computer program product comprising a non-transitory computer readable storage medium storing a computer program, which when executed by a processor, implements a method as described in the above embodiments.

Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.

Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory can include ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (Electrically Erasable PROM, EEPROM), or flash memory. Volatile memory can include random access memory (random access memory, RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (Dynamic Random Access Memory, DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDR SDRAM), enhanced SDRAM (Enhanced Synchronous DRAM, ESDRAM), synchronous Link DRAM (SLDRAM), memory bus Direct RAM (Rambus DRAM), and Direct memory bus dynamic RAM (DRDRAM).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments and that the acts and modules referred to are not necessarily required in the present application.

In various embodiments of the present application, it should be understood that the size of the sequence numbers of the above processes does not mean that the execution sequence of the processes is necessarily sequential, and the execution sequence of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer-accessible memory. Based on such understanding, the technical solution of the present application, or a part contributing to the prior art or all or part of the technical solution, may be embodied in the form of a software product stored in a memory, including several requests for a computer device (which may be a personal computer, a server or a network device, etc., in particular may be a processor in the computer device) to perform part or all of the steps of the above-mentioned method of the various embodiments of the present application.

The foregoing has described in detail the image processing method, apparatus, electronic device and computer readable storage medium disclosed in the embodiments of the present application, and specific examples have been applied to illustrate the principles and embodiments of the present application, where the foregoing description of the embodiments is only for aiding in understanding the method and core idea of the present application. Meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. An image processing method, comprising:

acquiring shooting images shot by a camera module, wherein the camera module comprises at least two cameras, and the shooting images are images obtained by fusion processing of multiple frames of original images;

editing the shot image according to the detected editing operation to obtain a first image, wherein the editing operation comprises one or more of an amplifying operation, a shrinking operation and a cutting operation;

judging whether the editing operation changes the image size of the photographed image or changes the image content size in the photographed image;

if yes, respectively editing N frames of original images corresponding to the photographed images according to the editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one of the cameras in the camera module, and N is a positive integer greater than or equal to 1;

2. The method of claim 1, wherein selecting the target image from the N frames of second images comprises:

And selecting a second image with the image quality arranged in the previous M frames from the second images with the image quality higher than the first image as a target image according to the arrangement sequence of the image quality from high to low, wherein M is a positive integer greater than or equal to 1 and less than or equal to N.

3. The method according to claim 2, wherein selecting the second image of the M frames having the image quality arranged in the previous frame as the target image includes:

and selecting the second image with the highest image quality as a target image.

4. The method according to claim 1, wherein the editing processing is performed on N frames of original images corresponding to the captured image according to the editing operation, respectively, to obtain N frames of second images, including:

identifying main content in the first image and acquiring image information of the main content in the first image;

and respectively editing N frames of original images corresponding to the shooting images according to the editing operation and the image information to obtain N frames of second images.

5. The method of claim 4, wherein the image information includes one or more of location information of the subject content in the first image, a ratio of occupied images, and a corresponding angle of view.

6. The method according to any one of claims 1 to 5, wherein the performing editing processing on N frames of original images corresponding to the captured image according to the editing operation to obtain N frames of second images includes:

determining a camera corresponding to the operation type of the editing operation;

and acquiring N frames of original images which are acquired by the corresponding cameras and correspond to the shooting images, and editing the acquired N frames of original images according to the editing operation to obtain N frames of second images.

7. The method of any one of claims 1 to 5, wherein the camera module comprises at least two of a tele camera, a wide camera, and a standard camera.

8. The method according to claim 1, wherein the photographed image is an image subjected to an image beautifying process;

after the selecting the target image from the N frames of second images, the method further includes:

acquiring beautification processing parameters of the shot image;

carrying out image beautifying processing on the target image according to the beautifying processing parameters;

the outputting the target image includes:

outputting the target image after the beautifying treatment.

9. An image processing apparatus, comprising:

the image acquisition module is used for acquiring shooting images shot by the camera module, the camera module comprises at least two cameras, and the shooting images are images obtained after fusion processing of multiple frames of original images;

the first editing module is used for carrying out editing processing on the shot image according to the detected editing operation to obtain a first image, wherein the editing operation comprises one or more of an amplifying operation, a shrinking operation and a cutting operation;

a second editing module for judging whether the editing operation will change the image size of the photographed image or change the image content size in the photographed image; if yes, respectively editing N frames of original images corresponding to the photographed images according to the editing operation to obtain N frames of second images, wherein each frame of original image is acquired through one of the cameras in the camera module, and N is a positive integer greater than or equal to 1;

10. An electronic device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to implement the method of any of claims 1 to 8.

11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method according to any one of claims 1 to 8.