WO2021185250A1 - 图像处理方法及装置 - Google Patents

图像处理方法及装置 Download PDF

Info

Publication number
WO2021185250A1
WO2021185250A1 PCT/CN2021/081094 CN2021081094W WO2021185250A1 WO 2021185250 A1 WO2021185250 A1 WO 2021185250A1 CN 2021081094 W CN2021081094 W CN 2021081094W WO 2021185250 A1 WO2021185250 A1 WO 2021185250A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
preview
preview image
terminal
target subject
Prior art date
Application number
PCT/CN2021/081094
Other languages
English (en)
French (fr)
Inventor
王银廷
周承涛
张熙
黄一宁
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21772093.7A priority Critical patent/EP4117272A4/en
Publication of WO2021185250A1 publication Critical patent/WO2021185250A1/zh
Priority to US17/948,397 priority patent/US20230019164A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/62—Control of parameters via user interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H04N23/635—Region indicators; Field of view indicators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof

Definitions

  • This application relates to the field of photography, and in particular to an image processing method and device.
  • the high-magnification zoom mode is usually used to shoot distant objects, the slight shake of the lens will also cause a large shake of the preview area. This makes it difficult for the user to compose the picture, and even moves the subject of the shot out of the screen. It often takes multiple shots to obtain an ideal picture, and the user experience is not good.
  • the embodiment of the application relates to an image processing method and device, which can be applied to shooting scenes such as high-magnification zoom shooting, to solve the problems of the preview interface shaking greatly caused by the shaking of the handheld terminal, and it is difficult to obtain a clear target subject. Shooting performance.
  • a computing device refers to a device that can be abstracted as a computer system.
  • a computing device that supports image processing functions can be called an image processing device or an image processing device.
  • the image processing apparatus may be the entire computer of the computing device, or may be a part of the computing device, such as a chip related to image processing functions, such as an image processor (Image Signal Processor, ISP) or a system chip.
  • the system chip is also called system-on-chip, or SoC chip.
  • the image processing device may be a terminal such as a smart phone, or may be a system chip or an image processor that can be set in the terminal.
  • the image processor can be integrated inside the SoC chip or not integrated with the SoC chip.
  • an image processing method comprising: acquiring a first preview image of a scene to be shot, the first preview image corresponding to the first shooting magnification; determining a target subject in the first preview image; Acquire a second original image in the to-be-shot scene.
  • the second original image is an image collected by the terminal’s image sensor when the terminal captures the to-be-shot scene based on a target lens.
  • the target lens is the second A lens called when shooting the scene to be shot at a shooting magnification, the second shooting magnification is greater than the first shooting magnification; a second preview image is determined according to the target subject and the second original image, and the second preview image includes The target subject; displaying the second preview image on the display screen of the terminal.
  • the first shooting magnification may be a shooting magnification specified by the user, or may be a default shooting magnification after the terminal turns on the camera.
  • the target lens may be the default lens preset by the system.
  • the terminal may re-determine the target lens according to the user-specified shooting magnification.
  • the target subject can be one object, or multiple objects, it can also be all objects in an area, or it can be understood as a target area.
  • the second shooting magnification is the shooting magnification specified by the user.
  • the preview stream of the terminal includes one or more frames of preview images, such as a second preview image.
  • the terminal may predict the position of the target subject in the second original image based on the preview image of one frame before the second original image, or the position of the target subject in multiple frames of preview images.
  • the above-mentioned multi-frame preview image may be a continuous multi-frame preview image, or a multi-frame preview image at periodic or aperiodic intervals.
  • the first preview image may be displayed on the display screen of the terminal.
  • the preview image is determined based on the target subject and the original image collected by the terminal sensor when the terminal is shooting the scene to be shot. Re-determine) to adaptively change the current preview image, instead of only determining the preview image based on the original image collected by the terminal's sensor when the terminal is shooting the scene to be photographed. It can reduce the probability of the target subject moving out of the preview screen due to terminal jitter.
  • determining the second preview image according to the target subject and the second original image specifically includes: according to the position of the target subject in the second original image, the second shooting magnification and The second original image determines the second preview image.
  • the position of the target subject in the image to be cropped is determined.
  • the terminal can determine the required digital zoom magnification m according to the second shooting magnification and the optical zoom magnification of the target lens, where m is greater than or equal to 1.
  • the digital zoom magnification m is the ratio of the second shooting magnification to the optical zoom magnification of the target lens.
  • the terminal can determine the size of the area to be cropped in the image to be cropped, and obtain the second preview image from the image to be cropped.
  • the terminal can determine the required digital zoom magnification m according to the second shooting magnification and the optical zoom magnification of the target lens, and then determine the size of the area to be cropped in the image to be cropped. According to the size of the area to be cropped, the position of the target subject, and the second shooting magnification, the position of the center point of the area to be cropped is determined, and then the second preview image is determined.
  • the second shooting magnification is a shooting magnification specified by the user, or a default shooting magnification when the terminal shoots the scene to be shot.
  • the above-mentioned image to be cropped may be a second original image, or an intermediate image obtained based on the second original image.
  • the second original image may be an image obtained after cropping adapted to the display specifications of the terminal, or may also be a second original image.
  • the original image has undergone preview image processing, or part of the preview image processing (such as the first preview image processing), or the second original image has undergone preview image processing or part of the preview image processing (such as the first preview image processing) , And the image after cropping adapted to the display specification of the terminal.
  • preview image processing or partial preview image processing may also be performed before the second preview image is obtained in the foregoing processing.
  • the first preview image processing and the second preview image processing may be the same or different.
  • the sequence of processing operations such as cropping out 1/m 2 size, magnifying m 2 times, preview image processing or partial image processing, and adapting to the display specifications of the terminal can be adjusted, which is not limited by this application.
  • the image obtained after the second original image is cropped to adapt to the display specifications of the terminal is used as the image to be cropped, and the position of the target subject in the to-be cropped is the center or reference point, and then the size of 1/m 2 is cropped first.
  • the method of zooming in by m 2 times and then performing preview image processing to obtain the second preview image can reduce the computing resources required by the terminal to process the foregoing image and increase the processing speed.
  • the method further includes:
  • the method further includes:
  • a first operation on the target subject in the second preview image is received, the first operation is used to determine the target position of the target subject in the new preview image, and the third original image of the scene to be shot is acquired.
  • the original image is the image collected by the terminal’s sensor when the terminal shoots the scene to be shot based on the target lens.
  • the acquisition time of the second original image is earlier than the third original image.
  • the third preview image includes the target subject, and the third preview image is displayed on the display screen of the terminal.
  • the new preview image includes a third preview image.
  • the first operation may be gesture, voice, facial, and other types of operations.
  • the terminal detects and receives the user's two-finger dragging the target subject in the second preview image through the touch screen, and determines the target subject according to the direction and distance of the user's two-finger dragging and the position of the target subject in the second preview image.
  • the target position of the target subject in the third preview image wherein, the target position changes with the position of the target subject in the second preview image, the direction is the same as the dragging direction of the user's two fingers, and the distance is proportional to the dragging distance of the user's two fingers.
  • the size of the ratio can be determined by the user
  • the system setting interface is set by precision granularity, or it can be set according to the usage habits of the user or a large number of users.
  • the second original image and the third original image are different images collected by the terminal's sensor when the same target lens is used to shoot the scene to be shot.
  • the collection time corresponding to the second original image and the third original image is not complete Consistent, the time interval is short.
  • the terminal determines the size of the area to be cropped and the magnification to be magnified according to the second shooting magnification, and according to the determined target position of the target subject in the third preview image, as well as the size of the area to be cropped and the magnification to be Based on the enlarged ratio, the third preview image is determined based on the third original image.
  • the above-mentioned image to be cropped may be a third original image or an intermediate image obtained based on the third original image.
  • the above solution can adjust the position of the target subject in the preview image according to the user's instruction operation on the target subject, meet the user's diversified composition requirements, and avoid the unity caused by the fixed position of the target subject in the preview image.
  • the method further includes: after the terminal enters the anti-shake mode, displaying a first preview area and a second preview area on the display screen of the terminal, where the first preview area is used for An auxiliary preview image is displayed, the second preview area is used to display the second preview image, and the auxiliary preview image includes content in the second preview image.
  • auxiliary preview mode can also be referred to as auxiliary preview mode, or multi-preview area mode, etc., which can be selected in the form of a specific mode in the user's camera setting interface or system setting interface. Make a limit.
  • the viewing angle range of the auxiliary preview image is larger than the viewing angle range of the second preview image.
  • the method further includes: when the second shooting magnification is greater than a preset threshold, displaying a first preview area and a second preview area on the display screen of the terminal, wherein the first preview The area is used to display an auxiliary preview image, the second preview area is used to display the second preview image, and the auxiliary preview image includes content in the second preview image.
  • the value setting of the preset threshold may be preset by the user, for example, the user may set it in the terminal system setting page.
  • the value setting of the preset threshold may also be determined according to an empirical value. For example, a user survey is conducted to determine a shooting magnification that is imperceptible to small hand shakes of the user. The value of the magnification is determined as the preset threshold.
  • the second preview area can also be used to display other preview pictures in the preview stream, such as the third preview picture.
  • the content displayed in the first preview area and the second preview area can be exchanged according to user operations, for example, receiving operations on the first preview area and the second preview area (such as swiping from the first preview area to Operation of the second preview area), in response to the above operation, exchange the contents displayed in the first preview area and the second preview area, that is, the first preview area is used to display the second preview image, and the first preview area is used to display the second preview image.
  • the auxiliary preview image is used to display the first preview area and the second preview area.
  • the size and display specifications of the first preview area can also be adjusted according to the user's instructions, for example, receiving an operation on the first preview area (such as two fingers sliding in the opposite direction in the first preview area at the same time), in response to the above operation,
  • the size of the first preview area is enlarged according to a certain ratio, where the size of the ratio can be set by the user through the precision granularity on the system setting interface, or can also be set according to the usage habits of the user or a large number of users.
  • the above solution can assist the display of the preview image, so that the user can perceive the position of the current preview image (such as the second preview image and the third preview image) in the current scene to be shot, so that the user can adjust the composition.
  • the size, display specification and display content of the preview area can be adjusted, which improves the flexibility of users.
  • the auxiliary preview image is determined based on the second original image.
  • the auxiliary preview image is determined based on the second original image, or an intermediate image obtained based on the second original image.
  • the viewing angle range of the auxiliary preview image should be larger than the viewing angle range of the second original image.
  • the range of the area to be cropped corresponding to the auxiliary preview image should be larger than the range of the area to be cropped corresponding to the second original image.
  • the auxiliary preview image is determined based on the auxiliary original image, where the auxiliary original image and the second original image call different lenses for the terminal, and when the scene to be shot is taken, the image of the terminal Different images collected by the sensor.
  • the image collected by the terminal's sensor carries a time stamp. After the collected image is processed by the preview image, it is sent to the display for display. Synchronization will be based on the timestamp.
  • the absolute time corresponding to images with the same time stamp may not be completely the same. That is, the acquisition time corresponding to the auxiliary original image and the second original image are not completely consistent, but the time interval is relatively short, such as a difference of 0.01 millisecond.
  • the acquisition time sequence corresponding to the auxiliary original image and the second original image can be adjusted, which is not limited in this application.
  • the first preview image includes a first object
  • determining the target subject in the first preview image specifically includes: determining the first object when the first object is a system preset object As the target subject.
  • the objects preset by the system include one or more objects.
  • the first object may be the first system preset object detected by the terminal, or it may be the system preset object with the highest priority detected by the terminal, or it may be the system preset object with the largest number detected by the terminal .
  • the objects preset by the system can be set by the user in the system setting interface, or according to the shooting habits of the current user or a large number of users, the objects with a higher shooting probability of the user can be set as the objects preset by the system.
  • the terminal By presetting the object category list of the target subject by the system, the terminal directly recognizes the target subject in the first preview image, reducing user interaction, high processing efficiency, and convenient user operation.
  • the first preview image includes a first object
  • determining the target subject in the first preview image specifically includes: receiving a second object for the first object in the first preview image. Operation, in response to the second operation, determining that the first object is the target subject.
  • the above-mentioned second operation may be one or more of gesture operations such as finger click, double tap, long press, knuckle tapping, etc., space gestures, etc., or voice operation or facial operation, which is not limited in this application.
  • gesture operations such as finger click, double tap, long press, knuckle tapping, etc., space gestures, etc., or voice operation or facial operation, which is not limited in this application.
  • the determining the target subject in the first preview image specifically includes: receiving a third operation for the first area in the first preview image, and in response to the third operation, determining The object in the first area is the target subject.
  • the above-mentioned third operation may be a gesture operation such as a single finger or two fingers encircling the first area, or a knuckle encircling the first area, or a space gesture or voice operation or facial operation, which is not limited in this application.
  • the terminal can also more accurately identify the target subject that the user needs to photograph, and reduce the probability of user correction.
  • the user can reassign the target subject.
  • the target subject is marked in the second preview image by a marking frame and/or a symbol marking.
  • the marking frame may be a regular shape, such as a square, a circle, etc., or it may be an irregular shape, such as the outline of the target body.
  • the symbol identification can be a metre-shaped symbol, a cross-shaped symbol, etc., or it can be a straight line or a curve.
  • the line thickness and color of the above-mentioned logo frame and symbol logo can be adjusted. Generally speaking, a thicker or brighter line can be used for logo to distinguish it from other objects in the picture.
  • the target subject can also be marked in other preview images in the preview stream.
  • the position of the target subject in the preview image is marked by the marking frame and/or the symbol mark, so that the user can understand the target subject currently determined by the terminal for the user to confirm or adjust, and avoid identification errors.
  • the area to be cropped is wrong.
  • an image processing method comprising: acquiring a first preview image of a scene to be shot, the first preview image corresponding to the first shooting magnification; determining a target area in the first preview image; Acquire a second original image in the to-be-shot scene.
  • the second original image is an image collected by the terminal’s image sensor when the terminal captures the to-be-shot scene based on a target lens.
  • the target lens is the second A lens called when shooting the scene to be shot at a shooting magnification, the second shooting magnification is greater than the first shooting magnification; a second preview image is determined according to the target area and the second original image, and the second preview image includes The target area; displaying the second preview image on the display screen of the terminal.
  • target area can also be understood as the target subject.
  • technical details, technical effects and possible implementations of the second aspect and its possible implementations please refer to the relevant descriptions in the first aspect and any of its possible implementations. , I won’t repeat it here.
  • an image processing device in a third aspect, includes a transceiving unit and a processing unit.
  • the transceiving unit is configured to acquire a first preview image in a scene to be shot, the first preview image corresponding to the first shot. Magnification; the processing unit is used to determine the target subject in the first preview image; the transceiver unit is also used to obtain a second original image of the scene to be shot, the second original image is the terminal based on the target lens to capture the When the scene is to be shot, the image captured by the image sensor of the terminal, the target lens is the lens called when the terminal is shooting the scene to be shot at the second shooting magnification, and the second shooting magnification is greater than the first shooting magnification
  • the processing unit is also used to determine a second preview image based on the target subject and the second original image, the second preview image includes the target subject; the processing unit is also used to enable the display screen of the terminal The second preview image is displayed.
  • the foregoing transceiver unit and processing unit can be implemented by software, for example, a computer program or instruction with the foregoing functions can be implemented, and the corresponding computer program or instruction can be stored in the internal memory of the terminal, and the processor can read the internal data of the memory.
  • Corresponding computer programs or instructions realize the above-mentioned functions.
  • the image processing device can also be realized by hardware.
  • the processing unit is a processor (such as an image processing unit GPU, a processor in a system chip, etc.), and the transceiver unit is a transceiver circuit or an interface circuit.
  • the image processing device in the embodiment of the present application may also be implemented by a combination of a processor and a software module.
  • the processing unit is further configured to determine a second preview image based on the target subject and the second original image, and specifically includes: the processing unit is further configured to determine whether the target subject is in the first preview image. Two positions in the original image, the second shooting magnification and the second original image determine a second preview image.
  • the transceiver unit is further configured to receive a first operation on the target subject in the second preview image, and the first operation is used to determine the target subject of the target subject in the new preview image. Position; the processing unit is also used to determine the updated second preview image according to the target position of the target subject in the new preview image and the second original image, and the updated second preview image includes the Target subject; the processing unit is also used to enable the display screen of the terminal to display the updated second preview image.
  • the transceiver unit is further configured to receive a first operation on the target subject in the second preview image, and the first operation is used to determine the target subject of the target subject in the new preview image. Position; the transceiver unit is also used to obtain a third original image of the scene to be photographed, the third original image is the image collected by the terminal's sensor when the terminal shoots the scene to be photographed based on the target lens, the second The acquisition time of the original image is earlier than the third original image; the processing unit is further configured to determine the third preview image according to the target position of the target subject in the new preview image and the third original image, The third preview image includes the target body; the processing unit is further configured to enable the display screen of the terminal to display the third preview image.
  • the processing unit is further configured to enable the first preview area and the second preview area to be displayed on the display screen of the terminal when the second shooting magnification is greater than a preset threshold, where the first preview area The preview area is used to display an auxiliary preview image, the second preview area is used to display the second preview image, and the auxiliary preview image includes the content of the second preview image.
  • the auxiliary preview image is determined based on the second original image.
  • the auxiliary preview image is determined based on the auxiliary original image, where the auxiliary original image and the second original image are different lenses for the terminal to call, and when shooting the scene to be shot, the image sensor of the terminal Different images collected.
  • the first preview image includes a first object
  • the processing unit is configured to determine a target subject in the first preview image, and specifically includes: the processing unit is configured to determine whether the first object is When the system presets the object, the first object is determined to be the target subject.
  • the first preview image includes a first object
  • the transceiver unit is configured to receive a second operation on the first object in the first preview image
  • the processing unit is configured to Determining the target subject in the first preview image specifically includes: the processing unit, configured to determine that the first object is the target subject in response to the second operation.
  • the transceiver unit is configured to receive a third operation on the first area in the first preview image; the processing unit is configured to determine the target subject in the first preview image, specifically including : A processing unit, configured to determine that the object in the first area is the target subject in response to the third operation.
  • the target subject is marked in the second preview image by a marking frame and/or a symbol marking.
  • the second shooting magnification is a user-specified shooting magnification.
  • the target body is located at a position other than the center position in the second preview image.
  • a terminal in a fourth aspect, includes: a touch screen, wherein the touch screen includes a touch-sensitive surface and a display; a camera; one or more processors; a memory; a plurality of application programs; and one or more computer programs, The one or more computer programs are stored in the memory, and the one or more computer programs include instructions.
  • the terminal executes the first aspect to the second aspect and any one of the possibilities.
  • an electronic device including a touch screen, a memory, one or more processors, multiple application programs, and one or more programs; wherein the one or more programs are stored in the memory; and features In that, when the one or more processors execute the one or more programs, the electronic device executes the image processing method in the first aspect to the second aspect and any one of the possible implementation manners.
  • a computer storage medium which is characterized by including computer instructions, which when the computer instructions run on an electronic device, cause the electronic device to execute the first aspect to the second aspect and any possible implementation manners This image processing method.
  • a computer program product which is characterized in that when the computer program product runs on a computer, the computer is caused to execute the image processing method according to any one of claims 1 to 11.
  • the image processing solution provided by the embodiments of the present application can be used to improve the stability of image shooting, especially when the user-specified shooting magnification is high, the original image to be processed by the terminal can be cropped based on the position of the target subject. Image to improve the performance of image capture. In addition, you can also adjust the position of the target subject in the preview image according to the user's adjustment operation to improve the flexibility of composition.
  • the display mode of multiple preview areas can also facilitate the user to adjust the target subject and know the current position of the target subject in the image collected by the camera, so as to reduce the probability of the target subject being drawn.
  • FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the application.
  • FIG. 2 is a schematic diagram of the software structure of an electronic device provided by an embodiment of the application.
  • FIG. 3 is a schematic flowchart of an image processing method in the prior art
  • FIG. 4 is a schematic diagram showing the display of the first frame of image of the shooting target subject in a high-magnification zoom scene in the prior art
  • FIG. 5 is a schematic diagram showing the display of the 50th frame image of the shooting target subject in a high-magnification zoom scene in the prior art
  • FIG. 6 is a schematic diagram showing the display of the 100th frame image of the shooting target subject in a high-magnification zoom scene in the prior art
  • FIG. 7 is an image user interface of a mobile phone provided by an embodiment of the application.
  • FIG. 8 is another graphical user interface of a mobile phone provided by an embodiment of the application.
  • Fig. 9 is another graphical user interface of a mobile phone provided by an embodiment of the application.
  • FIG. 10 is a schematic flowchart of an image processing method provided by an embodiment of the application.
  • FIG. 11A is a schematic structural diagram of a preview image processing algorithm provided by an embodiment of this application.
  • FIG. 11B is a schematic diagram of an image cropping method provided by an embodiment of this application.
  • FIG. 12 is another graphical user interface of a mobile phone provided by an embodiment of the application.
  • FIG. 13 is another graphical user interface of a mobile phone provided by an embodiment of the application.
  • FIG. 14 is a schematic diagram of a display of a shooting target subject in a high-magnification zoom scene provided by an embodiment of the application.
  • FIG. 15 is a user interface for a user to adjust a shooting magnification provided by an embodiment of the application
  • FIG. 16 is another user interface for adjusting the shooting magnification provided by an embodiment of the application.
  • FIG. 17 is a user interface of another mobile phone provided by an embodiment of the application.
  • FIG. 19 is a user interface for photographing provided by an embodiment of the application.
  • FIG. 20 is another user interface for photographing provided by an embodiment of the application.
  • FIG. 21 is a schematic structural diagram of an image processing device provided by an embodiment of the application.
  • FIG. 22 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the application.
  • the embodiment of the present application proposes an image processing solution, including an image processing method and device.
  • the image processing method proposed in the embodiment of this application can be applied to a computing device (for example, the image processing device or electronic device proposed in this application), which can track the target subject in the preview stream when taking photos or videos, and According to the position of the target subject, the preview image is cropped from the captured image, so that when the handheld camera is shooting a high-magnification zoom scene, the problem of the target subject moving out of the shooting screen caused by hand shake or camera shake is avoided, and the shooting is improved. stability.
  • the image display method provided by the embodiments of this application can be applied to mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) devices, notebook computers, and super mobile personal computers
  • AR augmented reality
  • VR virtual reality
  • UMPC ultra-mobile personal computer
  • netbooks netbooks
  • PDA personal digital assistant
  • FIG. 1 shows a schematic structural diagram of an electronic device 100.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100.
  • the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented by hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait.
  • AP application processor
  • modem processor modem processor
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the different processing units may be independent devices or integrated in one or more processors.
  • the controller may be the nerve center and command center of the electronic device 100.
  • the controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the processor 110 may include one or more interfaces.
  • the interface can include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transmitter (universal asynchronous) interface.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB Universal Serial Bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may include multiple sets of I2C buses.
  • the processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100.
  • the I2S interface can be used for audio communication.
  • the processor 110 may include multiple sets of I2S buses.
  • the processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170.
  • the audio module 170 may transmit audio signals to the wireless communication module 160 through an I2S interface, so as to realize the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communication to sample, quantize and encode analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • the UART interface is generally used to connect the processor 110 and the wireless communication module 160.
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function.
  • the audio module 170 may transmit audio signals to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with the display screen 194, the camera 193 and other peripheral devices.
  • the MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on.
  • the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100.
  • the processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect earphones and play audio through earphones. This interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules illustrated in the embodiment of the present application is merely a schematic description, and does not constitute a structural limitation of the electronic device 100.
  • the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger can be a wireless charger or a wired charger.
  • the charging management module 140 may receive the charging input of the wired charger through the USB interface 130.
  • the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance).
  • the power management module 141 may also be provided in the processor 110.
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
  • the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100.
  • the mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like.
  • the mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation via the antenna 1.
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194.
  • the modem processor may be an independent device.
  • the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
  • the wireless communication module 160 may also receive a signal to be sent from the processor 110, perform frequency modulation, amplify, and convert it into electromagnetic waves to radiate through the antenna 2.
  • the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), and the quasi-zenith satellite system (quasi). -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite-based augmentation systems
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is an image processing microprocessor, which is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations and is used for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos, and the like.
  • the display screen 194 includes a display panel.
  • the display panel can use liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc.
  • the electronic device 100 may include one or N display screens 194, and N is a positive integer greater than one.
  • the electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
  • the ISP is used to process the data fed back from the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing and is converted into an image visible to the naked eye.
  • ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193.
  • the camera 193 is used to capture still images or videos.
  • the object generates an optical image through the lens and is projected to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the electronic device 100 may include one or N cameras 193, and N is a positive integer greater than one.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects the frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • MPEG moving picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, and so on.
  • NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, and so on.
  • the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
  • the internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required by at least one function, and the like.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.
  • UFS universal flash storage
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal.
  • the audio module 170 can also be used to encode and decode audio signals.
  • the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.
  • the speaker 170A also called “speaker” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
  • the receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.
  • the microphone 170C also called “microphone”, “microphone”, is used to convert sound signals into electrical signals.
  • the user can make a sound by approaching the microphone 170C through the human mouth, and input the sound signal into the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement noise reduction functions in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.
  • the earphone interface 170D is used to connect wired earphones.
  • the earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, and a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA, CTIA
  • the pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal.
  • the pressure sensor 180A may be provided on the display screen 194.
  • the capacitive pressure sensor may include at least two parallel plates with conductive materials.
  • the electronic device 100 determines the intensity of the pressure according to the change in capacitance.
  • the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations that act on the same touch position but have different touch operation strengths may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.
  • the gyro sensor 180B may be used to determine the movement posture of the electronic device 100.
  • the angular velocity of the electronic device 100 around three axes ie, x, y, and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the gyro sensor 180B detects the shake angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse movement to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the air pressure sensor 180C is used to measure air pressure.
  • the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster.
  • the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D.
  • features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and apply to applications such as horizontal and vertical screen switching, pedometers, and so on.
  • the electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F to measure the distance to achieve fast focusing.
  • the proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.
  • the light emitting diode may be an infrared light emitting diode.
  • the electronic device 100 emits infrared light to the outside through the light emitting diode.
  • the electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, and the pocket mode will automatically unlock and lock the screen.
  • the ambient light sensor 180L is used to sense the brightness of the ambient light.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.
  • the temperature sensor 180J is used to detect temperature.
  • the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection.
  • the electronic device 100 when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature.
  • the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K may be disposed on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”.
  • the touch sensor 180K is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • the visual output related to the touch operation can be provided through the display screen 194.
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.
  • the bone conduction sensor 180M can acquire vibration signals.
  • the bone conduction sensor 180M can obtain the vibration signal of the vibrating bone mass of the human voice.
  • the bone conduction sensor 180M can also contact the human pulse and receive the blood pressure pulse signal.
  • the bone conduction sensor 180M may also be provided in the earphone, combined with the bone conduction earphone.
  • the audio module 170 can parse the voice signal based on the vibration signal of the vibrating bone block of the voice obtained by the bone conduction sensor 180M, and realize the voice function.
  • the application processor can analyze the heart rate information based on the blood pressure beating signal obtained by the bone conduction sensor 180M, and realize the heart rate detection function.
  • the button 190 includes a power-on button, a volume button, and so on.
  • the button 190 may be a mechanical button. It can also be a touch button.
  • the electronic device 100 may receive key input, and generate key signal input related to user settings and function control of the electronic device 100.
  • the motor 191 can generate vibration prompts.
  • the motor 191 can be used for incoming call vibration notification, and can also be used for touch vibration feedback.
  • touch operations applied to different applications can correspond to different vibration feedback effects.
  • Acting on touch operations in different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects.
  • Different application scenarios for example: time reminding, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.
  • the SIM card interface 195 is used to connect to the SIM card.
  • the SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100.
  • the electronic device 100 may support 1 or N SIM card interfaces, and N is a positive integer greater than 1.
  • the SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.
  • the same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards can be the same or different.
  • the SIM card interface 195 can also be compatible with different types of SIM cards.
  • the SIM card interface 195 may also be compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication.
  • the electronic device 100 adopts an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.
  • the camera 193 collects an image, and the ISP processes the data fed back by the camera 193, for example, to determine the target subject in the image.
  • the processor 110 or the NPU in the processor 110 may perform image segmentation on the image processed by the ISP, and determine the regions where different objects or different object types are respectively located on the image.
  • the processor 110 may track the determined target subject in the preview stream.
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiment of the present application takes an Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 by way of example.
  • FIG. 2 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface.
  • the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, and a notification manager.
  • the window manager is used to manage window programs.
  • the window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, take a screenshot, etc.
  • the content provider is used to store and retrieve data and make these data accessible to applications.
  • the data may include videos, images, audios, phone calls made and received, browsing history and bookmarks, phone book, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, and so on.
  • the view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface that includes a short message notification icon may include a view that displays text and a view that displays pictures.
  • the phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call status (including connecting, hanging up, etc.).
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and it can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, and so on.
  • the notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or a scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, text messages are prompted in the status bar, prompt sounds, electronic devices vibrate, and indicator lights flash.
  • Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.
  • the application layer and application framework layer run in a virtual machine.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides a combination of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.
  • the 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • zooming methods mainly include: optical zoom (optical zoom) and digital zoom (digital zoom).
  • optical zoom optical zoom
  • digital zoom digital zoom
  • the optical zoom is produced by changes in the positions of the lens, angle of view, and focus. That is, the object to be photographed is enlarged or reduced by the movement of the lens. The larger the optical zoom, the farther away the target subject can be from the lens.
  • the digital zoom uses a processor in a terminal (such as a mobile phone and a digital camera) to increase the area of each pixel in the collected picture to achieve the purpose of magnification.
  • the current optical zoom lenses that can be used on mobile phones have limited zoom capabilities.
  • optical zoom technology combined with digital zoom technology is still required for shooting.
  • the digital zoom technology refers to the technology of cropping the central area of the image captured by the optical zoom lens to the user-specified magnification according to the shooting magnification specified by the user.
  • Fig. 3 is a schematic flowchart of an image processing method in the prior art.
  • a user interface including a preview display area
  • the preview display area can be used to display the preview stream, where the preview stream includes Preview images in one or more frames.
  • the user can indicate the selection of different shooting magnifications through the controls on the UI, for example, 3x zoom, 20 zoom, and so on.
  • the mobile phone can determine the target lens (that is, the lens used to collect the image of the area to be shot) according to the shooting magnification indicated by the user, specifically, the number of target lenses and the corresponding magnification .
  • a mobile phone has two lenses, the optical zoom magnification of the main lens is 1x (1x), and the optical zoom magnification of the telephoto lens is 5x (5x).
  • the mobile phone detects that the user has instructed to use a shooting magnification of 20 times, the mobile phone determines that it needs to use a 5x optical zoom magnification telephoto lens (that is, the target lens) to collect the image.
  • the mobile phone shoots the scene to be photographed based on the target lens, and according to the display specification of the mobile phone, such as display size and display scale, the original image to be processed by the terminal is obtained after cropping.
  • the mobile phone can determine the field of view (FOV) desired by the user and the size of the area to be cropped according to the shooting magnification indicated by the user. From the center position of the original image to be processed by the terminal, according to the determined size of the area to be cropped, an image of a corresponding size is cropped, and then processed and enlarged to a corresponding preview resolution, and then displayed in the preview area.
  • FOV field of view
  • FIG. 4 and FIG. 5 to FIG. 6 respectively show the original image to be processed by the terminal and the first frame, the 50th frame, and the 100th frame image corresponding to the image displayed in the preview area.
  • the original image to be processed by the terminal refers to an image that is cropped to adapt to the display specifications of the mobile phone after the mobile phone shoots the scene to be photographed based on the target lens.
  • the mobile phone uses a lens with a 1x optical zoom magnification as the target lens.
  • the mobile phone cuts out 1/100 of the image from the center of the original image to be processed by the terminal, and zooms in at a ratio of 1:100, and displays the corresponding image in the preview area.
  • the original image 401 to be processed by the terminal includes an image 402a of the target subject.
  • the mobile phone determines the image size of the area 403 to be cropped according to the shooting magnification adopted by the user. Take the center of the original image 401 to be processed by the terminal as the center, and crop an image of a corresponding size according to the image size of the region 403 to be cropped. Finally, after image processing and magnification to the corresponding preview resolution, the image 404 is displayed through the preview area, and the image 404 displays the preview image 402b of the target subject.
  • the original image 501 to be processed by the terminal includes an image 502a of the target subject.
  • the area to be cropped 503 only covers part of the image 502a of the target subject. Therefore, the image 504 displayed in the preview area only displays the preview image 502b of a part of the target subject.
  • the original image 601 to be processed by the terminal includes an image 602a of the target subject. However, at this time, there is a distance between the area to be cropped 603 and the image 602a of the target subject, and the image 604 displayed in the preview area does not display the preview image of the target subject.
  • an embodiment of the present application proposes an image processing solution, including an image processing method and device.
  • an image processing solution including an image processing method and device.
  • FIG. 7 is a graphical user interface (graphical user interface, GUI) of a mobile phone provided by an embodiment of the application, and the GUI is a desktop 701 of the mobile phone.
  • GUI graphical user interface
  • the mobile phone detects that the user has clicked on the icon 702 of the camera application (application, APP) on the desktop 701, the camera application can be started, and another GUI as shown in (b) in Figure 5 is displayed.
  • This GUI can be called Shooting interface 703.
  • the shooting interface 703 may include a viewing frame 704.
  • a preview image can be displayed in the viewfinder frame 704 in real time, where the viewfinder frame 704 and its displayed content can become a preview area.
  • the size of the viewfinder frame 704 may be different in the photographing mode and the video recording mode (ie, the video shooting mode).
  • the finder frame shown in (b) in FIG. 7 may be the finder frame in the photographing mode.
  • the viewfinder frame 704 can be the entire touch screen.
  • the viewfinder frame 704 may display an image.
  • the shooting interface may also include a control 705 for indicating a shooting mode, a control 706 for indicating a video recording mode, and a shooting control 707.
  • the camera mode when the mobile phone detects that the user clicks the shooting control 707, the mobile phone executes the camera operation; in the video mode, when the mobile phone detects the user clicks the shooting control 707, the mobile phone executes the video shooting operation.
  • a still picture or a dynamic picture live photo
  • FIG. 8 shows another GUI of the mobile phone, and the GUI is the interface 801 in the mode of shooting still pictures.
  • the photographing interface for the still picture photographing mode may further include a control 802 for instructing to photograph a dynamic picture.
  • the mobile phone detects that the user clicks on the control 802, it switches from the still picture shooting mode to the dynamic picture shooting mode, and another GUI as shown in FIG. 8(b) is displayed.
  • the GUI is the interface 803 for the dynamic picture shooting mode.
  • the photographing interface for the dynamic picture photographing mode may further include a control 804 for instructing to photograph a still picture.
  • control 802 and the control 804 may be the same icon, and are distinguished by color highlighting.
  • control 802 and the control 804 may be the same icon and are distinguished by different types of lines, for example, a solid line and a dashed line, or a thick line and a thin line.
  • the GUI for entering the shooting dynamic picture mode has a variety of optional designs.
  • the shooting interface 901 also includes instructions for displaying other more modes. Control 902.
  • the mobile phone detects that the user selects the shooting control 902, for example, the user clicks the shooting control 902, or the mobile phone detects that the user slides the shooting control 902 to the center of the GUI, or the mobile phone detects that the user slides the shooting control 902 above the shooting key.
  • the GUI shown in (b) in Fig. 9 is displayed.
  • the GUI is an interface 903, and a variety of controls for indicating a specific shooting mode are displayed in the interface 903, including a control 904 for indicating a mode of shooting a dynamic picture.
  • the shooting interface 901 is displayed, and the dynamic picture shooting mode is entered.
  • image processing method provided in the embodiments of the present application can be applied to the shooting and processing scenes of still pictures, dynamic pictures, and videos, which is not limited in this application.
  • FIG. 10 is a schematic flowchart of an image processing method provided by an embodiment of the application. It should be understood that the image processing method may be executed by a computing device or an electronic device (for example, a terminal), and may also be executed by a device in the electronic device (for example, an image processing device). Among them, the image processing device may be an ISP or a system on chip (system on chip, SoC). Here, take terminal execution as an example to expand the introduction. As shown in FIG. 10, the image processing method 1000 includes:
  • S1001 The terminal acquires a first preview image of a scene to be shot, where the first preview image corresponds to a first shooting magnification.
  • the first shooting magnification may be the default shooting magnification (for example, 1x shooting magnification), or may be the shooting magnification used by the terminal again designated by the user during the shooting process.
  • FIG. 11A is a schematic structural diagram of a preview image processing algorithm provided by an embodiment of this application.
  • the preview image processing algorithm 1100 includes: a display specification processing module 1101 adapted to the terminal, a cropping module 1102, an enlargement module 1103, and a preview image processing module 1104. It should be understood that the order of the above-mentioned modules can be adjusted, and the preview image processing module 1104 can also be divided into a plurality of identical or different preview image processing modules to process images during the entire preview image processing process.
  • the terminal determines the first shooting magnification (such as 10 times), a lens with a 1x optical zoom magnification needs to be used for shooting.
  • the first original image is collected by the terminal's sensor.
  • the first original image is input into the preview image processing algorithm as shown in FIG. 11A to obtain the first preview image.
  • the first preview image is cropped according to the display specifications of the middle terminal to obtain the original image to be processed by the terminal.
  • the terminal may determine the size of the area to be cropped in the image to be cropped and the to-be-enlarged magnification of the area to be cropped according to the first shooting magnification.
  • the first preview image is obtained after preview image processing.
  • the image to be cropped may be the first original image, or the original image to be processed by the terminal, or may be an image obtained by subjecting the foregoing image to preview image processing or partial preview image processing (such as first preview image processing).
  • S1002 The terminal determines the target subject in the first preview image.
  • the target subject in the embodiments of the present application may be one object, or multiple objects, and may be all objects in a region (that is, the target subject is the target region), which is not limited here.
  • the target area may be a regular pattern, for example, a circle, a rectangle, a square, etc., or an irregular pattern.
  • the current target subject in the first preview image is determined according to the target subject in the preview image one or more frames before the first preview image in the preview stream.
  • the preview stream will be displayed on the display screen of the terminal, where the preview stream includes one frame of preview images or multiple frames of preview images.
  • the position of the target subject in the preview stream of adjacent or close frames of images will not have a big sudden change.
  • the terminal may predict the position of the target subject in the first preview image according to the position of the target subject in the preview image one or more frames before the first preview image in the preview stream.
  • the above-mentioned multi-frame preview image may be continuous multiple frames, or may be multiple frames at periodic intervals or multiple frames at non-periodical intervals.
  • the terminal detects that there is an object corresponding to the system preset object category in the first preview image, it is determined that the object is the target subject.
  • the terminal stores objects preset by the system, and the specific storage content may be object category tags.
  • the terminal performs image recognition on the object in the first preview image in the preview stream, obtains the object category label in the first preview image, and compares it with the object category label preset by the system.
  • the terminal performs image segmentation on the first preview image, and matches the segmented image with an image corresponding to the object category label preset by the system, where the image corresponding to the object category label preset by the system can be obtained by the cloud server , Or, obtained from local storage.
  • the cloud server Or, obtained from local storage.
  • the user may be prompted on the user interaction interface to select and confirm the target subject, and the object indicated by the user is determined as the target subject.
  • the user may be prompted on the user interaction interface to confirm the target subject, and the object indicated by the user is determined to be the target subject.
  • the specific storage content of the object preset by the system may also be an object image.
  • the terminal can perform image segmentation on the first preview image to directly match the segmented image with the object image preset by the system.
  • the object category list preset by the system may be that by learning the user's shooting habits, objects with higher shooting probability of the user are added to the object category list preset by the system. For example, according to the current user of the terminal, and/or a large number of images taken or browsed by other users before, by identifying and labeling the types of objects in the above-mentioned images, the types of objects photographed by the user and the corresponding shooting probabilities are obtained. The object categories whose shooting probability is greater than the system preset threshold are classified into the system preset object category list.
  • the object category list preset by the above system can be preset before leaving the factory, or can be set when the terminal system is upgraded, or it can be stored in a cloud server, when the terminal executes the image processing method (for example, image processing method 1000) When, read the list of object categories preset by the system from the cloud server.
  • the terminal may select the object preset by the system first detected in the first preview image. , Determined as the target subject.
  • the object preset by the system is set with a priority, and the terminal may determine the object preset by the system with the highest priority detected in the first preview image as the target subject.
  • the terminal may identify the most frequently detected object in the first preview image and corresponding to the object type preset by the system as the target subject.
  • algorithms such as saliency area detection can be used to analyze the first preview image to automatically determine the target subject that the user wants to shoot.
  • algorithms for saliency region detection can be used to extract image features and detect special regions with image features that are different from other regions. It is also possible to train a neural network, analyze the preview image, and infer the target subject that the user wants to shoot.
  • the terminal By presetting the object category list of the target subject by the system, the terminal directly recognizes the target subject in the first preview image, reducing user interaction, high processing efficiency, and convenient user operation.
  • the first object in response to an operation on the first object, it is determined that the first object is the target subject. That is: the user directly instructs a specific object as the target subject.
  • the user can indicate that the object is the target subject through a gesture of clicking or double-tapping or long-pressing the first object.
  • the user can indicate that the first object is the target subject by long-pressing the first object in a gesture action combined with a voice command.
  • the aforementioned gesture action may be a finger touch or a knuckle touch, a single finger operation or a multi-finger operation, and the specific form may be selected according to the actual situation of the terminal.
  • the terminal may detect the aforementioned gesture action through a sensor on the touch screen, and when the user's gesture action on the first object is detected, in response to the user's operation on the first object, it is determined that the first object is the target subject.
  • the user interaction perception is improved, and the terminal can also more accurately identify the target subject that the user needs to photograph, thereby improving the user experience.
  • the user can draw a specific area through gestures, and after the terminal detects the gestures on the touch screen, the terminal determines the objects in the specific area, and when there are objects in the specific area that correspond to the object category preset by the system, Make sure that the object is the target subject.
  • the user may designate a point in the first preview image, and determine the first area with this point as the center. The size of the first area may be determined according to the shooting magnification indicated by the user. In this way, the terminal does not need to perform object detection and image segmentation, which reduces the difficulty of the algorithm.
  • the object in the first area is the target subject.
  • the above-mentioned operations on the first area may be one or more of long-press, single-click, double-click, knuckle click, and knuckle delimiting the area.
  • the confirmed target subject may be marked on the shooting interface.
  • it can be identified by encircling or marking the confirmed target subject through shape marks such as focus frame or label frame, or symbol marks such as cross mark or Pzi sign.
  • shape of the mark such as the focus frame or the label frame may include, but is not limited to, a square, an ellipse, and the like. A simple square or ellipse is convenient for implementation, and requires low recognition accuracy for the confirmed target subject image.
  • Symbol marks such as a cross mark or a rice mark may be located in the central area of the target subject image, wherein the size of the mark may be adjusted or changed according to the size of the target image or the size of the shooting magnification.
  • it is more accurate to indicate the confirmed target subject by displaying the outline of the confirmed target subject.
  • the above-mentioned mark can move as the position of the target subject in the preview image moves.
  • the user interaction perception can be improved, and the accuracy of the terminal for recognizing the target subject can be improved, and the user experience can be improved.
  • the terminal acquires a second original image of the scene to be photographed, where the second original image is an image collected by the image sensor of the terminal when the terminal photographs the scene to be photographed based on a target lens, and the target lens is the terminal A lens called when shooting the scene to be shot at a second shooting magnification, where the second shooting magnification is greater than the first shooting magnification.
  • the second shooting magnification is the shooting magnification adopted by the terminal designated by the user. After the terminal determines the second shooting magnification, the target lens can be determined to obtain the second original image. After the second original image is tailored to the display specification of the terminal, the original image to be processed by the terminal is obtained. If the second shooting magnification is greater than the optical zoom magnification of the target lens, it is also necessary to combine the digital zoom technology to process the original image to be processed by the terminal.
  • S1004 The terminal determines a second preview image according to the target subject and the second original image, where the second preview image includes the target subject.
  • the second preview image is determined according to the second shooting magnification and the second original image, and the second preview image includes the target subject.
  • it can also be understood as: input the second original image into the preview image processing algorithm as shown in FIG. 11A to obtain the second preview image.
  • the specific implementation and calculation parameters of each module (such as the cropping module 1102) corresponding to the second preview image are not completely consistent with the specific implementation and calculation parameters of each module corresponding to the first preview image.
  • the terminal can determine the target lens, the FOV desired by the user, the digital zoom magnification to be adopted, and the size of the area to be cropped according to the second shooting magnification.
  • the position of the target subject in the second original image may be taken as the center, and the second intermediate image may be obtained by cropping based on the determined size of the area to be cropped, and then by zooming in to the preview resolution, the second preview is obtained after the preview image is processed image.
  • the order of the various modules in FIG. 11A can be adjusted, and the preview image processing module 1104 can also be divided into a plurality of identical or different preview image processing modules to process the image during the entire preview image processing process.
  • the preview image processing module 1104 may include one or more algorithmic processes, for example, denoising, equalizing, and improving contrast, etc., which may be executed separately or together. For example, you can first perform budget image processing such as denoising, and then perform preview image processing such as cropping, zooming, and then equalizing and improving contrast. This application does not limit this.
  • the updated preview stream (including the second preview image) can remain stable. But in this way, it will be difficult for the user to adjust the composition by changing the position of the phone, and the target subject will always remain in the center of the preview stream.
  • the terminal may determine the size of the area to be cropped and the position of the center point of the area to be cropped by using the position of the target subject in the image to be cropped as a reference point in the cropping process, and in combination with the second shooting magnification.
  • the second preview image makes the target subject not be fixed in the middle position of the preview image, so as to avoid the unity of composition.
  • FIG. 11B is a schematic diagram of an image cropping method provided by an embodiment of the application. See FIG. 11B. Is w and the height is h.
  • O is the center position of the image
  • T is the tracked target subject position
  • the center point of the area to be cropped is denoted as P
  • the width and height of the area to be cropped are denoted as w c and h c respectively
  • the zoom magnification used by the terminal instructed by the user Is r the coordinate of the center point P (x p , y p ) of the area to be cropped and the size of the area to be cropped satisfy the following formula:
  • the position of the target subject in the preview image can also be adjusted based on the user's instruction.
  • a first operation on the target subject in the second preview image is received, and in response to the first operation, the target position of the target subject in the third preview image is determined, and the third image of the scene to be shot is acquired.
  • An original image, the third original image is an image collected by a sensor of the terminal when the terminal shoots a scene to be photographed based on a target lens, and the collection time of the second original image is earlier than the third original image.
  • the first operation may be a gesture, voice, facial, or other type of operation.
  • the terminal detects and receives the user's two-finger dragging the target subject in the second preview image through the touch screen, and determines the target subject according to the direction and distance of the user's two-finger dragging and the position of the target subject in the second preview image.
  • the target position of the target subject in the third preview image wherein, the target position changes with the position of the target subject in the second preview image, the direction is the same as the dragging direction of the user's two fingers, and the distance is proportional to the dragging distance of the user's two fingers.
  • the size of the ratio can be determined by the user
  • the system setting interface is set by precision granularity, or it can be set according to the usage habits of the user or a large number of users.
  • the second original image and the third original image are different images collected by the terminal's sensor when the same target lens is used to shoot the scene to be shot.
  • the collection time corresponding to the second original image and the third original image is not complete Consistent, the time interval is short.
  • the terminal determines the size of the area to be cropped and the magnification to be magnified according to the second shooting magnification, and according to the determined target position of the target subject in the third preview image, as well as the size of the area to be cropped and the magnification to be Based on the enlarged ratio, the third preview image is determined based on the third original image.
  • the above-mentioned image to be cropped may be a third original image or an intermediate image obtained based on the third original image.
  • the above solution can adjust the position of the target subject in the preview image according to the user's instruction operation on the target subject, meet the user's diversified composition requirements, and avoid the unity caused by the fixed position of the target subject in the preview image.
  • S1005 Display the second preview image on the display screen of the terminal.
  • FIG. 12 is another GUI of a mobile phone provided by an embodiment of the application.
  • the shooting interface 1201 displayed on the display screen of the terminal includes only a preview area 1202, and the preview area 1202 is used to display the first 2.
  • Preview the image to assist the user in shooting if you hold the phone directly to shoot, if the hand shake causes the lens to shake significantly, the target subject may move out of the current preview area. At this time, the terminal can re-determine the target subject or prompt the user to re-determine the target subject.
  • At least two preview areas are displayed on the display screen of the terminal, for example, a first preview area and a second preview area are displayed, where the first preview area is used to display the auxiliary preview image, and the second preview area is used to display The second preview image, and the auxiliary preview image includes content in the second preview image.
  • the FOV of the auxiliary preview image should be greater than or equal to the FOV of the second preview image, and the auxiliary preview image is determined based on the second original image or an intermediate image obtained based on the second original image.
  • the range of the area to be cropped corresponding to the auxiliary preview image should be larger than the range of the area to be cropped corresponding to the second original image.
  • the auxiliary preview image may be an original image to be processed by the terminal after the second original image is cropped to adapt to the display specification of the terminal, and obtained through preview image processing.
  • the auxiliary preview image may be the original image to be processed by the terminal. According to the position of the target subject in the second original image, after cropping out an area of 1/n 2 size in the second original image, it is enlarged by n 2 times and passed through Preview the image obtained after image processing, where n is a positive integer, and n is less than the ratio of the second shooting magnification to the optical zoom magnification of the target lens corresponding to the second original image.
  • the auxiliary preview image may be determined based on the auxiliary original image, where the auxiliary original image and the second original image may be different images collected by the terminal's image sensor when the scene to be photographed by different lenses of the terminal, and the auxiliary preview image
  • the target subject can be identified by means of identification boxes or symbols, etc., and the specific identification method can refer to the relevant description in S1002.
  • the position and size of the first preview area can be adjusted according to the user's instructions.
  • the user's first operation on the first preview area is received, such as gesture operations such as dragging or voice.
  • the position of the first preview area can be changed from the original position to the position indicated by the user, or the user can receive a second operation on the first preview area, such as two fingers
  • the size of the first preview area can be reduced or enlarged according to the proportion of gesture operations such as sliding zooming or other types of operations such as voice commands.
  • the user can adjust the position of the shooting interface displayed in the multiple preview area of the first preview area by setting the default format (eg, size, position) of the first preview area.
  • FIG. 13 is another GUI of a mobile phone provided by an embodiment of this application.
  • the shooting interface 1301 includes a first preview area 1302 and a second preview area 1303.
  • the first preview area 1302 may be used to display an auxiliary preview image
  • the second preview area 1303 may be used to display a second preview image or a newly determined second preview image.
  • the position, size, shape, and area of the first preview area 1302 and the second preview area 1303 can be fixed or changed, and it is not limited here, and can be designed according to the actual situation of the terminal.
  • the positions of the first preview area 1302 and the second preview area 1303 can be exchanged according to the user's instruction.
  • the terminal will exchange the content displayed in the two preview areas after responding to the above gesture.
  • the first preview area 1302 is used to display the second preview image or The re-determined second preview image and other preview images in the preview stream
  • the second preview area 1303 is used to display the auxiliary preview stream.
  • the auxiliary preview image displayed in the first preview area 1302 is determined based on the auxiliary original image, where the auxiliary original image is an image collected by the terminal's sensor when the terminal calls the lens 1 to shoot the scene to be photographed.
  • the second preview image displayed in the second preview area 1303 is determined based on the second original image, where the second original image is an image collected by the terminal's sensor when the terminal calls the target lens (lens 2) to shoot the scene to be photographed.
  • the auxiliary original image and the second original image are the images collected by the sensor of the terminal when the terminal calls different lenses to shoot the scene to be photographed at the same time, and the corresponding FOVs are different.
  • the optical zoom magnification of the lens 1 is less than or equal to the optical zoom magnification of the lens 2.
  • the lens 1 is a lens with a 3x optical zoom magnification
  • the lens 2 is a lens with a 5x optical zoom magnification
  • the second shooting magnification ie, the shooting magnification specified by the user
  • the second original image is cropped to adapt to the display specifications of the terminal to obtain the original image to be processed by the terminal, and according to the position of the target subject in the second original image, an area of 1/9 size is cropped out of the original image to be processed by the terminal.
  • the second preview image is obtained after 9 times magnification and preview image processing.
  • the auxiliary preview image may be an image obtained after the auxiliary original image is cropped to adapt to the display specification of the terminal, or the auxiliary preview image may be an image obtained after the auxiliary original image is cropped to adapt to the display specification of the terminal, according to the target subject in the auxiliary original image After cropping out an area of 1/m 2 size, zoom in m 2 times and get the image after preview image processing.
  • m should be a positive integer less than 5, so that the FOV of the auxiliary preview image is greater than the second preview image.
  • the user can re-designate the target subject in the auxiliary preview image
  • the method of re-designating the target subject can be a gesture instruction, an air gesture instruction, a voice instruction, a facial instruction (for example, the direction of eyeball movement), and the like.
  • the user can simultaneously perceive the position of the target subject in the current second preview image and the position of the target subject on the original screen or auxiliary preview screen.
  • the user can move the phone based on the auxiliary preview image to obtain different fields of view. It can also reduce the probability of the target subject being drawn.
  • the image processing method 1000 may further include S1006,
  • S1006 The terminal saves the target image in response to the user clicking the shooting control.
  • the target image may be one frame of image, or multiple frames of images (such as video images, dynamic images, etc.). It should be understood that the FOV of the target image and the second preview image are the same.
  • the terminal may start the shooting processing algorithm.
  • the shooting processing algorithm module may input one frame of image or multiple frames of images (for example, multiple frames of continuous images).
  • the shooting processing algorithm and the preview processing algorithm may be different.
  • the specific implementation process including but not limited to the following examples:
  • the image input by the shooting processing algorithm module is: the image (such as the second original image) collected by the terminal's sensor when the terminal calls the target lens to shoot the area to be photographed, and/or the terminal calls the target lens to shoot the waiting area.
  • the image input by the photographing processing algorithm module is: the second original image, and/or, the second original image is adapted to the image obtained after cropping of the display specification of the terminal.
  • the image input by the photographing processing algorithm module is: an image in the preview stream (for example, a second preview image), where the image in the preview stream may be the whole of one or more frames of original images to be processed by the terminal, It can also be a cropped part of one or more frames of the original image to be processed by the terminal.
  • the image is cropped according to the display specifications and shooting magnification.
  • an image obtained by cropping according to the position of the target subject, the display specification, and the shooting magnification is obtained by cropping according to the position of the target subject, the display specification, and the shooting magnification.
  • the aforementioned shooting processing algorithms include, but are not limited to, one or more of the following processing: auto-exposure (AE), auto-focus (AF), auto-white balance (AWB), Dynamic range adjustment (DRC), lens shading correction (LSC), distortion correction, etc.
  • FIG. 14 is a schematic diagram of a display of a shooting target subject in a high-magnification zoom scene provided by an embodiment of the application.
  • the 50th frame of the preview image 1401 to be processed by the terminal includes a target subject 1402a, but the target subject 1402a is not in the center of the preview image 1401.
  • the center position of the region 1403 to be cropped is determined according to the position of the target subject 1402a in the preview image 1401, and the size of the region to be cropped 1403 of the first image is determined according to the position and the zoom factor of the current shooting.
  • the image in the region to be cropped 1403 is cropped to obtain the second image, and the target image can be obtained by performing image processing on the second image and zooming in to the corresponding preview resolution.
  • the target image in the image 1404 displayed in the preview area we can display the target image in the image 1404 displayed in the preview area, and the target image 1402b can be located at the center of the image 1404 displayed in the preview area.
  • the current preview stream is cropped according to the position of the target subject in the current preview image, a stable preview stream can be obtained, and the shaking of the preview image caused by hand shaking, even the target subject is avoided.
  • a stable preview stream can be obtained, and the shaking of the preview image caused by hand shaking, even the target subject is avoided.
  • FIG. 15 is a GUI for a user to adjust the shooting magnification provided by an embodiment of the application.
  • a GUI as shown in (a) in FIG. 15 is displayed, and the GUI may be referred to as a photographing interface 1501.
  • the user can slide up and down on the control 1502 with a finger to select the shooting magnification currently instructed by the mobile phone. After the user slides the control 1502 from bottom to top, another GUI as shown in (b) of FIG.
  • FIG. 15 is displayed, and the shooting magnification used by the mobile phone instructed by the user is changed from 1x (1x) to 40x (40x).
  • FIG. 16 is another GUI for a user to adjust the shooting magnification provided by an embodiment of the application.
  • the difference from the GUI shown in FIG. 15 is that the control 1502 for indicating the first shooting magnification in the shooting interface 1601 has a different shape from the control 1602 for indicating the first shooting magnification in the shooting interface 1501.
  • the user can slide left and right on the shooting control 1602 to adjust the shooting magnification currently adopted by the mobile phone.
  • the target lens determined by the terminal may be different.
  • the terminal tracks the change of the second shooting magnification (that is, the shooting magnification specified by the user), and when the terminal determines that the second shooting magnification is greater than a preset threshold, it triggers the mode of enabling the display of multiple preview areas.
  • the terminal is provided with a specific shooting mode for triggering the display of multiple preview areas, and when the user chooses to enter the specific shooting mode, the display of the multiple preview areas is triggered.
  • the shooting interface 1701 also includes a control 1702 for instructing to display other more modes.
  • the mobile phone detects that the user selects the shooting control 1702, for example, the user clicks the shooting control 1702, or the mobile phone detects that the user slides the shooting control 1702 to the center of the GUI, or the mobile phone detects that the user slides the shooting control 1702 above the shooting key.
  • the GUI shown in (b) in Fig. 17 is displayed.
  • the GUI is an interface 1703, and a variety of controls for indicating a specific shooting mode are displayed in the interface 1703, including a control 1704 for instructing to trigger a specific shooting mode for enabling the display of multiple preview areas, and the control 1704 is referred to herein as an auxiliary display.
  • the shooting interface 1701 is displayed, and the specific shooting mode that triggers the display of multiple preview areas is entered.
  • the specific shooting mode may be referred to as an auxiliary display mode, or a high-magnification zoom shooting mode, or an anti-shake mode, etc., which is not limited here.
  • the terminal after detecting that the user has performed a predefined action on the target object, the terminal triggers the display of multiple preview areas.
  • the predefined action may refer to actions such as joint pressing, or double-clicking, or single-clicking, or encircling the outer contour.
  • triggering the display of the multiple preview area may be a direct trigger, which directly jumps to the GUI displayed in the multiple preview area.
  • triggering the display of multiple preview areas may also be an indirect trigger.
  • indication information may be displayed on the display screen of the terminal to prompt the user to select whether to enter the multiple preview area display. After the user chooses to enter the multiple preview area display, then jump to the GUI displayed in the multiple preview area.
  • the user-specified target subject can be before entering the specific mode, or after entering the specific mode, that is, after the auxiliary preview stream has been displayed.
  • the GUI display mode of auxiliary preview stream is not used, that is, when the GUI has only one preview area, the user needs to specify the target subject before entering a specific mode, or the target subject needs to be specified on the screen corresponding to the low and medium magnification digital zoom magnification.
  • the target subject cannot be found in the current preview stream.
  • FIG. 18 is a schematic flowchart of another image processing method provided by an embodiment of the application. It should be understood that the image processing method may be executed by a computing device or an electronic device (for example, a terminal), and may also be executed by a device in the electronic device (for example, an image processing device). Among them, the image processing device can be an ISP or SoC. Here, take terminal execution as an example to expand the introduction. As shown in FIG. 18, the image processing method 1800 includes:
  • S1801 The terminal acquires a first preview image of a scene to be shot, where the first preview image corresponds to a first shooting magnification.
  • S1802 The terminal determines the target area in the first preview image.
  • the target area can be a regular pattern, such as a circle, a rectangle, a square, and so on. It can also be irregular graphics.
  • the first area is determined to be the target area. That is: the user directly designates a specific area as the target area.
  • the above-mentioned operations on the first area may be actions such as long press, single click, double click, knuckle click, and knuckle delimiting the area.
  • FIG. 19 is a GUI for taking pictures provided by an embodiment of the application.
  • the current preview image is displayed on the photographing interface 1901.
  • the area 1902 is an irregular shape.
  • the area 1902 is marked by a marking frame 1904 in the shooting interface 1903, and the shape of the marking frame 1904 is or close to the outer contour of the area 1902.
  • a prompt message may be displayed on the shooting interface 1903 to prompt the user to confirm whether the area encircled in the mark frame 1904 is the target area that the user currently needs to shoot.
  • the second area including the first area is the target area. That is: according to the specific area specified by the user, the area matching the regular shape is the target area.
  • FIG. 20 is another GUI for taking pictures provided by an embodiment of the application.
  • the current preview stream is displayed on the photographing interface 2001.
  • the area 2002 is an irregular shape.
  • the target area is marked by the marking frame 2004, where the target area includes the area 2002.
  • prompt information 2006 may also be displayed to prompt the user to confirm whether the area encircled in the mark frame 2004 is the target that the user currently needs to shoot area.
  • the aspect ratio of the target area may be determined according to the display specifications of the terminal.
  • there are multiple methods for displaying the prompt information and the user confirming that the user enters the anti-shake mode including but not limited to the following examples.
  • exit Control 2008. When the user clicks the exit control 2008, they can exit the anti-shake shooting mode and return to the original shooting mode.
  • the first area is determined to be the target area.
  • the terminal determining the target area in the first preview image may further include: determining the location of the target area.
  • the location of the target area may be the coordinates of the target area, such as the boundary coordinates of the target area, or the coordinates of the center position of the target area.
  • the terminal can track the location of the target area. Specifically, for example, the terminal may predict the position of the target area in subsequent frames in the preview stream based on the position of the target area in the i-th frame image in the preview stream, and/or the positions of several frames before and after the i-th frame image.
  • the terminal can use the image characteristics of the target area in the preview stream to express the characteristics of the target area, and then use the position of the target area in the i-th frame image and/or the positions of several frames before and after the i-th frame image as the starting point, in the preview stream Image feature matching is performed on similar positions in the subsequent frames, and the area with a higher matching degree is selected as the position of the target area in the subsequent frames.
  • the image features include but are not limited to one or more of the following features: color, shape, brightness, texture, corners, etc.
  • the terminal may analyze the image of the target area to determine whether the current target area has failed to track based on parameters such as confidence or similarity.
  • the user may be prompted on the display screen of the terminal to re-determine the target area.
  • an instruction message is displayed on the display screen of the terminal to prompt the user to re-determine the target area.
  • the center position may be marked on the display screen of the terminal, or it may stop at the last position in the preview stream before the target area disappears. Prompt by instruction information, or by the user to indicate whether the current target area needs to be corrected. If it is corrected, follow-up methods are executed according to the corrected target area.
  • the terminal acquires a second original image of the scene to be photographed, where the second original image is an image collected by the image sensor of the terminal when the terminal photographs the scene to be photographed based on a target lens, and the target lens is the terminal A lens called when shooting the scene to be shot at a second shooting magnification, where the second shooting magnification is greater than the first shooting magnification.
  • S1804 The terminal determines a second preview image according to the target area and the second original image, where the second preview image includes the target area.
  • S1805 Display the second preview image on the display screen of the terminal.
  • the image processing method 1800 may further include S1806,
  • S1806 The terminal saves the target image in response to the user clicking the shooting control.
  • the terminal recognizes the target area instead of the target subject, but it can also be understood as taking the entire target area as the target subject.
  • Example 1 After the terminal turns on the camera, the camera interface 1201 as shown in Figure 12 is displayed, where the camera interface 1201 includes a preview screen 1202, and the current terminal calls the default target lens (such as a 1x optical zoom lens) to take photos. Picture of the scene.
  • the user adjusts the shooting magnification indicated by the user, for example, from 1x to 20 times.
  • a prompt message is displayed on the display screen of the terminal. The prompt message is used for reminding
  • the terminal selects whether to enter the anti-shake shooting mode, and indicates the currently determined target subject through a mark (for example, a cross mark).
  • the user When the user is sure that the recognition of the target subject is correct, he can choose to enter the anti-shake shooting mode. Otherwise, the user can re-designate the target subject by long-pressing or knuckle drawing.
  • the second preview image obtained according to the method 1000 or the method 1800 is displayed in the shooting interface, and the shooting magnification can still be adjusted in the shooting interface on the shooting interface. If the user finds that the current target subject is not in the preview image of the shooting interface, he can swipe through gestures or tap the return control in the shooting interface to exit the currently displayed second preview image and return to the preview image with a lower digital zoom ratio. , Reassign the target subject.
  • the value setting of the preset threshold may be preset by the user, for example, the user may set it in the terminal system setting page.
  • the value setting of the preset threshold may also be determined according to an empirical value. For example, a user survey is conducted to determine a shooting magnification that is imperceptible to small hand shakes of the user. The value of this magnification is determined as the preset threshold.
  • the terminal saves the target image.
  • the target image is the same as the FOV of the second preview image corresponding to the instruction to take a picture, that is, it is an image obtained by cropping the same area based on the second original image to adapt to the display specifications of the terminal, but the target image is the same as the instruction to take a picture.
  • the corresponding image processing process can be different.
  • Example 2 After the terminal turns on the camera, the camera interface 1701 shown in Figure 17 is displayed. Among them, the camera interface 1701 includes a preview screen and a function option 1702. The current terminal calls the default target lens (such as 1x optical zoom). Lens) to take an image of the scene to be shot.
  • the user enters the shooting mode selection interface 1703 through the function option 1702, and enters the multi-preview as shown in Figure 13 by clicking on the auxiliary display mode (the auxiliary display mode can have other names, which are not limited by this application) Auxiliary display mode for area display, where the shooting interface 1301 includes a first preview area 1302 and a second preview area 1303.
  • the first preview area 1302 is used to display the auxiliary preview image
  • the second preview area 1303 is used to display the second preview image.
  • the user can determine the target subject in the auxiliary preview image. After the user determines the target subject, in the auxiliary preview stream, the target subject is identified by the identification box or the symbol mark. Specifically, when the user does not actively determine the target subject, the terminal may first recognize a target subject through the identification method mentioned in method 1000 as the default target subject identifier. During the shooting, the user can reassign the target subject.
  • the user can instruct to exit the auxiliary display mode through instructions, such as gesture instructions, voice instructions, facial instructions, and other types of instructions.
  • the terminal When the user instructs to take a photo, the terminal saves the target image.
  • the target image is the same as the FOV of the second preview image corresponding to the instruction to take a picture, that is, both are images obtained by cropping the same area based on the second original image to adapt to the display specifications of the terminal, but the target image is the same as the instruction
  • the corresponding image processing process of the second preview image corresponding to the photo may be different.
  • the saved target image may also include an auxiliary preview image, and/or an image obtained after cropping based on the second original image or the auxiliary original image adapted to the display specifications of the terminal, and an image obtained after photographing image processing.
  • whether to save the above-mentioned image can be specified by the user by setting optional options in the user setting interface.
  • the shooting interface is selected according to the user's instructions, such as gesture instructions (such as single-click, double-click, and the corresponding target images of the camera control are different), voice instructions, facial instructions, and other types of instructions.
  • Example 3 After the terminal turns on the camera, the camera interface 1201 shown in Figure 12 is displayed.
  • the camera interface 1201 includes a preview screen and a function option 1202.
  • the current terminal calls the default target lens (such as 1x optical zoom). Lens) to take an image of the scene to be shot.
  • the user adjusts the shooting magnification indicated by the user during use, for example, from 1x to 20 times.
  • the multiplier shown in FIG. 13 is displayed.
  • Preview area Wherein, the shooting interface 1301 includes a first preview area 1302 and a second preview area 1303.
  • the first preview area 1302 is used to display the auxiliary preview image
  • the second preview area 1303 is used to display the second preview image.
  • the user can determine the target subject in the auxiliary preview image. After the user determines the target subject, in the auxiliary preview stream, the target subject is identified by the identification box or the symbol mark. Specifically, when the user does not actively determine the target subject, the terminal may first recognize a target subject through the identification method mentioned in method 1000 as the default target subject identifier. During the shooting, the user can reassign the target subject. When the photographing magnification indicated by the user is lower than the second preset threshold, the photographing interface returns to the photographing interface 1201 shown in FIG. 12, wherein the preview image of the photographing interface 1201 is the second preview image.
  • the value setting of the first preset threshold and the second preset threshold may be preset by the user, for example, the user may set it on the terminal system setting page.
  • the value setting of the first preset threshold may also be determined according to an empirical value. For example, a user is investigated to determine a shooting magnification that is imperceptible to small hand shakes of the user, and the value of the magnification is determined as the first preset threshold. Investigate the user to determine how close the FOV size of the auxiliary preview image and the second preview image are displayed by the user, corresponding to the most intolerable shooting magnification, and determine the magnification as the second preset threshold.
  • FIG. 21 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the application.
  • the image processing apparatus may be a computing device or an electronic device (for example, a terminal), or may be a device in an electronic device (for example, an ISP or SoC).
  • the image processing method shown in FIG. 6 or FIG. 11 and the above-mentioned optional embodiments can be implemented.
  • the image processing apparatus 2100 includes: a transceiver unit 2101 and a processing unit 2102.
  • the transceiving unit 2101 is configured to execute any of the steps S1001 and S1003 in the above-mentioned image processing method 1000, S1801 and S1803 in the above-mentioned image processing method 1800, and any optional examples thereof.
  • the processing unit 2102 is configured to execute any one of steps S1002, S1004, S1005, and S1006 in the above-mentioned image processing method 1000, S1802, S1804, S1805, and S1806 in the above-mentioned image processing method 1800, and any optional examples thereof.
  • steps S1002, S1004, S1005, and S1006 in the above-mentioned image processing method 1000, S1802, S1804, S1805, and S1806 in the above-mentioned image processing method 1800, and any optional examples thereof.
  • the image processing apparatus in the embodiments of the present application can be implemented by software, for example, a computer program or instruction with the above-mentioned functions can be implemented, and the corresponding computer program or instruction can be stored in the internal memory of the terminal and read by the processor.
  • the corresponding computer program or instruction in the memory is taken to realize the above-mentioned functions.
  • the image processing apparatus in the embodiment of the present application may also be implemented by hardware.
  • the processing unit 2102 is a processor (such as an NPU, GPU, or a processor in a system chip), and the transceiver unit 2101 is a transceiver circuit or an interface circuit.
  • the image processing device in the embodiment of the present application may also be implemented by a combination of a processor and a software module.
  • the image processing apparatus may be a computing device or an electronic device (for example, a terminal), or a device in an electronic device (for example, an ISP or SoC) .
  • the image processing method shown in FIG. 6 or FIG. 11 and the above-mentioned optional embodiments can be implemented.
  • the image processing apparatus 2200 includes a processor 2201 and an interface circuit 2202 coupled with the processor 1001. It should be understood that although only one processor and one interface circuit are shown in FIG. 22.
  • the image processing apparatus 2200 may include other numbers of processors and interface circuits.
  • the interface circuit 2202 is used to communicate with other components of the terminal, such as a memory or other processors.
  • the processor 2201 is used for signal interaction with other components through the interface circuit 2202.
  • the interface circuit 2202 may be an input/output interface of the processor 2201.
  • the processor 2201 reads computer programs or instructions in the memory coupled to it through the interface circuit 2202, and decodes and executes these computer programs or instructions.
  • these computer programs or instructions may include the above-mentioned terminal function program, and may also include the above-mentioned function program of the image processing apparatus applied in the terminal.
  • the terminal or the image processing device in the terminal can realize the solution in the image processing method provided in the embodiment of the present application.
  • these terminal function programs are stored in a memory external to the image processing apparatus 2200.
  • the terminal function program is decoded and executed by the processor 2201, part or all of the content of the terminal function program is temporarily stored in the memory.
  • these terminal function programs are stored in the internal memory of the image processing apparatus 2200.
  • the image processing device 2200 may be set in the terminal of the embodiment of the present invention.
  • part of the content of these terminal function programs is stored in a memory outside the image processing apparatus 2200, and other parts of the content of these terminal function programs are stored in a memory inside the image processing apparatus 2200.
  • FIGS. 1 to 2 and 21 to 22 can be combined with each other, and the image processing devices shown in any one of FIGS. 1 to 2, and 21 to 22 and various optional implementations
  • the related design details of the examples can be referred to each other, and the image processing method shown in any one of FIG. 10 or FIG. 18 and related design details of each alternative embodiment may also be referred to. I will not repeat them here.
  • the image processing method and each optional embodiment shown in any one of FIG. 10 or FIG. 18, the image processing device shown in any one of FIGS. 1 to 2 and FIG. 21 to FIG. 22 and each optional embodiment are not only It can be used to process videos or images during shooting, and can also be used to process videos or images that have been shot. This application is not limited.
  • At least one refers to one or more, and “multiple” refers to two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A, only B, and both A and B , Where A and B can be singular or plural.
  • the character “/” generally indicates that the associated objects before and after are in an “or” relationship.
  • "The following at least one item (a)” or similar expressions refers to any combination of these items, including any combination of a single item (a) or a plurality of items (a).
  • At least one of a, b, or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c" ", where a, b, c can be single or multiple.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium, such as a floppy disk, a hard disk, and a magnetic tape; it may be an optical medium, such as a DVD, or a semiconductor medium, such as a solid state disk (SSD).
  • the memory refers to a device or circuit with data or information storage capability, and can provide instructions and data to the processor.
  • Memory includes read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), non-volatile random access memory (NVRAM), programmable read-only memory or electrically erasable and programmable Memory, registers, etc.
  • first and second are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined with “first” and “second” may explicitly or implicitly include one or more of these features. In the description of the present embodiment, unless otherwise specified, “plurality” means two or more.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Studio Devices (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请实施例公开了拍照领域中的一种图像处理的方法和装置，该方法包括：确定待拍摄场景中的目标主体，根据目标主体、用户指定的拍摄倍率以及终端采用目标镜头在该拍摄倍率拍摄时，终端的传感器采集得到的原始图像，确定出预览图像并在终端的显示屏上显示。其中，预览图像的显示方式可以是单预览区域显示，或者结合辅助预览图像形成多预览区域显示。上述方法可以用于拍照过程中，提高拍摄过程中，尤其是高倍率变焦拍摄场景中，图像的稳定性，降低手抖造成的目标主体出画概率。

Description

图像处理方法及装置
本申请要求于2020年03月20日提交中国国家知识产权局、申请号为202010202272.5、申请名称为“图像处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及拍照领域，尤其涉及一种图像处理方法及装置。
背景技术
拍照已经成为各种移动终端，如手机、平板电脑、智能眼镜、穿戴式设备等，最常用的功能之一。当用户需要拍摄位于远处的目标主体的特写时，例如，远处的鸟、花、屋檐等物体，往往需要采用高倍率变焦模式来拍摄。随着智能手机拍照功能的逐步提升，当下智能手机也可以采用高倍率变焦拍摄。但较于通过架设在三脚架上的智能手机来拍摄而言，用户直接通过手持智能手机利用高倍率变焦模式拍摄时，存在预览区域画面抖动的问题。并且由于高倍率变焦模式通常用于拍摄远距离物体，镜头的微小抖动也会带来预览区域的大幅度晃动。这使得用户难以构图，甚至会使所拍摄的目标主体移出画面，往往需要多次拍摄才可以获得比较理想的图片，用户体验不佳。
因此，针对高倍率变焦拍摄场景中的抖动问题，需要展开具体研究。
发明内容
本申请实施例涉及了一种图像处理方法及装置，可以应用于高倍率变焦拍摄等拍摄场景中，以解决手持终端抖动带来的预览界面大幅晃动，难以拍摄得到清晰的目标主体等问题，提高拍摄性能。
应理解，本申请实施例所提供的方案中，计算设备是指能够被抽象为计算机系统的设备。支持图像处理功能的计算设备，可称为图像处理设备或图像处理装置。图像处理装置可以是该计算设备的整机，也可以是该计算设备的部分器件，例如图像处理功能相关的芯片，如图像处理器(Image Signal Processor，ISP)或系统芯片。其中，其中，系统芯片也称为片上系统，或称为SoC芯片。具体地，图像处理装置可以是诸如智能手机这样的终端，也可以是能够被设置在终端中的系统芯片或图像处理器。在物理实现中，图像处理器可以集成在SoC芯片内部，也可以不与SoC芯片集成。
第一方面，提供了一种图像处理方法，该方法包括：获取待拍摄场景的第一预览图像，该第一预览图像对应于该第一拍摄倍率；在该第一预览图像中确定目标主体；获取该待拍摄场景中的第二原始图像，该第二原始图像为该终端基于目标镜头拍摄该待拍摄场景时，该终端的图像传感器所采集得到的图像，该目标镜头为该终端在第二拍摄倍率下拍摄该待拍摄场景时所调用的镜头，该第二拍摄倍率大于该第一拍摄倍率；根据该目标主体和该第二原始图像，确定出第二预览图像，该第二预览图像包括该目 标主体；在该终端的显示屏上显示该第二预览图像。
应理解，第一拍摄倍率可以为用户指定的拍摄倍率，也可以为终端开启相机后的默认拍摄倍率。在终端开启相机后，目标镜头可以是系统预设置的默认镜头，随着用户调整指定的拍摄倍率，终端可以根据用户指定的拍摄倍率，重新确定目标镜头。目标主体可以是一个物体，或多个物体，也可以是一片区域中的全部物体，也可以理解为目标区域。第二拍摄倍率为用户指定的拍摄倍率。
终端的预览流包括一帧或多帧预览图像，例如第二预览图像。终端可以根据第二原始图像前的一帧预览图像，或多帧预览图像中目标主体的位置，预测目标主体在第二原始图像中的位置。上述多帧预览图像可以是连续多帧预览图像，或者，周期或非周期间隔的多帧预览图像。通过跟踪目标主体在一帧或多帧预览图像(或原始图像)的位置，预测目标主体在下一帧或下几帧原始图像中的位置，进而确定出预览图像，降低目标主体在预览图像中抖动的概率，画面会更稳定。
在获取到第一预览图像后，可以在终端的显示屏上显示第一预览图像。
上述方案中，根据目标主体、以及终端在拍摄待拍摄场景时终端传感器所采集的原始图像，来确定出预览图像，可以根据目标主体的变化(例如，目标主体的位置变动，或，目标主体的重新确定)，适应性地改变当前预览图像，而非只根据终端在拍摄待拍摄场景时终端的传感器所采集的原始图像，确定预览图像。可以降低因终端抖动而导致的，目标主体移出预览画面的概率。
在一种可能的实施方式中，根据该目标主体和该第二原始图像，确定出第二预览图像，具体包括：根据该目标主体在该第二原始图像中的位置，该第二拍摄倍率和该第二原始图像，确定出第二预览图像。
其中，根据该目标主体在第二原始图像中的位置，确定出该目标主体在待裁剪图像中的位置。终端可以根据第二拍摄倍率和目标镜头的光学变焦倍率，确定出所需要的数码变焦倍率m，其中，m大于或等于1。例如，数码变焦倍率m为第二拍摄倍率与目标镜头的光学变焦倍率之比。进而，终端可以确定出待裁剪图像中的待裁剪区域的大小，并从待裁剪图像中得到第二预览图像。例如，以目标主体在待裁剪图像中的位置为中心，在待裁剪图像中裁剪出1/m 2大小后再放大m 2倍，得到第二预览图像。或者，以该目标主体在第二原始图像中的位置为参考点，根据第二拍摄倍率和第二原始图像，确定出待裁剪区域的位置，如待裁剪区域的中心位置，待裁剪区域的四角坐标等。
应理解，如上所述，终端可以根据第二拍摄倍率和目标镜头的光学变焦倍率，确定出所需要的数码变焦倍率m，进而确定出待裁剪图像中的待裁剪区域的大小。根据上述待裁剪区域的大小、目标主体的位置、第二拍摄倍率，确定出待裁剪区域的中心点的位置，进而确定出第二预览图像。
第二拍摄倍率为用户指定的拍摄倍率，或者，为终端拍摄该待拍摄场景时默认的拍摄倍率。
例如，以目标主体在待裁剪图像中的位置为参考点，在待裁剪图像中裁剪出1/m 2大小后再放大m 2倍，得到包括目标主体的第二预览图像，以使得目标主体并不固定在 预览图像的中心位置，增加构图的多样性。
其中，上述待裁剪图像可以是第二原始图像，或者，基于第二原始图像得到的中间图像，例如，第二原始图像适应于终端的显示规格裁剪后得到的图像，或者，还可以是第二原始图像经过预览图像处理，或部分预览图像处理(如第一预览图像处理)得到的图像，或者，还可以是第二原始图像经过预览图像处理或部分预览图像处理(如第一预览图像处理)，以及适应于终端的显示规格裁剪后得到的图像。另外，在上述处理过程中得到第二预览图像之前，还可以进行预览图像处理或部分预览图像处理(如第二预览图像处理)。第一预览图像处理和第二预览图像处理可以相同，可以不同。上述裁剪出1/m 2大小、放大m 2倍、预览图像处理或部分图像处理、适应于终端的显示规格等处理操作的顺序可以调整，本申请不做限定。一般来说，将第二原始图像适应于终端的显示规格裁剪后得到的图像作为待裁剪图像，以目标主体在待裁剪中的位置为中心或参考点，先裁剪出1/m 2大小后再放大m 2倍，再进行预览图像处理，得到第二预览图像的方式，可以降低终端处理上述图像所需的计算资源，提高处理速度。
在一种可能的实施方式中，在该终端的显示屏上显示该第二预览图像之后，该方法还包括：
接收对于该第二预览图像中的该目标主体的第一操作，该第一操作用于确定该目标主体在新的预览图像中的目标位置，根据该确定的该目标主体在新的预览图像中的目标位置，以及该第二原始图像，确定出更新后的第二预览图像，该更新后的第二预览图像包括该目标主体，在该终端的显示屏上显示该更新后的第二预览图像。
在一种可能的实施方式中，在该终端的显示屏上显示该第二预览图像之后，该方法还包括：
接收对于该第二预览图像中的该目标主体的第一操作，该第一操作用于确定该目标主体在新的预览图像中的目标位置，获取该待拍摄场景的第三原始图像，该第三原始图像为该终端基于目标镜头拍摄待拍摄场景时，该终端的传感器所采集到的图像，该第二原始图像的采集时间早于该第三原始图像，根据该目标主体在新的预览图像中的目标位置，以及该第三原始图像，确定出该第三预览图像，该第三预览图像包括该目标主体，在该终端的显示屏上显示该第三预览图像。
应理解，新的预览图像包括第三预览图像。第一操作可以是手势、语音、面部等类型的操作。例如，终端通过触摸屏检测并接收到用户双指拖拽第二预览图像中的目标主体，根据用户双指拖拽的方向和距离，以及该目标主体位于第二预览图像中的位置，确定出该目标主体在第三预览图像中的目标位置。其中，目标位置与该目标主体在第二预览图像中的位置变化，方向与用户双指拖拽的方向相同，距离与用户双指拖拽的距离成比例，其中，比例的大小可以由用户在系统设置界面通过精度粒度来设置，或者，还可以根据该用户或大量用户统计的使用习惯设置。
第二原始图像和第三原始图像为采用相同目标镜头拍摄待拍摄场景时，该终端的传感器所采集到的不同图像，一般来说，第二原始图像和第三原始图像对应的采集时间不完全一致，时间间隔较短。
如上所示，终端根据第二拍摄倍率确定出待裁剪区域的大小和待放大的倍数，根 据该确定的该目标主体在该第三预览图像中的目标位置，以及上述待裁剪区域的大小和待放大的比例，基于第三原始图像，确定出第三预览图像。其中，上述待裁剪图像可以是第三原始图像，或基于第三原始图像得到的中间图像。这里，可以参照上述基于第二原始图像得到的中间图像，此处不再重复赘述。
上述方案，可以根据用户对目标主体的指示操作，调整目标主体在预览图像中的位置，满足用户的多样化构图需求，避免目标主体在预览图像中的位置固定造成的单一性。
在一种可能的实施方式中，该方法还包括：在该终端进入防抖模式后，在该终端的显示屏上显示第一预览区域和第二预览区域，其中，该第一预览区域用于显示辅助预览图像，该第二预览区域用于显示该第二预览图像，该辅助预览图像包括该第二预览图像中的内容。
应理解，上述防抖模式也可以称为辅助预览模式，或者，多预览区域模式等，可以在用户拍照的设置界面或系统设置界面中，以特定模式的形式选择进入，本申请对于上述名称不做限定。另外，该辅助预览图像的视角范围大于该第二预览图像的视角范围。
在一种可能的实施方式中，该方法还包括：在该第二拍摄倍率大于预设阈值时，在该终端的显示屏上显示第一预览区域和第二预览区域，其中，该第一预览区域用于显示辅助预览图像，该第二预览区域用于显示该第二预览图像，该辅助预览图像包括该第二预览图像中的内容。
应理解，预设阈值的取值设定可以由用户预先设置，例如，用户可以在终端系统设置页面中设置。或者，预设阈值的取值设定还可以根据经验值确定。例如，对用户进行调研，确定用户对手部微小抖动无感知的拍摄倍率。将该倍率的取值确定为预设阈值。
类似地，第二预览区域还可以用于显示预览流中的其他预览画面，例如第三预览画面。该第一预览区域和该第二预览区域所显示的内容可以根据用户的操作进行交换，例如，接收对于第一预览区域和该第二预览区域的操作(如单指从第一预览区域滑动至第二预览区域的操作)，响应于上述操作，交换该第一预览区域和该第二预览区域所显示的内容，即第一预览区域用于显示该第二预览图像，第一预览区域用于该辅助预览图像。
该第一预览区域的大小、显示规格还可以根据用户的指示进行调整，例如，接收对于第一预览区域的操作(如双指同时在第一预览区域向相反方向滑动)，响应于上述操作，按照一定比例放大第一预览区域的大小，其中，比例的大小可以由用户在系统设置界面通过精度粒度来设置，或者，还可以根据该用户或大量用户统计的使用习惯设置。
上述方案，可以通过辅助预览图像的显示，使用户感知到当前的预览图像(如第二预览图像，第三预览图像)在当前待拍摄场景中的位置，便于用户调整构图。预览区域对应的大小、显示规格和显示内容的可调，提高了用户使用的灵活性。
在一种可能的实施方式中，该辅助预览图像基于该第二原始图像确定。例如，该辅助预览图像基于该第二原始图像，或基于该第二原始图像得到的中间图像确定。该辅助预览图像的视角范围应大于该第二原始图像的视角范围。在相同条件下(如对于同一个待裁剪图像)，该辅助预览图像所对应的待裁剪区域的范围应大于该第二原始图像所对应的待裁剪区域的范围。
通过同一个原始图像来确定辅助预览图像和第二原始图像，可以只调用相同的目标镜头，而无需调用其他镜头来拍摄待拍摄场景，并且可以采用相似或相同的图像处理流程或方法来处理上述图像，降低硬件成本和软件算法成本，提升了处理速度。
在一种可能的实施方式中，该辅助预览图像基于辅助原始图像确定，其中，该辅助原始图像与该第二原始图像为该终端调用不同镜头，在拍摄该待拍摄场景时，该终端的图像传感器所采集到的不同图像。
对于具有多镜头的终端，希望使上述不同镜头同时对待拍摄场景进行拍摄，终端的传感器所采集到的图像中携带有时间戳，在将采集的图像经过预览图像处理后，送至显示器显示时，会基于时间戳进行同步。但由于不同镜头及传感器的精度、帧率等参数差异，时间戳相同的图像，所对应的绝对时间不一定完全一致。即：该辅助原始图像与该第二原始图像所对应的采集时间不完全一致，但时间间隔较短，如相差0.01毫秒。另外，该辅助原始图像和该第二原始图像所对应的采集时间顺序可以调整，本申请不做限定。
在一种可能的实施方式中，该第一预览图像包括第一物体，在该第一预览图像中确定目标主体，具体包括：在该第一物体为系统预设物体时，确定该第一物体为目标主体。
应理解，系统预设置的物体包括一个或多个物体。该第一物体可以为终端检测到的第一个系统预设物体，或者，可以为终端检测到的优先级最高的系统预设物体，或者，可以为终端检测到的数量最多的系统预设物体。
系统预设置的物体可以由用户在系统设置界面设定，也可以根据当前用户或大量用户的拍摄习惯，将用户拍摄概率较高的物体设置为系统预设置的物体。
通过系统预设置目标主体的物体类别清单的方式，终端直接识别出第一预览图像中的目标主体，减少用户交互，处理效率高，用户操作便捷。
在一种可能的实施方式中，该第一预览图像包括第一物体，该在该第一预览图像中确定目标主体，具体包括：接收对于所述第一预览图像中的第一物体的第二操作，响应于该第二操作，确定该第一物体为目标主体。
上述第二操作可以是手指的单击、双击、长按、指关节敲击等手势操作、隔空手势等中的一个或多个，也可以是语音操作或面部操作，本申请不做限定。
在一种可能的实施方式中，该在该第一预览图像中确定目标主体，具体包括：接收对于该第一预览图像中的第一区域的第三操作，响应于所述第三操作，确定该第一区域中的物体为目标主体。
上述第三操作可以是单指或双指圈出第一区域，或者指关节圈出第一区域等手势 操作，或者隔空手势或语音操作或面部操作，本申请不做限定。
通过上述用户直接指示特定物体或特定区域为目标主体的方式，提高了用户交互感知，也可以使终端更精确地识别到用户需求拍摄的目标主体，减少用户修正的概率。
另外，在拍摄过程中，用户可以重新指定目标主体。
在一种可能的实施方式中，该目标主体在该第二预览图像中，通过标识框，和/或，符号标识来标记。
其中，标识框可以是规则形状，例如方形、圆形等形状，也可以是不规则形状，例如目标主体的外廓。符号标识可以是米字型符号、十字形符号等符号，也可以是直线或曲线。上述标识框和符号标识的线条粗细和颜色可以调整，一般来说，可以采用稍粗或亮色的线条来标识，以与画面中的其他物体区分。类似的，该目标主体在预览流中的其他预览图像中也可以标记出来。
上述方案中，通过标识框，和/或，符号标识来标记目标主体在预览图像中的位置，可以让用户了解到终端当前确定出来的目标主体，以供用户确认或调整，避免识别错误造成的待裁剪区域错误。
第二方面，提供了一种图像处理方法，该方法包括：获取待拍摄场景的第一预览图像，该第一预览图像对应于该第一拍摄倍率；在该第一预览图像中确定目标区域；获取该待拍摄场景中的第二原始图像，该第二原始图像为该终端基于目标镜头拍摄该待拍摄场景时，该终端的图像传感器所采集得到的图像，该目标镜头为该终端在第二拍摄倍率下拍摄该待拍摄场景时所调用的镜头，该第二拍摄倍率大于该第一拍摄倍率；根据该目标区域和该第二原始图像中确定出第二预览图像，该第二预览图像包括该目标区域；在该终端的显示屏上显示该第二预览图像。
应理解，目标区域也可以理解为目标主体，第二方面及其可能的实施方式的技术细节、技术效果和可能的实施方式可以参考第一方面及其任一种可能的实施方式中的相关描述，此处不再重复赘述。
第三方面，提供了一种图像处理装置，该装置包括：收发单元以及处理单元，该收发单元，用于获取待拍摄场景中的第一预览图像，该第一预览图像对应于该第一拍摄倍率；该处理单元，用于在该第一预览图像中确定目标主体；该收发单元，还用于获取该待拍摄场景的第二原始图像，该第二原始图像为该终端基于目标镜头拍摄该待拍摄场景时，该终端的图像传感器所采集得到的图像，该目标镜头为该终端在第二拍摄倍率下拍摄该待拍摄场景时所调用的镜头，该第二拍摄倍率大于该第一拍摄倍率；该处理单元，还用于根据该目标主体和该第二原始图像，确定出第二预览图像，该第二预览图像包括该目标主体；该处理单元，还用于使能该终端的显示屏显示该第二预览图像。
应理解，上述收发单元和处理单元可以由软件实现，例如，具有上述功能的计算机程序或指令来实现，相应计算机程序或指令可以存储在终端内部的存储器中，通过处理器读取该存储器内部的相应计算机程序或指令来实现上述功能。或者，图像处理 装置还可以由硬件来实现。其中处理单元为处理器(如图像处理单元GPU、系统芯片中的处理器等)，收发单元为收发电路或接口电路。或者，本申请实施例中的图像处理装置还可以由处理器和软件模块的结合实现。第三方面及其可能的实施方式的技术细节、技术效果和可能的实施方式可以参考第一方面、第二方面及其任一种可能的实施方式中的相关描述，此处不再重复赘述。
一种可能的实施方式中，该处理单元，还用于根据该目标主体和该第二原始图像，确定出第二预览图像，具体包括：该处理单元，还用于根据该目标主体在该第二原始图像中的位置，该第二拍摄倍率和该第二原始图像，确定出第二预览图像。
一种可能的实施方式中，该收发单元，还用于接收对于该第二预览图像中的该目标主体的第一操作，该第一操作用于确定该目标主体在新的预览图像中的目标位置；该处理单元，还用于根据该目标主体在新的预览图像中的目标位置，以及该第二原始图像，确定出更新后的第二预览图像，该更新后的第二预览图像包括该目标主体；该处理单元，还用于使能该终端的显示屏显示该更新后的第二预览图像。
一种可能的实施方式中，该收发单元，还用于接收对于该第二预览图像中的该目标主体的第一操作，该第一操作用于确定该目标主体在新的预览图像中的目标位置；该收发单元，还用于获取该待拍摄场景的第三原始图像，该第三原始图像为该终端基于目标镜头拍摄待拍摄场景时，该终端的传感器所采集到的图像，该第二原始图像的采集时间早于该第三原始图像；该处理单元，还用于根据所述目标主体在新的预览图像中的目标位置，以及该第三原始图像，确定出该第三预览图像，该第三预览图像包括该目标主体；该处理单元，还用于使能该终端的显示屏显示该第三预览图像。
一种可能的实施方式中，该处理单元还用于在该第二拍摄倍率大于预设阈值时，使能该终端的显示屏上显示第一预览区域和第二预览区域，其中，该第一预览区域用于显示辅助预览图像，该第二预览区域用于显示该第二预览图像，该辅助预览图像包括该第二预览图像的内容。
一种可能的实施方式中，该辅助预览图像是基于该第二原始图像确定的。
一种可能的实施方式中，该辅助预览图像基于辅助原始图像确定，其中，该辅助原始图像与该第二原始图像为该终端调用不同镜头，拍摄该待拍摄场景时，该终端的图像传感器所采集到的不同图像。
一种可能的实施方式中，该第一预览图像包括第一物体，该处理单元，用于在该第一预览图像中确定目标主体，具体包括：该处理单元，用于在该第一物体为系统预设物体时，确定该第一物体为目标主体。
一种可能的实施方式中，该第一预览图像包括第一物体，该收发单元，用于接收对于该第一预览图像中的该第一物体的第二操作；该处理单元，用于在该第一预览图像中确定目标主体，具体包括：该处理单元，用于响应于该第二操作，确定该第一物体为目标主体。
一种可能的实施方式中，该收发单元，用于接收对于该第一预览图像中的第一区域的第三操作；该处理单元，用于在该第一预览图像中确定目标主体，具体包括：处理单元，用于响应于该第三操作，确定该第一区域中的物体为目标主体。
一种可能的实施方式中，该目标主体在该第二预览图像中，通过标识框，和/或，符号标识来标记。
一种可能的实施方式中，该第二拍摄倍率为用户指定的拍摄倍率。
一种可能的实施方式中，该目标主体位于该第二预览图像中除中心位置以外的其他位置。
第四方面，提供一种终端，该终端包括：触摸屏，其中，该触摸屏包括触敏表面和显示器；摄像头；一个或多个处理器；存储器；多个应用程序；以及一个或多个计算机程序，其中该一个或多个计算机程序被存储在该存储器中，该一个或多个计算机程序包括指令，当该指令被该终端执行时，使得该终端执行第一方面至第二方面及任一种可能的实施方式中该的图像处理方法。
第五方面，提供一种电子设备，包括触摸屏，存储器，一个或多个处理器，多个应用程序，以及一个或多个程序；其中该一个或多个程序被存储在该存储器中；其特征在于，该一个或多个处理器在执行该一个或多个程序时，使得该电子设备执行第一方面至第二方面及任一种可能的实施方式中该的图像处理方法。
第六方面，提供一种计算机存储介质，其特征在于，包括计算机指令，当该计算机指令在电子设备上运行时，使得该电子设备执行第一方面至第二方面及任一种可能的实施方式中该的图像处理方法。
第七方面，提供一种计算机程序产品，其特征在于，当该计算机程序产品在计算机上运行时，使得该计算机执行如权利要求1至11中任一该的图像处理方法。
第四方面至第七方面中的任一方面、及其可能的实施方式的技术细节、技术效果和可能的实施方式可以参考第一方面、第二方面及其任一种可能的实施方式中的相关描述，此处不再重复赘述。
应理解，本申请实施例提供的图像处理方案，可以用于提高图像拍摄的稳定性，尤其是在用户指定的拍摄倍率为高倍率时，可以通过基于目标主体的位置，裁剪待终端处理的原始图像，以提高图像拍摄的性能。另外，还可以根据用户的调整操作，调整目标主体在预览图像中的位置，提高构图的灵活性。通过多预览区域的显示方式还可以便于用户调整目标主体，以及知晓当前目标主体在摄像头采集的图像中的位置，以降低目标主体出画概率。
附图说明
图1为本申请实施例提供的一种电子设备的硬件结构示意图；
图2为本申请实施例提供的一种电子设备的软件结构示意图；
图3为现有技术中一种图像处理方法的流程示意图；
图4为现有技术中高倍率变焦场景下拍摄目标主体的第1帧图像的显示示意图；
图5为现有技术中高倍率变焦场景下拍摄目标主体的第50帧图像的显示示意图；
图6为现有技术中高倍率变焦场景下拍摄目标主体的第100帧图像的显示示意图；
图7为本申请实施例提供的手机的一种图像用户界面；
图8为本申请实施例提供的手机的另一种图像用户界面；
图9为本申请实施例提供的手机的另一种图像用户界面；
图10为本申请实施例提供的一种图像处理方法的流程示意图；
图11A为本申请实施例提供的一种预览图像处理算法的结构示意图；
图11B为本申请实施例提供的一种图像裁剪方式示意图；
图12为本申请实施例提供的手机的另一种图像用户界面；
图13为本申请实施例提供的手机的另一种图像用户界面；
图14为本申请实施例提供的一种高倍率变焦场景下拍摄目标主体的显示示意图；
图15为本申请实施例提供的一种用户调整拍摄倍率的用户界面；
图16为本申请实施例提供的另一种用户调整拍摄倍率的用户界面；
图17为本申请实施例提供的另一种手机的用户界面；
图18为本申请实施例提供的另一种图像处理方法的流程示意图；
图19为本申请实施例提供的一种拍照的用户界面；
图20为本申请实施例提供的另一种拍照的用户界面；
图21为本申请实施例提供的一种图像处理装置的结构示意图；
图22为本申请实施例提供的另一种图像处理装置的结构示意图。
具体实施方式
下面结合附图并举实施例，对本申请提供的技术方案作进一步说明。应理解，本申请实施例中提供的系统结构和业务场景主要是为了说明本申请的技术方案的可能的实施方式，不应被解读为对本申请的技术方案的唯一限定。本领域普通技术人员可知，随着系统结构的演进和新业务场景的出现，本申请提供的技术方案对类似技术问题同样适用。
本申请实施例提出一种图像处理方案，包括图像处理的方法及装置。其中，本申请实施例提出的图像处理方法可以应用于计算设备(例如，本申请提出的图像处理装置或电子设备)上，能够在拍摄照片或视频时，通过跟踪预览流中的目标主体，并根据目标主体的位置，在拍摄得到的图像中裁剪得到预览图像，从而在手持拍摄装置拍摄高倍率变焦场景时，避免因手抖或者拍摄装置抖动造成的目标主体移出拍摄画面等问题，提高拍摄的稳定性。
本申请实施例提供的图像显示方法可以应用于手机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等电子设备上，本申请实施例对电子设备的具体类型不作任何限制。
示例性的，图1示出了电子设备100的结构示意图。电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以通过硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。所述I2S接口和所述PCM接口都可以用于音频通信。
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，外部存储器，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在 其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system， GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子 设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行电子设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。
扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。
受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。电子设备100可以设置至少一个麦克风170C。在另一些实施例中，电子设备100可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，电子设备100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测所述触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看 短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测电子设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消电子设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。
气压传感器180C用于测量气压。在一些实施例中，电子设备100通过气压传感器180C测得的气压值计算海拔高度，辅助定位和导航。
磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中，当电子设备100是翻盖机时，电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态，设置翻盖自动解锁等特性。
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。
距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定电子设备100附近有物体。当检测到不充分的反射光时，电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中，电子设备100利用温度传感器180J检测的温度，执行温度处理策略。例如，当温度传感器180J上报的温度超过阈值，电子设备100执行降低位于温度传感器180J附近的处理器的性能，以便降低功耗实施热保护。在另一些实施例中，当温度低于另一阈值时，电子设备100对电池142加热，以避免低温导致电子设备100异常关机。在其他一些实施例中，当温度低于又一阈值时，电子设备100对电池142的输出电压执行升压，以避免低温导致的异常关机。
触摸传感器180K，也称“触控面板”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
骨传导传感器180M可以获取振动信号。在一些实施例中，骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏，接收血压跳动信号。在一些实施例中，骨传导传感器180M也可以设置于耳机中，结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号，解析出语音信号，实现语音功能。应用处理器可以基于所述骨传导传感器180M获取的血压跳动信号解析心率信息，实现心率检测功能。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。
在本申请实施例中，在拍摄场景下，摄像头193采集图像，ISP对摄像头193反馈的数据进行处理，例如，确定图像中的目标主体。可选的，处理器110或处理器110中的NPU可以对ISP处理后的图像进行图像分割，确定图像上不同物体或不同物体类型分别所在的区域。另外，处理器110可以在预览流中对已确定的目标主体进行追踪。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。
图2是本申请实施例的电子设备100的软件结构框图。分层架构将软件分成若干 个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。应用程序层可以包括一系列应用程序包。
如图2所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图2所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供电子设备100的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG， PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
2D图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
目前，变焦方式主要包括：光学变焦(optical zoom)和数码变焦(digital zoom)。其中，光学变焦是通过镜头、视角和焦点三方的位置变化而产生的。即，通过镜片的移动来放大或缩小待拍摄物体。光学变焦的倍数越大，所能拍摄的目标主体可以距离镜头越远。而数码变焦是通过终端(如手机、数码相机)内的处理器，将采集到的图片内的每个像素面积增大，从而达到放大的目的。但目前手机上所能采用的光学变焦镜头的变焦能力有限，对于高倍率变焦场景，仍需要光学变焦技术结合数码变焦技术来进行拍摄。其中，数码变焦技术指的是根据用户所指定的拍摄倍率，将光学变焦镜头所采集到图像的中心区域裁剪后放大到用户指定倍率的技术。
图3为现有技术中一种图像处理方法的流程示意图。如图3所示，在用户打开相机应用后，手机显示屏上会显示包括预览显示区域的用户界面(user interference，UI)，其中，预览显示区域可以用于显示预览流，其中，预览流包括一帧或多帧预览图像。用户可以通过该UI上的控件来指示选择采用不同拍摄倍率，例如，3倍变焦，20变焦等。在确定用户指示采用的拍摄倍率后，手机可以根据用户所指示采用的拍摄倍率，确定目标镜头(即用于采集待拍摄区域图像的镜头)，具体地，确定目标镜头的个数和对应的倍率。例如，手机有两个镜头，主镜头的光学变焦倍率为1倍(1x)、长焦镜头的光学变焦倍率为5倍(5x)。手机检测到用户指示采用的拍摄倍率为20倍时，手机确定需要通过一个5倍光学变焦倍率的长焦镜头(即目标镜头)来采集图像。另外，手机基于目标镜头拍摄待拍摄场景，并根据手机的显示规格(display specification)，例如显示尺寸(display size)和显示比例(display scale)，进行裁剪后得到待终端处理的原始图像。手机根据用户所指示的拍摄倍率，可以确定出用户所希望拍到的视角范围(field of view，FOV)和待裁剪区域的大小。从待终端处理的原始图像的中心位置，根据上述确定出的待裁剪区域的大小，裁剪得到相应大小的图像，再经过图像处理、放大到相应的预览分辨率后，通过预览区域显示。
图4至图6为一组现有技术中高倍率变焦场景下拍摄目标主体的显示示意图。其中，图4、图5至图6分别示出了待终端处理的原始图像、以及预览区域所显示的图像所对应的第1帧，第50帧，以及第100帧图像。其中，待终端处理的原始图像指的是手机基于目标镜头拍摄待拍摄场景后，适应于手机的显示规格裁剪得到的图像。
这里，假设用户所指示采用的拍摄倍率为10倍，手机采用了为1倍光学变焦倍率的镜头作为目标镜头。手机根据如图3所示的图像处理方法，从待终端处理的原始图像中的中心位置，裁剪出1/100的图像，并按照1:100的比例进行放大，在预览区域中 显示相应图像。
具体地，如图4所示，对于所拍摄的第1帧图像，待终端处理的原始图像401中包括目标主体的图像402a。手机根据用户指示采用的拍摄倍率，确定待裁剪区域403的图像大小。以待终端处理的原始图像401的中心为中心，根据待裁剪区域403的图像大小，裁剪得到对应大小的图像。最终经过图像处理、放大到相应的预览分辨率后，通过预览区域显示图像404，该图像404显示出目标主体的预览图像402b。
类似的，如图5所示，对于所拍摄的第50帧图像，待终端处理的原始图像501中包括目标主体的图像502a。此时的待裁剪区域503只覆盖了部分目标主体的图像502a。因此，在预览区域中所显示的图像504只显示了部分目标主体的预览图像502b。如图6所示，对于所拍摄的第100帧图像，待终端处理的原始图像601中包括目标主体的图像602a。但此时的待裁剪区域603与目标主体的图像602a存在距离，预览区域所显示的图像604并未显示出目标主体的预览图像。
对于高倍率变焦拍摄场景下，手部的抖动会使得所拍摄的目标主体在预览区域的位置发生变化，这使得用户难以构图，甚至会使所拍摄的目标主体移出画面，往往需要多次拍摄才可以获得比较理想的图片，用户体验不佳。
基于此，本申请实施例提出一种图像处理方案，包括图像处理的方法及装置。通过对待拍摄区域中的目标主体采用跟踪裁剪技术，来降低目标主体在预览图像中的抖动，提高拍摄的稳定性。由于这些技术方案解决问题的原理相同或相似，在如下具体实施例的介绍中，某些重复之处可能不再赘述，但应视为这些具体实施例之间已有相互引用，可以相互结合。
图7为本申请实施例提供的手机的一种图像用户界面(graphical user interface，GUI)，该GUI为手机的桌面701。当手机检测到用户点击桌面701上的相机应用(application，APP)的图标702的操作后，可以启动相机应用，显示如图5中的(b)所示的另一GUI，该GUI可以称为拍摄界面703。该拍摄界面703上可以包括取景框704。在预览状态下，该取景框704内可以实时显示预览图像，这里该取景框704以及其显示的内容可以成为预览区域。可以理解的是，在拍照模式和录像模式(即视频拍摄模式)下，取景框704的大小可以不同。例如，图7中的(b)所示的取景框可以为拍照模式下的取景框。在录像模式下，取景框704可以为整个触摸屏。
示例性的，参见图7中的(b)，在手机启动相机后，取景框704可以显示有图像。另外，拍摄界面上还可以包括用于指示拍照模式的控件705，用于指示录像模式的控件706，以及拍摄控件707。在拍照模式下，当手机检测到用户点击该拍摄控件707的操作后，手机执行拍照操作；在录像模式下，当手机检测到用户点击该拍摄控件707的操作后，手机执行拍摄视频的操作。其中，可选的，在拍照模式下，可以拍摄静态图片或者动态图片(live photo)。
图8中的(a)示出了手机的另一种GUI，该GUI为拍摄静态图片模式的界面801。在手机启动相机后，在拍照模式下，对于拍摄静态图片模式的拍摄界面上还可以包括用于指示拍摄动态图片的控件802。当手机检测到用户点击该控件802，由拍摄静态图 片模式转换为拍摄动态图片模式，显示如图8中(b)所示的另一GUI，该GUI为拍摄动态图片模式的界面803。类似的，在手机启动相机后，在拍照模式下，对于拍摄动态图片模式的拍摄界面上还可以包括用于指示拍摄静态图片的控件804。当手机检测到用户点击该控件804，由拍摄静态图片模式转换为拍摄动态图片模式，显示如图8中(a)所示的GUI。其中，可选的，控件802与控件804可以为相同图标，并以带有颜色的高亮区分。可选的，控件802与控件804可以为相同图标，并以不同类型的线条区分，例如，实线与虚线，或者，粗线与细线。
在具体实施过程中，进入拍摄动态图片模式的GUI还有多种可选的设计，示例性的，参见图9中的(a)，拍摄界面901上还包括用于指示显示其他更多模式的控件902。当手机检测到用户选中该拍摄控件902，例如，用户点击该拍摄控件902，或者手机检测到用户将拍摄控件902滑动至GUI中央，或者手机检测到用户将拍摄控件902滑动至拍摄键上方。显示如图9中的(b)所示的GUI。该GUI为界面903，界面903中显示有多种用于指示特定拍摄模式的控件，其中包括用于指示拍摄动态图片模式的控件904。当手机检测到用户点击该拍摄控件904，显示拍摄界面901，并进入拍摄动态图片模式。
应理解，本申请实施例所提供的图像处理方法可以应用于静态图片、动态图片、视频的拍摄以及处理场景中，本申请不做限定。
图10为本申请实施例提供的一种图像处理方法的流程示意图。应理解，该图像处理方法可以由计算设备或电子设备(例如，终端)执行，也可以由电子设备内的装置(例如，图像处理装置)执行。其中，图像处理装置可以是ISP或系统芯片(system on chip，SoC)。这里，以终端执行为例，展开介绍。如图10所示，该图像处理方法1000包括：
S1001：终端获取待拍摄场景的第一预览图像，该第一预览图像对应于第一拍摄倍率。
其中，第一拍摄倍率可以是默认拍摄倍率(例如，1倍拍摄倍率)，也可以是用户在拍摄过程中重新指定终端采用的拍摄倍率。
在用户打开终端的拍摄应用(如相机)后，终端会进入默认拍摄倍率并确定目标镜头为默认镜头(如1倍光学变焦倍率)对待拍摄场景进行拍摄，随着用户调整指定的拍摄倍率，终端可以根据用户指定的拍摄倍率，重新确定目标镜头。图11A为本申请实施例提供的一种预览图像处理算法的结构示意图。参见图11A所示，预览图像处理算法1100包括：适应于终端的显示规格处理模块1101，裁剪模块1102，放大模块1103，预览图像处理模块1104。其中，应理解的是，上述模块的顺序可以调整，预览图像处理模块1104还可以分为多个相同或不同的预览图像处理模块，在整个预览图像处理的过程中对图像进行处理。
例如，终端确定第一拍摄倍率(如10倍)，需要采用一个1倍光学变焦倍率的镜头进行拍摄，在终端采用上述镜头拍摄待拍摄场景时，通过终端的传感器采集到第一原始图像。将第一原始图像输入至如图11A所示的预览图像处理算法中，得到第一预览图像。具体地，第一预览图像根据中终端的显示规格进行裁剪，得到待终端处理的 原始图像。终端可以根据第一拍摄倍率确定出待裁剪图像中的待裁剪区域的大小、以及待裁剪区域的待放大倍数。进而，以待裁剪图像的中心位置为待裁剪区域的中心，根据上述确定的待裁剪区域的大小和待放大倍数，执行裁剪、放大等动作，再经过预览图像处理后得到第一预览图像。其中，待裁剪图像可以为第一原始图像，也可以为待终端处理的原始图像，还可以是将上述图像经过预览图像处理或部分预览图像处理(如第一预览图像处理)得到的图像。
S1002：终端在该第一预览图像中确定目标主体。
应理解，本申请实施例中的目标主体可以是一个物体，也可以是多个物体，可以是一个区域内的全部物体(即目标主体为目标区域)，此处不做限定。另外，目标区域可以是规则图形，例如，圆形、长方形、正方形等，也可以是不规则图形。
目标主体的确认方式在具体实施过程中有多种实施方式，下面给出几种示例，本申请所保护的范围包括但不限于以下示例：
可选的，根据预览流中在第一预览图像之前的一帧或多帧预览图像中的目标主体，确定出当前第一预览图像中的目标主体。
终端的显示屏上会显示预览流，其中，预览流包括一帧预览图像或多帧预览图像。一般来说，目标主体在预览流中相邻或相近几帧图像的位置不会发生大的突变。终端可以根据预览流中的在第一预览图像之前的一帧或多帧预览图像中目标主体的位置，预测第一预览图像中目标主体的位置。其中，上述多帧预览图像可以是连续的多帧，也可以是周期性间隔的多帧或非周期性间隔的多帧。
可选的，当终端检测出第一预览图像中，存在对应于系统预设物体类别的物体，在确定该物体为目标主体。
具体地，例如，终端存储有系统预设置的物体，具体的存储内容可以是物体类别标签。终端对于预览流中的第一预览图像的物体进行图像识别，获得第一预览图像中的物体类别标签，并与系统预设置的物体类别标签比较。或者，终端对于第一预览图像进行图像分割，将分割后的图像与对应于系统预设置的物体类别标签的图像进行匹配，其中，对应于系统预设置的物体类别标签的图像可以由云端服务器获取，或者，由本地存储器中获取。当第一预览图像中存在对应于系统预设置的物体类别的一个物体时，确定相应物体为目标主体。当第一预览图像中存在对应于系统预设置的物体类别的多个物体时，可以在用户交互界面上提示用户进行目标主体的选择与确认，确定用户指示的物体为目标主体。当第一预览图像中不存在对应于系统预设置的物体类别的物体时，可以在用户交互界面上提示用户进行目标主体的确认，确定用户指示的物体为目标主体。
可以理解的是，系统预设置的物体具体的存储内容也可以是物体图像。终端可以通过对第一预览图像进行图像分割，将分割后的图像直接与系统预设置的物体图像进行匹配。
其中，上述系统预设置的物体类别清单，可以是通过学习用户拍摄习惯，将用户拍摄概率较高的物体添加至系统预设置的物体类别清单。例如，根据终端的当前用户，和/或，大量其他用户之前所拍摄的图像或浏览的图像，通过对上述图像内物体类别的识别、标注，获得用户拍摄物体类别以及对应的拍摄概率。将其中拍摄概率大于系统 预设阈值的物体类别划分至系统预设置的物体类别清单。另外，上述系统预设置的物体类别清单，可以在出厂前预先设置，也可以通过终端系统升级时设置，还可以是存储在云端服务器中，当终端执行图像处理方法(例如，图像处理方法1000)时，从云端服务器中读取上述系统预设置的物体类别清单。
可以理解的是，系统预设置的物体可能不止一个，终端在检测到第一预览图像中存在不止一个系统预设置的物体，终端可以将第一预览图像中最先检测到的系统预设置的物体，确定为目标主体。或者，上述系统预设置的物体设置有优先级，终端可以将第一预览图像中检测到的对应优先级最高的系统预设置的物体，确定为目标主体。或者，终端可以将第一预览图像中检测到出现最多的对应于系统预设置的物体类型的物体，确认为目标主体。
可选的，可以利用显著性区域检测等算法，对第一预览图像进行分析，自动判断用户想用拍摄的目标主体。显著性区域检测相关算法有很多，可以利用传统方法，提取图像特征，检测图像特征与其它区域不同的特殊区域。也可以训练神经网络，分析预览图像，推断用户想要拍摄的目标主体。
通过系统预设置目标主体的物体类别清单的方式，终端直接识别出第一预览图像中的目标主体，减少用户交互，处理效率高，用户操作便捷。
可选的，响应于对于第一物体的操作，确定第一物体为目标主体。即：由用户直接指示特定物体为目标主体。
具体地，例如，用户可以通过单击或双击或长按第一物体的手势动作，来指示该物体为目标主体。再例如，用户可以通过长按第一物体的手势动作，并结合语音命令来指示第一物体为目标主体。其中，上述手势动作可以是手指触碰，也可以是指关节触碰，可以是单指操作，也可以是多指操作，具体形式可以根据终端实际情况进行选择。
终端可以通过触摸屏上的传感器来检测上述手势动作，在检测到用户对第一物体的手势动作时，响应于用户对于第一物体的操作，确定第一物体为目标主体。
通过上述用户直接指示特定物体为目标主体的方式，提高了用户交互感知，也可以使终端更精确地识别到用户需求拍摄的目标主体，提高用户体验。
可选的，响应于用户对于第一预览图像中第一区域的操作，当第一区域存在对于系统预先设置物体类别的物体，则确定该物体为目标主体。
具体地，例如，用户可以通过手势划出特定区域，终端通过触摸屏上的传感器检测到上述手势后，确定特定区域内的物体，在该特定区域内存在于系统预先设置的物体类别对应的物体时，确定该物体为目标主体。再例如，用户可以指定第一预览图像中的一个点，以该点为中心确定第一区域。第一区域的大小可以根据用户指示的拍摄倍率来确定。这种方式下，终端无需进行物体检测和图像分割，降低算法难度。
可选的，响应于用户对于第一预览图像中的第一区域的操作，确定第一区域中的物体为目标主体。上述对于第一区域的操作可以是长按、单击、双击、指关节单击、指关节划定区域等动作中的一种或多种。
应理解，可选的，确认的目标主体可以在拍摄界面上予以标识。例如，通过对焦框或标注框等形状标识，或者十字标识或者米字标识等符号标识，圈出或标识出所确 认的目标主体来予以标识。其中，对焦框或标注框等标识的形状可以包括但不限于方形、椭圆形等，简单的方形、椭圆形便于实施，对于所确认的目标主体图像识别精度需求低。十字标识或米字标识等符号标识可以是位于目标主体图像的中心区域，其中，标识的大小可以根据目标图像的大小或者根据拍摄倍率的大小调整或变动。再例如，对确认的目标主体进行外廓显示标识，这种方式示意确认的目标主体更为准确。其中，上述标识可以随着目标主体在预览图像中的位置移动而移动。
通过上述方法，可以提高用户交互感知，并提高终端识别目标主体的精确度，提高用户体验。
S1003：终端获取该待拍摄场景的第二原始图像，其中，该第二原始图像为该终端基于目标镜头拍摄该待拍摄场景时，该终端的图像传感器所采集的图像，该目标镜头为该终端在第二拍摄倍率下拍摄该待拍摄场景时所调用的镜头，该第二拍摄倍率大于该第一拍摄倍率。
其中，第二拍摄倍率为用户指定终端采用的拍摄倍率，在终端确定第二拍摄倍率后，可以确定目标镜头，获得第二原始图像。第二原始图像适应于终端的显示规格裁剪后，得到待终端处理的原始图像。若第二拍摄倍率大于目标镜头的光学变焦倍率，则还需要结合数码变焦技术来处理待终端处理的原始图像。
S1004：终端根据该目标主体和该第二原始图像，确定出第二预览图像，该第二预览图像包括该目标主体。
应理解，S1004在具体实施过程中，有多种实施方式。这里给出几种示例，各示例可以单独或结合实施，但本申请所提供的方案包括但不限于下列示例。
可选的，根据目标主体在第二原始图像中的位置，根据第二拍摄倍率和第二原始图像，确定出第二预览图像，第二预览图像包括目标主体。这里，也可以理解为：将第二原始图像输入如图11A所示的预览图像处理算法中，得到第二预览图像。其中，第二预览图像所对应的各个模块(如裁剪模块1102)的具体实施方式和计算参数，与第一预览图像所对应的各个模块的具体实施方式和计算参数不完全一致。例如，终端根据第二拍摄倍率可以确定出目标镜头、用户所希望拍到的FOV、需要采用的数码变焦倍率以及待裁剪区域的大小。进而，可以以目标主体在第二原始图像中的位置为中心，基于确定出的待裁剪区域的大小，裁剪得到第二中间图像，再通过放大至预览分辨率，预览图像处理后得到第二预览图像。其中，应理解的是，图11A中的各个模块的顺序可以调整，预览图像处理模块1104还可以分为多个相同或不同的预览图像处理模块，在整个预览图像处理的过程中对图像进行处理。预览图像处理模块1104可以包括一个或多个算法过程，例如，去噪、均衡、提高对比度等，可以分别执行，也可以在一起执行。例如，可以先进行去噪等预算图像处理过程，再进行裁剪、放大、而后进行均衡、提高对比度等预览图像处理过程。本申请对此不做限定。
采用这种方式，只要目标主体存在于预览流中，排除因跟踪算法不稳定、目标主体运动等原因导致的小的抖动，更新后的预览流(包括第二预览图像)可以保持稳定。但这样的方式，用户将无法难以通过手机的位置变化调整构图，目标主体将一直保持在预览流的画面中心。
可选的，终端可以在裁剪过程中，以目标主体在待裁剪图像中的位置为参考点， 结合第二拍摄倍率，确定出待裁剪区域的大小和待裁剪区域的中心点位置，进而确定出第二预览图像，使得目标主体不固定在预览图像的中间位置，以避免构图的单一性。例如，图11B为本申请实施例提供的一种图像裁剪方式示意图，参见图11B，图11B示意出摄像头采集后并适应显示规格裁剪后的图像，摄像头的光学变焦倍率为1倍，该图像宽为w，高为h。O为图像的中心位置，T为跟踪到的目标主体位置，待裁剪区域的中心点记作P，待裁剪区域的宽和高分别记作w c和h c，用户所指示终端采用的变焦倍率为r，则待裁剪区域的中心点P坐标(x p,y p)以及待裁剪区域大小满足以下公式：
Figure PCTCN2021081094-appb-000001
Figure PCTCN2021081094-appb-000002
Figure PCTCN2021081094-appb-000003
Figure PCTCN2021081094-appb-000004
为了提升用户构图的多样性，还可以基于用户的指示来调整目标主体在预览图像中的位置。可选的，接收对于该第二预览图像中的该目标主体的第一操作，响应于该第一操作，确定该目标主体在第三预览图像中的目标位置，获取该待拍摄场景的第三原始图像，该第三原始图像为该终端基于目标镜头拍摄待拍摄场景时，该终端的传感器所采集到的图像，该第二原始图像的采集时间早于该第三原始图像。
应理解，第一操作可以是手势、语音、面部等类型的操作。例如，终端通过触摸屏检测并接收到用户双指拖拽第二预览图像中的目标主体，根据用户双指拖拽的方向和距离，以及该目标主体位于第二预览图像中的位置，确定出该目标主体在第三预览图像中的目标位置。其中，目标位置与该目标主体在第二预览图像中的位置变化，方向与用户双指拖拽的方向相同，距离与用户双指拖拽的距离成比例，其中，比例的大小可以由用户在系统设置界面通过精度粒度来设置，或者，还可以根据该用户或大量用户统计的使用习惯设置。
第二原始图像和第三原始图像为采用相同目标镜头拍摄待拍摄场景时，该终端的传感器所采集到的不同图像，一般来说，第二原始图像和第三原始图像对应的采集时间不完全一致，时间间隔较短。
如上所示，终端根据第二拍摄倍率确定出待裁剪区域的大小和待放大的倍数，根据该确定的该目标主体在该第三预览图像中的目标位置，以及上述待裁剪区域的大小和待放大的比例，基于第三原始图像，确定出第三预览图像。其中，上述待裁剪图像可以是第三原始图像，或基于第三原始图像得到的中间图像。这里，可以参照上述基于第二原始图像得到的中间图像，此处不再重复赘述。
上述方案，可以根据用户对目标主体的指示操作，调整目标主体在预览图像中的位置，满足用户的多样化构图需求，避免目标主体在预览图像中的位置固定造成的单一性。
S1005：在终端的显示屏上显示该第二预览图像。
应理解，如何在终端的显示屏上显示第二预览图像在具体实施过程中，有多种实施方式。这里给出几种示例，各示例可以单独或结合实施，但本申请所提供的方案包括但不限于下列示例。
可选的，在终端的显示屏上显示一个预览区域，该预览区域用于显示第二预览图像，以及预览流中的其他预览图像，这种方式实施起来较为简单。例如，图12为本申请实施例提供的手机的另一种GUI，如图12所示，终端的显示屏上显示的拍摄界面1201中仅包括一个预览区域1202，该预览区域1202用于显示第二预览图像，以辅助用户进行拍摄。但直接手持手机进行拍摄，若手部抖动带来镜头大幅抖动，目标主体可能会移出当前的预览区域。此时，终端可以重新确定目标主体或提示用户重新确定目标主体。
可选的，在终端的显示屏上显示至少两个预览区域，例如，显示第一预览区域和第二预览区域，其中，第一预览区域用于显示辅助预览图像，第二预览区域用于显示第二预览图像，辅助预览图像包括第二预览图像中的内容。这里，辅助预览图像的FOV应大于或等于第二预览图像的FOV，该辅助预览图像基于该第二原始图像，或基于该第二原始图像得到的中间图像确定得到。在相同条件下(如对于同一个待裁剪图像)，该辅助预览图像所对应的待裁剪区域的范围应大于该第二原始图像所对应的待裁剪区域的范围。例如，辅助预览图像可以是第二原始图像适应于终端的显示规格裁剪后得到待终端处理的原始图像，经过预览图像处理得到的。再例如，辅助预览图像可以是待终端处理的原始图像，根据目标主体在第二原始图像中的位置，在第二原始图像中裁剪出1/n 2大小的区域后，放大n 2倍、经过预览图像处理后得到的图像，其中，n为正整数，且n小于第二拍摄倍率与第二原始图像对应的目标镜头的光学变焦倍率的比值。或者，辅助预览图像可以是基于辅助原始图像确定得到的，其中，辅助原始图像和第二原始图像可以由终端的不同镜头拍摄待拍摄场景时，终端的图像传感器采集得到的不同图像，辅助预览图像中可以通过标识框或符号等方式来标识出目标主体，具体的标识方式可以参考S1002中的相关描述。
另外，第一预览区域的位置和大小可以根据用户的指示进行调整，例如，在多预览区域显示的拍摄界面中，接收用户对于第一预览区域的第一操作，如拖拽等手势操作或语音命令等类型的操作，响应于上述第一操作，可以将第一预览区域的位置由原来的位置变更至用户所指示的位置，或者，接收用户对于第一预览区域的第二操作，如双指滑动缩放等手势操作或语音命令等类型的操作，响应于上述第二操作，可以将第一预览区域的大小根据比例进行缩小或放大。或者，在系统设置界面中，用户可以通过设置第一预览区域的默认格式(如，大小、位置)来调整第一预览区域在多预览区域显示的拍摄界面的位置。
例如，图13为本申请实施例提供的手机的另一种GUI。如图13所示，该拍摄界面1301包括第一预览区域1302和第二预览区域1303。其中，第一预览区域1302可以用于显示辅助预览图像，第二预览区域1303可以用于显示第二预览图像或重新确定的第二预览图像。应理解的是，在具体实施过程中，第一预览区域1302和第二预览区域1303的位置、大小、形状以及区域可以固定，也可以改变，此处不做限定，可以根 据终端的实际情况设计。另外，第一预览区域1302和第二预览区域1303的位置可以根据用户的指示进行交换。如用户手势从第一预览区域1302滑动至第二预览区域1303，终端响应于上述手势后，将上述两个预览区域显示的内容交换，例如，第一预览区域1302用于显示第二预览图像或者重新确定的第二预览图像等预览流中的预览图像，第二预览区域1303用于显示辅助预览流。
例如，第一预览区域1302中显示的辅助预览图像是基于辅助原始图像确定的，其中，辅助原始图像为终端调用镜头1拍摄待拍摄场景时，终端的传感器所采集的图像。第二预览区域1303中显示的第二预览图像是基于第二原始图像确定的，其中，第二原始图像为终端调用目标镜头(镜头2)拍摄待拍摄场景时，终端的传感器所采集的图像。辅助原始图像和第二原始图像为终端调用不同镜头在同一时刻拍摄待拍摄场景时，终端的传感器所采集的图像，对应的FOV不同。一般来说，镜头1的光学变焦倍率小于或等于镜头2的光学变焦倍率。这里，假设镜头1为采用3倍光学变焦倍率的镜头，镜头2为采用5倍光学变焦倍率的镜头，第二拍摄倍率(即用户指定的拍摄倍率)为15倍。第二原始图像适应于终端的显示规格裁剪后得到待终端处理的原始图像，并根据目标主体在第二原始图像中的位置，在待终端处理的原始图像中裁剪出1/9大小的区域，并放大9倍、经过预览图像处理后得到第二预览图像。辅助预览图像可以是辅助原始图像适应于终端的显示规格裁剪后得到的图像，或者，辅助预览图像可以是辅助原始图像适应于终端的显示规格裁剪后得到的图像，根据目标主体在辅助原始图像中的位置，裁剪出1/m 2大小的区域后，放大m 2倍、经过预览图像处理后得到的图像。其中，m应为小于5的正整数，从而使得辅助预览图像的FOV大于第二预览图像。另外，用户可以在辅助预览图像中重新指定目标主体，重新指定目标主体的方式可以是手势指令、隔空手势指令、语音指令、面部指令(例如，眼球移动方向)等。
通过辅助预览图像的方式，可以使用户同时感知目标主体在当前第二预览图像中的位置和目标主体在原画面或辅助预览画面的位置，用户可以基于辅助预览图像移动手机来获取不同的视野范围，也可以降低目标主体出画概率。
可选的，图像处理方法1000还可以包括S1006，
S1006：终端响应于用户点击拍摄控件，保存目标图像。
其中，目标图像可以是一帧图像，也可以是多帧图像(例如视频图像，动态图像等)。应理解的是，目标图像与第二预览图像的FOV相同。
应理解，在用户点击拍摄控件后，终端可以启动拍摄处理算法。在具体实施过程中，根据不同拍摄处理算法的需求，该拍摄处理算法模块可以输入一帧图像或多帧图像(如，多帧连续图像)。其中，拍摄处理算法与预览处理算法可以不同。在具体实施过程中，包括但不限于下列几种示例：
可选的，拍摄处理算法模块输入的图像为：终端在调用目标镜头拍摄待拍摄区域时，终端的传感器所采集的图像(如第二原始图像)，和/或，终端在调用目标镜头拍摄待拍摄区域同一时刻，终端调用其他镜头(一个或多个)拍摄待拍摄区域时，终端的传感器所采集的图像。
可选的，拍摄处理算法模块输入的图像为：第二原始图像，和/或，第二原始图像适应于终端的显示规格裁剪后得到的图像。
可选的，拍摄处理算法模块输入的图像为：预览流中的图像(例如，第二预览图像)，其中，预览流中的图像可以是一帧或多帧待终端处理的原始图像的整体，也可以是一帧或多帧待终端处理的原始图像裁剪后的部分。例如，在待终端处理的图像中心，根据显示规格和拍摄倍率裁剪得到的图像。或者，在待终端处理的图像中，根据目标主体的位置、显示规格和拍摄倍率裁剪得到的图像。
应理解的是，拍摄处理算法模块输入的图像不同，对应的拍摄处理算法也不同，可以根据图像裁剪的区域不同，进行调整。上述拍摄处理算法包括但不限于下列处理中的一种或多种：自动曝光(auto-exposure，AE)、自动聚焦(auto-focus，AF)、自动白平衡(auto-white balance，AWB)、动态范围调整(dynamic range correction，DRC)、镜头阴影矫正(lens shading correction，LSC)、畸变校正等。
图14为本申请实施例提供的一种高倍率变焦场景下拍摄目标主体的显示示意图。如图14所示，待终端处理的第50帧预览图像1401中包括目标主体1402a，但目标主体1402a并不处于该预览图像1401的中心位置。采用图像处理方法1000，根据目标主体1402a在该预览图像1401的位置，确定待裁剪区域1403的中心位置，根据该位置结合当前拍摄的变焦倍数，确定第一图像的待裁剪区域1403的大小。裁剪待裁剪区域1403内的图像，得到第二图像，可以通过对第二图像进行图像处理、放大到相应的预览分辨率后，得到目标图像。这里，我们可以在预览区域所显示的图像1404中显示该目标图像，并且目标图像1402b可以位于预览区域所显示的图像1404的中心位置。
应理解，采用上述图像处理方法1000，根据目标主体在当前预览图像中的位置，裁剪当前预览流，可以获得稳定的预览流，避免因手部抖动而带来的预览画面晃动，甚至是目标主体出画等情况的出现。
应理解，在具体实施过程中，用户调整拍摄倍率的GUI有多种设计方式，这里，给出几种示例，本申请实施例包括但不限于下列示例。
图15为本申请实施例提供的一种用户调整拍摄倍率的GUI。在手机启动相机后，在拍照模式下，显示如图15中的(a)所示的GUI，该GUI可以称为拍摄界面1501。拍摄界面1501上存在用于指示第一拍摄倍率的控件1502，其中，第一拍摄倍率为用户指示手机采用的拍摄倍率。用户可以通过手指在控件1502上下滑动，以选定当前指示手机采用的拍摄倍率。在用户将控件1502自下向上滑后，显示如图15中的(b)所示的另一GUI，用户所指示手机采用的拍摄倍率由1倍(1x)变为40倍(40x)。图16为本申请实施例提供的另一种用户调整拍摄倍率的GUI。与图15所示的GUI所区别的是，拍摄界面1601中用于指示第一拍摄倍率的控件1502与拍摄界面1501中用于指示第一拍摄倍率的控件1602形状不同。用户可以通过对该拍摄控件1602左右滑动以调整当前手机采用的拍摄倍率。
应理解，随着用户指定的拍摄倍率变化，终端所确定的目标镜头可以不同。
另外，S1000中提出的多预览区域显示的GUI的触发可以有多种方式。
可选的，终端跟踪第二拍摄倍率(即用户指定的拍摄倍率)的变化，当终端判断出第二拍摄倍率大于预设阈值时，触发启用多预览区域显示的方式。
可选的，终端设置有用于触发启用多预览区域显示的特定拍摄模式，当用户选择进入该特定拍摄模式时，触发多预览区域的显示。例如，参见图17中的(a)，拍摄界面1701上还包括用于指示显示其他更多模式的控件1702。当手机检测到用户选中该拍摄控件1702，例如，用户点击该拍摄控件1702，或者手机检测到用户将拍摄控件1702滑动至GUI中央，或者手机检测到用户将拍摄控件1702滑动至拍摄键上方。显示如图17中的(b)所示的GUI。该GUI为界面1703，界面1703中显示有多种用于指示特定拍摄模式的控件，其中包括用于指示触发启用多预览区域显示的特定拍摄模式的控件1704，该控件1704这里称为辅助显示。当手机检测到用户点击该拍摄控件1704，显示拍摄界面1701，并进入触发启用多预览区域显示的特定拍摄模式。其中，特定拍摄模式可以称为辅助显示模式，或者，高倍率变焦拍摄模式，或者，防抖模式等，此处不做限定。
可选的，终端检测到用户对于目标物体进行了预定义的动作后，触发多预览区域的显示。其中，预定义的动作可以是指关节按压，或双击，或单击，或圈定外轮廓等动作。
这里，触发多预览区域的显示可以是直接触发，直接跳转多预览区域显示的GUI。或者，触发多预览区域的显示还可以是间接触发，在满足上述方式中的一种或多种后，可以在终端的显示屏上显示指示信息，用于提示用户选择是否进入多预览区域显示。在用户选择进入多预览区域显示后，再跳转多预览区域显示的GUI。
另外，应理解的是，若采用辅助预览流的GUI显示方式，用户指定目标主体可以在进入特定模式前，也可以在进入特定模式后，即已显示辅助预览流后。若不采用辅助预览流的GUI显示方式，即GUI仅有一个预览区域时，用户指定目标主体需要在进入特定模式前，或者需要在显示对应于低、中倍率数码变焦倍率的画面上指定目标主体，避免因当前预览流中的画面被放大，目标主体丢失，在当前预览流中无法找到目标主体。需要退出特定模式后，或者，在退出高倍率变焦倍率的画面返回低、中倍率数码变焦倍率的画面后，重新确定目标主体，再进入该特定模式。
图18为本申请实施例提供的另一种图像处理方法的流程示意图。应理解，该图像处理方法可以由计算设备或电子设备(例如，终端)执行，也可以由电子设备内的装置(例如，图像处理装置)执行。其中，图像处理装置可以是ISP或SoC。这里，以终端执行为例，展开介绍。如图18所示，该图像处理方法1800包括：
S1801：终端获取待拍摄场景的第一预览图像，该第一预览图像对应于第一拍摄倍率。
S1802：终端在该第一预览图像中确定目标区域。
其中，目标区域可以是规则图形，例如圆形、长方形、正方形等。也可以是不规则图形。
在具体实施过程中，如何确定目标区域有多种可选的实施方法。这里给出几种示例，各示例可以单独或结合实施，但本申请所提供的方案包括但不限于下列示例。
1)可选的，响应于用户对于第一区域的操作，确定第一区域为目标区域。即：由用户直接指定特定区域为目标区域。
具体地，可选的，上述对于第一区域的操作可以是长按、单击、双击、指关节单击、指关节划定区域等动作。
例如，图19为本申请实施例提供的一种拍照的GUI。参见图19中的(a)，在手机启动相机后，在拍照模式下，拍摄界面1901上显示了当前预览图像。在手机检测到用户用手指划定区域1902时，其中，区域1902为不规则形状。显示图19中的(b)，在拍摄界面1903中通过标记框1904来标记该区域1902，标记框1904的形状为或者接近于区域1902的外部轮廓。另外，该拍摄界面1903上还可以显示提示信息，来提示用户确认标记框1904中所圈定的区域是否为用户当前需要拍摄的目标区域。
2)可选的，响应于用户对第一区域的操作，确定包括第一区域的第二区域为目标区域。即：根据用户指定的特定区域，匹配出规则形状的区域为目标区域。
例如，图20为本申请实施例提供的另一种拍照的GUI。参见图20中的(a)，在手机启动相机后，在拍照模式下，拍摄界面2001上显示了当前预览流。在手机检测到用户用手指划定区域2002时，其中，区域2002为不规则形状。显示图20中的(b)，在拍摄界面2003中通过标记框2004来标记目标区域，其中，目标区域包括区域2002。或者，显示图20中的(c)，该拍摄界面2005上除了显示标记框2004外，还可以显示提示信息2006，来提示用户确认标记框2004中所圈定的区域是否为用户当前需要拍摄的目标区域。应理解的是，目标区域的长宽比例可以根据终端的显示规格确定。这里，在具体实施过程中，提示信息的显示方法和用户确认进入防抖模式的方法有多种，包括但不限于以下示例。用户点击提示信息2006选择进入防抖拍摄模式，或者在提示信息2006上向右滑动手指以进入防抖拍摄模式，显示图20中的(d)，其中拍摄界面2007中显示更新后的预览流，以及退出控件2008。当用户点击退出控件2008时，可以退出防抖拍摄模式，返回原来的拍摄模式。或者在提示信息2006上向左滑动手指以退出防抖拍摄模式，避免误触进入防抖拍摄模式，返回原来的拍摄模式。应理解的是，退出控件2008的形状和样式可以在具体实施过程中设计，本申请中不做限定。
3)可选的，检测到第一区域内存在对应于系统预先设置物体类别的物体，则确定第一区域为目标区域。这里，可以参考图10中的S1001的相关表述。
终端在该第一预览图像中确定目标区域，还可以包括：确定出目标区域的位置。
在具体实施过程中，目标区域的位置可以是目标区域的坐标，如目标区域的边界坐标，或，目标区域的中心位置的坐标。在确定目标区域后，终端可以跟踪目标区域的位置。具体地，例如，终端可以根据目标区域在预览流中的第i帧图像的位置，和/或，第i帧图像前后几帧图像的位置，预测预览流中后续几帧中目标区域的位置。终端可以利用预览流中目标区域的图像特征对目标区域进行特征表达，然后，以目标区域在第i帧图像中的位置，和/或第i帧图像前后几帧的位置为起点，在预览流中后续几帧中相近的位置进行图像特征匹配，选择匹配度较高的区域为目标区域在该后续几帧的位置。其中，图像特征包括但不限于下列特征中的一种或多种：颜色、形状、亮度、纹理、角点等。
应理解的是，目标区域中的物体可能会有移动或变化，可选的，终端可以对目标区域的图像分析，基于置信度或相似度等参数确定当前目标区域是否跟踪失败。
可选的，在终端跟踪目标区域失败后，可以在终端的显示屏上提示用户重新确定 目标区域。例如，在终端跟踪目标区域失败后，在终端的显示屏中显示指示消息来示意用户重新确定目标区域。或者，在终端跟踪目标区域失败后，在终端的显示屏中标识出当前终端所预测目标区域最可能出现的位置，在在终端跟踪目标区域失败后，在终端的显示屏中标识出预览流的中心位置，由用户自己指出当前目标区域是否需要修正。如果修正，则根据修正后的目标区域执行后续方法。
可选的，如果跟踪的目标区域移出预览流，终端无法跟踪，则可以在终端的显示屏中标识到中心位置，或者停在目标区域消失前最后所处于预览流中的位置。通过指示信息提示，或由用户自己指出当前目标区域是否需要修正。如果修正，则根据修正后的目标区域执行后续方法。
S1803：终端获取该待拍摄场景的第二原始图像，其中，该第二原始图像为该终端基于目标镜头拍摄该待拍摄场景时，该终端的图像传感器所采集的图像，该目标镜头为该终端在第二拍摄倍率下拍摄该待拍摄场景时所调用的镜头，该第二拍摄倍率大于该第一拍摄倍率。
S1804：终端根据该目标区域和该第二原始图像，确定出第二预览图像，该第二预览图像包括该目标区域。
S1805：在终端的显示屏上显示该第二预览图像。
可选的，图像处理方法1800还可以包括S1806，
S1806：终端响应于用户点击拍摄控件，保存目标图像。
应理解，S1801至S1806的技术细节和有益效果可以参考S1001至S1006中的相关表述。图像处理方法1800与图像处理方法1000的区别包括：
终端识别的是目标区域而非目标主体，但也可以理解为将目标区域的整体作为目标主体。
为了便于理解，下面给出几个具体的示例，各示例可以结合。本申请实施例包括但不限于下列示例。
示例一：终端开启相机后，显示如图12所示的拍照界面1201，其中，拍照界面1201中包括一个预览画面1202，当前终端调用默认的目标镜头(如1倍光学变焦镜头)来拍摄待拍摄场景的图片。用户在使用过程中，调整用户指示的拍摄倍率，例如，从1倍调整至20倍，在用户指示的拍摄倍率超过预设阈值时，终端的显示屏上显示提示消息，该提示消息用于提示终端选择是否进入防抖拍摄模式，并通过标识(例如十字标识)来指示出当前确定的目标主体。用户确定该目标主体的识别正确时，可以选择进入防抖拍摄模式。否则，用户可以通过长按或指关节划定等动作，重新指定目标主体。在用户进入防抖拍摄模式后，拍摄界面中显示根据方法1000或方法1800中得到的第二预览图像，并且在该拍摄界面依然可以在拍摄界面中调整拍摄倍率。若用户发现当前目标主体不在该拍摄界面的预览图像中，则可以通过手势滑动或者点击拍摄界面中的返回控件来退出当前显示的第二预览图像，回到对应数码变焦倍率更低的预览图像中，重新指定目标主体。应理解，预设阈值的取值设定可以由用户预先设置，例如，用户可以在终端系统设置页面中设置。或者，预设阈值的取值设定还可以根据经验值确定。例如，对用户进行调研，确定用户对手部微小抖动无感知的拍摄倍率。将 该倍率的取值确定为预设阈值。
在用户指示拍照时，终端保存目标图像。其中，目标图像与指示拍照时对应的第二预览图像的FOV相同，即，都是基于第二原始图像适应终端的显示规格裁剪后得到的图像，裁剪相同区域得到的，但目标图像与指示拍照时对应的第二预览图像，所对应的图像处理过程可以不同。
示例二：终端开启相机后，显示如图17所示的拍照界面1701，其中，其中，拍照界面1701中包括一个预览画面和一个功能选项1702，当前终端调用默认的目标镜头(如1倍光学变焦镜头)来拍摄待拍摄场景的图像。在拍摄过程中，用户通过功能选项1702进入拍摄模式的选择界面1703，通过点击辅助显示模式(其中，辅助显示模式可以有其他名称，本申请不做限定)，进入如图13所示的多预览区域显示的辅助显示模式，其中，拍摄界面1301中包括第一预览区域1302和第二预览区域1303，第一预览区域1302用于显示辅助预览图像，第二预览区域1303用于显示第二预览图像。用户可以在辅助预览图像中，确定目标主体。在用户确定目标主体后，辅助预览流中，通过标识框或符号标识来标识出目标主体。具体的，在用户未主动确定目标主体时，终端可以通过方法1000中所提到的识别方式，先识别出一个目标主体，作为默认目标主体标识。在拍摄过程中，用户可以重新指定目标主体。
应理解的是，在使用过程中，用户可以通过指令，例如手势指令、语音指令、面部指令等类型的指令，来指示退出上述辅助显示模式。
在用户指示拍照时，终端保存目标图像。其中，目标图像与指示拍照时对应的第二预览图像的FOV相同，即，都是基于第二原始图像适应终端的显示规格裁剪后得到的图像，裁剪相同区域得到的，但目标图像与与指示拍照时对应的第二预览图像，所对应的图像处理过程可以不同。
可以理解的是，保存的目标图像还可以包括辅助预览图像，和/或，基于第二原始图像或辅助原始图像适应于终端的显示规格裁剪后得到的图像，经过拍照图像处理后得到的图像。具体的，是否保存上述图像，可以在用户设置界面通过设置可选项由用户指定。或者，在拍摄界面根据用户的指令选择，例如手势指令(如单击、双击拍照控件对应的目标图像不同)、语音指令、面部指令等类型的指令。
示例三：终端开启相机后，显示如图12所示的拍照界面1201，其中，其中，拍照界面1201中包括一个预览画面和一个功能选项1202，当前终端调用默认的目标镜头(如1倍光学变焦镜头)来拍摄待拍摄场景的图像。在拍摄过程中，用户在使用过程中，调整用户指示的拍摄倍率，例如，从1倍调整至20倍，在用户指示的拍摄倍率超过第一预设阈值时，显示如图13所示的多预览区域。其中，拍摄界面1301中包括第一预览区域1302和第二预览区域1303，第一预览区域1302用于显示辅助预览图像，第二预览区域1303用于显示第二预览图像。用户可以在辅助预览图像中，确定目标主体。在用户确定目标主体后，辅助预览流中，通过标识框或符号标识来标识出目标主体。具体的，在用户未主动确定目标主体时，终端可以通过方法1000中所提到的识别方式，先识别出一个目标主体，作为默认目标主体标识。在拍摄过程中，用户可以重新指定目标主体。在用户指示的拍摄倍率低于第二预设阈值时，拍摄界面恢复至图12 所示的拍照界面1201，其中，拍照界面1201的预览画面为第二预览画面。
应理解，第一预设阈值和第二预设阈值的取值设定可以由用户预先设置，例如，用户可以在终端系统设置页面中设置。或者，第一预设阈值的取值设定还可以根据经验值确定。例如，对用户进行调研，确定用户对手部微小抖动无感知的拍摄倍率，将该倍率的取值确定为第一预设阈值。对用户进行调研，确定用户对辅助预览图像和第二预览图像显示FOV大小相近程度，对应最不能容忍的拍摄倍率，将该倍率确定为第二预设阈值。
图21为本申请实施例提供的一种图像处理装置的结构示意图，该图像处理装置可以计算设备或电子设备(例如，终端)，也可以是电子设备内的装置(例如，ISP或SoC)。并且可以实现如图6或图11所示的图像处理方法以及上述各可选实施例。如图21所示，图像处理装置2100包括：收发单元2101和处理单元2102。
收发单元2101用于执行上述图像处理方法1000中S1001、S1003、上述图像处理方法1800中S1801、S1803中任一步骤以及其中任一可选的示例。处理单元2102，用于执行上述图像处理方法1000中S1002、S1004、S1005、S1006、上述图像处理方法1800中S1802、S1804、S1805、S1806中任一步骤以及其中任一可选的示例。具体参见方法示例中的详细描述，此处不做赘述。
应理解的是，本申请实施例中的图像处理装置可以由软件实现，例如，具有上述功能的计算机程序或指令来实现，相应计算机程序或指令可以存储在终端内部的存储器中，通过处理器读取该存储器内部的相应计算机程序或指令来实现上述功能。或者，本申请实施例中的图像处理装置还可以由硬件来实现。其中处理单元2102为处理器(如NPU、GPU、系统芯片中的处理器)，收发单元2101为收发电路或接口电路。或者，本申请实施例中的图像处理装置还可以由处理器和软件模块的结合实现。
应理解，本申请实施例中的装置处理细节可以参考图10、图18的相关表述，本申请实施例将不再重复赘述。
图22为本申请实施例提供的另一种图像处理装置的结构示意图，该图像处理装置可以计算设备或电子设备(例如，终端)，也可以是电子设备内的装置(例如，ISP或SoC)。并且可以实现如图6或图11所示的图像处理方法以及上述各可选实施例。如图22所示，图像处理装置2200包括：处理器2201，与处理器1001耦合的接口电路2202。应理解，虽然图22中仅示出了一个处理器和一个接口电路。图像处理装置2200可以包括其他数目的处理器和接口电路。
其中，接口电路2202用于与终端的其他组件连通，例如存储器或其他处理器。处理器2201用于通过接口电路2202与其他组件进行信号交互。接口电路2202可以是处理器2201的输入/输出接口。
例如，处理器2201通过接口电路2202读取与之耦合的存储器中的计算机程序或指令，并译码和执行这些计算机程序或指令。应理解，这些计算机程序或指令可包括上述终端功能程序，也可以包括上述应用在终端内的图像处理装置的功能程序。当相 应功能程序被处理器2201译码并执行时，可以使得终端或在终端内的图像处理装置实现本申请实施例所提供的图像处理方法中的方案。
可选的，这些终端功能程序存储在图像处理装置2200外部的存储器中。当上述终端功能程序被处理器2201译码并执行时，存储器中临时存放上述终端功能程序的部分或全部内容。
可选的，这些终端功能程序存储在图像处理装置2200内部的存储器中。当图像处理装置2200内部的存储器中存储有终端功能程序时，图像处理装置2200可被设置在本发明实施例的终端中。
可选的，这些终端功能程序的部分内容存储在图像处理装置2200外部的存储器中，这些终端功能程序的其他部分内容存储在图像处理装置2200内部的存储器中。
应理解，图1至图2、图21至图22任一所示的图像处理装置可以互相结合，图1至图2、图21至图22任一所示的图像处理装置以及各可选实施例相关设计细节可互相参考，也可以参考图10或图18任一所示的图像处理方法以及各可选实施例相关设计细节。此处不再重复赘述。
应理解，图10或图18任一所示的图像处理方法以及各可选实施例，图1至图2、图21至图22任一所示的图像处理装置以及各可选实施例，不仅可以用于在拍摄中处理视频或图像，还可以用于处理已经拍摄完成的视频或图像。本申请不做限定。
实施例及附图中的术语“第一”、“第二”、“第三”、“第四”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。此外，术语“包括”和“具有”以及他们的任何变形，意图在于表示不排他的包含，例如，包含了一系列步骤或单元。方法、系统、产品或设备不必仅限于字面列出的那些步骤或单元，而是可包括没有字面列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，吧b，c可以是单个，也可以是多个。
应理解，在本申请中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。本申请提到的“耦合”一词，用于表达不同组件之间的互通或互相作用，可以包括直接相连或通过其他组件间接相连。
在本申请的上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可 以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤等)或无线(例如红外、无线电、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，例如，软盘、硬盘和磁带；可以是光介质，例如DVD；也可以是半导体介质，例如固态硬盘(Solid State Disk，SSD)等。
本申请实施例中，存储器，是指具有数据或信息存储能力的器件或电路，并可向处理器提供指令和数据。存储器包括只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、非易失性随机存取存储器(NVRAM)，可编程只读存储器或者电可擦写可编程存储器、寄存器等。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。
需要说明的是，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。

Claims (28)

  1. 一种应用于终端的图像处理方法，其特征在于，所述方法包括：
    获取待拍摄场景的第一预览图像，所述第一预览图像对应于第一拍摄倍率；
    在所述第一预览图像中确定目标主体；
    获取所述待拍摄场景的第二原始图像，所述第二原始图像为所述终端基于目标镜头拍摄所述待拍摄场景时，所述终端的图像传感器所采集得到的图像，所述目标镜头为所述终端在第二拍摄倍率下拍摄所述待拍摄场景时所调用的镜头，所述第二拍摄倍率大于所述第一拍摄倍率；
    根据所述目标主体和所述第二原始图像，确定出第二预览图像，所述第二预览图像包括所述目标主体；
    在所述终端的显示屏上显示所述第二预览图像。
  2. 根据权利要求1所述的图像处理方法，其特征在于，所述根据所述目标主体和所述第二原始图像，确定出第二预览图像，具体包括：
    根据所述目标主体在所述第二原始图像中的位置，以及所述第二拍摄倍率和所述第二原始图像，确定出所述第二预览图像。
  3. 根据权利要求1至2任一所述的图像处理方法，其特征在于，在所述终端的显示屏上显示所述第二预览图像之后，所述方法还包括：
    接收对于所述第二预览图像中的所述目标主体的第一操作，其中，所述第一操作用于确定所述目标主体在新的预览图像中的目标位置；
    获取所述待拍摄场景的第三原始图像，所述第三原始图像为所述终端基于目标镜头拍摄待拍摄场景时，所述终端的传感器所采集到的图像，所述第二原始图像的采集时间早于所述第三原始图像；
    根据所述目标主体在新的预览图像中的目标位置，以及所述第三原始图像，确定出所述第三预览图像，所述第三预览图像包括所述目标主体；
    在所述终端的显示屏上显示所述第三预览图像。
  4. 根据权利要求1至3任一所述的图像处理方法，其特征在于，所述方法还包括：
    在所述第二拍摄倍率大于预设阈值时，在所述终端的显示屏上显示第一预览区域和第二预览区域，其中，所述第一预览区域用于显示辅助预览图像，所述第二预览区域用于显示所述第二预览图像，所述辅助预览图像包括所述第二预览图像的内容。
  5. 根据权利要求1至4任一所述的图像处理方法，其特征在于，所述辅助预览图像是基于所述第二原始图像确定的。
  6. 根据权利要求1至4任一所述的图像处理方法，其特征在于，所述辅助预览图像基于辅助原始图像确定，其中，所述辅助原始图像与所述第二原始图像为所述终端调用不同镜头，拍摄所述待拍摄场景时，所述终端的图像传感器所采集到的不同图像。
  7. 根据权利要求1至6任一所述的图像处理方法，其特征在于，所述第一预览图像包括第一物体，所述在所述第一预览图像中确定目标主体，具体包括：
    在所述第一物体为系统预设物体时，确定所述第一物体为目标主体。
  8. 根据权利要求1至6任一所述的图像处理方法，其特征在于，所述第一预览图像包括第一物体，所述在所述第一预览图像中确定目标主体，具体包括：
    接收对于所述第一预览图像中的所述第一物体的第二操作；
    响应于所述第二操作，确定所述第一物体为目标主体。
  9. 根据权利要求1至6任一所述的图像处理方法，其特征在于，所述在所述第一预览图像中确定目标主体，具体包括：
    接收对于所述第一预览图像中的第一区域的第三操作；
    响应于所述第三操作，确定所述第一区域中的物体为目标主体。
  10. 根据权利要求1至9任一所述的图像处理方法，其特征在于，所述目标主体在所述第二预览图像中，通过标识框，和/或，符号标识来标记。
  11. 根据权利要求1至10任一所述的图像处理方法，其特征在于，所述第二拍摄倍率为用户指定的拍摄倍率。
  12. 根据权利要求1至11任一所述的图像处理方法，其特征在于，所述目标主体位于所述第二预览图像中除中心位置以外的其他位置。
  13. 一种应用于终端的图像处理装置，其特征在于，所述图像处理装置包括：收发单元以及处理单元，
    所述收发单元，用于获取待拍摄场景中的第一预览图像，所述第一预览图像对应于所述第一拍摄倍率；
    所述处理单元，用于在所述第一预览图像中确定目标主体；
    所述收发单元，还用于获取所述待拍摄场景的第二原始图像，所述第二原始图像为所述终端基于目标镜头拍摄所述待拍摄场景时，所述终端的图像传感器所采集得到的图像，所述目标镜头为所述终端在第二拍摄倍率下拍摄所述待拍摄场景时所调用的镜头，所述第二拍摄倍率大于所述第一拍摄倍率；
    所述处理单元，还用于根据所述目标主体和所述第二原始图像，确定出第二预览图像，所述第二预览图像包括所述目标主体；
    所述处理单元，还用于使能所述终端的显示屏显示所述第二预览图像。
  14. 根据权利要求13所述的图像处理装置，其特征在于，所述处理单元，还用于根据所述目标主体和所述第二原始图像，确定出第二预览图像，具体包括：
    所述处理单元，还用于根据所述目标主体在所述第二原始图像中的位置、所述第二拍摄倍率和所述第二原始图像，确定出所述第二预览图像。
  15. 根据权利要求13至14任一所述的图像处理装置，其特征在于，
    所述收发单元，还用于接收对于所述第二预览图像中的所述目标主体的第一操作，所述第一操作用于确定所述目标主体在新的预览图像中的位置；
    所述收发单元，还用于获取所述待拍摄场景的第三原始图像，所述第三原始图像为所述终端基于目标镜头拍摄待拍摄场景时，所述终端的传感器所采集到的图像，所述第二原始图像的采集时间早于所述第三原始图像；
    所述处理单元，还用于根据所述确定的所述目标主体在所述第三预览图像中的目标位置，以及所述第三原始图像，确定出所述第三预览图像，所述第三预览图像包括所述目标主体；
    所述处理单元，还用于使能所述终端的显示屏显示所述第三预览图像。
  16. 根据权利要求13至15任一所述的图像处理装置，其特征在于，所述处理单元还用于在所述第二拍摄倍率大于预设阈值时，使能所述终端的显示屏上显示第一预览区域和第二预览区域，其中，所述第一预览区域用于显示辅助预览图像，所述第二预览区域用于显示所述第二预览图像，所述辅助预览图像包括所述第二预览图像的内容。
  17. 根据权利要求13至16任一所述的图像处理装置，其特征在于，所述辅助预览图像是基于所述第二原始图像确定的。
  18. 根据权利要求13至16任一所述的图像处理装置，其特征在于，所述辅助预览图像基于辅助原始图像确定，其中，所述辅助原始图像与所述第二原始图像为所述终端调用不同镜头，拍摄所述待拍摄场景时，所述终端的图像传感器所采集到的不同图像。
  19. 根据权利要求13至18任一所述的图像处理装置，其特征在于，所述第一预览图像包括第一物体，所述处理单元，用于在所述第一预览图像中确定目标主体，具体包括：
    所述处理单元，用于在所述第一物体为系统预设物体时，确定所述第一物体为目标主体。
  20. 根据权利要求13至18任一所述的图像处理装置，其特征在于，所述第一预览图像包括第一物体，所述收发单元，用于接收对于所述第一预览图像中的所述第一物体的第二操作；所述处理单元，用于在所述第一预览图像中确定目标主体，具体包括：
    所述处理单元，用于响应于所述第二操作，确定所述第一物体为目标主体。
  21. 根据权利要求13至18任一所述的图像处理装置，其特征在于，所述收发单元，用于接收对于所述第一预览图像中的第一区域的第三操作；
    所述处理单元，用于在所述第一预览图像中确定目标主体，具体包括：
    处理单元，用于响应于所述第三操作，确定所述第一区域中的物体为目标主体。
  22. 根据权利要求13至21任一所述的图像处理装置，其特征在于，所述目标主体在所述第二预览图像中，通过标识框，和/或，符号标识来标记。
  23. 根据权利要求13至22任一所述的图像处理装置，其特征在于，所述第二拍摄倍率为用户指定的拍摄倍率。
  24. 根据权利要求13至23任一所述的图像处理装置，其特征在于，所述目标主体位于所述第二预览图像中除中心位置以外的其他位置。
  25. 一种终端，其特征在于，触摸屏，其中，所述触摸屏包括触敏表面和显示器；摄像头；一个或多个处理器；存储器；多个应用程序；以及一个或多个计算机程序，其中所述一个或多个计算机程序被存储在所述存储器中，所述一个或多个计算机程序包括指令，当所述指令被所述终端执行时，使得所述终端执行权利要求1至12任一所述的图像处理方法。
  26. 一种电子设备，包括触摸屏，存储器，一个或多个处理器，多个应用程序，以及一个或多个程序；其中所述一个或多个程序被存储在所述存储器中；其特征在于，所述一个或多个处理器在执行所述一个或多个程序时，使得所述电子设备实现如权利要求1至12中任一项所述的图像处理方法。
  27. 一种计算机存储介质，其特征在于，包括计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行如权利要求1至12中任一所述的图像处理方法。
  28. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1至12中任一所述的图像处理方法。
PCT/CN2021/081094 2020-03-20 2021-03-16 图像处理方法及装置 WO2021185250A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21772093.7A EP4117272A4 (en) 2020-03-20 2021-03-16 IMAGE PROCESSING METHOD AND APPARATUS
US17/948,397 US20230019164A1 (en) 2020-03-20 2022-09-20 Image Processing Method and Apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010202272.5 2020-03-20
CN202010202272.5A CN113497881B (zh) 2020-03-20 2020-03-20 图像处理方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/948,397 Continuation US20230019164A1 (en) 2020-03-20 2022-09-20 Image Processing Method and Apparatus

Publications (1)

Publication Number Publication Date
WO2021185250A1 true WO2021185250A1 (zh) 2021-09-23

Family

ID=77768035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/081094 WO2021185250A1 (zh) 2020-03-20 2021-03-16 图像处理方法及装置

Country Status (4)

Country Link
US (1) US20230019164A1 (zh)
EP (1) EP4117272A4 (zh)
CN (2) CN113497881B (zh)
WO (1) WO2021185250A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117272654A (zh) * 2023-09-26 2023-12-22 重庆交通大学 小客车专用高速公路限速标志位置设置方法
US11956530B2 (en) * 2020-04-17 2024-04-09 Samsung Electronics Co., Ltd. Electronic device comprising multi-camera, and photographing method
WO2024076362A1 (en) * 2022-10-04 2024-04-11 Google Llc Stabilized object tracking at high magnification ratios

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116939363B (zh) * 2022-03-29 2024-04-26 荣耀终端有限公司 图像处理方法与电子设备
CN114640798B (zh) * 2022-05-09 2022-10-04 荣耀终端有限公司 图像处理方法、电子设备及计算机存储介质
CN116051368B (zh) * 2022-06-29 2023-10-20 荣耀终端有限公司 图像处理方法及其相关设备
CN115103126B (zh) * 2022-07-22 2024-06-04 维沃移动通信有限公司 拍摄预览方法、装置、电子设备及存储介质
CN118102099B (zh) * 2024-04-17 2024-07-23 江西省自然资源事业发展中心 一种自然资源监管视频拍摄防抖方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106909274A (zh) * 2017-02-27 2017-06-30 努比亚技术有限公司 一种图像显示方法和装置
US20180069983A1 (en) * 2016-09-06 2018-03-08 Lg Electronics Inc. Terminal and controlling method thereof
CN108134903A (zh) * 2017-12-25 2018-06-08 广东欧珀移动通信有限公司 拍摄方法及相关产品
CN108391035A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种拍摄方法、装置与设备
CN110365897A (zh) * 2019-07-08 2019-10-22 Oppo广东移动通信有限公司 图像修正方法和装置、电子设备、计算机可读存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4730569B2 (ja) * 2009-03-27 2011-07-20 カシオ計算機株式会社 撮影装置、撮像方法、及びプログラム
JP6103526B2 (ja) * 2013-03-15 2017-03-29 オリンパス株式会社 撮影機器，画像表示機器，及び画像表示機器の表示制御方法
EP3018528A1 (en) * 2014-11-04 2016-05-11 Jia-Tin-Chang Co., Ltd. Pivotable camera module and electronic device having the camera module
CN110769149B (zh) * 2015-04-23 2021-05-11 苹果公司 用于处理来自多相机的内容的方法、电子设备和存储介质
CN106937045B (zh) * 2017-02-23 2020-08-14 华为机器有限公司 一种预览图像的显示方法、终端设备及计算机存储介质
CN109002787B (zh) * 2018-07-09 2021-02-23 Oppo广东移动通信有限公司 图像处理方法和装置、存储介质、电子设备
CN112954219A (zh) * 2019-03-18 2021-06-11 荣耀终端有限公司 一种多路录像方法及设备
CN110445978B (zh) * 2019-06-24 2020-12-15 华为技术有限公司 一种拍摄方法及设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180069983A1 (en) * 2016-09-06 2018-03-08 Lg Electronics Inc. Terminal and controlling method thereof
CN106909274A (zh) * 2017-02-27 2017-06-30 努比亚技术有限公司 一种图像显示方法和装置
CN108134903A (zh) * 2017-12-25 2018-06-08 广东欧珀移动通信有限公司 拍摄方法及相关产品
CN108391035A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种拍摄方法、装置与设备
CN110365897A (zh) * 2019-07-08 2019-10-22 Oppo广东移动通信有限公司 图像修正方法和装置、电子设备、计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4117272A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11956530B2 (en) * 2020-04-17 2024-04-09 Samsung Electronics Co., Ltd. Electronic device comprising multi-camera, and photographing method
WO2024076362A1 (en) * 2022-10-04 2024-04-11 Google Llc Stabilized object tracking at high magnification ratios
CN117272654A (zh) * 2023-09-26 2023-12-22 重庆交通大学 小客车专用高速公路限速标志位置设置方法

Also Published As

Publication number Publication date
CN113497881B (zh) 2022-11-08
CN113497881A (zh) 2021-10-12
EP4117272A4 (en) 2023-07-19
CN115734071A (zh) 2023-03-03
US20230019164A1 (en) 2023-01-19
EP4117272A1 (en) 2023-01-11

Similar Documents

Publication Publication Date Title
WO2021093793A1 (zh) 一种拍摄方法及电子设备
WO2021185250A1 (zh) 图像处理方法及装置
WO2020073959A1 (zh) 图像捕捉方法及电子设备
CN112714901B (zh) 系统导航栏的显示控制方法、图形用户界面及电子设备
CN115866121B (zh) 应用界面交互方法、电子设备和计算机可读存储介质
WO2020134869A1 (zh) 电子设备的操作方法和电子设备
WO2021000881A1 (zh) 一种分屏方法及电子设备
US20230007186A1 (en) Video Shooting Method and Electronic Device
WO2021104485A1 (zh) 一种拍摄方法及电子设备
WO2021129198A1 (zh) 一种长焦场景下的拍摄方法及终端
WO2021143269A1 (zh) 一种长焦场景下的拍摄方法及移动终端
WO2020238759A1 (zh) 一种界面显示方法和电子设备
WO2021238370A1 (zh) 显示控制方法、电子设备和计算机可读存储介质
WO2022262475A1 (zh) 拍摄方法、图形用户界面及电子设备
CN115967851A (zh) 快速拍照方法、电子设备及计算机可读存储介质
CN112150499A (zh) 图像处理方法及相关装置
WO2022206589A1 (zh) 一种图像处理方法以及相关设备
WO2022002213A1 (zh) 翻译结果显示方法、装置及电子设备
CN114173005B (zh) 一种应用布局控制方法、装置、终端设备及计算机可读存储介质
CN116709018B (zh) 一种变焦条分割方法及电子设备
WO2023160224A9 (zh) 一种拍摄方法及相关设备
CN118331469A (zh) 一种截图方法
CN116521033A (zh) 图片分享或分类方法、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21772093

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021772093

Country of ref document: EP

Effective date: 20221005

NENP Non-entry into the national phase

Ref country code: DE