WO2022007500A1 - 图像的处理方法、装置、电子设备、可读介质及移动终端 - Google Patents

图像的处理方法、装置、电子设备、可读介质及移动终端 Download PDF

Info

Publication number
WO2022007500A1
WO2022007500A1 PCT/CN2021/093706 CN2021093706W WO2022007500A1 WO 2022007500 A1 WO2022007500 A1 WO 2022007500A1 CN 2021093706 W CN2021093706 W CN 2021093706W WO 2022007500 A1 WO2022007500 A1 WO 2022007500A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
magnification
offset
roi
current
Prior art date
Application number
PCT/CN2021/093706
Other languages
English (en)
French (fr)
Inventor
刘钦
Original Assignee
北京迈格威科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京迈格威科技有限公司 filed Critical 北京迈格威科技有限公司
Publication of WO2022007500A1 publication Critical patent/WO2022007500A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to an image processing method, apparatus, electronic device, readable medium, and mobile terminal.
  • the camera device is usually a fixed-focus lens, which can only achieve digital zoom.
  • the camera device is usually a fixed-focus lens, which can only achieve digital zoom.
  • more and more mobile terminals are equipped with two cameras in the industry.
  • the shooting performance is improved by combining digital zoom with optical zoom to satisfy consumers.
  • the combination of digital zoom and optical zoom means that one camera device is used to enlarge the image, and then switch to another camera device for imaging.
  • the above method is also called optical digital combined zoom.
  • each camera device has a certain error during assembly, and its actual focal length is also different from the theoretical reference value given by the lens manufacturer; and for the optical axis, the two camera devices are assembled in At the same time, the optical axis will also have errors.
  • the optical axes of the two camera devices are not absolutely parallel, resulting in inconsistent directions observed by the two camera devices. Bad user experience.
  • the Baseline baseline, which can also be understood as a distance
  • the Baseline will cause parallax between the two cameras, and the parallax will cause the image content to change significantly when the two cameras are switched.
  • the purpose of the present disclosure is to provide an image processing method, apparatus, electronic device, and computer-readable medium, so as to alleviate the technical problem of image content jumping during imaging switching of a camera in the prior art.
  • an embodiment of the present disclosure provides an image processing method, which is applied to a mobile terminal, where the mobile terminal includes: a first camera device and a second camera device, and the method includes: acquiring the first camera device The collected first image; and determine the target ROI area in the first image; calculate the ROI offset of the target ROI area in the first image and the second image, wherein the second image is the image captured by the second camera device synchronously when the first camera device collects the first image; obtains the current magnification of the first image, and slices the ROI offset according to the current magnification processing, to obtain the current superimposed offset for performing image stereoscopic correction on the first image at the current moment; performing image stereoscopic correction on the first image based on the current superimposed offset to obtain the corrected first image ; if the current magnification is a preset switching magnification, the mobile terminal is switched from the first camera device to the second camera device for display; wherein, when switching, the first image is imaged The sum of the stereo corrected overlay offsets is the ROI offset
  • the method further includes: performing texture detection on the image located in the target ROI area in the first image to obtain texture intensity; calculating the texture intensity of the target ROI area in the first image and the second image
  • the ROI offset includes: if it is determined that the texture intensity of the image located in the target ROI area in the first image is greater than the preset texture intensity, calculating the target ROI area in the first image and the second image. The ROI offset of the target ROI area described in .
  • performing texture detection on the image located in the target ROI area in the first image, and obtaining the texture intensity includes: calculating the target value based on the pixel points located in the target ROI area in the first image, and The texture intensity is determined based on the target value, wherein the texture intensity includes a standard deviation calculated from any of the following values: pixel gray value, pixel RGB value.
  • calculating the ROI offset of the target ROI area in the first image and the second image includes: determining the image feature points located in the target ROI area in the first image, and obtaining the first image feature point; determine the matching feature points of the first image feature points in the second image to obtain the second image feature points; based on the pixel distance between the first image feature points and the second image feature points , and determine the ROI offset.
  • performing slicing processing on the ROI offset according to the current magnification, and obtaining the current superimposed offset for performing image stereoscopic correction on the first image at the current moment includes: acquiring a target magnification, wherein the The target magnification includes: an initial magnification and a preset switching magnification; the initial magnification is the magnification corresponding to when the first camera device performs a digital zoom operation on the first image when the ROI offset is determined for the first time;
  • the preset switching magnification represents the image magnification when the first camera is switched to the second camera; the ROI is offset according to the size relationship between the current magnification and the target magnification Slicing is performed to obtain the current stacking offset of the image stereoscopic correction performed on the first image at the current moment.
  • the ROI offset is sliced according to the size relationship between the current magnification and the target magnification, so as to obtain the current superimposed offset of the image stereoscopic correction performed on the first image at the current moment.
  • the method includes: if the current magnification is smaller than the initial magnification, determining that the current stacking offset is 0.
  • the ROI offset is sliced according to the size relationship between the current magnification and the target magnification, so as to obtain the current superimposed offset of the image stereoscopic correction performed on the first image at the current moment. It also includes: if the current magnification is greater than the initial magnification and smaller than the preset switching magnification, the calculation formula of the current overlay offset is: Wherein, s cur represents the current overlay offset, S ROI represents the ROI offset, ul cur represents the current magnification, ul0 represents the initial magnification of the first image, and s1 represents the preset Switch the magnification.
  • the ROI offset is sliced according to the size relationship between the current magnification and the target magnification, so as to obtain the current superimposed offset of the image stereoscopic correction performed on the first image at the current moment. It also includes: if the current magnification is greater than the preset switching magnification, the current overlay offset is the ROI offset.
  • performing image stereo correction on the first image based on the current overlay offset, and obtaining the corrected first image includes: constructing a target transformation matrix based on the current overlay offset and an image enlargement parameter;
  • the image enlargement parameters include the center point of the first image and a preset switching magnification; calculate the product between the target transformation matrix and the homogeneous coordinates of the first image at the current moment, and determine according to the product calculation result The first image after image stereo correction is performed.
  • the method further includes: after calculating the ROI offset, detecting whether the ROI offset has changed according to a preset time interval; if it is detected that the ROI offset has changed, Then determine the target ROI offset based on the changed ROI offset and the current overlay offset of the first image; obtain a first magnification, wherein the first magnification is the detection of the ROI offset The magnification of the first image when the amount occurs; re-determine the current overlay offset of the first image according to the first magnification and the target ROI offset, and re-determine the current overlay offset according to the re-determined current overlay offset. Perform image stereo correction on the first image by the shift amount.
  • redetermining the current overlay offset of the first image according to the first magnification and the target ROI offset includes: if the first magnification and the second magnification are the same, then the The current stacking offset of the first image is the same as the stacking offset at the previous moment, wherein the second magnification is the image magnification corresponding to the stereoscopic correction operation of the image at the previous moment.
  • redetermining the current overlay offset of the first image according to the first magnification and the target ROI offset includes: if the first magnification and the second magnification are different, Then, the current overlay offset of the first image is determined according to the first magnification and the offset of the target ROI.
  • an embodiment of the present disclosure provides an image processing device, which is provided in a mobile terminal, where the mobile terminal includes: a first camera device and a second camera device, and the device includes: a first acquisition unit configured to acquiring a first image collected by the first camera device; and determining a target ROI region in the first image; a computing unit configured to calculate the ROI of the target ROI region in the first image and the second image an offset, wherein the second image is an image captured by the second camera device synchronously when the first camera device captures the first image; the second acquisition unit is configured to acquire an image of the first image At the current magnification, the ROI offset is sliced according to the current magnification, so as to obtain the current superimposed offset of the image stereoscopic correction of the first image at the current moment; the image stereoscopic correction unit is configured to be based on the The current overlay offset is used to perform image stereoscopic correction on the first image to obtain the corrected first image; the switching display unit is configured so that if the current magnification is
  • the slice processing unit is further configured to: acquire a target magnification, wherein the target magnification includes: an initial magnification and a preset switching magnification; the initial magnification is when the ROI offset is determined for the first time , the corresponding magnification when the first camera device performs a digital zoom operation on the first image; the preset switching magnification represents the image magnification when the first camera device is switched to the second camera device; according to the The ROI offset is sliced according to the size relationship between the current magnification and the target magnification, so as to obtain the current superimposed offset of the image stereoscopic correction performed on the first image at the current moment.
  • the target magnification includes: an initial magnification and a preset switching magnification
  • the initial magnification is when the ROI offset is determined for the first time , the corresponding magnification when the first camera device performs a digital zoom operation on the first image
  • the preset switching magnification represents the image magnification when the first camera device is switched to the second camera device
  • the device further includes a recalibration unit, the recalibration unit is configured to: after calculating the ROI offset, detect whether the ROI offset has changed according to a preset time interval; If it is found that the ROI offset has changed, the target ROI offset is determined based on the changed ROI offset and the current superimposed offset of the first image; the first magnification is obtained, wherein the first magnification is A magnification is the magnification of the first image when the occurrence of the ROI offset is detected; the current overlay offset of the first image is re-determined according to the first magnification and the target ROI offset and performing image stereoscopic correction on the first image according to the re-determined current stacking offset.
  • the recalibration unit is configured to: after calculating the ROI offset, detect whether the ROI offset has changed according to a preset time interval; If it is found that the ROI offset has changed, the target ROI offset is determined based on the changed ROI offset and the current superimposed offset of the first image; the first magnification is obtained, wherein the first magnification
  • an embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program Implement the steps of the method of any one of the first aspects above.
  • an embodiment of the present disclosure provides a computer-readable medium having a non-volatile program code executable by a processor, the program code causing the processor to execute any one of the foregoing first aspects. steps of the method.
  • an embodiment of the present disclosure provides a mobile terminal, including a processor, and a first camera device and a second camera device respectively connected to the processor, and the processor executes any one of the foregoing first aspects. The steps of the method described in item.
  • FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of the principle of image correction of a dual-camera device according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of a digital zoom operation according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a distance between dual-camera devices according to an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of a change in the distance between a parallax and a dual-camera device according to an embodiment of the present disclosure
  • FIG. 6 is a flowchart of an image processing method according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of the principle of texture detection in a target ROI region according to an embodiment of the present disclosure.
  • FIG. 8 is a flowchart of another image processing method according to an embodiment of the present disclosure.
  • FIG. 9 is a flowchart of still another image processing method according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure.
  • an electronic device 100 for implementing an embodiment of the present disclosure is described with reference to FIG. 1 , and the electronic device can be used to execute the image processing method provided by each embodiment of the present disclosure.
  • electronic device 100 includes one or more processing devices 102, one or more storage devices 104, input devices 106, output devices 108, and camera devices 110, these components being communicated via a bus system 112 and/or other forms of A connection mechanism (not shown) interconnects. It should be noted that the components and structures of the electronic device 100 shown in FIG. 1 are only exemplary and not restrictive, and the electronic device may also have other components and structures as required.
  • the processing device 102 may be a Central Processing Unit (CPU) or other form of processing unit with data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired operations. function.
  • CPU Central Processing Unit
  • the processing device 102 may be a Central Processing Unit (CPU) or other form of processing unit with data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired operations. function.
  • CPU Central Processing Unit
  • the storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (Random Access Memory, RAM) and/or cache memory (cache).
  • the non-volatile memory may include, for example, a read-only memory (Read-Only Memory, ROM), a hard disk, a flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present disclosure described below. and/or other desired functionality.
  • Various application programs and various data such as various data used and/or generated by the application program, etc. may also be stored in the computer-readable storage medium.
  • the input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.
  • the output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.
  • the camera device 110 may capture images desired by the user (eg, photos, videos, etc.), and store the captured images in the storage device 104 for use by other components.
  • the example electronic device for implementing the image processing method according to the embodiment of the present disclosure may be implemented on a mobile terminal such as a smart phone, a tablet computer, a wearable electronic device, and the like.
  • the digital zoom operation is performed at this time.
  • the scale of the enlarged image content is the same as that of the FOV image content captured by the camera with smaller FOV, switch to another camera for imaging, and the user continues to enlarge the preview image displayed on the screen, and then continues to execute Digital zoom operation.
  • the optical zoom operation At the moment of switching the camera, it is the optical zoom operation.
  • the digital zoom ensures a smooth and natural magnification process, close to the experience of real optical zoom in the process of magnifying image content;
  • the Baseline (baseline, which can also be understood as a distance) of the two cameras is required to be as small as possible: the Baseline will cause parallax between the two cameras, and the parallax will make the two cameras When the camera device is switched, its content jumps, and its principle will be explained later;
  • the optical axis is required to be parallel, that is, the direction of the optical axis captured by the two camera devices is the same;
  • each camera device has a certain error when assembling, and its actual focal length is also different from the theoretical reference value given by the lens manufacturer; and for the optical axis, the two camera devices are assembled in At the same time, the optical axis will also produce errors.
  • the optical axes of the two camera devices are not absolutely parallel, resulting in inconsistent directions observed by the two camera devices. Bad user experience. Therefore, the "optical digital joint zoom" algorithm usually requires the factory to calibrate the two camera devices in the production of mobile phones, and calculate the accurate internal and external parameters of the camera device, wherein the internal parameters include the true focal length of the lens; the external parameters include the camera. The optical axis angle between the devices, and then the calibrated data is stored in the mobile phone chip.
  • the mobile phone When the mobile phone is used by the user after leaving the factory, such as when the user uses the mobile phone to perform photography, videography, etc., the mobile phone can read the data stored in advance in the The calibration data in the chip is used to obtain relevant information, and transformation operations such as stereo rotation and scale transformation are performed on the image. This image transformation operation can be called image stereo correction, and finally an image with parallel optical axes is obtained.
  • FIG. 2 The schematic diagram of stereo correction is shown in Figure 2, in which the rhombus-shaped plane is the picture actually captured by two non-parallel cameras, and the rectangular-shaped plane is the absolutely parallel picture corrected by the algorithm using the calibration data. Among them, once the optical axes are parallel, the corresponding imaging contents before and after the switching of the camera device will not generate a sense of rotation and unnecessary translation due to the Baseline.
  • H zoom is a 3*3 image magnification matrix, s is the magnification, (cx, cy) is the image center; I input is the image before digital zoom, I output is the image after digital zoom, (u, v, 1) is the homogeneous coordinate of the input image, (u', v', 1) is the homogeneous coordinate of the output image.
  • Function 2 Switching logic.
  • the camera device Wide is a camera device with a large FOV
  • the camera device Tele is a camera device with a small FOV.
  • the camera device Wide is enlarged to a FOV equal to the FOV of the camera device Tele, it is switched to the camera device Tele.
  • switch level 2. Its calculation formula is as follows:
  • Tele refers to a camera with a small FOV
  • Wide refers to a camera with a large FOV
  • f is the focal length
  • width is the width of the sensor of the camera. If these data are calculated by calibration, the accuracy of the switch level will be higher.
  • the digital zoom is performed by the camera device with a large FOV; otherwise, the digital zoom is performed by the camera device with a small FOV.
  • Function 3 Eliminate rotation. In order to make the video image as smooth as possible when switching the camera device, it is necessary to perform stereo correction on the image based on the calibration data to eliminate the sense of rotation and jump caused by the optical axis error.
  • the transformation formula is as follows:
  • H is a transformation matrix
  • K is an internal parameter of the camera device obtained by calibration
  • R is a rotation angle obtained by calibration.
  • Figure 4 shows the top view of the two cameras after stereo correction: the two "bold line segments in the X-axis direction" represent the sensor projection surfaces of the two cameras, O R and O T represent the optical center of the lens, The distance between the two optical centers is Baseline. At this time, the optical axes of the two cameras are absolutely parallel, and the focus is the same (the scale is the same).
  • a point P in space, the distance from the dual-camera system is Z
  • the two image planes are respectively imaged as p and p'
  • the left end of the "bold line segment in the X-axis direction" is the coordinate starting point, p
  • the coordinate of is x R
  • the coordinate of p' is x T
  • D (x R -x T ) the disparity of the point P in the space in the dual-camera system.
  • the parallax D changes with the change of the distance Z. When Z is smaller, that is, the distance is closer, the parallax is larger; otherwise, the parallax is smaller, and the parallax is 0 when the distance is infinite.
  • the relationship can be represented by FIG. 5 .
  • each pixel in the image will show different degrees of parallax jump due to the different distances between the imaged spatial point and the camera ( As shown in point P in Fig. 4, when switching from the camera on the left to the camera on the right, P is in the image and will jump from point p to point p'). Only objects that are very far away (approximately infinity) in the scene will not produce jumps; and the closer the objects are, the more obvious the jumps in parallax will be.
  • a mobile phone uses a mobile phone to take pictures, in addition to the distant scenery such as the distant scenery, he will focus on the distance.
  • an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and, Although a logical order is shown in the flowcharts, in some cases steps shown or described may be performed in an order different from that herein.
  • FIG. 6 is a flowchart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 6 , the method includes the following steps:
  • Step S602 acquiring a first image collected by the first camera device; and determining a target ROI region in the first image.
  • the first camera device here and the second camera device described in the following steps are camera devices in a mobile terminal (eg, a mobile phone), wherein the fields of view of the first camera device and the second camera device are The angles are different, for example, the field of view of the first camera is larger than the field of view of the second camera.
  • the first camera device and the second camera device may be dual-camera devices in a dual-camera mobile phone.
  • the target ROI region refers to the region of interest of the user.
  • the mobile terminal usually when the user clicks a certain position on the screen to focus, it means that the user wants to pay attention to this area, so the mobile terminal will use this area as the target ROI area. That is, when the user manually focuses, the ROI area can be determined based on the user's touch position on the screen. If the user does not focus manually, the mobile terminal will automatically use the central area of the screen as the target ROI area, or automatically detect the position of the target object such as a face on the screen, and determine the target ROI area according to the detection results.
  • the ROI area may be the ROI area of the Wide lens (the first camera device) or the ROI area of the Tele lens (the second camera device).
  • Step S604 calculating the ROI offset of the target ROI region in the first image and the second image, wherein the second image is the first image collected by the second camera in the first camera time-synchronized captured images.
  • the ROI offset refers to the position offset of the target ROI region in the first image and the second image.
  • it can be expressed as the position offset of the feature points in the target ROI region in the first image and the feature points in the target ROI region in the second image.
  • Step S606 Acquire the current magnification of the first image, perform slice processing on the ROI offset according to the current magnification, and obtain the current overlay offset of the image stereoscopic correction performed on the first image at the current moment .
  • the current magnification refers to the magnification of the first image at the current moment in the process of performing the digital zoom operation on the first image.
  • slicing processing refers to performing slicing processing on the ROI offset, so as to determine the current overlay offset corresponding to the current moment.
  • Step S608 Perform image stereoscopic correction on the first image based on the current overlay offset to obtain the corrected first image.
  • stereoscopic correction may be performed on the first image based on the current overlay offset to obtain the corrected first image.
  • Step S610 if the current magnification is a preset switching magnification, the mobile terminal is switched from the first camera device to the second camera device for display; wherein, when switching, the first image
  • the sum of the overlay offsets for image stereo correction is the ROI offset.
  • the preset switching magnification refers to the switching magnification switch level described in the above content, which represents the magnification corresponding to the first image when the mobile terminal is switched from the first camera device to the second camera device.
  • a first image captured by a first camera device is acquired; and a target ROI area in the first image is determined, and then, the difference between the target ROI area in the first image and the second image is calculated
  • the ROI offset is obtained, and the current magnification of the first image is obtained, and the ROI offset is sliced according to the current magnification to obtain the current superimposed offset of the stereoscopic correction of the first image at the current moment; then, based on the current Perform image stereoscopic correction on the first image by superimposing the offset to obtain the corrected first image; wherein, if the current magnification is a preset switching magnification, the mobile terminal is switched from the first camera device to the first image Two cameras are displayed.
  • the current overlay offset is determined based on the current magnification of the first image and based on the current magnification, and stereoscopic correction is performed on the first image based on the current overlay offset.
  • the target ROI in the first image may also be determined.
  • the image in the area is subjected to texture detection to obtain the texture intensity.
  • a target value may be calculated based on the pixel points located in the target ROI region in the first image, and the texture intensity may be determined based on the target value, wherein the texture intensity includes The standard deviation calculated from the following values: pixel gray value or pixel RGB value. That is, the texture intensity can be calculated based on the gray value of the pixel, or can be calculated based on the RGB value of the pixel.
  • the content in the target ROI area can be analyzed to ensure that the content in the target ROI area is textured, so that the offset data can be calculated in the next step. is reliable.
  • the texture detection in the embodiment of the present disclosure mainly calculates the standard deviation based on the gray value or RGB value of the image pixels in the current target ROI area, and then determines whether the image in the target ROI area contains texture through the standard deviation, wherein, The standard deviation can be the above target value.
  • the formula for calculating the standard deviation can be: g i represents the gray value or pixel RGB value of the ith pixel in the target ROI area, and H and W are the length and width of the target ROI area.
  • the texture intensity of the image in the target ROI area is based on the texture intensity. is greater than the preset texture strength, if it is determined based on the texture strength that the texture strength of the image located in the target ROI area in the first image is greater than the preset texture strength, then it is determined that the target ROI area is within the first The offset in the image and the second image to get the ROI offset.
  • the target ROI area is composed of pixels in rows H and columns W, where each pixel has a corresponding gray value gi, so the standard deviation is calculated by using the gray value gi.
  • an empirical value may be set as the control threshold Tolerance (that is, the preset texture intensity). When the standard deviation ⁇ >Tolerance (that is, the preset texture intensity), the texture in the target ROI area is relatively low.
  • the texture in the target ROI area is weak, indicating that the currently obtained target ROI area is not suitable for calculating the ROI offset, and then wait for a new image to be re-acquired before making a judgment.
  • the target ROI area in the first image after obtaining the target ROI area in the first image according to the above-described process, and performing texture detection on the image located in the target ROI area, it is possible to calculate the target ROI area in the first image. and the offset in the second image to get the ROI offset.
  • step S604 calculating the ROI offset of the target ROI region in the first image and the second image includes the following process:
  • Step S6041 determining the image feature points located in the target ROI region in the first image, and obtaining the first image feature points;
  • Step S6042 determining the matching feature points of the first image feature points in the second image to obtain the second image feature points
  • Step S6043 Determine the ROI offset based on the pixel distance between the first image feature point and the second image feature point.
  • the target ROI area After the target ROI area is determined, it is necessary to capture the images captured by the first camera device and the second camera device synchronously, that is, the first image and the second image. Then, find the matching feature points of the two images in the target ROI area. For example, first, determine the image feature points located in the target ROI area in the first image to obtain the first image feature points, wherein the first image feature points may be the feature points of objects contained in the target ROI area, for example, a human The feature points of the face, etc., are not specifically limited in this embodiment of the present disclosure. Next, the matching feature points of the first image feature points are determined in the second image to obtain the second image feature points.
  • the pixel distance between the first image feature point and the second image feature point is calculated, and the pixel distance is used as the ROI offset.
  • the first image feature point and the second image feature point can be calculated.
  • the coordinate difference between matching pixels in the middle is taken as the pixel distance.
  • the target ROI area belongs to the same depth in space, that is, the distance between the area and the camera is approximately the same, rather than a part of the area. In the foreground area, part in the background area. At this time, it is necessary to first determine the camera device corresponding to the target ROI area to determine the image content of the target ROI area, and then go to another image for matching, and obtain the offset of the area.
  • the above process will be exemplified below.
  • the first image currently displayed in the mobile terminal is captured by the first camera device with a larger FOV, and the user manually focuses on a certain area (that is, the target ROI area). ).
  • the preview image of the screen of the mobile terminal is the image (first image) of the Wide camera (first camera), and the target ROI area is obtained by focusing on the touch screen.
  • the embodiment of the present disclosure obtains the image content in the target ROI area based on the target ROI area and the first image, and then combines the Tele images (ie, the second camera) captured at the same time, and based on feature point extraction and matching algorithms, to obtain multiple sets of Matching point sets (wherein, multiple sets of matching point sets are the above-mentioned first image feature points and second image feature points).
  • the image captured by Tele (that is, the second camera) is not displayed in the mobile terminal, when previewing the image captured or captured, the Wide (that is, the first camera) A camera device) and Tele (ie, the second camera device) are turned on at the same time, and are always ready to display their respective captured images on the screen.
  • the feature point extraction and matching algorithm may be any of the following: SIFT (Scale-Invariant Feature Transform), ORB (Oriented FAST and Rotated BRIEF), BRISK (Binary Robust Invariant Scalable Keypoints ), etc., the embodiments of the present disclosure do not limit a specific matching algorithm.
  • the process of determining the ROI offset ( ⁇ u, ⁇ v) can be referred to as follows:
  • each set of matching point pairs includes the first image feature point and its matched second image feature points.
  • the current magnification of the first image can be obtained, and slice processing is performed on the ROI offset according to the current magnification, so as to obtain the stereoscopic image of the first image at the current moment. Corrected current stack offset.
  • step S606 the step of performing slice processing on the ROI offset according to the current magnification to obtain the current overlay offset of the image stereoscopic correction performed on the first image at the current moment include:
  • Step S6061 acquiring a target magnification, wherein the target magnification includes: an initial magnification and a preset switching magnification; the initial magnification is when the ROI offset is determined for the first time, and the first camera device is used for the first image. the corresponding magnification when the digital zoom operation is performed; the preset switching magnification represents the image magnification when the first camera device is switched to the second camera device;
  • Step S6062 slicing the ROI offset according to the size relationship between the current magnification and the target magnification to obtain the current superimposed offset of the image stereoscopic correction performed on the first image at the current moment .
  • the ROI offset in order to ensure the smoothness of the entire digital zooming process when the digital zooming process is performed on the first image, after the ROI offset is obtained, the ROI offset needs to be sliced. During the process of enlarging the first image, ROI offsets of different proportions (ie, overlapping offsets) are gradually superimposed on the first image, until the first image is enlarged to the switching magnification, and the total offset superposition is completed.
  • the ROI offset can be sliced by comparing the size relationship between the current magnification and the target magnification, so as to obtain the stereoscopic image of the first image at the current moment. Corrected current stack offset. By slicing the ROI offset, it can be ensured that the first camera device and the second camera device have no jumps in the image in the target ROI area when switching, and the specific implementation is as follows:
  • the initial magnification user level corresponding to the first camera device performing the digital zoom operation on the first image is ul 0
  • the size of the ROI offset is S ROI
  • the preset switching magnification switch level is abbreviated as sl.
  • the current magnification user level is ul cur
  • the current superimposed offset on each frame of image at the current moment is s cur
  • the results of s cur can be divided into the following situations:
  • the current overlay offset is determined to be 0.
  • the calculation formula of the current overlay offset is: Wherein, s cur represents the current overlay offset, S ROI represents the ROI offset, ul cur represents the current magnification, ul0 represents the initial magnification of the first image, and s1 represents the preset Switch the magnification.
  • This formula realizes the slicing effect of the ROI offset, that is, the distance ratio between the current magnification and the preset switching magnification switch level corresponds to the ROI offset S ROI , and realizes the next frame of ul 0 and ul 0 The offset between them is small enough to prevent visible translation jumps in the content of the image; at the same time, when ul cur is close to sl, s cur is also basically equal to S ROI .
  • the current overlay offset is the ROI offset.
  • s cur S ROI .
  • the screen of the mobile terminal displays the second image collected by the second camera, but the current overlay offset is still superimposed on the first image, so that the user can switch from the second camera back to the first camera.
  • the device When the device is installed, it can still ensure that the image in the target ROI area displayed on the screen has no jump.
  • the current overlay offset required for performing the image stereoscopic correction operation on the first image at the current moment is determined in the manner described in the above cases 1 to 3, the current overlay offset can be determined based on the current overlay offset.
  • An image stereo correction operation is performed on the first image.
  • performing image stereo correction on the first image based on the current overlay offset, and obtaining the corrected first image includes the following steps (1) to (2):
  • the image enlargement parameter includes the center point of the first image and a preset switching magnification
  • the specific implementation of the digital zoom operation is to construct an image transformation matrix H zoom according to a certain zoom ratio, and then realize the transformation of the image through the image transformation matrix H zoom .
  • the specific formulas for constructing the image transformation matrix and realizing the transformation of the image through the image transformation matrix are as follows:
  • the formula of the target transformation matrix constructed based on the current overlay offset and the image magnification parameters required to perform the image stereo correction operation on the first image at the current moment is:
  • the image can be zoomed and the image can be translated at the same time, and the offset calculated by the above process can be applied to the video frame. in the input image.
  • the target ROI area can be updated, and the target ROI area can be updated by the user. Therefore, whether the scene changes or the focus point changes, it is possible to change the ROI offset. Therefore, in the embodiment of the present disclosure, the process of calculating the ROI offset also requires that it be repeated at a certain frequency in the entire "optical digital joint zoom" process. Therefore, it is also necessary to consider how the calculation of the overlay offset should be updated when the ROI offset is changed.
  • the method further includes the following steps S1 to S4:
  • Step S1 detecting whether the ROI offset has changed according to a preset time interval
  • Step S2 if it is detected that the ROI offset has changed, the target ROI offset is determined based on the ROI offset after the change and the current overlay offset of the first image;
  • Step S3 acquiring a first magnification, wherein the first magnification is the magnification of the first image when the occurrence of the ROI offset is detected;
  • Step S4 re-determine the current overlay offset of the first image according to the first magnification and the target ROI offset, and perform the first image based on the re-determined current overlay offset.
  • Image stereo correction re-determine the current overlay offset of the first image according to the first magnification and the target ROI offset, and perform the first image based on the re-determined current overlay offset.
  • the ROI offset S ROI S ROI_0 ; at the moment before the ROI offset update, the magnification userlevel of the first image (that is, the embodiment of the present disclosure The second magnification in ) is ul cur_0 . If ul 0 ⁇ ul cur_0 ⁇ sl, at this time, the current overlay offset s cur_0 corresponding to the stereo correction of the image is as follows shown, otherwise s cur_0 shall be equal to 0 or S ROI_0 .
  • the magnification of the first image when the occurrence of the ROI offset is detected that is, the first magnification can be continuously obtained.
  • the current overlay offset of the first image is re-determined according to the first magnification and the target ROI offset, and the image stereo correction is performed on the first image according to the re-determined current overlay offset.
  • the current overlay offset of the first image is the same as the overlay offset of the previous moment, wherein the second magnification is the image of the previous moment
  • the image frame has already superimposed the offset of the superimposed offset s cur_0.
  • the current overlay offset of the first image is determined according to the first magnification and the offset of the target ROI.
  • the image processing method mainly includes the following three parts:
  • the target ROI region refers to the region of interest of the user.
  • the mobile terminal usually when the user clicks a certain position on the screen to focus, it means that the user wants to pay attention to this area, so the mobile terminal will use this area as the target ROI area. That is, when the user manually focuses, the ROI area can be determined based on the user's touch position on the screen. If the user does not focus manually, the mobile terminal will automatically use the central area of the screen as the target ROI area, or automatically detect the position of the target object such as a face on the screen, and determine the target ROI area according to the detection results. These are all commonly used ways for mobile terminals to obtain the target ROI area.
  • the ROI area may be the ROI area of the Wide lens (the first camera device) or the ROI area of the Tele lens (the second camera device).
  • the target ROI region in the first image ie, the Wide image frame
  • the second image ie, the Tele image frame
  • ROI texture verification In order to ensure that the acquired target ROI area is matchable (that is, it has texture features), before matching, the texture of the area will be verified first.
  • the verification method mainly calculates the standard deviation based on the gray value or RGB value of each pixel in the area. If the standard deviation is less than the preset texture intensity, it is considered that there is no texture in the area, and the current calculation is exited; otherwise, the next step is calculated.
  • the first image feature point may be determined in the first image by the above algorithm, and the matching feature point of the first image feature point may be determined in the second image to obtain the second image feature point.
  • the image in the "optical digital joint zoom” process can be translated based on the data, so that when the image is zoomed in enough to switch the camera, there is no jump in the target ROI area.
  • the acquisition of the ROI offset is carried out in the process of "optical digital joint zoom”; on the other hand, since the target ROI area can also be controlled by the user, that is, the target ROI area can be changed, so the ROI
  • the calculation of the offset needs to be performed frequently and synchronously with the zooming process to adapt to changes in the scene and changes in the target ROI area. Therefore, how to apply the obtained ROI offset in the zooming process can not only ensure that the target ROI area does not jump during switching, but also make the entire digital zooming process smooth, which is the most important technical issue.
  • an embodiment of the present disclosure proposes a method for smoothly eliminating the ROI offset, which can not only ensure the smooth enlargement of the digital zoom process, avoid jumps due to the appearance and update of the ROI offset, but also ensure that the camera device is switched. When the target ROI area has no jumps.
  • the act of eliminating the ROI offset is mainly for the process of image translation. Since the FOV of Wide is much larger than that of Tele, in order to ensure that Wide and Tele are aligned at the target ROI area when switching cameras, the calculated ROI offset mainly acts on the Wide image frame. Therefore, the specific implementation process is as follows:
  • the digital zoom process usually zooms in or out on the center of the image based on the magnification (user level) before getting the ROI offset for the first time.
  • the ROI offset is obtained for the first time, according to the distance between the current user level and the switch level, the value of the offset is sliced, and in the process of continuing to enlarge the user level, the overlapping offsets of different proportions are superimposed.
  • the ROI offset changes due to scene changes or when the user refocuses to a different depth from the camera. Therefore, it is also necessary to consider the case where the ROI offset changes.
  • the ROI offset changes due to scene changes or when the user refocuses to a different depth from the camera. Therefore, it is also necessary to consider the case where the ROI offset changes.
  • the user level2 at this time has also been enlarged by a certain rate compared to the user level1 when the ROI offset was obtained last time, that is, the image of the current frame has been based on the last time.
  • the superimposed offset is shifted by a certain pixel value, then it is necessary to subtract the currently shifted ROI offset based on the currently updated ROI offset as the ROI offset at the current moment, and re-create the ROI
  • the offset is sliced, so that even after the ROI offset is updated, the digital zoom can be smoothed over while still ensuring that there is no jump at the ROI when switching cameras.
  • An embodiment of the present disclosure further provides an image processing apparatus, and the image processing apparatus is mainly used to execute the image processing method provided by the above content of the embodiment of the present disclosure.
  • the following describes the image processing apparatus provided by the embodiment of the present disclosure in detail. introduce.
  • FIG. 10 is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure.
  • the image processing apparatus mainly includes: a first acquisition unit 10 , a calculation unit 20 , a second acquisition unit 30 , and slice processing unit 40, image correction unit 50 and switching display unit 60, wherein:
  • a first acquisition unit configured to acquire a first image collected by the first camera device; and determine a target ROI region in the first image
  • the calculation unit is configured to calculate the ROI offset of the target ROI region in the first image and the second image, wherein the second image is the first image captured by the second camera in the first camera. An image captured simultaneously with one image;
  • a second acquisition unit configured to acquire the current magnification of the first image
  • a slicing processing unit configured to perform slicing processing on the ROI offset according to the current magnification, so as to obtain a current superimposed offset for performing image stereoscopic correction on the first image at the current moment;
  • an image stereo correction unit configured to perform image stereo correction on the first image based on the current overlay offset to obtain the corrected first image
  • a switching display unit configured to switch the mobile terminal from the first camera device to the second camera device for display if the current magnification is a preset switching ratio; wherein, when switching, the The sum of the superimposed offsets for the stereoscopic correction of the first image is the ROI offset.
  • the current overlay offset is determined based on the current magnification of the first image and based on the current magnification, and stereoscopic correction is performed on the first image based on the current overlay offset.
  • the device further includes a texture detection unit: the texture detection unit is configured to: perform texture detection on the image located in the target ROI area in the first image to obtain the texture intensity; the calculation unit is further configured to: if determined It is found that the texture intensity of the image located in the target ROI area in the first image is greater than the preset texture intensity, then calculate the ROI of the target ROI area in the first image and the second image of the target ROI area Offset.
  • the texture detection unit is configured to: perform texture detection on the image located in the target ROI area in the first image to obtain the texture intensity
  • the calculation unit is further configured to: if determined It is found that the texture intensity of the image located in the target ROI area in the first image is greater than the preset texture intensity, then calculate the ROI of the target ROI area in the first image and the second image of the target ROI area Offset.
  • the texture detection unit is further configured to: calculate a target value based on the pixel points located in the target ROI area in the first image, and determine the texture intensity based on the target value, wherein the texture intensity It includes the standard deviation calculated by any of the following values: pixel gray value, pixel RGB value.
  • the computing unit is further configured to: determine image feature points located in the target ROI region in the first image to obtain first image feature points; determine the first image feature in the second image The matching feature points of the points are obtained to obtain a second image feature point; and the ROI offset is determined based on the pixel distance between the first image feature point and the second image feature point.
  • the slice processing unit is configured to: obtain a target magnification, wherein the target magnification includes: an initial magnification and a preset switching magnification; the initial magnification is the first magnification when the ROI offset is determined for the first time.
  • the magnitude relationship between the magnification ratio and the target magnification ratio is used to slice the ROI offset, so as to obtain the current superimposed offset of the stereoscopic correction of the first image at the current moment.
  • the slice processing unit is further configured to: if the current magnification is smaller than the initial magnification, determine that the current overlay offset is 0.
  • the slice processing unit is further configured to: if the current magnification is greater than the initial magnification and smaller than the preset switching magnification, the calculation formula of the current overlay offset is: Wherein, s cur represents the current overlay offset, S ROI represents the ROI offset, ul cur represents the current magnification, ul0 represents the initial magnification of the first image, and s1 represents the preset Switch the magnification.
  • the slice processing unit is further configured to: if the current magnification is greater than the preset switching magnification, the current overlay offset is the ROI offset.
  • the image stereo correction unit is configured to: construct a target transformation matrix based on the current overlay offset and image enlargement parameters; the image enlargement parameters include the center point of the first image and a preset switching magnification; calculate The product between the target transformation matrix and the homogeneous coordinates of the first image at the current moment, and the first image after stereoscopic correction of the image is determined according to the product calculation result.
  • the device further includes a recalibration unit, and the recalibration unit is configured to: after calculating the ROI offset, detect whether the ROI offset has changed according to a preset time interval; If the ROI offset has changed, the target ROI offset is determined based on the changed ROI offset and the current overlay offset of the first image; obtain a first magnification, wherein the first magnification The magnification is the magnification of the first image when the occurrence of the ROI offset is detected; the current overlay offset of the first image is re-determined according to the first magnification and the target ROI offset, and performing image stereoscopic correction on the first image according to the re-determined current stacking offset.
  • the recalibration unit is further configured to: if the first magnification and the second magnification are the same, the current stacking offset of the first image is the same as the stacking offset at the previous moment, wherein, The second magnification is the image magnification corresponding to the image stereoscopic correction operation at the previous moment.
  • the recalibration unit is further configured to: if the first magnification and the second magnification are different, determine the first image according to the first magnification and the offset of the target ROI The current overlay offset of .
  • An embodiment of the present disclosure further provides a mobile terminal, including a processor, and a first camera device and a second camera device respectively connected to the processor.
  • the mobile terminal may also be referred to as a dual-camera terminal, such as a dual-camera terminal. camera phone, etc.
  • the processor in the mobile terminal can execute the steps of the image processing method described in any one of the above.
  • the terms “installed”, “connected” and “connected” should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or the internal communication between the two components.
  • installed should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or the internal communication between the two components.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Studio Devices (AREA)

Abstract

本公开提供了一种图像的处理方法、装置、电子设备、可读介质及移动终端，涉及图像处理的技术领域，包括：获取第一摄像装置采集到的第一图像；并确定第一图像中的目标ROI区域；计算第一图像和第二图像中目标ROI区域的ROI偏移量；获取第一图像的当前放大倍率，根据当前放大倍率对ROI偏移量进行切片处理，得到当前时刻对第一图像进行图像立体校正的当前叠加偏移量；基于当前叠加偏移量对第一图像进行图像立体校正，得到校正之后的第一图像；若当前放大倍率为预设切换倍率，则移动终端由第一摄像装置切换至第二摄像装置进行显示，本公开缓解了采用现有技术中在进行摄像装置成像切换时跳变明显的技术问题。

Description

图像的处理方法、装置、电子设备、可读介质及移动终端
相关申请的交叉引用
本公开要求于2020年7月7日提交中国专利局的申请号为2020106495056、名称为“图像的处理方法、装置、电子设备和计算机可读介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及图像处理的技术领域，尤其是涉及一种图像的处理方法、装置、电子设备、可读介质及移动终端。
背景技术
对于移动终端而言，其摄像装置通常都是定焦镜头，只能实现数字变焦。而随着消费者对诸如手机等移动终端的拍摄、摄像等功能的要求越来越高，为了解决数字变焦在高变焦倍率下的缺陷，业界则越来越多的在移动终端上搭载两颗或多颗焦距不同的摄像装置进行搭配，如在移动终端上搭载两颗FOV(field angle，视场角)的镜头，诸如一颗镜头FOV(field angle，视场角)＝80°，一颗镜头FOV＝40°。此时通过将数字变焦与光学变焦相结合的方式来提升拍摄性能，以此来满足消费者。数字变焦与光学变焦相结合是指通过一个摄像装置进行图像的放大，再切换至另外一个摄像装置进行成像，上述方法又称为光学数字联合变焦。但事实上，对于焦距而言，每颗摄像装置在组装时都有一定误差，其真实焦距与镜头厂商给的理论参考值也是有差异的；而对光轴而言，两颗摄像装置组装在一起时，其光轴也会产生误差，两颗摄像装置的光轴不是绝对的平行，导致两颗摄像装置所观测的方向不完全一致，进而在切换摄像装置后最终成像效果不佳，带来不好的用户体验。例如，两颗摄像装置的Baseline(基线，也可理解为距离)会使得两颗摄像装置产生视差，视差便会使得两颗摄像装置在切换时，图像内容产生较为明显的跳变。
公开内容
鉴于此，本公开的目的在于提供一种图像的处理方法、装置、电子设备和计算机可读介质，以缓解了采用现有技术中在进行摄像装置成像切换时图像内容跳变的技术问题。
第一方面，本公开实施例提供了一种图像的处理方法，应用于移动终端，所述移动终端包括：第一摄像装置和第二摄像装置，所述方法包括：获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域；计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像；获取所述第一图像的当前放大倍率，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量；基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像；若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
进一步地，所述方法还包括：对所述第一图像中位于所述目标ROI区域内的图像进行 纹理检测，得到纹理强度；计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量包括：若确定出所述第一图像中位于所述目标ROI区域内的图像的纹理强度大于预设纹理强度，则计算所述目标ROI区域在所述第一图像和第二图像中所述目标ROI区域的ROI偏移量。
进一步地，对所述第一图像中位于所述目标ROI区域内的图像进行纹理检测，得到纹理强度包括：基于所述第一图像中位于所述目标ROI区域内的像素点计算目标数值，并基于所述目标数值确定所述纹理强度，其中，所述纹理强度包括以下任一种数值计算得到的标准差：像素灰度值、像素RGB值。
进一步地，计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量包括：确定所述第一图像中位于所述目标ROI区域内的图像特征点，得到第一图像特征点；在所述第二图像中确定所述第一图像特征点的匹配特征点，得到第二图像特征点；基于所述第一图像特征点和所述第二图像特征点之间的像素距离，确定所述ROI偏移量。
进一步地，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量包括：获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切为所述第二摄像装置时的图像放大倍率；根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
进一步地，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量包括：若所述当前放大倍率小于所述初始放大倍率，则确定所述当前叠加偏移量为0。
进一步地，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量还包括：若所述当前放大倍率大于所述初始放大倍率，且小于所述预设切换倍率，则所述当前叠加偏移量的计算公式为
Figure PCTCN2021093706-appb-000001
其中，s cur表示所述当前叠加偏移量，S ROI表示所述ROI偏移量，ul cur表示所述当前放大倍率，ul0表示所述第一图像的初始放大倍率，sl表示所述预设切换倍率。
进一步地，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量还包括：若所述当前放大倍率大于所述预设切换倍率，则所述当前叠加偏移量为所述ROI偏移量。
进一步地，基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像包括：基于所述当前叠加偏移量和图像放大参数构建目标变换矩阵；所述图像放大参数中包含所述第一图像的中心点和预设切换倍率；计算所述目标变换矩阵和当前时刻所述第一图像的齐次坐标之间的乘积，并根据乘积计算结果确定进行图像立体校正之后的所述第一图像。
进一步地，所述方法还包括：在计算出所述ROI偏移量之后，按照预设时间间隔检测所述ROI偏移量是否发生了变化；若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生时所述第一图像的放大倍率；根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
进一步地，根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量包括：若所述第一放大倍率和第二放大倍率相同，则所述第一图像的当前叠加偏移量和上一时刻的叠加偏移量相同，其中，所述第二放大倍率为上一时刻图像立体校正操作所对应的图像放大倍率。
进一步地，根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量包括：若所述第一放大倍率和所述第二放大倍率不相同，则根据所述第一放大倍率和所述目标ROI偏移量确定所述第一图像的当前叠加偏移量。
第二方面，本公开实施例提供了一种图像的处理装置，设置于移动终端，所述移动终端包括：第一摄像装置和第二摄像装置，所述装置包括：第一获取单元，配置成获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域；计算单元，配置成计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像；第二获取单元，配置成获取所述第一图像的当前放大倍率，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量；图像立体校正单元，配置成基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像；切换显示单元，配置成若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
进一步地，所述切片处理单元进一步配置成：获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切为所述第二摄像装置时的图像放大倍率；根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
进一步地，所述装置还包括重新校正单元，所述重新校正单元配置成：在计算出所述ROI偏移量之后，按照预设时间间隔检测所述ROI偏移量是否发生了变化；若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生时所述第一图像的放大倍率；根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
第三方面，本公开实施例提供了一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述第一方面中任一项所述的方法的步骤。
第四方面，本公开实施例提供了一种具有处理器可执行的非易失的程序代码的计算机可读介质，所述程序代码使所述处理器执行上述第一方面中任一项所述的方法的步骤。
第五方面，本公开实施例提供了一种移动终端，包括处理器，以及与所述处理器分别连接的第一摄像装置和第二摄像装置，所述处理器执行上述第一方面中任一项所述的方法的步骤。
本公开的其他特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本公开而了解。本公开的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是根据本公开实施例的一种电子设备的示意图；
图2是根据本公开实施例的一种双摄装置图像校正的原理示意图；
图3是根据本公开实施例的一种数字变焦操作的流程示意图；
图4是根据本公开实施例的一种双摄装置之间的距离的示意图；
图5是根据本公开实施例的一种视差和双摄装置之间的距离的变化示意图；
图6是根据本公开实施例的一种图像的处理方法的流程图；
图7是根据本公开实施例的一种目标ROI区域进行纹理检测的原理示意图；
图8是根据本公开实施例的另一种图像的处理方法的流程图；
图9是根据本公开实施例的又一种图像的处理方法的流程图；
图10是根据本公开实施例的一种图像的处理装置的示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合附图对本公开的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
首先，参照图1来描述用于实现本公开实施例的电子设备100，该电子设备可以用于运行本公开各实施例提供的图像的处理方法。
如图1所示，电子设备100包括一个或多个处理设备102、一个或多个存储装置104、输入装置106、输出装置108以及摄像装置110，这些组件通过总线系统112和/或其它形式的连接机构(未示出)互连。应当注意，图1所示的电子设备100的组件和结构只是示例性的，而非限制性的，根据需要，所述电子设备也可以具有其他组件和结构。
所述处理设备102可以是中央处理单元(Central Processing Unit，CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元，并且可以控制所述电子设备100中的其 它组件以执行期望的功能。
所述存储装置104可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(Random Access Memory，RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(Read-Only Memory，ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理设备102可以运行所述程序指令，以实现下文所述的本公开实施例中(由处理器实现)的客户端功能以及/或者其它期望的功能。在所述计算机可读存储介质中还可以存储各种应用程序和各种数据，例如所述应用程序使用和/或产生的各种数据等。
所述输入装置106可以是用户用来输入指令的装置，并且可以包括键盘、鼠标、麦克风和触摸屏等中的一个或多个。
所述输出装置108可以向外部(例如，用户)输出各种信息(例如，图像或声音)，并且可以包括显示器、扬声器等中的一个或多个。
所述摄像装置110可以拍摄用户期望的图像(例如照片、视频等)，并且将所拍摄的图像存储在所述存储装置104中以供其它组件使用。
示例性地，用于实现根据本公开实施例的图像的处理方法的示例电子设备可以被实现为诸如智能手机、平板电脑、可穿戴电子设备等移动终端上。
在介绍本申请的图像处理方法之前，首先介绍光学数字联合变焦的处理过程，描述如下：
用户在使用手机进行拍照或摄像的过程中，随着用户通过屏幕开始对具有较大FOV的摄像装置采集的预览图像进行放大，此时执行数字变焦操作。当放大后的图像内容的尺度与具有较小FOV的摄像装置采集的FOV图像内容尺度相同时，切换为另一个摄像装置进行成像，此时用户继续放大显示在屏幕上的预览图像，便继续执行数字变焦操作。在切换摄像装置这一瞬间，为光学变焦操作。该操作相比于单颗摄像装置进行纯数字变焦操作的优势在于：当放大倍率较大时，该操作会使用一颗焦距更大的摄像装置进行成像，特别对于中远景，其成像质量一定会高于单摄数字变焦的成像质量。
该操作除了要求满足两个摄像装置切换前后的屏幕内容的尺度相同以外，为了尽可能的逼近真实的光学变焦的体验感，通常还会要求满足以下几点：
(1)、数字变焦保证放大过程平滑自然，接近真实光学变焦在放大图像内容过程中的体验；
(2)、当切换摄像装置时，保证图像内容尺度相同的同时，前后两帧无旋转感。
(3)、当切换摄像装置时，保证图像内容尺度相同的同时，前后两帧尽可能少的跳变(图像内容平移)。
因此，为了满足以上要求，需要满足以下设置要求：
(1)、减少切换摄像装置时的跳变，则要求两个摄像装置的Baseline(基线，也可理解为距离)尽可能小：Baseline会使得两个摄像装置产生视差，视差便会使得两颗摄像装置在切换时，其内容产生跳变，后面会对其原理进行阐述；
(2)、使切换摄像装置前后图像无旋转感，则要求光轴平行，即两颗摄像装置拍摄的光轴方向是相同的；
(3)、保证数字变焦放大过程平滑，以及切换摄像装置时图像内容的尺度相同，则要求准确的确定摄像装置的焦距信息，以便准确计算出两个不同FOV的摄像装置的尺度比例。
但事实上，对于焦距而言，每个摄像装置在组装时都有一定误差，其真实焦距与镜头厂商给的理论参考值也是有差异的；而对光轴而言，两个摄像装置组装在一起时，其光轴也会产生误差，两颗摄像装置的光轴不是绝对的平行，导致两个摄像装置所观测的方向不完全一致，进而在切换摄像装置后最终成像效果不佳，带来不好的用户体验。因此，“光学数字联合变焦”算法通常会要求工厂在手机生产中，对两个摄像装置进行标定，计算出摄像装置准确的内外参数，其中内参数包括了镜头的真实焦距；外参数包括了摄像装置之间的光轴夹角，然后将标定得到的数据保存在手机芯片中，手机在出厂后被用户使用时，诸如用户在采用手机执行拍照、摄像等操作时，手机可读取预先保存在芯片中的标定数据来获取相关信息，并对图像进行立体旋转和尺度变换等变换操作，可将该图像变换操作称之为图像立体校正，最后得到光轴平行的图像。
立体校正的示意图如图2所示，其中，菱形形状的平面是不平行的两颗摄像装置真实拍摄得到的图片，矩形形状的平面是利用标定数据，经算法校正后的绝对平行的图片。其中，一旦光轴平行，摄像装置在切换前后各自对应的摄像内容便不会产生旋转感和因Baseline以外多余的平移。
因此，要实现光学数字联合变焦，主要需要实现如下三个功能：
功能一：数字变焦。基于任一放大倍率(user level，(取值范围为：1.0～))，实现对图像的数字变焦。如下式所示：
Figure PCTCN2021093706-appb-000002
和
Figure PCTCN2021093706-appb-000003
H zoom为3*3的图像放大矩阵，s为放大倍率，(cx,cy)为图像中心；I input为数字变焦前的图，I output为数字变焦后的图，(u,v,1)为输入图像的齐次坐标，(u',v',1)为输出图像的齐次坐标。
功能二：切换逻辑。如图3所示，摄像装置Wide为FOV较大的摄像装置，摄像装置Tele为FOV较小的摄像装置，当摄像装置Wide放大到FOV等于摄像装置Tele的FOV时，切换为摄像装置Tele。在实现时，通常不会去比较FOV是否相同，而是计算切换时对应的切换倍率switch level，如图3所示，switch level＝2。其计算公式如下：
Figure PCTCN2021093706-appb-000004
其中，Tele指FOV小的摄像装置，Wide指FOV大的摄像装置，f为焦距，width为摄像装置的sensor的宽度，若这些数据为标定计算出来的，那么switch level的准确性会更高。当user level<switch level时，由FOV大的摄像装置做数字变焦；否则由FOV小的摄像装置做数字变焦。
功能三：消除旋转。为了在切换摄像装置时可以尽可能的使视频画面平滑过度，需要基于标定数据，对图像进行立体校正，消除光轴误差带来的旋转感和跳变感。变换公式如下：
Figure PCTCN2021093706-appb-000005
Figure PCTCN2021093706-appb-000006
其中，H为变换矩阵，K为标定获取的摄像装置内参数，R为标定获取的旋转角度。
以上三个功能，是基于两个FOV差异较大的摄像装置实现“光学数字联合平滑变焦”的核心部分。接下来重点介绍Baseline对“光学数字联合变焦”算法的影响。通过上述描述可知，在执行图像立体校正操作之后，能够实现两个摄像装置所拍摄图像的图像内容角度相同，且图像内容尺度相同。但由于两个摄像装置之间存在距离，虽然标定数据可以消除尺度误差和光轴平行误差，但是两个摄像装置之间的视差却无法通过标定来去除。如图4所示的为两个摄像装置经立体校正后的俯视图：其中两条“X轴方向的粗体线段”代表两个摄像装置的sensor投影面，O R与O T代表镜头光心，两个光心之间的距离为Baseline，此时两个摄像装置的光轴绝对平行，且focus相同(尺度相同)。
如图4所示，空间中一点P，距离双摄系统的距离为Z，分别在两个像平面成像为p和p’,以“X轴方向的粗体线段”左端为坐标起始点，p的坐标为x R，p’的坐标为x T，因此我们称D＝(x R–x T)为空间中点P在该双摄系统中的视差Disparity。视差D随着距离Z的变化而变化，当Z越小，即距离越近，视差越大；反之，视差越小，当距离无穷远时，视差为0。其关系可由图5所表示。
因此，在“光学数字联合变焦”算法的应用中，当切换摄像装置时，图像中的每个像素点都会因其成像的空间点距离摄像装置的距离不同，而呈现不同程度的视差跳变(如图4中点P，如果从左边的摄像装置切换至右边摄像装置时，P在图像中，会从点p跳至点p’)。只有场景中特别远(近似无穷远处)的物体不会产生跳变；而越近的物体，视差的跳变会越明显。而用户在使用手机摄像时，除了拍摄远处的风景等远景会重点关注远处外，大多数拍摄场景所关注的内容都在距离摄像装置不远的前景。此时在放大并切换摄像装置的时候，所关注的前景必然会发生跳变，给用户带来不好的体验感。这也是大多数主流算法尽管标定数据很准确，但仍然能在使用联合变焦功能时，感受到屏幕中场景的跳变。基于此，在本公开实施例中，提出了一种图像的处理方法，该方法能够使得视频效果既保证了在两个摄像装置切换时，用户关注的ROI的内容无跳变，又保证了整个数字变焦放大过程中的平滑性，进而缓解了采用现有技术中在进行摄像装置成像切换时跳变明显的技术问题。
根据本公开实施例，提供了一种图像的处理方法的实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图6是根据本公开实施例的一种图像的处理方法的流程图，如图6所示，该方法包括如下步骤：
步骤S602，获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域。
在本申请中，此处的第一摄像装置和下述步骤中描述的第二摄像装置为移动终端(例如，手机)中的摄像装置，其中，第一摄像装置和第二摄像装置的视场角不相同，例如，第一摄像装置的视场角大于第二摄像装置的视场角。具体地，第一摄像装置和第二摄像装 置可以为双摄手机中的双摄装置。
在本公开实施例中，目标ROI区域是指用户感兴趣区域。对于可触屏的移动终端而言，通常当用户点击屏幕某一位置进行对焦时，说明用户想关注这一块区域，因此移动终端会将该区域作为目标ROI区域。也即，在用户手动对焦时，可以基于用户在屏幕上的触碰位置确定ROI区域。如果用户没有手动对焦，移动终端会自动将屏幕中心区域作为目标ROI区域，或者自动检测屏幕中诸如人脸等目标对象的位置，根据检测结果确定目标ROI区域等。在“光学数字联合变焦”算法中，虽然同时打开两个摄像装置(例如，第一摄像装置和第二摄像装置)，但移动终端的屏幕只会显示其中一个摄像装置的画面，因此这里的目标ROI区域可能是Wide镜头(第一摄像装置)的ROI区域，也可能是Tele镜头(第二摄像装置)的ROI区域。
步骤S604，计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像。
在本公开实施例中，ROI偏移量是指目标ROI区域在所述第一图像和第二图像中的位置偏移量。例如，可以表示为第一图像中目标ROI区域内的特征点和第二图像中目标ROI区域内的特征点的位置偏移量。
步骤S606，获取所述第一图像的当前放大倍率，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
当前放大倍率是指在对第一图像进行数字变焦操作的过程中，当前时刻对第一图像进行放大的倍率。在本公开实施例中，切片处理是指对ROI偏移量进行切分处理，从而确定出当前时刻所对应的当前叠加偏移量。
步骤S608，基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像。
在本公开实施例中，在确定出当前叠加偏移量之后，就可以基于当前叠加偏移量对第一图像进行图像立体校正，得到校正之后的所述第一图像。
步骤S610，若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
在本公开实施例中，预设切换倍率是指上述内容中所描述的切换倍率switch level，表示将移动终端由第一摄像装置切换至第二摄像装置时，第一图像所对应的放大倍率。
在本公开实施例中，首先，获取第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域，接下来，计算第一图像和第二图像中目标ROI区域的ROI偏移量，并获取第一图像的当前放大倍率，根据当前放大倍率对ROI偏移量进行切片处理，得到当前时刻对第一图像进行图像立体校正的当前叠加偏移量；然后，基于当前叠加偏移量对第一图像进行图像立体校正，得到校正之后的所述第一图像；其中，若当前放大倍率为预设切换倍率，则移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示。通过上述描述可知，在本公开实施例中，通过基于第一图像的当前放大倍率，并基于当前放大倍率确定当前叠加偏移量，并通过该当前叠加偏移量对第一图像进行图像立体校正的方式，能够保证在两个摄像装置切换时，用户关注的ROI的内容无跳变，进而缓解了采用现有技术中在进行摄像装置成像切换时跳变明显的技术问题。
在本公开实施例中，在获取所述第一摄像装置采集到的第一图像；并确定所述第一图 像中的目标ROI区域之后，还可以对所述第一图像中位于所述目标ROI区域内的图像进行纹理检测，得到纹理强度。
在一个可选的实施方式中，可以基于所述第一图像中位于所述目标ROI区域内的像素点计算目标数值，并基于所述目标数值确定所述纹理强度，其中，所述纹理强度包括以下数值计算得到的标准差：像素灰度值或者像素RGB值。也即，纹理强度可以是基于像素灰度值计算得到，也可以基于像素RGB值计算得到。
通过上述描述可知，在按照上述所描述的方式确定目标ROI区域之后，可以对目标ROI区域内的内容进行分析，确保目标ROI区域内的内容是有纹理的，以便下一步计算偏移量的数据是可靠的。
本公开实施例中的纹理检测，主要基于当前目标ROI区域内的图像像素灰度值或RGB值，来计算标准差，进而通过该标准差来确定目标ROI区域内的图像是否包含纹理，其中，标准差可以为上述目标数值。
其中，标准差的计算公式可以为：
Figure PCTCN2021093706-appb-000007
g i表示目标ROI区域内第i个像素点的灰度值或者像素RGB值，H和W为目标ROI区域的长度和宽度。
在本公开实施例中，在按照上述所描述的方式得到目标ROI区域内的图像的纹理强度之后，就可以基于该纹理强度判断第一图像中位于所述目标ROI区域内的图像的纹理强度是否大于预设纹理强度，若基于所述纹理强度确定出所述第一图像中位于所述目标ROI区域内的图像的纹理强度大于预设纹理强度，则确定所述目标ROI区域在所述第一图像和第二图像中的偏移量，得到ROI偏移量。
例如，如图7所示，假设目标ROI区域是由H行，W列的像素点组成，其中，每个像素点都有对应的灰度值gi，因此通过灰度值gi来计算标准差就可以得到这n＝H*W个像素值的稳定性。若目标ROI区域内的纹理较弱，那么其灰度值或RGB值一定较为接近，标准差较小，反之标准差较大。在本公开实施例中，可以设定经验值作为管控阈值Tolerancce(也即，预设纹理强度)，当标准差σ>Tolerance(也即，预设纹理强度)时，目标ROI区域内的纹理较强，可用于后续计算偏移量所用；否则目标ROI区域内的纹理较弱，说明当前获取的目标ROI区域不适用于计算ROI偏移量，则等待重新获取新的图像，再进行判断。
在本公开实施例中，在按照上述所描述的过程得到第一图像中的目标ROI区域，并对位于目标ROI区域内的图像进行纹理检测之后，就可以计算目标ROI区域在所述第一图像和第二图像中的偏移量，得到ROI偏移量。
在一个可选的实施方式中，步骤S604，计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量包括如下过程：
步骤S6041，确定所述第一图像中位于所述目标ROI区域内的图像特征点，得到第一图像特征点；
步骤S6042，在所述第二图像中确定所述第一图像特征点的匹配特征点，得到第二图像特征点；
步骤S6043，基于所述第一图像特征点和所述第二图像特征点之间的像素距离，确定所述ROI偏移量。
当确定目标ROI区域后，则需要抓取第一摄像装置和第二摄像装置同步拍摄的图像， 即第一图像和第二图像。然后，在目标ROI区域内寻找这两张图像的匹配特征点。例如，首先，确定第一图像中位于目标ROI区域内的图像特征点，得到第一图像特征点，其中，该第一图像特征点可以为目标ROI区域内所包含对象的特征点，例如，人脸的特征点等等，本公开实施例对此不作具体限定。接下来，在第二图像中确定第一图像特征点的匹配特征点，得到第二图像特征点。然后，计算第一图像特征点和第二图像特征点之间的像素距离，并将该像素距离作为ROI偏移量，在一些实施例中，可以计算第一图像特征点和第二图像特征点中相匹配像素点之间的坐标差值作为像素距离。
需要说明的是，在本公开实施例中，在计算ROI偏移量时，通常假设该目标ROI区域在空间中属于同一深度，即该区域距离摄像装置的距离大致相同，而不是该区域中一部分在前景区域，一部分在背景区域。此时，需要先确定目标ROI区域所对应的摄像装置，以确定该目标ROI区域的图像内容，再到另一张图像中去进行匹配，并获取该区域的偏移量。
下面将举例说明上述过程，在以下示例中，假设当前移动终端中显示的第一图像为FOV更大的第一摄像装置所采集到的，且用户手动对焦到某个区域(即，目标ROI区域)。
如图8所示，移动终端的屏幕的预览画面为Wide摄像装置(第一摄像装置)的图像(第一图像)，经触屏对焦得到目标ROI区域。本公开实施例基于目标ROI区域和第一图像得到目标ROI区域中的图像内容，再结合同一时间拍摄的Tele图像(即，第二摄像装置)，并基于特征点提取与匹配算法，得到多组匹配点集(其中，多组匹配点集即为上述第一图像特征点和第二图像特征点)。需要说明的是，在本公开实施例中，虽然在移动终端中不显示Tele(即，第二摄像装置)所拍摄到的图像，但在预览拍照或摄像所得的图像时，Wide(即，第一摄像装置)与Tele(即，第二摄像装置)同时打开，并时刻准备着各自采集到的图像被显示在屏幕上。需要说明的是，在本公开实施例中，特征点提取与匹配算法可以为以下任一种：SIFT(Scale-Invariant Feature Transform)，ORB(Oriented FAST and Rotated BRIEF)，BRISK(Binary Robust Invariant Scalable Keypoints)等，本公开实施例不限定具体的匹配算法。
在本公开实施例中，确定ROI偏移量(△u，△v)的流程可参照如下所示：
首先，利用鲁棒的特征点检测与匹配算法计算出N组匹配点对；即第一图像特征点和第二图像特征点；换言之，每组匹配点对都包含有第一图像特征点以及与其匹配的第二图像特征点。
然后，利用剔除错误匹配点的算法RANSAC对上述匹配点对进行误匹配点的剔除，保留下准确的匹配点对；
最后，求解保留下的准确的多组匹配点对之间的像素距离，并计算像素距离的平均值，将计算结果作为ROI偏移量。
在本公开实施例中，在得到ROI偏移量之后，就可以获取第一图像的当前放大倍率，并根据当前放大倍率对ROI偏移量进行切片处理，得到当前时刻对第一图像进行图像立体校正的当前叠加偏移量。
在一个可选的实施方式中，步骤S606，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量的步骤包括：
步骤S6061，获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切 为所述第二摄像装置时的图像放大倍率；
步骤S6062，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
在本公开实施例中，为了在对第一图像执行数字变焦过程中，保证整个数字变焦过程的平滑，当得到ROI偏移量之后，需要将ROI偏移量进行切片处理。在对第一图像放大过程中逐渐叠加不同比例的ROI偏移量(即，叠加偏移量)至第一图像，直至将第一图像放大到切换倍率时，总偏移量叠加完成。
在一些实施方式中，在本公开实施例中，可以通过比较当前放大倍率和目标放大倍率之间的大小关系来对ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移。通过对ROI偏移量进行切片处理，能够保证第一摄像装置和第二摄像装置在切换时目标ROI区域内的图像无跳变，具体实施方式如下：
由于ROI偏移量的计算与“光学数字联合变焦”同步进行，假设首次计算出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的初始放大倍率user level为ul 0，ROI偏移量的大小为S ROI，预设切换倍率switch level简写为sl。当前放大倍率user level为ul cur，当前时刻叠加到每一帧图像上的当前叠加偏移量为s cur，s cur的结果可以分为以下几种情况：
情况一、
若所述当前放大倍率小于所述初始放大倍率，则确定所述当前叠加偏移量为0。
也就是说，如果ul cur<ul 0，那么可以将当前叠加偏移量s cur设置为零。
情况二、
若所述当前放大倍率大于所述初始放大倍率，且小于所述预设切换倍率，则所述当前叠加偏移量的计算公式为
Figure PCTCN2021093706-appb-000008
其中，s cur表示所述当前叠加偏移量，S ROI表示所述ROI偏移量，ul cur表示所述当前放大倍率，ul0表示所述第一图像的初始放大倍率，sl表示所述预设切换倍率。
也就是说，如果ul 0<ul cur<sl时，那么当前叠加偏移量s cur可以通过公式
Figure PCTCN2021093706-appb-000009
来确定。
该公式即实现了ROI偏移量的切片效果，即将当前放大倍率与预设切换倍率switch level之间的距离比，对应到ROI偏移量S ROI上面，实现ul 0与ul 0的下一帧之间的偏移量足够小，防止图像的内容发生可见的平移跳变；同时当ul cur接近于sl时，s cur也基本等于S ROI。
情况三、
若所述当前放大倍率大于所述预设切换倍率，则所述当前叠加偏移量为所述ROI偏移量。
在本公开实施例中，若ul cur>sl，则s cur＝S ROI。需要说明的是，此时移动终端的屏幕显示第二摄像装置采集到的第二图像，但该当前叠加偏移量仍叠加到第一图像上，以便用户 从第二摄像装置切换回第一摄像装置时，仍可保证屏幕上显示的目标ROI区域内的图像无跳变。
在本公开实施例中，在按照上述情况一至情况三所描述的方式确定当前时刻对第一图像进行图像立体校正操作所需的当前叠加偏移量之后，就可以基于所述当前叠加偏移量对所述第一图像执行图像立体校正操作。
在一个可选的实施方式中，基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像包括如下步骤(1)～(2)：
(1)、基于所述当前叠加偏移量和图像放大参数构建目标变换矩阵；所述图像放大参数中包含所述第一图像的中心点和预设切换倍率；
(2)、计算所述目标变换矩阵和当前时刻所述第一图像的齐次坐标之间的乘积，并根据乘积计算结果确定进行图像立体校正之后的所述第一图像。
通过上述描述可知，数字变焦操作的具体实现方式是按照一定的缩放比例，构建图像变换矩阵H zoom，进而通过图像变换矩阵H zoom实现图像的变换。其中，构建图像变换矩阵和通过图像变换矩阵实现图像的变换的具体公式如下所示：
Figure PCTCN2021093706-appb-000010
因此，在确定出当前时刻对第一图像执行图像立体校正操作所需的当前叠加偏移量s cur＝(△u,△v)之后，仍然需要构建变换矩阵H shift，并将变换矩阵H shift叠加到图像变换矩阵H zoom中，其中，图像变换矩阵H zoom中包含图像放大参数。通过上述处理方式能够在对第一图像执行立体校正的同时，完成图像的平移。
在本公开实施例中，变换矩阵H shift公式如下：
Figure PCTCN2021093706-appb-000011
基于当前时刻对第一图像执行图像立体校正操作所需的当前叠加偏移量和图像放大参数所构建得到的目标变换矩阵的公式为：
Figure PCTCN2021093706-appb-000012
通过以上构建及变换得到新的Hzoom'(即，目标变换矩阵)后，即可在数字变焦流程中，实现图像放大的同时也实现图像的平移，将以上流程计算的偏移量作用在视频帧的输入图像中。
需要说明的是，在本公开实施例中，目标ROI区域是可以更新的，且目标ROI区域可以由用户来实现更新。因此无论是场景改变，还是对焦点改变，都有可能改变ROI偏移量。因此，在本公开实施例中，计算ROI偏移量的过程也要求在整个“光学数字联合变焦”过程中，按照一定的频率重复的进行。因此，也需要考虑当ROI偏移量改变后，叠加偏移量的 计算应该如何更新。
基于此，该方法还包括如下步骤S1～步骤S4：
步骤S1，按照预设时间间隔检测所述ROI偏移量是否发生了变化；
步骤S2，若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；
步骤S3，获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生时所述第一图像的放大倍率；
步骤S4，根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
在一些实施方式中，假设在ROI偏移量更新前，ROI偏移量S ROI＝S ROI_0；在ROI偏移量更新的前一时刻，第一图像的放大倍率userlevel(即，本公开实施例中的第二放大倍率)为ul cur_0。若ul 0<ul cur_0<sl，，此时，图像立体校正对应的当前叠加偏移量s cur_0如式
Figure PCTCN2021093706-appb-000013
所示，否则s cur_0应等于0或S ROI_0。
在ROI偏移量更新后，假设变化之后的ROI偏移量为S ROI_1，但由于此时图像帧已经按照S ROI_0为基准，叠加了叠加偏移量s cur_0的偏移量。因此，图像立体校正操作对应的目标ROI偏移量S ROI为：S ROI＝S ROI_1-s cur_0。
在得到目标ROI偏移量之后，可以继续获取检测到ROI偏移量发生时第一图像的放大倍率，即第一放大倍率。然后，根据第一放大倍率和目标ROI偏移量重新确定第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对第一图像进行图像立体校正。
因此，在此基础上，根据第一放大倍率和目标ROI偏移量重新确定所述第一图像的当前叠加偏移量可以分为以下几种情况：
若所述第一放大倍率和第二放大倍率相同，则所述第一图像的当前叠加偏移量和上一时刻的叠加偏移量相同，其中，所述第二放大倍率为上一时刻图像立体校正操作所对应的图像放大倍率。
在此情况下，ROI偏移量更新后，但放大倍率user level并没有改变，即第一放大倍率ul cur＝第二放大倍率ul cur_0，此时，当前时刻对第一图像执行图像立体校正操作所需的当前叠加偏移量和上一时刻对第一图像执行图像立体校正操作所需的叠加偏移量相同，即：s cur＝s cur_0。
在此情况下，虽然第一放大倍率ul cur可能大于初始放大倍率ul 0且小于预设切换倍率sl，但此时图像帧已经叠加了叠加偏移量s cur_0的偏移量，若按照切片公式来计算当前时刻对第一图像执行图像立体校正操作的叠加偏移量s cur的话，ROI偏移量S ROI则会发生改变，即放大倍率user level没有发生改变的时候，屏幕中的内容却产生了跳变，这种跳变是肉眼可能感知的。因此，当ROI偏移量更新后，需要先判断用户控制的放大倍率user level(第一放大倍率)有没有发生变化，如果还未改变，则仍保留原s cur＝s cur_0。
若所述第一放大倍率和所述第二放大倍率不相同，则根据所述第一放大倍率和所述目标ROI偏移量确定所述第一图像的当前叠加偏移量。
在此情况下，当第一放大倍率user level发生改变后，则继续按照上述情况一至情况三 中所描述的方式确定当前时刻图像立体校正操作的叠加偏移量，此处不再展开描述。
如图9所示，在本公开实施例中，图像的处理方法主要包含以下三个部分：
一、确定目标ROI区域
在本公开实施例中，目标ROI区域是指用户感兴趣区域。对于可触屏的移动终端而言，通常当用户点击屏幕某一位置进行对焦时，说明用户想关注这一块区域，因此移动终端会将该区域作为目标ROI区域。也即，在用户手动对焦时，可以基于用户在屏幕上的触碰位置确定ROI区域。如果用户没有手动对焦，移动终端会自动将屏幕中心区域作为目标ROI区域，或者自动检测屏幕中诸如人脸等目标对象的位置，根据检测结果确定目标ROI区域等。这些都是移动终端常用的获取目标ROI区域的方式。在“光学数字联合变焦”算法中，虽然同时打开两个摄像装置(例如，第一摄像装置和第二摄像装置)，但移动终端的屏幕只会显示其中一个摄像装置的画面，因此这里的目标ROI区域可能是Wide镜头(第一摄像装置)的ROI区域，也可能是Tele镜头(第二摄像装置)的ROI区域。
二、进行纹理检测
通过上述描述可知，在确定出第一图像(也即，Wide图像帧)中的目标ROI区域之后，基于目标ROI区域在所述第一图像和第二图像(也即，Tele图像帧)中的偏移量，得到ROI偏移量的过程可以总结如下：
ROI纹理性验证：为了确保获取的目标ROI区域是可匹配的(即有纹理特征)，在进行匹配前，首先会对该区域的纹理性进行验证。验证方法主要是基于该区域每个像素的灰度值或RGB值计算标准差，如果标准差小于预设纹理强度，认为该区域没有纹理，则退出当前计算；否则进入下一步计算。
图像特征点的提取与匹配：相关算法有很多，如SIFT，ORB，BRISK等，本公开实施例不明确定义具体寻找匹配点的方法。在本公开实施例中，可以通过上述算法在第一图像中确定第一图像特征点，并在第二图像中确定第一图像特征点的匹配特征点，得到第二图像特征点。
三、计算ROI偏移量
完成图像特征点的提取与匹配后，通常会得到不止一个匹配点，此时为了剔除错误的匹配点，还需要基于诸如RANSAC(Random Sample Consensus)等算法进行一次错误匹配点的剔除。得到正确的匹配点对后，计算每一个点对的像素偏移量，再求其平均值，得到最终的偏移量(即，ROI偏移量)。
四、平滑消除ROI的偏移量
在得到ROI偏移量后，便可以基于该数据对“光学数字联合变焦”过程中的图像进行平移，使得当图像放大到可以切换摄像装置时，在目标ROI区域处无跳变。但一方面由于ROI偏移量的获取是在“光学数字联合变焦”的过程中进行的；另一方面由于目标ROI区域也是可以由用户控制的，也即目标ROI区域是可变化的，所以ROI偏移量的计算需要频繁进行并与变焦过程同步进行，以适应场景的变化和目标ROI区域的变化。因此，获取到的ROI偏移量应如何应用在变焦过程中既能保证切换时目标ROI区域无跳变，还能让整个数字变焦过程平滑，是最重要的技术问题。
首先，假设在当前帧获取到ROI偏移量后，不可能直接将该ROI偏移量直接作用于下一帧，因为这样的话，会直接导致当前帧和下一帧之间产生跳变，而非平滑变焦；但同时需要保证当变焦到达摄像装置切换帧时，ROI偏移量已经作用于当前帧，使得下一帧切换 为另一个摄像装置时，图像内容在目标ROI区域处没有跳变。因此，本公开实施例提出一种平滑消除ROI偏移量的方法，既能保证数字变焦过程的平滑放大，不因ROI偏移量的出现和更新时产生跳变，又能确保在切换摄像装置的时候，目标ROI区域无跳变。
在此需要说明的是，消除ROI偏移量的行为主要是针对图像进行平移的过程。由于Wide的FOV远大于Tele，为了确保Wide在切换摄像装置时与Tele在目标ROI区域处对齐，所以基于计算得到的ROI偏移量主要作用于Wide图像帧。因此具体实现流程如下：
数字变焦过程在第一次得到ROI偏移量之前，通常都基于放大倍率(user level)对图像进行中心放大或缩小。当第一次得到ROI偏移量后，按照当前user level与switch level之间的距离，对偏移量的值进行切片，在user level继续放大的过程中，叠加不同比例的叠加偏移量，使得user level在越接近switch level时，偏移量越大，反之越小，直至user level＝switch level时，ROI偏移量完全作用于图像帧。由于ROI偏移量已经被切片，因此可以保证数字变焦过程的平滑，不至于在放大时，图像产生跳变。该步骤还需注意的是，若user level没有变化，则保持当前的偏移量持续作用到图像帧；若user level>switch level，此时通常显示Tele图像帧，因此不需要对Tele图像帧叠加偏移量。
由于场景变化或用户重新对焦到距离摄像装置不同深度的位置时，ROI偏移量会发生变化。因此，还需要考虑到ROI偏移量变化的情况。此时，假设在ROI偏移量被更新的时候，此时的user level2相比上一次得到ROI偏移量时的user level1也已经放大了一定的倍率，即当前帧的图像已经基于上一次的叠加偏移量有了一定像素值的平移，那么则需要基于当前已更新的ROI偏移量减去当前已经平移后的ROI偏移量，作为当前时刻的ROI偏移量，并重新对该ROI偏移量进行切片，以实现即便ROI偏移量更新后，也能在数字变焦平滑过度的同时仍保证切换摄像装置时ROI处没有跳变。
本公开实施例还提供了一种图像的处理装置，该图像的处理装置主要用于执行本公开实施例上述内容所提供的图像的处理方法，以下对本公开实施例提供的图像的处理装置做具体介绍。
图10是根据本公开实施例的一种图像的处理装置的示意图，如图10所示，该图像的处理装置主要包括：第一获取单元10，计算单元20，第二获取单元30，切片处理单元40，图像校正单元50和切换显示单元60，其中：
第一获取单元，配置成获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域；
计算单元，配置成计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像；
第二获取单元，配置成获取所述第一图像的当前放大倍率；
切片处理单元，配置成根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量；
图像立体校正单元，配置成基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像；
切换显示单元，配置成若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
通过上述描述可知，在本公开实施例中，通过基于第一图像的当前放大倍率，并基于当前放大倍率确定当前叠加偏移量，并通过该当前叠加偏移量对第一图像进行图像立体校正的方式，能够保证在两个摄像装置切换时，用户关注的ROI的内容无跳变，进而缓解了采用现有技术中在进行摄像装置成像切换时跳变明显的技术问题。
可选地，该装置还包括纹理检测单元：纹理检测单元配置成：对所述第一图像中位于所述目标ROI区域内的图像进行纹理检测，得到纹理强度；计算单元还配置成：若确定出所述第一图像中位于所述目标ROI区域内的图像的纹理强度大于预设纹理强度，则计算所述目标ROI区域在所述第一图像和第二图像中所述目标ROI区域的ROI偏移量。
可选地，纹理检测单元进一步配置成：基于所述第一图像中位于所述目标ROI区域内的像素点计算目标数值，并基于所述目标数值确定所述纹理强度，其中，所述纹理强度包括以下任一种数值计算得到的标准差：像素灰度值、像素RGB值。
可选地，计算单元还配置成：确定所述第一图像中位于所述目标ROI区域内的图像特征点，得到第一图像特征点；在所述第二图像中确定所述第一图像特征点的匹配特征点，得到第二图像特征点；基于所述第一图像特征点和所述第二图像特征点之间的像素距离，确定所述ROI偏移量。
可选地，切片处理单元配置成：获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切为所述第二摄像装置时的图像放大倍率；根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
可选地，切片处理单元还配置成：若所述当前放大倍率小于所述初始放大倍率，则确定所述当前叠加偏移量为0。
可选地，切片处理单元还配置成：若所述当前放大倍率大于所述初始放大倍率，且小于所述预设切换倍率，则所述当前叠加偏移量的计算公式为
Figure PCTCN2021093706-appb-000014
其中，s cur表示所述当前叠加偏移量，S ROI表示所述ROI偏移量，ul cur表示所述当前放大倍率，ul0表示所述第一图像的初始放大倍率，sl表示所述预设切换倍率。
可选地，切片处理单元还配置成：若所述当前放大倍率大于所述预设切换倍率，则所述当前叠加偏移量为所述ROI偏移量。
可选地，图像立体校正单元配置成：基于所述当前叠加偏移量和图像放大参数构建目标变换矩阵；所述图像放大参数中包含所述第一图像的中心点和预设切换倍率；计算所述目标变换矩阵和当前时刻所述第一图像的齐次坐标之间的乘积，并根据乘积计算结果确定进行图像立体校正之后的所述第一图像。
可选地，该装置还包括重新校正单元，重新校正单元配置成：在计算出所述ROI偏移量之后，按照预设时间间隔检测所述ROI偏移量是否发生了变化；若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生 时所述第一图像的放大倍率；根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
可选地，重新校正单元进一步配置成：若所述第一放大倍率和第二放大倍率相同，则所述第一图像的当前叠加偏移量和上一时刻的叠加偏移量相同，其中，所述第二放大倍率为上一时刻图像立体校正操作所对应的图像放大倍率。
可选地，重新校正单元进一步配置成：若所述第一放大倍率和所述第二放大倍率不相同，则根据所述第一放大倍率和所述目标ROI偏移量确定所述第一图像的当前叠加偏移量。
本公开实施例所提供的装置，其实现原理及产生的技术效果和前述方法实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。
本公开实施例还提供了一种移动终端，包括处理器，以及与所述处理器分别连接的第一摄像装置和第二摄像装置，该移动终端也可称为双摄终端，诸如可以为双摄手机等。移动终端内的处理器可执行上述任一项所述的图像的处理方法的步骤。
另外，在本公开实施例的描述中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本公开中的具体含义。
在本公开的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计 算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。
工业实用性
本公开提出的技术方案中，通过基于第一图像的当前放大倍率，并基于当前放大倍率确定当前叠加偏移量，并通过该当前叠加偏移量对第一图像进行图像立体校正的方式，能够保证在两个摄像装置切换时，用户关注的ROI的内容无跳变，进而缓解了采用现有技术中在进行摄像装置成像切换时跳变明显的技术问题。

Claims (18)

  1. 一种图像的处理方法，应用于移动终端，所述移动终端包括：第一摄像装置和第二摄像装置，其特征在于，所述方法包括：
    获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域；
    计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像；
    获取所述第一图像的当前放大倍率，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量；
    基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像；
    若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
  2. 根据权利要求1所述的方法，其特征在于，
    所述方法还包括：对所述第一图像中位于所述目标ROI区域内的图像进行纹理检测，得到纹理强度；
    计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量包括：若确定出所述第一图像中位于所述目标ROI区域内的图像的纹理强度大于预设纹理强度，则计算所述目标ROI区域在所述第一图像和第二图像中所述目标ROI区域的ROI偏移量。
  3. 根据权利要求2所述的方法，其特征在于，对所述第一图像中位于所述目标ROI区域内的图像进行纹理检测，得到纹理强度包括：
    基于所述第一图像中位于所述目标ROI区域内的像素点计算目标数值，并基于所述目标数值确定所述纹理强度，其中，所述纹理强度包括以下任一种数值计算得到的标准差：像素灰度值、像素RGB值。
  4. 根据权利要求1或2所述的方法，其特征在于，计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量包括：
    确定所述第一图像中位于所述目标ROI区域内的图像特征点，得到第一图像特征点；
    在所述第二图像中确定所述第一图像特征点的匹配特征点，得到第二图像特征点；
    基于所述第一图像特征点和所述第二图像特征点之间的像素距离，确定所述ROI偏移量。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量包括：
    获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切为所述第二摄像装置时的图像放大倍率；
    根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
  6. 根据权利要求5所述的方法，其特征在于，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量包括：
    若所述当前放大倍率小于所述初始放大倍率，则确定所述当前叠加偏移量为0。
  7. 根据权利要求5或6所述的方法，其特征在于，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量还包括：
    若所述当前放大倍率大于所述初始放大倍率，且小于所述预设切换倍率，则所述当前叠加偏移量的计算公式为
    Figure PCTCN2021093706-appb-100001
    其中，s cur表示所述当前叠加偏移量，S ROI表示所述ROI偏移量，ul cur表示所述当前放大倍率，ul0表示所述第一图像的初始放大倍率，sl表示所述预设切换倍率。
  8. 根据权利要求5至7任一项所述的方法，其特征在于，根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量还包括：
    若所述当前放大倍率大于所述预设切换倍率，则所述当前叠加偏移量为所述ROI偏移量。
  9. 根据权利要求1至8任一项所述的方法，其特征在于，基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像包括：
    基于所述当前叠加偏移量和图像放大参数构建目标变换矩阵；所述图像放大参数中包含所述第一图像的中心点和预设切换倍率；
    计算所述目标变换矩阵和当前时刻所述第一图像的齐次坐标之间的乘积，并根据乘积计算结果确定进行图像立体校正之后的所述第一图像。
  10. 根据权利要求1至9任一项所述的方法，其特征在于，所述方法还包括：
    在计算出所述ROI偏移量之后，按照预设时间间隔检测所述ROI偏移量是否发生了变化；
    若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；
    获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生时所述第一图像的放大倍率；
    根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
  11. 根据权利要求10所述的方法，其特征在于，根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量包括：
    若所述第一放大倍率和第二放大倍率相同，则所述第一图像的当前叠加偏移量和上一时刻的叠加偏移量相同，其中，所述第二放大倍率为上一时刻图像立体校正操作所对应的图像放大倍率。
  12. 根据权利要求11所述的方法，其特征在于，根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量包括：
    若所述第一放大倍率和所述第二放大倍率不相同，则根据所述第一放大倍率和所述目标ROI偏移量确定所述第一图像的当前叠加偏移量。
  13. 一种图像的处理装置，设置于移动终端，所述移动终端包括：第一摄像装置和第二摄像装置，其特征在于，所述装置包括：
    第一获取单元，配置成获取所述第一摄像装置采集到的第一图像；并确定所述第一图像中的目标ROI区域；
    计算单元，配置成计算所述第一图像和第二图像中所述目标ROI区域的ROI偏移量，其中，所述第二图像为所述第二摄像装置在所述第一摄像装置采集第一图像时同步拍摄到的图像；
    第二获取单元，配置成获取所述第一图像的当前放大倍率；
    切片处理单元，配置成根据所述当前放大倍率对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量；
    图像立体校正单元，配置成基于所述当前叠加偏移量对所述第一图像进行图像立体校正，得到校正之后的所述第一图像；
    切换显示单元，配置成若所述当前放大倍率为预设切换倍率，则所述移动终端由所述第一摄像装置切换至所述第二摄像装置进行显示；其中，在切换时，对所述第一图像进行图像立体校正的叠加偏移量的总和为所述ROI偏移量。
  14. 根据权利要求13所述的装置，其特征在于，所述切片处理单元进一步配置成：
    获取目标放大倍率，其中，所述目标放大倍率包括：初始放大倍率和预设切换倍率；所述初始放大倍率为首次确定出ROI偏移量的时候，第一摄像装置对第一图像执行数字变焦操作时所对应的放大倍率；所述预设切换倍率表示将所述第一摄像装置切为所述第二摄像装置时的图像放大倍率；
    根据所述当前放大倍率与所述目标放大倍率之间的大小关系对所述ROI偏移量进行切片处理，得到当前时刻对所述第一图像进行图像立体校正的当前叠加偏移量。
  15. 根据权利要求13或14所述的装置，其特征在于，所述装置还包括重新校正单元，所述重新校正单元配置成：
    在计算出所述ROI偏移量之后，按照预设时间间隔检测所述ROI偏移量是否发生了变化；
    若检测出所述ROI偏移量发生了变化，则基于变化之后的ROI偏移量和所述第一图像的当前叠加偏移量确定目标ROI偏移量；
    获取第一放大倍率，其中，所述第一放大倍率为检测到所述ROI偏移量发生时所述第一图像的放大倍率；
    根据所述第一放大倍率和所述目标ROI偏移量重新确定所述第一图像的当前叠加偏移量，并根据重新确定出的当前叠加偏移量对所述第一图像进行图像立体校正。
  16. 一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述权利要求1至12中任一项所述的方法的步骤。
  17. 一种具有处理器可执行的非易失的程序代码的计算机可读介质，其特征在于，所述程序代码使所述处理器执行上述权利要求1至12中任一项所述的方法的步骤。
  18. 一种移动终端，其特征在于，包括处理器，以及与所述处理器分别连接的第一摄像装置和第二摄像装置，所述处理器执行上述权利要求1至12中任一项所述的方法的步骤。
PCT/CN2021/093706 2020-07-07 2021-05-13 图像的处理方法、装置、电子设备、可读介质及移动终端 WO2022007500A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010649505.6A CN111935398B (zh) 2020-07-07 2020-07-07 图像的处理方法、装置、电子设备和计算机可读介质
CN202010649505.6 2020-07-07

Publications (1)

Publication Number Publication Date
WO2022007500A1 true WO2022007500A1 (zh) 2022-01-13

Family

ID=73313391

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/093706 WO2022007500A1 (zh) 2020-07-07 2021-05-13 图像的处理方法、装置、电子设备、可读介质及移动终端

Country Status (2)

Country Link
CN (1) CN111935398B (zh)
WO (1) WO2022007500A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115200555A (zh) * 2022-09-19 2022-10-18 中国科学院长春光学精密机械与物理研究所 一种动态摄影测量内外方位元素标定方法及装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111935398B (zh) * 2020-07-07 2022-04-22 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质
CN114531539B (zh) * 2020-11-23 2024-03-19 华为技术有限公司 拍摄方法及电子设备
CN113630549B (zh) * 2021-06-18 2023-07-14 北京旷视科技有限公司 变焦控制方法、装置、电子设备和计算机可读存储介质
CN115550544B (zh) * 2022-08-19 2024-04-12 荣耀终端有限公司 图像处理方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959553A (zh) * 2016-05-30 2016-09-21 维沃移动通信有限公司 一种摄像头的切换方法及终端
CN107277360A (zh) * 2017-07-17 2017-10-20 惠州Tcl移动通信有限公司 一种双摄像头切换进行变焦的方法、移动终端及存储装置
CN107896303A (zh) * 2017-10-23 2018-04-10 努比亚技术有限公司 一种图像采集方法、系统和设备及计算机可读存储介质
US10429608B1 (en) * 2016-09-23 2019-10-01 Apple Inc. Primary-subordinate camera focus based on lens position sensing
CN111935398A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103096119B (zh) * 2011-10-28 2015-10-07 浙江大华技术股份有限公司 一种摄像头变倍检测方法及装置
CN106161941B (zh) * 2016-07-29 2022-03-11 南昌黑鲨科技有限公司 双摄像头自动追焦方法、装置及终端
CN205883406U (zh) * 2016-07-29 2017-01-11 深圳众思科技有限公司 双摄像头自动追焦装置及终端
CN107770433B (zh) * 2016-08-15 2020-08-04 广州立景创新科技有限公司 影像获取装置及其影像平顺缩放方法
CN106303258A (zh) * 2016-09-19 2017-01-04 深圳市金立通信设备有限公司 一种基于双摄像头的拍摄方法和终端
CN108833768A (zh) * 2018-05-10 2018-11-16 信利光电股份有限公司 一种多摄像头的拍摄方法、拍摄终端和可读存储介质
CN109379537A (zh) * 2018-12-30 2019-02-22 北京旷视科技有限公司 滑动变焦效果实现方法、装置、电子设备及计算机可读存储介质
CN110784651B (zh) * 2019-11-15 2021-06-29 维沃移动通信有限公司 一种防抖方法及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959553A (zh) * 2016-05-30 2016-09-21 维沃移动通信有限公司 一种摄像头的切换方法及终端
US10429608B1 (en) * 2016-09-23 2019-10-01 Apple Inc. Primary-subordinate camera focus based on lens position sensing
CN107277360A (zh) * 2017-07-17 2017-10-20 惠州Tcl移动通信有限公司 一种双摄像头切换进行变焦的方法、移动终端及存储装置
CN107896303A (zh) * 2017-10-23 2018-04-10 努比亚技术有限公司 一种图像采集方法、系统和设备及计算机可读存储介质
CN111935398A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115200555A (zh) * 2022-09-19 2022-10-18 中国科学院长春光学精密机械与物理研究所 一种动态摄影测量内外方位元素标定方法及装置
CN115200555B (zh) * 2022-09-19 2022-11-25 中国科学院长春光学精密机械与物理研究所 一种动态摄影测量内外方位元素标定方法及装置

Also Published As

Publication number Publication date
CN111935398A (zh) 2020-11-13
CN111935398B (zh) 2022-04-22

Similar Documents

Publication Publication Date Title
WO2022007500A1 (zh) 图像的处理方法、装置、电子设备、可读介质及移动终端
WO2021208371A1 (zh) 多摄变焦控制方法、装置、电子系统及存储介质
US20230353869A1 (en) Imaging apparatus and setting screen thereof
US10389948B2 (en) Depth-based zoom function using multiple cameras
US9973672B2 (en) Photographing for dual-lens device using photographing environment determined using depth estimation
CN112672022B (zh) 视频支持和切换/无切换动态控制的双孔径变焦摄影机
CN105453136B (zh) 使用自动聚焦反馈进行立体侧倾校正的系统、方法及设备
JP5592006B2 (ja) 三次元画像処理
TWI433530B (zh) 具有立體影像攝影引導的攝影系統與方法及自動調整方法
WO2020007320A1 (zh) 多视角图像的融合方法、装置、计算机设备和存储介质
US9167224B2 (en) Image processing device, imaging device, and image processing method
CN112261387B (zh) 用于多摄像头模组的图像融合方法及装置、存储介质、移动终端
KR20140108078A (ko) 논-스테레오스코픽 카메라를 이용하여 스테레오스코픽 이미지를 생성하는 방법과 기기 및 장치
TWI629550B (zh) 影像擷取裝置及其縮放影像的方法
CN111935397B (zh) 图像的处理方法、装置、电子设备和计算机可读介质
JP2014531860A (ja) 立体視画像ペアの条件付き表示のための方法及び装置
WO2021189804A1 (zh) 图像矫正方法、装置和电子系统
WO2019006762A1 (zh) 一种图像捕捉装置及方法
JP6732440B2 (ja) 画像処理装置、画像処理方法、及びそのプログラム
JP2013192152A (ja) 撮像装置、撮像システムおよび画像処理方法
CN112053403B (zh) 双摄像头的光轴夹角确定方法及装置
JP2014134723A (ja) 画像処理装置、画像処理方法およびプログラム
WO2024119319A1 (en) Electronic device, method of controlling electronic device, and computer readable storage medium
US9076215B2 (en) Arithmetic processing device
JP2024002631A (ja) 画像処理装置、撮像装置、画像処理方法、コンピュータのプログラムおよび記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21837149

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28-04-2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21837149

Country of ref document: EP

Kind code of ref document: A1